System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting a destination and a travel method by a user; means for transmitting the input information to a server; means for calculating an optimal route based on the destination and the travel method; means for searching a sightseeing spot and a restaurant in the vicinity of the route from a database; means for calculating a time and a cost added by stopping at each stopover spot; means for selecting an optimal stopover plan in consideration of past use data of the user; and means for presenting the stopover plan to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] While it is relatively easy to research tourist spots near a destination while traveling, finding detour spots while traveling can be extremely time-consuming. For a trip where travel is the majority of the time, enjoying the journey itself can improve the quality of the trip. However, current technology makes it difficult to plan such enjoyable experiences while traveling, and many travelers miss out on these opportunities. Furthermore, information about tourist spots in depopulated areas is not provided adequately, limiting the contribution to regional revitalization. The present invention aims to solve these problems, provide travelers with an enjoyable experience while traveling, and contribute to the revitalization of depopulated areas. [Means for solving the problem]

[0005] The present invention proposes a system that includes a means for a user to input a destination and a mode of transportation, a means for transmitting the input information to a server, a means for calculating an optimal route based on the destination and mode of transportation, a means for searching a database for tourist spots and restaurants around the route, a means for calculating the additional time and cost of stopping at each detour spot, a means for selecting an optimal detour plan taking into account the user's past usage data, and a means for presenting the detour plan to the user. This allows users to easily plan detours and enjoy their trip while traveling. Furthermore, by proactively introducing tourist spots in depopulated areas, the system can contribute to regional revitalization.

[0006] "User" refers to a traveler who uses the system to input their travel destination and mode of transportation and receive suggestions for detour plans.

[0007] "Destination" refers to the place where a traveler ultimately arrives.

[0008] "Mode of travel" refers to the means of transportation used by travelers to reach their destination.

[0009] "Server" refers to a central processing unit that receives input information and calculates optimal routes and generates detour plans.

[0010] The term "terminal" refers to a device on which a user performs input operations, transmits necessary information to a server, and displays information from the server.

[0011] "Route" refers to the path to reach a destination.

[0012] "Tourist attractions" refer to tourist destinations and famous places that may be of interest to travelers.

[0013] "Restaurants" refers to establishments where travelers can eat meals.

[0014] A "database" refers to a collection of data that stores information on tourist spots, restaurants, and so on.

[0015] "Detour spots" refer to tourist attractions and restaurants that the user can stop at along their travel route.

[0016] "Time" refers to the additional time required to visit a location.

[0017] "Cost" refers to the additional economic costs involved in visiting a location.

[0018] "Usage Data" refers to information about a user's past usage history and preferences.

[0019] A "side trip plan" refers to a collection of side trip spots suggested along a user's travel route.

[0020] "Presenting" refers to visually displaying information sent from the server to the user. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] This invention is a system that suggests optimal routes and detour spots when travelers input their destination and transportation method, providing an enjoyable travel experience while on the move. Furthermore, it contributes to regional revitalization by introducing tourist spots in depopulated areas.

[0043] First, the user launches the application on a device such as a smartphone or tablet. The user then inputs their destination and travel method on the application interface. This input information is then sent from the device to the server.

[0044] Once the server receives the destination and mode of travel, it calculates the optimal route. Using a map API, the server generates a route from the starting point to the destination. The server then queries a database to find tourist attractions and restaurants near the route. This database contains information on a variety of tourist attractions and restaurants.

[0045] The server then calculates the additional time and cost of each detour stop, again using the map API to calculate the route changes and travel time required for each stop, as well as any additional costs associated with each stop (entrance fees, food and drink, etc.).

[0046] At this stage, the server takes into account the user's past usage and preference data. It references the user's history and analyzes their interest in specific categories (e.g., historical places, natural scenery, etc.). It then applies machine learning algorithms to select the most suitable detour plan for the user.

[0047] The server then generates a detour plan based on the selected itinerary and sends it to the device. The device then displays the plan on the application interface, allowing the user to visually confirm the detour plan. Detailed information about each stop, as well as additional time and costs, are also displayed, allowing the user to easily approve or customize the plan.

[0048] As a concrete example, consider a user traveling by car from Osaka to Kyoto. When the user inputs the destination "Kyoto" and the mode of travel "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches a database for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost required for each. Based on this information, and taking into account the user's tendency to prefer historical places in the past, it proposes a plan that includes the Great Buddha of Nara and the teahouse. The terminal visually presents this to the user, who then approves it as the final plan.

[0049] In this way, the present invention is a system that improves the quality of travel and contributes to regional revitalization by suggesting optimal routes and detour spots based on the destination and mode of travel.

[0050] The processing flow will be explained below.

[0051] Step 1:

[0052] The user launches the application. The device displays a user interface and provides a screen for inputting the destination and travel method.

[0053] Step 2:

[0054] The user inputs the destination and transportation method. Specifically, the user inputs "Kyoto" in the text box and selects "Car" from the drop-down menu.

[0055] Step 3:

[0056] The device sends the entered destination and travel method to the server. The device packages the input data as an HTTP request and sends it to the server.

[0057] Step 4:

[0058] The server receives the input information and calculates the optimal route. The server calls the map API and generates a route from the starting point (the user's current location) to the destination (Kyoto).

[0059] Step 5:

[0060] The server searches for tourist attractions and restaurants around the route. The server queries a database to obtain a list of tourist attractions (e.g., the Great Buddha of Nara) and restaurants (e.g., local teahouses) within a certain range of the route.

[0061] Step 6:

[0062] The server calculates the additional time and cost of stopping at each detour spot. The server again uses the map API to calculate the route and travel time that will be changed by stopping at each spot. It also calculates additional costs such as entrance fees and food and drink costs at each spot.

[0063] Step 7:

[0064] The server selects the optimal detour plan based on the user's past usage data and preference data. The server refers to the user's past history and analyzes their interest in specific categories using machine learning algorithms.

[0065] Step 8:

[0066] The server generates an optimal detour plan and sends it to the device. Specifically, it generates data in a structured data format (e.g., JSON) including information such as detour spots, additional time, and costs, and sends it to the device as an HTTP response.

[0067] Step 9:

[0068] The device receives the plan from the server and displays it on the user interface. The device analyzes the received data, displays detour spots on the map with markers, and visually presents detailed information about each spot.

[0069] Step 10:

[0070] The user can review the presented detour plan and customize it as needed. The user can add or remove spots, change the order, and the device will resubmit the finalized plan to the server.

[0071] Step 11:

[0072] The user approves the final plan and begins the trip. The device saves the final plan locally and enters navigation mode. The user travels along the plan, and the device provides necessary navigation and detour information.

[0073] Example 1

[0074] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0075] Conventional travel planning systems require users to manually set their destination and route, which requires a lot of time and effort when selecting detour spots and calculating travel time and additional costs. Furthermore, it is difficult to provide optimal detour plans that take into account the user's travel preferences and past usage. As a result, users' travel experiences are limited, and they are unable to fully utilize local tourist spots.

[0076] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0077] In this invention, the server includes a means for selecting an optimal detour plan taking into account the user's past usage data and preference data, a means for using a map API to calculate the optimal route, and a means for analyzing the user's preference data using a machine learning algorithm. This allows the user to automatically receive optimal routes and detour spots without having to manually create complex plans, thereby improving the quality of the travel experience.

[0078] 1. "User" refers to a person who uses the system to set a destination and specify a method of travel.

[0079] 2. "Terminal" refers to a mobile information terminal such as a smartphone or tablet, which transmits information entered by the user to a server.

[0080] 3. "Server" refers to the central processing unit that receives and analyzes information entered by the user and calculates and generates optimal routes and detour plans.

[0081] 4. "Destination" refers to the final destination set by a user as the purpose of a trip or movement.

[0082] 5. "Method of transportation" refers to the means of transportation specified by the user (e.g., car, train, bus, etc.).

[0083] 6. "Optimal route" refers to the most efficient and effective route from a starting point to a destination.

[0084] 7. "Tourist spot" refers to a famous place or tourist destination that travelers visit.

[0085] 8. "Restaurant" refers to an establishment where travelers stop to have a meal.

[0086] 9. "Database" refers to an information aggregation system in which information on tourist spots and restaurants is systematically stored.

[0087] 10. "Detour Spot" refers to a tourist spot or restaurant located on the user's travel route that can be visited along the way.

[0088] 11. "Additional time" refers to the extra travel time or time spent at a detour spot.

[0089] 12. "Additional costs" refers to fees such as entrance fees and food and drink costs incurred by stopping at detour spots.

[0090] 13. "Preference Data" refers to information regarding the categories and types of places that a User has previously visited.

[0091] 14. "Optimal detour plan" refers to a plan that combines the most suitable tourist spots and restaurants to stop at while traveling to the user's destination.

[0092] 15. "Visual display" refers to displaying information in an easily viewable manner as text or images on a user interface.

[0093] 16. "Map API" means an application program interface that provides map information and enables route calculation and geographic data retrieval.

[0094] 17. "Machine learning algorithm" refers to a computational method that learns patterns from past data and makes predictions or classifications for new data.

[0095] MODE FOR CARRYING OUT THE INVENTION

[0096] This invention is a system that suggests optimal routes and detour spots when travelers input their destination and transportation method, providing an enjoyable travel experience while on the move. Furthermore, by introducing tourist spots in depopulated areas, it contributes to regional revitalization.

[0097] Hardware and Software

[0098] 1. Terminal

[0099] Mobile information terminals such as smartphones and tablets

[0100] Any device with internet connectivity

[0101] 2. Server

[0102] A high-performance computer that acts as a central processing unit

[0103] Map APIs, database systems, and computers capable of running machine learning algorithms

[0104] software

[0105] 1. Map API

[0106] Google (registered trademark) Maps API, etc.

[0107] 2. Database

[0108] A database that stores information about tourist attractions and restaurants and allows you to run SQL queries

[0109] 3. Machine Learning Algorithms

[0110] Analyze user preference data using algorithms such as K-means clustering

[0111] How it works

[0112] 1. Device operation

[0113] The user operates a smartphone or tablet and launches a travel assistance application.

[0114] Enter your destination and transportation method on the application interface.

[0115] 2. Sending input information

[0116] The device collects the input information and sends it to the server, using an internet connection.

[0117] 3. Calculating the optimal route

[0118] The server calls the Google Maps API or similar to calculate the optimal route from the starting point to the destination.

[0119] 4. Search for tourist attractions and restaurants

[0120] The server searches a database of tourist attractions and restaurants around the route, including categories and location information.

[0121] 5. Calculation of additional time and costs

[0122] The server calculates route changes, travel time, and additional costs for each detour, using information obtained from the map API and other databases.

[0123] 6. Considering user preferences

[0124] The server analyzes the user's past usage data and preference data, and applies machine learning algorithms to select the optimal detour plan for the user.

[0125] 7. Detour Plan Generation

[0126] The server generates a detour plan based on the optimal route, combining related detour spots and their detailed information, and sends the generated plan to the device.

[0127] 8. Visual Indications

[0128] The device receives the plan and displays it visually on the application interface with a map, where the user can review the details of each stop, the additional time and cost, and then approve or customize it.

[0129] Specific examples

[0130] Consider an example of a user traveling by car from Osaka to Kyoto. The user inputs the destination "Kyoto" and the mode of travel "car," and the server calculates the optimal route from Osaka to Kyoto. It then searches the database for detour spots, such as the Great Buddha of Nara and famous teahouses in the area, and calculates the additional time and cost involved for each. Based on this information and taking into account the user's past preference for historical places, the server proposes a plan that includes the Great Buddha of Nara and the teahouse. The terminal visually presents this to the user, who then approves it as the final plan.

[0131] Prompt Sentence Examples

[0132] "I'm traveling by car from Osaka to Kyoto. Please suggest the best route, including a stop at the Great Buddha and a teahouse in Nara."

[0133] "Please create a driving plan from Osaka to Kyoto, including tourist spots and restaurants."

[0134] In this way, the present invention is a system that suggests optimal routes and detour spots based on the destination and mode of travel, improving the quality of travel while also contributing to regional revitalization.

[0135] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0136] Step 1:

[0137] The user starts the application and inputs the destination and the method of travel.

[0138] The user launches the application on a smartphone or tablet and inputs their destination and mode of transportation on the interface. The input data is the destination (e.g., Kyoto) and mode of transportation (e.g., car). This input provides the initial data for calculating the optimal route in the next step.

[0139] Step 2:

[0140] The terminal sends the input information to the server.

[0141] The terminal sends the destination and travel method information entered by the user to the server in the form of an HTTP request. The input here is the destination and travel method from the user, and the output is the request data sent to the server.

[0142] Step 3:

[0143] The server calculates the optimal route.

[0144] Based on the received destination and travel method, the server calls a map API such as Google Maps API to calculate the optimal route from the starting point to the destination. The input for this process is the transmitted destination and travel method, and the output is the optimal route information. The server analyzes the route data returned from the API and obtains an efficient route from the starting point to the destination.

[0145] Step 4:

[0146] The server searches for tourist attractions and restaurants around the route.

[0147] The server searches a database for tourist attractions and restaurants near the calculated route. The input is the optimal route information, and the output is a list of relevant tourist attractions and restaurants. The server issues an SQL query to retrieve relevant spots from the database.

[0148] Step 5:

[0149] The server calculates the additional time and cost of stopping at the detour spot.

[0150] The server again uses the map API to calculate route changes and travel times for each detour spot. It also retrieves additional costs, such as entrance fees and food and drink costs, for each spot from a separate database. The input for this process is a list of tourist spots and restaurants and route data, and the output is additional time and cost information for each spot.

[0151] Step 6:

[0152] The server takes into account the user's past usage and preference data.

[0153] The server retrieves the user's past travel history and preference data from a database and applies machine learning algorithms to analyze it. The input is the user's history and preference data, and the output is a detour plan that is suitable for the user.

[0154] Step 7:

[0155] The server generates an optimal detour plan and transmits it to the terminal.

[0156] The server generates a detour plan that combines related detour spots and their detailed information based on optimal route information and user preference data. The generated detour plan is sent to the terminal. The input is tourist spot information obtained from the database, calculated time and cost information, and user preference data, and the output is the detour plan data.

[0157] Step 8:

[0158] The terminal visually displays the detour plan and the user approves or customizes the plan.

[0159] The terminal displays the received detour plan on the application interface. The user can check the detailed information of each spot, the additional time and cost, and approve or customize the plan. The input is the detour plan received from the server, and the output is the visual display to the user.

[0160] (Application example 1)

[0161] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0162] When travelers select their destination and mode of transportation, there is a demand for systems that can automatically suggest optimal routes and detour spots, providing an enjoyable experience while traveling. It is also expected that this will contribute to raising awareness of tourist spots in depopulated areas and revitalizing the region. Furthermore, when users use autonomous vehicles, a system is needed that allows them to easily check and customize detour plans while traveling.

[0163] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0164] In this invention, the server includes a means for a user to input a destination and a mode of transportation, a means for transmitting the input information to the server, and a means for calculating an optimal route based on the destination and mode of transportation, thereby enabling a user of an autonomous vehicle to visually check, approve, or customize a detour plan.

[0165] "User" means an individual or organization that uses the System to plan and manage a trip.

[0166] The "destination" is the final destination of the trip set by the user.

[0167] "Transportation method" refers to the means of transportation selected by the user (e.g., car, train, bus, etc.).

[0168] A "server" is a computer system that receives user input and processes the data.

[0169] "Input information" is data relating to the destination and travel method specified by the user.

[0170] An "optimal route" is the most efficient route from a starting point to a destination.

[0171] A "database" is a collection of data that stores information on tourist spots, restaurants, etc.

[0172] "Places" are tourist spots, restaurants, and other stops along the route.

[0173] "Time and cost" refers to the additional travel time and cost incurred by stopping at each detour location.

[0174] "Usage data" is information relating to a user's past behavioral history and preferences.

[0175] A "side trip plan" is a travel plan that includes suggested destinations along the optimal route.

[0176] "Transportation" refers to the means of transportation or vehicles used by the user.

[0177] A "display" is a display device that displays a user interface.

[0178] "Detailed information" is additional information about the detour location (e.g., required time, cost, overview).

[0179] This invention is a system that suggests optimal routes and detour spots when a traveler inputs their destination and mode of transportation. Based on the information entered by the user, the system calculates the optimal route to the destination, searches for tourist spots and restaurants around the route, and calculates the travel time and cost required for each. The system also selects optimal detour plans taking into account the user's past usage data and preference data, and displays them on a display inside the self-driving vehicle.

[0180] Program processing explanation

[0181] The hardware includes a server, a user terminal, and a display in the autonomous vehicle, while the software uses Python, Google Maps API, and Google Places API.

[0182] When a user inputs their destination and transportation method, the input data is sent to the server, which uses this information to calculate the optimal route to the destination using the Google Maps API. The server then retrieves tourist attractions and restaurants near the route from a database via the Google Places API, including details such as location, rating, and category.

[0183] The server then calculates the additional time and cost of each detour stop, again using the Google Maps API to calculate the route changes and travel time required for each stop, as well as any additional costs associated with each stop (entrance fees, food and drink, etc.).

[0184] To take into account the user's past usage data and preference data, the server applies machine learning algorithms that analyze the places the user has visited in the past and the categories they like, and provide the user with the most suitable detour plan.

[0185] Finally, the server generates the selected detour plan and sends it to the user terminal, which visually displays the plan on a display inside the autonomous vehicle, allowing the user to review the plan's details and approve or customize it as needed.

[0186] Specific examples

[0187] For example, consider a user traveling from Osaka to Kyoto by car. When the user inputs the destination "Kyoto" and the travel method "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches the database for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost involved for each. Based on this information, and taking into account the user's past preference for historical places, it proposes a plan that includes the Great Buddha of Nara and the teahouse. The device visually presents this to the user, who then approves it as the final plan.

[0188] Prompt Sentence Examples

[0189] Generative AI model prompt:

[0190] We are considering a system that suggests optimal routes and detour spots when a user inputs their destination and transportation method. This system uses the Google Maps API and Google Places API to obtain information on optimal routes and tourist spots.

[0191] Generate a Python program using the following information:

[0192] Departure point: "Osaka"

[0193] Destination: "Kyoto"

[0194] Transportation method: "car"

[0195] Get the best route and view nearby attractions.

[0196] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0197] Step 1:

[0198] The user inputs the destination and the mode of travel.

[0199] Input: Destination and method of travel (e.g., "Kyoto", "car").

[0200] Operation: The user enters a destination and a method of travel on the application interface and presses the "Submit" button.

[0201] Output: The input information is sent to the server.

[0202] Step 2:

[0203] The server receives the input information and calculates the optimal route.

[0204] Input: User submitted destination and travel method.

[0205] How it works: The server uses the Google Maps API to get the best route from the start point to the destination.

[0206] Output: Optimal route data.

[0207] Step 3:

[0208] The server searches for tourist attractions and restaurants around the route.

[0209] Input: Optimal route data.

[0210] How it works: The server uses the Google Places API to obtain information about tourist attractions and restaurants near the route.

[0211] Output: A list of attractions and restaurants.

[0212] Step 4:

[0213] The server calculates the additional time and cost of stopping at each detour location.

[0214] Input: List of attractions and restaurants and optimal route data.

[0215] How it works: The server again uses the Google Maps API to calculate route changes and travel times for each stop, as well as any additional costs associated with each stop (e.g., entrance fees, food and drink, etc.).

[0216] Output: Additional time and cost information for each detour location.

[0217] Step 5:

[0218] The server selects the optimal detour plan taking into consideration the user's past usage data and preference data.

[0219] Input: A list of tourist attractions and restaurants, additional time and cost information for each detour, and the user's past usage data.

[0220] How it works: The server uses machine learning algorithms to analyze the user's preferences and past behavioral history and automatically select the most suitable detour plan.

[0221] Output: The selected detour plan.

[0222] Step 6:

[0223] The server generates the selected detour plan and transmits it to the terminal.

[0224] Input: The selected detour plan.

[0225] Operation: The server generates a detour plan and sends it to the user terminal.

[0226] Output: The detour plan received by the device.

[0227] Step 7:

[0228] The terminal displays detailed information about the detour plan on a display inside the user's self-driving vehicle.

[0229] Input: Detour plan sent from the server.

[0230] How it works: The device receives detour plans and details and visually displays them on a display inside the autonomous vehicle, allowing the user to review and customize them in real time.

[0231] Output: Detailed information about the detour plan displayed on the display.

[0232] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0233] This invention is a system that suggests optimal routes and detour spots when a user inputs their destination and transportation method, and furthermore, recognizes the user's emotions to personalize the suggestions. This provides an enjoyable travel experience while traveling and also contributes to regional revitalization.

[0234] First, the user launches the application on a device such as a smartphone or tablet. The user then inputs their destination and travel method on the application interface. This input information is then sent from the device to the server.

[0235] Next, the server receives the destination and mode of travel and calculates the optimal route. The server uses a map API to generate a route from the starting point to the destination. The server then performs a database query to find tourist attractions and restaurants near the route. This database contains information on a variety of tourist attractions and restaurants.

[0236] The server then calculates the additional time and cost of each detour stop, which involves again using the map API to calculate the route changes and travel time required for each stop, as well as any additional costs associated with each stop (entrance fees, food and drink, etc.).

[0237] The server then uses an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and voice data to determine emotions such as joy, sadness, and surprise in real time. This emotion data is sent to the server and, together with the user's usage data and preference data, is reflected in the selection of detour plans.

[0238] The server considers the emotional data and selects the best detour plan for the user's current emotional state. For example, if the user is tired, it will recommend relaxing places, and if the user is excited, it will recommend active tourist spots. Using a machine learning algorithm, it generates the best detour plan that combines the user's preferences and emotional state.

[0239] The server generates a detour plan based on the selected destinations and sends it to the device. The device displays the plan on the application interface, where the user can view detailed information about the detour spots, as well as additional time and costs. The user can then approve the final plan and customize it as needed.

[0240] As a concrete example, consider a user traveling by car from Osaka to Kyoto. When the user inputs the destination "Kyoto" and the mode of travel "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches the database for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost required for each. If the emotion engine recognizes that the user has a tendency to like historical places in the past and that their current emotional state is relaxed, the server will suggest a plan that includes the Great Buddha of Nara and the teahouse. The device visually presents this to the user, who then approves it as the final plan.

[0241] In this way, the present invention improves the quality of travel by suggesting optimal routes and detour spots based on the destination and mode of travel, and also by taking the user's emotions into consideration. Furthermore, by proactively introducing tourist spots in depopulated areas, the system also contributes to regional revitalization.

[0242] The processing flow will be explained below.

[0243] Step 1:

[0244] The user launches the application. The device displays a user interface and provides a screen for inputting the destination and travel method.

[0245] Step 2:

[0246] The user inputs the destination and transportation method. Specifically, the user inputs "Kyoto" in the text box and selects "Car" from the drop-down menu.

[0247] Step 3:

[0248] The device sends the entered destination and travel method to the server. The device packages the input data as an HTTP request and sends it to the server.

[0249] Step 4:

[0250] The server receives the input information and calculates the optimal route. The server calls the map API and generates a route from the starting point (the user's current location) to the destination (Kyoto).

[0251] Step 5:

[0252] The server searches for tourist attractions and restaurants around the route. The server queries a database to obtain a list of tourist attractions (e.g., the Great Buddha of Nara) and restaurants (e.g., local teahouses) within a certain range of the route.

[0253] Step 6:

[0254] The server calculates the additional time and cost of stopping at each detour spot. The server again uses the map API to calculate the route and travel time that will be changed by stopping at each spot. It also calculates additional costs such as entrance fees and food and drink costs at each spot.

[0255] Step 7:

[0256] To recognize the user's emotions, the device's built-in emotion engine analyzes facial expressions and voice data. Specifically, the device's camera and microphone are used to capture the user's facial expressions and voice, which are then analyzed in real time by the emotion engine.

[0257] Step 8:

[0258] The emotion engine sends the analysis results (e.g., joy, sadness, surprise, etc.) to the server. The device includes the analysis results in an HTTP request and sends it.

[0259] Step 9:

[0260] The server receives the emotion data and selects the optimal detour plan based on the user's past usage data and preference data. It uses a machine learning algorithm to select corresponding detour spots based on the user's current emotion.

[0261] Step 10:

[0262] The server generates an optimal detour plan and sends it to the device. Specifically, it generates information such as detour spots, additional time, and costs in a structured data format (e.g., JSON) and sends it to the device as an HTTP response.

[0263] Step 11:

[0264] The device receives the plan from the server and displays it on the user interface. The device analyzes the received data, displays detour spots on the map with markers, and visually presents detailed information about each spot.

[0265] Step 12:

[0266] The user can review the presented detour plan and customize it as needed. The user can add or remove spots, change the order, and the device will resubmit the finalized plan to the server.

[0267] Step 13:

[0268] The user approves the final plan and begins the trip. The device saves the final plan locally and enters navigation mode. The user travels along the plan, and the device provides necessary navigation and detour information.

[0269] Example 2

[0270] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0271] Conventional travel planning systems require users to search for detour spots and manually calculate travel time and costs, which is time-consuming and does not take into account the preferences and emotional state of individual users. Furthermore, it is difficult to individually optimize the travel experience, and these systems do not contribute sufficiently to regional revitalization. The present invention aims to solve these problems by providing optimal travel plans tailored to the individual needs and emotional state of users, and by introducing tourist spots in depopulated areas, contributing to regional revitalization.

[0272] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0273] In this invention, the server includes a means for analyzing facial expressions and voice data to recognize the user's emotions, a means for selecting an optimal detour plan based on the user's past usage data and emotional data, and a means for presenting the detour plan to the user. This allows the server to propose an optimal detour plan based on the user's current emotional state, improving the quality of the trip. Furthermore, personalized plans based on the user's preferences and past data are provided, enabling a travel experience that meets individual needs. Furthermore, by actively introducing local tourist attractions and dining facilities, the server can contribute to regional revitalization.

[0274] "Destination" refers to the location where the user ultimately wants to arrive.

[0275] "Transportation method" refers to the means of transportation (e.g., car, train, walking, etc.) that a user uses to get to a destination.

[0276] "Terminal" refers to an electronic device that is directly operated by a user (e.g., smartphone, tablet, PC, etc.).

[0277] "Server" refers to a remote computer that processes information sent from a terminal and provides the necessary data.

[0278] "Route" refers to the route traveled from a starting point to a destination.

[0279] "Tourist attractions" refer to specific places that are worth visiting for tourists (e.g., historical buildings, natural landscapes, theme parks, etc.).

[0280] "Food and beverage establishment" refers to a place that serves food and drinks (e.g., restaurant, cafe, food stall, etc.).

[0281] "Information base" refers to a database system that stores information on tourist attractions and dining establishments.

[0282] A "side trip point" refers to a place where a user stops temporarily on the way to a destination.

[0283] "Emotion recognition" refers to the technology of analyzing facial expressions and voice data to determine a user's emotional state.

[0284] "Past usage data" refers to historical information about a user's use of the system.

[0285] "Emotion data" refers to information about the user's emotional state analyzed from facial expressions and voice data.

[0286] A "side trip plan" refers to a recommended route that includes stops at tourist attractions and dining facilities on the way to the destination.

[0287] "User interface" refers to the part that provides the screen and operating method for the user to operate the system.

[0288] This invention is a system that, when a user inputs a destination and a mode of transportation, suggests optimal routes and detours, and further personalizes the suggestions by recognizing the user's emotions. This system is composed of a device such as a smartphone or tablet and a server located in a remote location.

[0289] First, the user launches a dedicated application on a device such as a smartphone or tablet. The user inputs their destination and method of transportation on the application interface. This input information is sent from the device to a server. The server receives this information and uses a map API (e.g., Google Maps API) to calculate the optimal route from the starting point to the destination.

[0290] Next, the server searches for tourist attractions and restaurants near the calculated route from an information base, which contains information on various tourist attractions and restaurants (such as name, location, opening hours, category, etc.). The server executes an SQL query to retrieve the relevant spots.

[0291] The server then calculates the additional time and cost of each detour, again using the map API to calculate the route changes and travel time required to visit each stop, as well as any additional costs associated with each stop (entrance fees, food, drink, etc.).

[0292] Next, the device captures the user's facial expression and voice data and sends them to the server, which then uses an emotion engine (e.g., facial expression recognition technology or voice analysis technology) to analyze the user's emotional state. The analysis results are obtained in real time and the user's emotional state (e.g., joy, sadness, surprise, etc.) is determined.

[0293] The server combines the emotional data with the user's past usage data and uses machine learning algorithms to generate optimal detour plans, recommending quiet and relaxing places if the user is relaxed, or active tourist spots if the user is excited.

[0294] The generated detour plan is sent from the server to the device and displayed on the application interface. The user can review the plan and check detailed information (such as additional time and costs). Furthermore, the user can approve the final plan and customize it as needed.

[0295] As a concrete example, consider a user traveling by car from Osaka to Kyoto. When the user inputs the destination "Kyoto" and the mode of travel "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches the information base for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost required for each. If the emotion engine recognizes that the user has a tendency to like historical places in the past and that their current emotional state is relaxed, the server will suggest a plan that includes the Great Buddha of Nara and the teahouse. The device visually presents this information to the user, and the user ultimately approves the plan.

[0296] In this way, the present invention improves the quality of travel by suggesting optimal routes and detours based on the destination and mode of travel, and by taking into account the user's emotions. Furthermore, by proactively introducing tourist spots in depopulated areas, the system also contributes to regional revitalization.

[0297] The following can be used as an example prompt:

[0298] Using the example of a user traveling by car from Osaka to Kyoto, please explain in detail all the processing steps, from inputting the destination and mode of transportation to generating an optimal plan of detour spots. Please also mention the specific operations and APIs and algorithms used. For example, calculating the optimal route using a map API, or analyzing the user's emotions using an emotion recognition engine.

[0299]

[0300] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0301] Step 1:

[0302] The user starts the application using a smartphone or tablet device and inputs the destination "Kyoto" and the method of transportation "car." The input information is entered in text format into a form on the device. When the user has completed the input, they press the "Send" button to send the information to the server. The input data includes "Destination: Kyoto" and "Method of transportation: Car." The output is the input information sent to the server.

[0303] Step 2:

[0304] The terminal converts the "destination" and "transportation method" information entered by the user into JSON format and sends it to the server via an HTTP POST request. The input is the "destination" and "transportation method" data from the user, and the output is the JSON-formatted request data sent to the server. Specifically, the terminal sends the data to an endpoint such as "https: / / example.com / api / route".

[0305] Step 3:

[0306] The server receives the HTTP request and extracts the "destination" and "transportation method" information from the JSON data. It then uses a map API (for example, Google Maps API) to calculate the optimal route from the starting point (the user's current location) to the destination "Kyoto." The input is the user's location information, destination, and transportation method, and the output is the optimal route information. Specifically, the server sends a request to the map API and receives route data in response.

[0307] Step 4:

[0308] The server analyzes the route information returned from the map API and obtains the optimal route. Based on this route information, the server executes an SQL query to search the information base for tourist attractions and restaurants near the route. The input is the route data obtained from the map API, and the output is a list of relevant tourist attractions and restaurants. Specifically, the server executes queries such as "SELECT FROM spots WHERE location NEAR route_coordinates".

[0309] Step 5:

[0310] The server calculates the additional time and cost of stopping at each detour point. This involves using the map API again to calculate the route changes and travel time required to stop at each spot. It also calculates any associated additional costs, such as admission fees and food and beverage costs, for each spot. The input is a list of tourist attractions and restaurants, and the output is the additional time and cost data for each detour point. Specifically, the server executes functions such as "Calculate End-to-End Route Time" and "Fetch Additional Costs."

[0311] Step 6:

[0312] The device captures the user's facial expressions and voice data using input devices such as the device's camera and microphone. The captured data is sent to the server in real time. The input is the user's facial expressions and voice data, and the output is digital data sent to the server. Specifically, the device executes functions such as "Start Camera Capture" and "Record Audio."

[0313] Step 7:

[0314] The server uses an emotion recognition engine to analyze the received facial expression and voice data. Based on the analysis results, the server determines the user's emotional state in real time. The input is the facial expression and voice data sent from the device, and the output is data on the user's emotional state. Specifically, the server executes algorithms such as "Analyze Facial Expression" and "Evaluate Audio Tone."

[0315] Step 8:

[0316] The server uses a machine learning algorithm to generate an optimal detour plan based on the user's past usage data and current emotional data. For example, if the user is relaxed, it will suggest a quiet and relaxing place, and if the user is excited, it will recommend an active tourist spot. The input is the user's past data and emotional data, and the output is the optimal detour plan. Specifically, the server executes processes such as "Generate Personalized Plan" and "Optimize Travel Itinerary."

[0317] Step 9:

[0318] The server sends the generated detour plan to the device. The input is the generated detour plan, and the output is the plan data to be sent to the device. Specifically, the server executes a command such as "Send Plan to Device."

[0319] Step 10:

[0320] The terminal displays the received detour plan on the application interface. The user can check the plan details (e.g., additional time and cost). The input is the plan data sent from the server, and the output is the interface display visually presented to the user. Specifically, the terminal executes functions such as "Display Plan on Screen" and "Show Plan Details."

[0321] (Application example 2)

[0322] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0323] Conventional route guidance systems have the ability to suggest optimal routes to a destination and detour spots, but they are unable to provide personalized suggestions that take into account the user's emotions and real-time situation. This limits the user experience and poses challenges in providing a more fulfilling travel experience. Furthermore, suggestions that do not take emotions into account are often inappropriate for the user's situation, potentially reducing satisfaction.

[0324] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input a destination and a method of travel, means for transmitting the input information to the server, means for calculating an optimal route based on the destination and method of travel, means for selecting an optimal detour plan taking into account the user's past usage data and current emotional state, and means for capturing and analyzing the user's facial expressions in real time using face recognition technology with a smartphone or a head-mounted display. This makes it possible to grasp the user's emotional state in real time and make personalized suggestions that are optimal for the situation.

[0325] A "user" is a person who uses the system by inputting a destination and a mode of travel.

[0326] A "destination" is a place that a user wishes to visit.

[0327] A "mode of travel" is the means of transportation a user chooses to reach a destination.

[0328] "Means" refers to a method or apparatus by which a system performs a particular function.

[0329] A "server" is a central processing unit that performs route calculations, database searches, sentiment analysis, and the like.

[0330] "Input information" refers to data that a user provides to the system regarding a destination and a method of travel.

[0331] The "optimal route" is the route that most effectively reaches the destination entered by the user.

[0332] "Tourist attractions" are places of interest or famous places that users can visit.

[0333] A "restaurant" is a store where users can purchase food and drinks.

[0334] A "database" is a system that organizes and stores information about tourist spots and restaurants.

[0335] A "side trip spot" is a place where a user can stop by on the way to their destination.

[0336] "Time" is the travel time added by stopping at a detour spot.

[0337] "Cost" is the additional monetary cost incurred by stopping at a detour spot.

[0338] "Usage data" refers to data and behavioral history of a user's past use.

[0339] "Emotional state" is a state that indicates the user's emotions in real time.

[0340] A "side trip plan" is a list of suggested side trip spots based on the optimal route.

[0341] A "smartphone" is a type of mobile phone and is a device that a user can use to access the system.

[0342] A "head-mounted display (HMD)" is a device worn by a user to visually obtain displayed information.

[0343] "Facial recognition technology" is a technology that captures and analyzes a user's facial expressions.

[0344] "Capture" is the act of acquiring an image or data.

[0345] "Analysis" is the act of analyzing the acquired data in detail.

[0346] "Real time" is a time frame in which processing occurs immediately without delay.

[0347] This invention is a system that suggests optimal routes and detour spots when a user inputs their destination and transportation method, and further personalizes the suggestions by recognizing the user's emotions. Specifically, this system is realized using the following hardware and software.

[0348] Hardware used:

[0349] 1. Smartphone

[0350] It is used by the user to input the destination and travel method and send it to the server.

[0351] It has a camera function and is used to capture the user's facial expressions.

[0352] 2. Head-Mounted Display (HMD)

[0353] This device allows users to visually confirm information and is used to capture the user's facial expressions in real time using facial recognition technology.

[0354] 3. Server

[0355] This is the central processing unit that handles major processes such as route calculation, database search, and emotion analysis.

[0356] Software used:

[0357] 1. Facial Recognition API

[0358] Examples: Amazon Rekognition, Google Cloud Vision API

[0359] 2. Map API

[0360] Example: Google Maps API

[0361] 3. Database Management System

[0362] Example: PostgreSQL

[0363] 4. Machine Learning Algorithms

[0364] Example: TENSORFLOW(R)

[0365] Process flow:

[0366] Step 1: Obtain user data

[0367] The user inputs the destination and travel method using a smartphone or HMD, and this information is sent from the device to the server.

[0368] Step 2: Route calculation and spot search by server

[0369] The server uses a map API to calculate the optimal route from the departure point to the destination and generate route information, while simultaneously searching a database for tourist spots and restaurants along the calculated route.

[0370] Step 3: Emotion Recognition

[0371] The user's face is captured using a smartphone or HMD camera, and emotions are analyzed in real time using a facial recognition API.

[0372] Step 4: Generate optimal detour plans

[0373] The server uses a machine learning algorithm to select the optimal detour spots based on the user's past usage data and analyzed emotional data, and generates a detour plan.

[0374] Step 5: Present to the user

[0375] The generated detour plan is sent from the server to the device and visually displayed to the user via a smartphone or HMD, where the user can review the proposal and customize it as needed.

[0376] Examples:

[0377] For example, imagine a user traveling from Osaka to Kyoto in an autonomous vehicle. The user uses their smartphone to input their destination "Kyoto" and their mode of travel "car." The HMD then recognizes the user's face, and the emotion engine analyzes whether they are in a relaxed state. Based on this data, the server suggests detour spots where the user can relax, such as the Great Buddha of Nara or a famous local teahouse. The user can visually confirm these suggestions and either adopt them as an optimal travel plan or customize them.

[0378] Example prompt sentence:

[0379] Suggest the best travel route and detour spots using the following information: The user's destination is "Kyoto" and the mode of transportation is "car." The user's current emotional state is relaxed. List recommended detour spots.

[0380] This allows us to provide a personalized travel experience that utilizes the user's emotions and travel information.

[0381] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0382] Step 1: Obtain user data

[0383] The user inputs their destination and travel method using a smartphone or HMD. The input information (destination and travel method) is sent to the server via the device. This allows the server to receive basic instructions about the user's trip and obtain data for the next processing step.

[0384] Step 2: Route calculation

[0385] The server calculates the optimal route using a map API (e.g., Google Maps API) based on the input information it receives. Specifically, it receives the starting point and destination as input and outputs the optimal route taking into account the shortest distance and required time. This output includes the most efficient route information for the user.

[0386] Step 3: Search for spots

[0387] Based on the calculated optimal route, the server searches for tourist attractions and restaurants near the route using a database query. The database (e.g., PostgreSQL) stores information on various tourist attractions and restaurants, and the search query effectively extracts this information. The input is the optimal route, and the output is a list of detour spots.

[0388] Step 4: Calculate additional time and costs

[0389] The server calculates the additional time and cost of stopping at each listed detour spot. It uses the map API again, taking as input the route changes and travel time required for each stop, and outputs the additional time and cost involved.

[0390] Step 5: Emotion Recognition

[0391] The user's facial expressions are captured by a smartphone or HMD camera, and emotions are analyzed in real time using a facial recognition API (e.g., Amazon Rekognition). The input is the captured facial expression data, and the output is the user's emotional state (e.g., relaxed, excited, etc.). This allows the server to understand the user's current emotional state.

[0392] Step 6: Choose a detour plan

[0393] The server uses a machine learning algorithm (e.g., TensorFlow) to select optimal detour spots based on the user's past usage data and current emotional data. The input is past usage data and the user's current emotional state, and the output is an optimal list of detour spots. The machine learning algorithm analyzes the past data and emotional data to suggest the most suitable spots for the user.

[0394] Step 7: Generate and submit your detour plan

[0395] The server sends the generated detour plan to the terminal. The input is a list of selected detour spots, and the output is the detour plan displayed on the user's terminal, allowing the user to review the final plan and customize it as needed.

[0396] Step 8: Present to the user

[0397] The device then displays the received detour plan on the user interface of the smartphone or HMD. Specifically, the interface allows the user to visually check detailed information about the spots, the additional time and cost, etc. This allows the user to understand the proposed detour plan and achieve the optimal travel experience.

[0398] Through these steps, the system can provide a personalized travel experience that utilizes the user's emotions and travel information.

[0399] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0400] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0401] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0402] [Second embodiment]

[0403] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0404] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0405] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0406] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0407] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0408] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0409] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0410] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0411] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0412] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0413] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0414] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0415] This invention is a system that suggests optimal routes and detour spots when travelers input their destination and transportation method, providing an enjoyable travel experience while on the move. Furthermore, it contributes to regional revitalization by introducing tourist spots in depopulated areas.

[0416] First, the user launches the application on a device such as a smartphone or tablet. The user then inputs their destination and travel method on the application interface. This input information is then sent from the device to the server.

[0417] Once the server receives the destination and mode of travel, it calculates the optimal route. Using a map API, the server generates a route from the starting point to the destination. The server then queries a database to find tourist attractions and restaurants near the route. This database contains information on a variety of tourist attractions and restaurants.

[0418] The server then calculates the additional time and cost of each detour stop, again using the map API to calculate the route changes and travel time required for each stop, as well as any additional costs associated with each stop (entrance fees, food and drink, etc.).

[0419] At this stage, the server takes into account the user's past usage and preference data. It references the user's history and analyzes their interest in specific categories (e.g., historical places, natural scenery, etc.). It then applies machine learning algorithms to select the most suitable detour plan for the user.

[0420] The server then generates a detour plan based on the selected itinerary and sends it to the device. The device then displays the plan on the application interface, allowing the user to visually confirm the detour plan. Detailed information about each stop, as well as additional time and costs, are also displayed, allowing the user to easily approve or customize the plan.

[0421] As a concrete example, consider a user traveling by car from Osaka to Kyoto. When the user inputs the destination "Kyoto" and the mode of travel "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches a database for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost required for each. Based on this information, and taking into account the user's tendency to prefer historical places in the past, it proposes a plan that includes the Great Buddha of Nara and the teahouse. The terminal visually presents this to the user, who then approves it as the final plan.

[0422] In this way, the present invention is a system that improves the quality of travel and contributes to regional revitalization by suggesting optimal routes and detour spots based on the destination and mode of travel.

[0423] The processing flow will be explained below.

[0424] Step 1:

[0425] The user launches the application. The device displays a user interface and provides a screen for inputting the destination and travel method.

[0426] Step 2:

[0427] The user inputs the destination and transportation method. Specifically, the user inputs "Kyoto" in the text box and selects "Car" from the drop-down menu.

[0428] Step 3:

[0429] The device sends the entered destination and travel method to the server. The device packages the input data as an HTTP request and sends it to the server.

[0430] Step 4:

[0431] The server receives the input information and calculates the optimal route. The server calls the map API and generates a route from the starting point (the user's current location) to the destination (Kyoto).

[0432] Step 5:

[0433] The server searches for tourist attractions and restaurants around the route. The server queries a database to obtain a list of tourist attractions (e.g., the Great Buddha of Nara) and restaurants (e.g., local teahouses) within a certain range of the route.

[0434] Step 6:

[0435] The server calculates the additional time and cost of stopping at each detour spot. The server again uses the map API to calculate the route and travel time that will be changed by stopping at each spot. It also calculates additional costs such as entrance fees and food and drink costs at each spot.

[0436] Step 7:

[0437] The server selects the optimal detour plan based on the user's past usage data and preference data. The server refers to the user's past history and analyzes their interest in specific categories using machine learning algorithms.

[0438] Step 8:

[0439] The server generates an optimal detour plan and sends it to the device. Specifically, it generates data in a structured data format (e.g., JSON) including information such as detour spots, additional time, and costs, and sends it to the device as an HTTP response.

[0440] Step 9:

[0441] The device receives the plan from the server and displays it on the user interface. The device analyzes the received data, displays detour spots on the map with markers, and visually presents detailed information about each spot.

[0442] Step 10:

[0443] The user can review the presented detour plan and customize it as needed. The user can add or remove spots, change the order, and the device will resubmit the finalized plan to the server.

[0444] Step 11:

[0445] The user approves the final plan and begins the trip. The device saves the final plan locally and enters navigation mode. The user travels along the plan, and the device provides necessary navigation and detour information.

[0446] Example 1

[0447] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0448] Conventional travel planning systems require users to manually set their destination and route, which requires a lot of time and effort when selecting detour spots and calculating travel time and additional costs. Furthermore, it is difficult to provide optimal detour plans that take into account the user's travel preferences and past usage. As a result, users' travel experiences are limited, and they are unable to fully utilize local tourist spots.

[0449] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0450] In this invention, the server includes a means for selecting an optimal detour plan taking into account the user's past usage data and preference data, a means for using a map API to calculate the optimal route, and a means for analyzing the user's preference data using a machine learning algorithm. This allows the user to automatically receive optimal routes and detour spots without having to manually create complex plans, thereby improving the quality of the travel experience.

[0451] 1. "User" refers to a person who uses the system to set a destination and specify a method of travel.

[0452] 2. "Terminal" refers to a mobile information terminal such as a smartphone or tablet, which transmits information entered by the user to a server.

[0453] 3. "Server" refers to the central processing unit that receives and analyzes information entered by the user and calculates and generates optimal routes and detour plans.

[0454] 4. "Destination" refers to the final destination set by a user as the purpose of a trip or movement.

[0455] 5. "Method of transportation" refers to the means of transportation specified by the user (e.g., car, train, bus, etc.).

[0456] 6. "Optimal route" refers to the most efficient and effective route from a starting point to a destination.

[0457] 7. "Tourist spot" refers to a famous place or tourist destination that travelers visit.

[0458] 8. "Restaurant" refers to an establishment where travelers stop to have a meal.

[0459] 9. "Database" refers to an information aggregation system in which information on tourist spots and restaurants is systematically stored.

[0460] 10. "Detour Spot" refers to a tourist spot or restaurant located on the user's travel route that can be visited along the way.

[0461] 11. "Additional time" refers to the extra travel time or time spent at a detour spot.

[0462] 12. "Additional costs" refers to fees such as entrance fees and food and drink costs incurred by stopping at detour spots.

[0463] 13. "Preference Data" refers to information regarding the categories and types of places that a User has previously visited.

[0464] 14. "Optimal detour plan" refers to a plan that combines the most suitable tourist spots and restaurants to stop at while traveling to the user's destination.

[0465] 15. "Visual display" refers to displaying information in an easily viewable manner as text or images on a user interface.

[0466] 16. "Map API" means an application program interface that provides map information and enables route calculation and geographic data retrieval.

[0467] 17. "Machine learning algorithm" refers to a computational method that learns patterns from past data and makes predictions or classifications for new data.

[0468] MODE FOR CARRYING OUT THE INVENTION

[0469] This invention is a system that suggests optimal routes and detour spots when travelers input their destination and transportation method, providing an enjoyable travel experience while on the move. Furthermore, by introducing tourist spots in depopulated areas, it contributes to regional revitalization.

[0470] Hardware and Software

[0471] 1. Terminal

[0472] Mobile information terminals such as smartphones and tablets

[0473] Any device with internet connectivity

[0474] 2. Server

[0475] A high-performance computer that acts as a central processing unit

[0476] Map APIs, database systems, and computers capable of running machine learning algorithms

[0477] software

[0478] 1. Map API

[0479] Google Maps API, etc.

[0480] 2. Database

[0481] A database that stores information about tourist attractions and restaurants and allows you to run SQL queries

[0482] 3. Machine Learning Algorithms

[0483] Analyze user preference data using algorithms such as K-means clustering

[0484] How it works

[0485] 1. Device operation

[0486] The user operates a smartphone or tablet and launches a travel assistance application.

[0487] Enter your destination and transportation method on the application interface.

[0488] 2. Sending input information

[0489] The device collects the input information and sends it to the server, using an internet connection.

[0490] 3. Calculating the optimal route

[0491] The server calls the Google Maps API or similar to calculate the optimal route from the starting point to the destination.

[0492] 4. Search for tourist attractions and restaurants

[0493] The server searches a database of tourist attractions and restaurants around the route, including categories and location information.

[0494] 5. Calculation of additional time and costs

[0495] The server calculates route changes, travel time, and additional costs for each detour, using information obtained from the map API and other databases.

[0496] 6. Considering user preferences

[0497] The server analyzes the user's past usage data and preference data, and applies machine learning algorithms to select the optimal detour plan for the user.

[0498] 7. Detour Plan Generation

[0499] The server generates a detour plan based on the optimal route, combining related detour spots and their detailed information, and sends the generated plan to the device.

[0500] 8. Visual Indications

[0501] The device receives the plan and displays it visually on the application interface with a map, where the user can review the details of each stop, the additional time and cost, and then approve or customize it.

[0502] Specific examples

[0503] Consider an example of a user traveling by car from Osaka to Kyoto. The user inputs the destination "Kyoto" and the mode of travel "car," and the server calculates the optimal route from Osaka to Kyoto. It then searches the database for detour spots, such as the Great Buddha of Nara and famous teahouses in the area, and calculates the additional time and cost involved for each. Based on this information and taking into account the user's past preference for historical places, the server proposes a plan that includes the Great Buddha of Nara and the teahouse. The terminal visually presents this to the user, who then approves it as the final plan.

[0504] Prompt Sentence Examples

[0505] "I'm traveling by car from Osaka to Kyoto. Please suggest the best route, including a stop at the Great Buddha and a teahouse in Nara."

[0506] "Please create a driving plan from Osaka to Kyoto, including tourist spots and restaurants."

[0507] In this way, the present invention is a system that suggests optimal routes and detour spots based on the destination and mode of travel, improving the quality of travel while also contributing to regional revitalization.

[0508] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0509] Step 1:

[0510] The user starts the application and inputs the destination and the method of travel.

[0511] The user launches the application on a smartphone or tablet and inputs their destination and mode of transportation on the interface. The input data is the destination (e.g., Kyoto) and mode of transportation (e.g., car). This input provides the initial data for calculating the optimal route in the next step.

[0512] Step 2:

[0513] The terminal sends the input information to the server.

[0514] The terminal sends the destination and travel method information entered by the user to the server in the form of an HTTP request. The input here is the destination and travel method from the user, and the output is the request data sent to the server.

[0515] Step 3:

[0516] The server calculates the optimal route.

[0517] Based on the received destination and travel method, the server calls a map API such as Google Maps API to calculate the optimal route from the starting point to the destination. The input for this process is the transmitted destination and travel method, and the output is the optimal route information. The server analyzes the route data returned from the API and obtains an efficient route from the starting point to the destination.

[0518] Step 4:

[0519] The server searches for tourist attractions and restaurants around the route.

[0520] The server searches a database for tourist attractions and restaurants near the calculated route. The input is the optimal route information, and the output is a list of relevant tourist attractions and restaurants. The server issues an SQL query to retrieve relevant spots from the database.

[0521] Step 5:

[0522] The server calculates the additional time and cost of stopping at the detour spot.

[0523] The server again uses the map API to calculate route changes and travel times for each detour spot. It also retrieves additional costs, such as entrance fees and food and drink costs, for each spot from a separate database. The input for this process is a list of tourist spots and restaurants and route data, and the output is additional time and cost information for each spot.

[0524] Step 6:

[0525] The server takes into account the user's past usage and preference data.

[0526] The server retrieves the user's past travel history and preference data from a database and applies machine learning algorithms to analyze it. The input is the user's history and preference data, and the output is a detour plan that is suitable for the user.

[0527] Step 7:

[0528] The server generates an optimal detour plan and transmits it to the terminal.

[0529] The server generates a detour plan that combines related detour spots and their detailed information based on optimal route information and user preference data. The generated detour plan is sent to the terminal. The input is tourist spot information obtained from the database, calculated time and cost information, and user preference data, and the output is the detour plan data.

[0530] Step 8:

[0531] The terminal visually displays the detour plan and the user approves or customizes the plan.

[0532] The terminal displays the received detour plan on the application interface. The user can check the detailed information of each spot, the additional time and cost, and approve or customize the plan. The input is the detour plan received from the server, and the output is the visual display to the user.

[0533] (Application example 1)

[0534] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0535] When travelers select their destination and mode of transportation, there is a demand for systems that can automatically suggest optimal routes and detour spots, providing an enjoyable experience while traveling. It is also expected that this will contribute to raising awareness of tourist spots in depopulated areas and revitalizing the region. Furthermore, when users use autonomous vehicles, a system is needed that allows them to easily check and customize detour plans while traveling.

[0536] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0537] In this invention, the server includes a means for a user to input a destination and a mode of transportation, a means for transmitting the input information to the server, and a means for calculating an optimal route based on the destination and mode of transportation, thereby enabling a user of an autonomous vehicle to visually check, approve, or customize a detour plan.

[0538] "User" means an individual or organization that uses the System to plan and manage a trip.

[0539] The "destination" is the final destination of the trip set by the user.

[0540] "Transportation method" refers to the means of transportation selected by the user (e.g., car, train, bus, etc.).

[0541] A "server" is a computer system that receives user input and processes the data.

[0542] "Input information" is data relating to the destination and travel method specified by the user.

[0543] An "optimal route" is the most efficient route from a starting point to a destination.

[0544] A "database" is a collection of data that stores information on tourist spots, restaurants, etc.

[0545] "Places" are tourist spots, restaurants, and other stops along the route.

[0546] "Time and cost" refers to the additional travel time and cost incurred by stopping at each detour location.

[0547] "Usage data" is information relating to a user's past behavioral history and preferences.

[0548] A "side trip plan" is a travel plan that includes suggested destinations along the optimal route.

[0549] "Transportation" refers to the means of transportation or vehicles used by the user.

[0550] A "display" is a display device that displays a user interface.

[0551] "Detailed information" is additional information about the detour location (e.g., required time, cost, overview).

[0552] This invention is a system that suggests optimal routes and detour spots when a traveler inputs their destination and mode of transportation. Based on the information entered by the user, the system calculates the optimal route to the destination, searches for tourist spots and restaurants around the route, and calculates the travel time and cost required for each. The system also selects optimal detour plans taking into account the user's past usage data and preference data, and displays them on a display inside the self-driving vehicle.

[0553] Program processing explanation

[0554] The hardware includes a server, a user terminal, and a display in the autonomous vehicle, while the software uses Python, Google Maps API, and Google Places API.

[0555] When a user inputs their destination and transportation method, the input data is sent to the server, which uses this information to calculate the optimal route to the destination using the Google Maps API. The server then retrieves tourist attractions and restaurants near the route from a database via the Google Places API, including details such as location, rating, and category.

[0556] The server then calculates the additional time and cost of each detour stop, again using the Google Maps API to calculate the route changes and travel time required for each stop, as well as any additional costs associated with each stop (entrance fees, food and drink, etc.).

[0557] To take into account the user's past usage data and preference data, the server applies machine learning algorithms that analyze the places the user has visited in the past and the categories they like, and provide the user with the most suitable detour plan.

[0558] Finally, the server generates the selected detour plan and sends it to the user terminal, which visually displays the plan on a display inside the autonomous vehicle, allowing the user to review the plan's details and approve or customize it as needed.

[0559] Specific examples

[0560] For example, consider a user traveling from Osaka to Kyoto by car. When the user inputs the destination "Kyoto" and the travel method "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches the database for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost involved for each. Based on this information, and taking into account the user's past preference for historical places, it proposes a plan that includes the Great Buddha of Nara and the teahouse. The device visually presents this to the user, who then approves it as the final plan.

[0561] Prompt Sentence Examples

[0562] Generative AI model prompt:

[0563] We are considering a system that suggests optimal routes and detour spots when a user inputs their destination and transportation method. This system uses the Google Maps API and Google Places API to obtain information on optimal routes and tourist spots.

[0564] Generate a Python program using the following information:

[0565] Departure point: "Osaka"

[0566] Destination: "Kyoto"

[0567] Transportation method: "car"

[0568] Get the best route and view nearby attractions.

[0569] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0570] Step 1:

[0571] The user inputs the destination and the mode of travel.

[0572] Input: Destination and method of travel (e.g., "Kyoto", "car").

[0573] Operation: The user enters a destination and a method of travel on the application interface and presses the "Submit" button.

[0574] Output: The input information is sent to the server.

[0575] Step 2:

[0576] The server receives the input information and calculates the optimal route.

[0577] Input: User submitted destination and travel method.

[0578] How it works: The server uses the Google Maps API to get the best route from the start point to the destination.

[0579] Output: Optimal route data.

[0580] Step 3:

[0581] The server searches for tourist attractions and restaurants around the route.

[0582] Input: Optimal route data.

[0583] How it works: The server uses the Google Places API to obtain information about tourist attractions and restaurants near the route.

[0584] Output: A list of attractions and restaurants.

[0585] Step 4:

[0586] The server calculates the additional time and cost of stopping at each detour location.

[0587] Input: List of attractions and restaurants and optimal route data.

[0588] How it works: The server again uses the Google Maps API to calculate route changes and travel times for each stop, as well as any additional costs associated with each stop (e.g., entrance fees, food and drink, etc.).

[0589] Output: Additional time and cost information for each detour location.

[0590] Step 5:

[0591] The server selects the optimal detour plan taking into consideration the user's past usage data and preference data.

[0592] Input: A list of tourist attractions and restaurants, additional time and cost information for each detour, and the user's past usage data.

[0593] How it works: The server uses machine learning algorithms to analyze the user's preferences and past behavioral history and automatically select the most suitable detour plan.

[0594] Output: The selected detour plan.

[0595] Step 6:

[0596] The server generates the selected detour plan and transmits it to the terminal.

[0597] Input: The selected detour plan.

[0598] Operation: The server generates a detour plan and sends it to the user terminal.

[0599] Output: The detour plan received by the device.

[0600] Step 7:

[0601] The terminal displays detailed information about the detour plan on a display inside the user's self-driving vehicle.

[0602] Input: Detour plan sent from the server.

[0603] How it works: The device receives detour plans and details and visually displays them on a display inside the autonomous vehicle, allowing the user to review and customize them in real time.

[0604] Output: Detailed information about the detour plan displayed on the display.

[0605] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0606] This invention is a system that suggests optimal routes and detour spots when a user inputs their destination and transportation method, and furthermore, recognizes the user's emotions to personalize the suggestions. This provides an enjoyable travel experience while traveling and also contributes to regional revitalization.

[0607] First, the user launches the application on a device such as a smartphone or tablet. The user then inputs their destination and travel method on the application interface. This input information is then sent from the device to the server.

[0608] Next, the server receives the destination and mode of travel and calculates the optimal route. The server uses a map API to generate a route from the starting point to the destination. The server then performs a database query to find tourist attractions and restaurants near the route. This database contains information on a variety of tourist attractions and restaurants.

[0609] The server then calculates the additional time and cost of each detour stop, which involves again using the map API to calculate the route changes and travel time required for each stop, as well as any additional costs associated with each stop (entrance fees, food and drink, etc.).

[0610] The server then uses an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and voice data to determine emotions such as joy, sadness, and surprise in real time. This emotion data is sent to the server and, together with the user's usage data and preference data, is reflected in the selection of detour plans.

[0611] The server considers the emotional data and selects the best detour plan for the user's current emotional state. For example, if the user is tired, it will recommend relaxing places, and if the user is excited, it will recommend active tourist spots. Using a machine learning algorithm, it generates the best detour plan that combines the user's preferences and emotional state.

[0612] The server generates a detour plan based on the selected destinations and sends it to the device. The device displays the plan on the application interface, where the user can view detailed information about the detour spots, as well as additional time and costs. The user can then approve the final plan and customize it as needed.

[0613] As a concrete example, consider a user traveling by car from Osaka to Kyoto. When the user inputs the destination "Kyoto" and the mode of travel "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches the database for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost required for each. If the emotion engine recognizes that the user has a tendency to like historical places in the past and that their current emotional state is relaxed, the server will suggest a plan that includes the Great Buddha of Nara and the teahouse. The device visually presents this to the user, who then approves it as the final plan.

[0614] In this way, the present invention improves the quality of travel by suggesting optimal routes and detour spots based on the destination and mode of travel, and also by taking the user's emotions into consideration. Furthermore, by proactively introducing tourist spots in depopulated areas, the system also contributes to regional revitalization.

[0615] The processing flow will be explained below.

[0616] Step 1:

[0617] The user launches the application. The device displays a user interface and provides a screen for inputting the destination and travel method.

[0618] Step 2:

[0619] The user inputs the destination and transportation method. Specifically, the user inputs "Kyoto" in the text box and selects "Car" from the drop-down menu.

[0620] Step 3:

[0621] The device sends the entered destination and travel method to the server. The device packages the input data as an HTTP request and sends it to the server.

[0622] Step 4:

[0623] The server receives the input information and calculates the optimal route. The server calls the map API and generates a route from the starting point (the user's current location) to the destination (Kyoto).

[0624] Step 5:

[0625] The server searches for tourist attractions and restaurants around the route. The server queries a database to obtain a list of tourist attractions (e.g., the Great Buddha of Nara) and restaurants (e.g., local teahouses) within a certain range of the route.

[0626] Step 6:

[0627] The server calculates the additional time and cost of stopping at each detour spot. The server again uses the map API to calculate the route and travel time that will be changed by stopping at each spot. It also calculates additional costs such as entrance fees and food and drink costs at each spot.

[0628] Step 7:

[0629] To recognize the user's emotions, the device's built-in emotion engine analyzes facial expressions and voice data. Specifically, the device's camera and microphone are used to capture the user's facial expressions and voice, which are then analyzed in real time by the emotion engine.

[0630] Step 8:

[0631] The emotion engine sends the analysis results (e.g., joy, sadness, surprise, etc.) to the server. The device includes the analysis results in an HTTP request and sends it.

[0632] Step 9:

[0633] The server receives the emotion data and selects the optimal detour plan based on the user's past usage data and preference data. It uses a machine learning algorithm to select corresponding detour spots based on the user's current emotion.

[0634] Step 10:

[0635] The server generates an optimal detour plan and sends it to the device. Specifically, it generates information such as detour spots, additional time, and costs in a structured data format (e.g., JSON) and sends it to the device as an HTTP response.

[0636] Step 11:

[0637] The device receives the plan from the server and displays it on the user interface. The device analyzes the received data, displays detour spots on the map with markers, and visually presents detailed information about each spot.

[0638] Step 12:

[0639] The user can review the presented detour plan and customize it as needed. The user can add or remove spots, change the order, and the device will resubmit the finalized plan to the server.

[0640] Step 13:

[0641] The user approves the final plan and begins the trip. The device saves the final plan locally and enters navigation mode. The user travels along the plan, and the device provides necessary navigation and detour information.

[0642] Example 2

[0643] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0644] Conventional travel planning systems require users to search for detour spots and manually calculate travel time and costs, which is time-consuming and does not take into account the preferences and emotional state of individual users. Furthermore, it is difficult to individually optimize the travel experience, and these systems do not contribute sufficiently to regional revitalization. The present invention aims to solve these problems by providing optimal travel plans tailored to the individual needs and emotional state of users, and by introducing tourist spots in depopulated areas, contributing to regional revitalization.

[0645] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0646] In this invention, the server includes a means for analyzing facial expressions and voice data to recognize the user's emotions, a means for selecting an optimal detour plan based on the user's past usage data and emotional data, and a means for presenting the detour plan to the user. This allows the server to propose an optimal detour plan based on the user's current emotional state, improving the quality of the trip. Furthermore, personalized plans based on the user's preferences and past data are provided, enabling a travel experience that meets individual needs. Furthermore, by actively introducing local tourist attractions and dining facilities, the server can contribute to regional revitalization.

[0647] "Destination" refers to the location where the user ultimately wants to arrive.

[0648] "Transportation method" refers to the means of transportation (e.g., car, train, walking, etc.) that a user uses to get to a destination.

[0649] "Terminal" refers to an electronic device that is directly operated by a user (e.g., smartphone, tablet, PC, etc.).

[0650] "Server" refers to a remote computer that processes information sent from a terminal and provides the necessary data.

[0651] "Route" refers to the route traveled from a starting point to a destination.

[0652] "Tourist attractions" refer to specific places that are worth visiting for tourists (e.g., historical buildings, natural landscapes, theme parks, etc.).

[0653] "Food and beverage establishment" refers to a place that serves food and drinks (e.g., restaurant, cafe, food stall, etc.).

[0654] "Information base" refers to a database system that stores information on tourist attractions and dining establishments.

[0655] A "side trip point" refers to a place where a user stops temporarily on the way to a destination.

[0656] "Emotion recognition" refers to the technology of analyzing facial expressions and voice data to determine a user's emotional state.

[0657] "Past usage data" refers to historical information about a user's use of the system.

[0658] "Emotion data" refers to information about the user's emotional state analyzed from facial expressions and voice data.

[0659] A "side trip plan" refers to a recommended route that includes stops at tourist attractions and dining facilities on the way to the destination.

[0660] "User interface" refers to the part that provides the screen and operating method for the user to operate the system.

[0661] This invention is a system that, when a user inputs a destination and a mode of transportation, suggests optimal routes and detours, and further personalizes the suggestions by recognizing the user's emotions. This system is composed of a device such as a smartphone or tablet and a server located in a remote location.

[0662] First, the user launches a dedicated application on a device such as a smartphone or tablet. The user inputs their destination and method of transportation on the application interface. This input information is sent from the device to a server. The server receives this information and uses a map API (e.g., Google Maps API) to calculate the optimal route from the starting point to the destination.

[0663] Next, the server searches for tourist attractions and restaurants near the calculated route from an information base, which contains information on various tourist attractions and restaurants (such as name, location, opening hours, category, etc.). The server executes an SQL query to retrieve the relevant spots.

[0664] The server then calculates the additional time and cost of each detour, again using the map API to calculate the route changes and travel time required to visit each stop, as well as any additional costs associated with each stop (entrance fees, food, drink, etc.).

[0665] Next, the device captures the user's facial expression and voice data and sends them to the server, which then uses an emotion engine (e.g., facial expression recognition technology or voice analysis technology) to analyze the user's emotional state. The analysis results are obtained in real time and the user's emotional state (e.g., joy, sadness, surprise, etc.) is determined.

[0666] The server combines the emotional data with the user's past usage data and uses machine learning algorithms to generate optimal detour plans, recommending quiet and relaxing places if the user is relaxed, or active tourist spots if the user is excited.

[0667] The generated detour plan is sent from the server to the device and displayed on the application interface. The user can review the plan and check detailed information (such as additional time and costs). Furthermore, the user can approve the final plan and customize it as needed.

[0668] As a concrete example, consider a user traveling by car from Osaka to Kyoto. When the user inputs the destination "Kyoto" and the mode of travel "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches the information base for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost required for each. If the emotion engine recognizes that the user has a tendency to like historical places in the past and that their current emotional state is relaxed, the server will suggest a plan that includes the Great Buddha of Nara and the teahouse. The device visually presents this information to the user, and the user ultimately approves the plan.

[0669] In this way, the present invention improves the quality of travel by suggesting optimal routes and detours based on the destination and mode of travel, and by taking into account the user's emotions. Furthermore, by proactively introducing tourist spots in depopulated areas, the system also contributes to regional revitalization.

[0670] The following can be used as an example prompt:

[0671] Using the example of a user traveling by car from Osaka to Kyoto, please explain in detail all the processing steps, from inputting the destination and mode of transportation to generating an optimal plan of detour spots. Please also mention the specific operations and APIs and algorithms used. For example, calculating the optimal route using a map API, or analyzing the user's emotions using an emotion recognition engine.

[0672]

[0673] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0674] Step 1:

[0675] The user starts the application using a smartphone or tablet device and inputs the destination "Kyoto" and the method of transportation "car." The input information is entered in text format into a form on the device. When the user has completed the input, they press the "Send" button to send the information to the server. The input data includes "Destination: Kyoto" and "Method of transportation: Car." The output is the input information sent to the server.

[0676] Step 2:

[0677] The terminal converts the "destination" and "transportation method" information entered by the user into JSON format and sends it to the server via an HTTP POST request. The input is the "destination" and "transportation method" data from the user, and the output is the JSON-formatted request data sent to the server. Specifically, the terminal sends the data to an endpoint such as "https: / / example.com / api / route".

[0678] Step 3:

[0679] The server receives the HTTP request and extracts the "destination" and "transportation method" information from the JSON data. It then uses a map API (for example, Google Maps API) to calculate the optimal route from the starting point (the user's current location) to the destination "Kyoto." The input is the user's location information, destination, and transportation method, and the output is the optimal route information. Specifically, the server sends a request to the map API and receives route data in response.

[0680] Step 4:

[0681] The server analyzes the route information returned from the map API and obtains the optimal route. Based on this route information, the server executes an SQL query to search the information base for tourist attractions and restaurants near the route. The input is the route data obtained from the map API, and the output is a list of relevant tourist attractions and restaurants. Specifically, the server executes queries such as "SELECT FROM spots WHERE location NEAR route_coordinates".

[0682] Step 5:

[0683] The server calculates the additional time and cost of stopping at each detour point. This involves using the map API again to calculate the route changes and travel time required to stop at each spot. It also calculates any associated additional costs, such as admission fees and food and beverage costs, for each spot. The input is a list of tourist attractions and restaurants, and the output is the additional time and cost data for each detour point. Specifically, the server executes functions such as "Calculate End-to-End Route Time" and "Fetch Additional Costs."

[0684] Step 6:

[0685] The device captures the user's facial expressions and voice data using input devices such as the device's camera and microphone. The captured data is sent to the server in real time. The input is the user's facial expressions and voice data, and the output is digital data sent to the server. Specifically, the device executes functions such as "Start Camera Capture" and "Record Audio."

[0686] Step 7:

[0687] The server uses an emotion recognition engine to analyze the received facial expression and voice data. Based on the analysis results, the server determines the user's emotional state in real time. The input is the facial expression and voice data sent from the device, and the output is data on the user's emotional state. Specifically, the server executes algorithms such as "Analyze Facial Expression" and "Evaluate Audio Tone."

[0688] Step 8:

[0689] The server uses a machine learning algorithm to generate an optimal detour plan based on the user's past usage data and current emotional data. For example, if the user is relaxed, it will suggest a quiet and relaxing place, and if the user is excited, it will recommend an active tourist spot. The input is the user's past data and emotional data, and the output is the optimal detour plan. Specifically, the server executes processes such as "Generate Personalized Plan" and "Optimize Travel Itinerary."

[0690] Step 9:

[0691] The server sends the generated detour plan to the device. The input is the generated detour plan, and the output is the plan data to be sent to the device. Specifically, the server executes a command such as "Send Plan to Device."

[0692] Step 10:

[0693] The terminal displays the received detour plan on the application interface. The user can check the plan details (e.g., additional time and cost). The input is the plan data sent from the server, and the output is the interface display visually presented to the user. Specifically, the terminal executes functions such as "Display Plan on Screen" and "Show Plan Details."

[0694] (Application example 2)

[0695] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0696] Conventional route guidance systems have the ability to suggest optimal routes to a destination and detour spots, but they are unable to provide personalized suggestions that take into account the user's emotions and real-time situation. This limits the user experience and poses challenges in providing a more fulfilling travel experience. Furthermore, suggestions that do not take emotions into account are often inappropriate for the user's situation, potentially reducing satisfaction.

[0697] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input a destination and a method of travel, means for transmitting the input information to the server, means for calculating an optimal route based on the destination and method of travel, means for selecting an optimal detour plan taking into account the user's past usage data and current emotional state, and means for capturing and analyzing the user's facial expressions in real time using face recognition technology with a smartphone or a head-mounted display. This makes it possible to grasp the user's emotional state in real time and make personalized suggestions that are optimal for the situation.

[0698] A "user" is a person who uses the system by inputting a destination and a mode of travel.

[0699] A "destination" is a place that a user wishes to visit.

[0700] A "mode of travel" is the means of transportation a user chooses to reach a destination.

[0701] "Means" refers to a method or apparatus by which a system performs a particular function.

[0702] A "server" is a central processing unit that performs route calculations, database searches, sentiment analysis, and the like.

[0703] "Input information" refers to data that a user provides to the system regarding a destination and a method of travel.

[0704] The "optimal route" is the route that most effectively reaches the destination entered by the user.

[0705] "Tourist attractions" are places of interest or famous places that users can visit.

[0706] A "restaurant" is a store where users can purchase food and drinks.

[0707] A "database" is a system that organizes and stores information about tourist spots and restaurants.

[0708] A "side trip spot" is a place where a user can stop by on the way to their destination.

[0709] "Time" is the travel time added by stopping at a detour spot.

[0710] "Cost" is the additional monetary cost incurred by stopping at a detour spot.

[0711] "Usage data" refers to data and behavioral history of a user's past use.

[0712] "Emotional state" is a state that indicates the user's emotions in real time.

[0713] A "side trip plan" is a list of suggested side trip spots based on the optimal route.

[0714] A "smartphone" is a type of mobile phone and is a device that a user can use to access the system.

[0715] A "head-mounted display (HMD)" is a device worn by a user to visually obtain displayed information.

[0716] "Facial recognition technology" is a technology that captures and analyzes a user's facial expressions.

[0717] "Capture" is the act of acquiring an image or data.

[0718] "Analysis" is the act of analyzing the acquired data in detail.

[0719] "Real time" is a time frame in which processing occurs immediately without delay.

[0720] This invention is a system that suggests optimal routes and detour spots when a user inputs their destination and transportation method, and further personalizes the suggestions by recognizing the user's emotions. Specifically, this system is realized using the following hardware and software.

[0721] Hardware used:

[0722] 1. Smartphone

[0723] It is used by the user to input the destination and travel method and send it to the server.

[0724] It has a camera function and is used to capture the user's facial expressions.

[0725] 2. Head-Mounted Display (HMD)

[0726] This device allows users to visually confirm information and is used to capture the user's facial expressions in real time using facial recognition technology.

[0727] 3. Server

[0728] This is the central processing unit that handles major processes such as route calculation, database search, and emotion analysis.

[0729] Software used:

[0730] 1. Facial Recognition API

[0731] Examples: Amazon Rekognition, Google Cloud Vision API

[0732] 2. Map API

[0733] Example: Google Maps API

[0734] 3. Database Management System

[0735] Example: PostgreSQL

[0736] 4. Machine Learning Algorithms

[0737] Example: TensorFlow

[0738] Process flow:

[0739] Step 1: Obtain user data

[0740] The user inputs the destination and travel method using a smartphone or HMD, and this information is sent from the device to the server.

[0741] Step 2: Route calculation and spot search by server

[0742] The server uses a map API to calculate the optimal route from the departure point to the destination and generate route information, while simultaneously searching a database for tourist spots and restaurants along the calculated route.

[0743] Step 3: Emotion Recognition

[0744] The user's face is captured using a smartphone or HMD camera, and emotions are analyzed in real time using a facial recognition API.

[0745] Step 4: Generate optimal detour plans

[0746] The server uses a machine learning algorithm to select the optimal detour spots based on the user's past usage data and analyzed emotional data, and generates a detour plan.

[0747] Step 5: Present to the user

[0748] The generated detour plan is sent from the server to the device and visually displayed to the user via a smartphone or HMD, where the user can review the proposal and customize it as needed.

[0749] Examples:

[0750] For example, imagine a user traveling from Osaka to Kyoto in an autonomous vehicle. The user uses their smartphone to input their destination "Kyoto" and their mode of travel "car." The HMD then recognizes the user's face, and the emotion engine analyzes whether they are in a relaxed state. Based on this data, the server suggests detour spots where the user can relax, such as the Great Buddha of Nara or a famous local teahouse. The user can visually confirm these suggestions and either adopt them as an optimal travel plan or customize them.

[0751] Example prompt sentence:

[0752] Suggest the best travel route and detour spots using the following information: The user's destination is "Kyoto" and the mode of transportation is "car." The user's current emotional state is relaxed. List recommended detour spots.

[0753] This allows us to provide a personalized travel experience that utilizes the user's emotions and travel information.

[0754] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0755] Step 1: Obtain user data

[0756] The user inputs their destination and travel method using a smartphone or HMD. The input information (destination and travel method) is sent to the server via the device. This allows the server to receive basic instructions about the user's trip and obtain data for the next processing step.

[0757] Step 2: Route calculation

[0758] The server calculates the optimal route using a map API (e.g., Google Maps API) based on the input information it receives. Specifically, it receives the starting point and destination as input and outputs the optimal route taking into account the shortest distance and required time. This output includes the most efficient route information for the user.

[0759] Step 3: Search for spots

[0760] Based on the calculated optimal route, the server searches for tourist attractions and restaurants near the route using a database query. The database (e.g., PostgreSQL) stores information on various tourist attractions and restaurants, and the search query effectively extracts this information. The input is the optimal route, and the output is a list of detour spots.

[0761] Step 4: Calculate additional time and costs

[0762] The server calculates the additional time and cost of stopping at each listed detour spot. It uses the map API again, taking as input the route changes and travel time required for each stop, and outputs the additional time and cost involved.

[0763] Step 5: Emotion Recognition

[0764] The user's facial expressions are captured by a smartphone or HMD camera, and emotions are analyzed in real time using a facial recognition API (e.g., Amazon Rekognition). The input is the captured facial expression data, and the output is the user's emotional state (e.g., relaxed, excited, etc.). This allows the server to understand the user's current emotional state.

[0765] Step 6: Choose a detour plan

[0766] The server uses a machine learning algorithm (e.g., TensorFlow) to select optimal detour spots based on the user's past usage data and current emotional data. The input is past usage data and the user's current emotional state, and the output is an optimal list of detour spots. The machine learning algorithm analyzes the past data and emotional data to suggest the most suitable spots for the user.

[0767] Step 7: Generate and submit your detour plan

[0768] The server sends the generated detour plan to the terminal. The input is a list of selected detour spots, and the output is the detour plan displayed on the user's terminal, allowing the user to review the final plan and customize it as needed.

[0769] Step 8: Present to the user

[0770] The device then displays the received detour plan on the user interface of the smartphone or HMD. Specifically, the interface allows the user to visually check detailed information about the spots, the additional time and cost, etc. This allows the user to understand the proposed detour plan and achieve the optimal travel experience.

[0771] Through these steps, the system can provide a personalized travel experience that utilizes the user's emotions and travel information.

[0772] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0773] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0774] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0775] [Third embodiment]

[0776] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0777] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0778] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0779] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0780] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0781] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0782] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0783] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0784] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0785] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0786] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0787] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0788] This invention is a system that suggests optimal routes and detour spots when travelers input their destination and transportation method, providing an enjoyable travel experience while on the move. Furthermore, it contributes to regional revitalization by introducing tourist spots in depopulated areas.

[0789] First, the user launches the application on a device such as a smartphone or tablet. The user then inputs their destination and travel method on the application interface. This input information is then sent from the device to the server.

[0790] Once the server receives the destination and mode of travel, it calculates the optimal route. Using a map API, the server generates a route from the starting point to the destination. The server then queries a database to find tourist attractions and restaurants near the route. This database contains information on a variety of tourist attractions and restaurants.

[0791] The server then calculates the additional time and cost of each detour stop, again using the map API to calculate the route changes and travel time required for each stop, as well as any additional costs associated with each stop (entrance fees, food and drink, etc.).

[0792] At this stage, the server takes into account the user's past usage and preference data. It references the user's history and analyzes their interest in specific categories (e.g., historical places, natural scenery, etc.). It then applies machine learning algorithms to select the most suitable detour plan for the user.

[0793] The server then generates a detour plan based on the selected itinerary and sends it to the device. The device then displays the plan on the application interface, allowing the user to visually confirm the detour plan. Detailed information about each stop, as well as additional time and costs, are also displayed, allowing the user to easily approve or customize the plan.

[0794] As a concrete example, consider a user traveling by car from Osaka to Kyoto. When the user inputs the destination "Kyoto" and the mode of travel "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches a database for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost required for each. Based on this information, and taking into account the user's tendency to prefer historical places in the past, it proposes a plan that includes the Great Buddha of Nara and the teahouse. The terminal visually presents this to the user, who then approves it as the final plan.

[0795] In this way, the present invention is a system that improves the quality of travel and contributes to regional revitalization by suggesting optimal routes and detour spots based on the destination and mode of travel.

[0796] The processing flow will be explained below.

[0797] Step 1:

[0798] The user launches the application. The device displays a user interface and provides a screen for inputting the destination and travel method.

[0799] Step 2:

[0800] The user inputs the destination and transportation method. Specifically, the user inputs "Kyoto" in the text box and selects "Car" from the drop-down menu.

[0801] Step 3:

[0802] The device sends the entered destination and travel method to the server. The device packages the input data as an HTTP request and sends it to the server.

[0803] Step 4:

[0804] The server receives the input information and calculates the optimal route. The server calls the map API and generates a route from the starting point (the user's current location) to the destination (Kyoto).

[0805] Step 5:

[0806] The server searches for tourist attractions and restaurants around the route. The server queries a database to obtain a list of tourist attractions (e.g., the Great Buddha of Nara) and restaurants (e.g., local teahouses) within a certain range of the route.

[0807] Step 6:

[0808] The server calculates the additional time and cost of stopping at each detour spot. The server again uses the map API to calculate the route and travel time that will be changed by stopping at each spot. It also calculates additional costs such as entrance fees and food and drink costs at each spot.

[0809] Step 7:

[0810] The server selects the optimal detour plan based on the user's past usage data and preference data. The server refers to the user's past history and analyzes their interest in specific categories using machine learning algorithms.

[0811] Step 8:

[0812] The server generates an optimal detour plan and sends it to the device. Specifically, it generates data in a structured data format (e.g., JSON) including information such as detour spots, additional time, and costs, and sends it to the device as an HTTP response.

[0813] Step 9:

[0814] The device receives the plan from the server and displays it on the user interface. The device analyzes the received data, displays detour spots on the map with markers, and visually presents detailed information about each spot.

[0815] Step 10:

[0816] The user can review the presented detour plan and customize it as needed. The user can add or remove spots, change the order, and the device will resubmit the finalized plan to the server.

[0817] Step 11:

[0818] The user approves the final plan and begins the trip. The device saves the final plan locally and enters navigation mode. The user travels along the plan, and the device provides necessary navigation and detour information.

[0819] Example 1

[0820] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0821] Conventional travel planning systems require users to manually set their destination and route, which requires a lot of time and effort when selecting detour spots and calculating travel time and additional costs. Furthermore, it is difficult to provide optimal detour plans that take into account the user's travel preferences and past usage. As a result, users' travel experiences are limited, and they are unable to fully utilize local tourist spots.

[0822] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0823] In this invention, the server includes a means for selecting an optimal detour plan taking into account the user's past usage data and preference data, a means for using a map API to calculate the optimal route, and a means for analyzing the user's preference data using a machine learning algorithm. This allows the user to automatically receive optimal routes and detour spots without having to manually create complex plans, thereby improving the quality of the travel experience.

[0824] 1. "User" refers to a person who uses the system to set a destination and specify a method of travel.

[0825] 2. "Terminal" refers to a mobile information terminal such as a smartphone or tablet, which transmits information entered by the user to a server.

[0826] 3. "Server" refers to the central processing unit that receives and analyzes information entered by the user and calculates and generates optimal routes and detour plans.

[0827] 4. "Destination" refers to the final destination set by a user as the purpose of a trip or movement.

[0828] 5. "Method of transportation" refers to the means of transportation specified by the user (e.g., car, train, bus, etc.).

[0829] 6. "Optimal route" refers to the most efficient and effective route from a starting point to a destination.

[0830] 7. "Tourist spot" refers to a famous place or tourist destination that travelers visit.

[0831] 8. "Restaurant" refers to an establishment where travelers stop to have a meal.

[0832] 9. "Database" refers to an information aggregation system in which information on tourist spots and restaurants is systematically stored.

[0833] 10. "Detour Spot" refers to a tourist spot or restaurant located on the user's travel route that can be visited along the way.

[0834] 11. "Additional time" refers to the extra travel time or time spent at a detour spot.

[0835] 12. "Additional costs" refers to fees such as entrance fees and food and drink costs incurred by stopping at detour spots.

[0836] 13. "Preference Data" refers to information regarding the categories and types of places that a User has previously visited.

[0837] 14. "Optimal detour plan" refers to a plan that combines the most suitable tourist spots and restaurants to stop at while traveling to the user's destination.

[0838] 15. "Visual display" refers to displaying information in an easily viewable manner as text or images on a user interface.

[0839] 16. "Map API" means an application program interface that provides map information and enables route calculation and geographic data retrieval.

[0840] 17. "Machine learning algorithm" refers to a computational method that learns patterns from past data and makes predictions or classifications for new data.

[0841] MODE FOR CARRYING OUT THE INVENTION

[0842] This invention is a system that suggests optimal routes and detour spots when travelers input their destination and transportation method, providing an enjoyable travel experience while on the move. Furthermore, by introducing tourist spots in depopulated areas, it contributes to regional revitalization.

[0843] Hardware and Software

[0844] 1. Terminal

[0845] Mobile information terminals such as smartphones and tablets

[0846] Any device with internet connectivity

[0847] 2. Server

[0848] A high-performance computer that acts as a central processing unit

[0849] Map APIs, database systems, and computers capable of running machine learning algorithms

[0850] software

[0851] 1. Map API

[0852] Google Maps API, etc.

[0853] 2. Database

[0854] A database that stores information about tourist attractions and restaurants and allows you to run SQL queries

[0855] 3. Machine Learning Algorithms

[0856] Analyze user preference data using algorithms such as K-means clustering

[0857] How it works

[0858] 1. Device operation

[0859] The user operates a smartphone or tablet and launches a travel assistance application.

[0860] Enter your destination and transportation method on the application interface.

[0861] 2. Sending input information

[0862] The device collects the input information and sends it to the server, using an internet connection.

[0863] 3. Calculating the optimal route

[0864] The server calls the Google Maps API or similar to calculate the optimal route from the starting point to the destination.

[0865] 4. Search for tourist attractions and restaurants

[0866] The server searches a database of tourist attractions and restaurants around the route, including categories and location information.

[0867] 5. Calculation of additional time and costs

[0868] The server calculates route changes, travel time, and additional costs for each detour, using information obtained from the map API and other databases.

[0869] 6. Considering user preferences

[0870] The server analyzes the user's past usage data and preference data, and applies machine learning algorithms to select the optimal detour plan for the user.

[0871] 7. Detour Plan Generation

[0872] The server generates a detour plan based on the optimal route, combining related detour spots and their detailed information, and sends the generated plan to the device.

[0873] 8. Visual Indications

[0874] The device receives the plan and displays it visually on the application interface with a map, where the user can review the details of each stop, the additional time and cost, and then approve or customize it.

[0875] Specific examples

[0876] Consider an example of a user traveling by car from Osaka to Kyoto. The user inputs the destination "Kyoto" and the mode of travel "car," and the server calculates the optimal route from Osaka to Kyoto. It then searches the database for detour spots, such as the Great Buddha of Nara and famous teahouses in the area, and calculates the additional time and cost involved for each. Based on this information and taking into account the user's past preference for historical places, the server proposes a plan that includes the Great Buddha of Nara and the teahouse. The terminal visually presents this to the user, who then approves it as the final plan.

[0877] Prompt Sentence Examples

[0878] "I'm traveling by car from Osaka to Kyoto. Please suggest the best route, including a stop at the Great Buddha and a teahouse in Nara."

[0879] "Please create a driving plan from Osaka to Kyoto, including tourist spots and restaurants."

[0880] In this way, the present invention is a system that suggests optimal routes and detour spots based on the destination and mode of travel, improving the quality of travel while also contributing to regional revitalization.

[0881] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0882] Step 1:

[0883] The user starts the application and inputs the destination and the method of travel.

[0884] The user launches the application on a smartphone or tablet and inputs their destination and mode of transportation on the interface. The input data is the destination (e.g., Kyoto) and mode of transportation (e.g., car). This input provides the initial data for calculating the optimal route in the next step.

[0885] Step 2:

[0886] The terminal sends the input information to the server.

[0887] The terminal sends the destination and travel method information entered by the user to the server in the form of an HTTP request. The input here is the destination and travel method from the user, and the output is the request data sent to the server.

[0888] Step 3:

[0889] The server calculates the optimal route.

[0890] Based on the received destination and travel method, the server calls a map API such as Google Maps API to calculate the optimal route from the starting point to the destination. The input for this process is the transmitted destination and travel method, and the output is the optimal route information. The server analyzes the route data returned from the API and obtains an efficient route from the starting point to the destination.

[0891] Step 4:

[0892] The server searches for tourist attractions and restaurants around the route.

[0893] The server searches a database for tourist attractions and restaurants near the calculated route. The input is the optimal route information, and the output is a list of relevant tourist attractions and restaurants. The server issues an SQL query to retrieve relevant spots from the database.

[0894] Step 5:

[0895] The server calculates the additional time and cost of stopping at the detour spot.

[0896] The server again uses the map API to calculate route changes and travel times for each detour spot. It also retrieves additional costs, such as entrance fees and food and drink costs, for each spot from a separate database. The input for this process is a list of tourist spots and restaurants and route data, and the output is additional time and cost information for each spot.

[0897] Step 6:

[0898] The server takes into account the user's past usage and preference data.

[0899] The server retrieves the user's past travel history and preference data from a database and applies machine learning algorithms to analyze it. The input is the user's history and preference data, and the output is a detour plan that is suitable for the user.

[0900] Step 7:

[0901] The server generates an optimal detour plan and transmits it to the terminal.

[0902] The server generates a detour plan that combines related detour spots and their detailed information based on optimal route information and user preference data. The generated detour plan is sent to the terminal. The input is tourist spot information obtained from the database, calculated time and cost information, and user preference data, and the output is the detour plan data.

[0903] Step 8:

[0904] The terminal visually displays the detour plan and the user approves or customizes the plan.

[0905] The terminal displays the received detour plan on the application interface. The user can check the detailed information of each spot, the additional time and cost, and approve or customize the plan. The input is the detour plan received from the server, and the output is the visual display to the user.

[0906] (Application example 1)

[0907] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0908] When travelers select their destination and mode of transportation, there is a demand for systems that can automatically suggest optimal routes and detour spots, providing an enjoyable experience while traveling. It is also expected that this will contribute to raising awareness of tourist spots in depopulated areas and revitalizing the region. Furthermore, when users use autonomous vehicles, a system is needed that allows them to easily check and customize detour plans while traveling.

[0909] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0910] In this invention, the server includes a means for a user to input a destination and a mode of transportation, a means for transmitting the input information to the server, and a means for calculating an optimal route based on the destination and mode of transportation, thereby enabling a user of an autonomous vehicle to visually check, approve, or customize a detour plan.

[0911] "User" means an individual or organization that uses the System to plan and manage a trip.

[0912] The "destination" is the final destination of the trip set by the user.

[0913] "Transportation method" refers to the means of transportation selected by the user (e.g., car, train, bus, etc.).

[0914] A "server" is a computer system that receives user input and processes the data.

[0915] "Input information" is data relating to the destination and travel method specified by the user.

[0916] An "optimal route" is the most efficient route from a starting point to a destination.

[0917] A "database" is a collection of data that stores information on tourist spots, restaurants, etc.

[0918] "Places" are tourist spots, restaurants, and other stops along the route.

[0919] "Time and cost" refers to the additional travel time and cost incurred by stopping at each detour location.

[0920] "Usage data" is information relating to a user's past behavioral history and preferences.

[0921] A "side trip plan" is a travel plan that includes suggested destinations along the optimal route.

[0922] "Transportation" refers to the means of transportation or vehicles used by the user.

[0923] A "display" is a display device that displays a user interface.

[0924] "Detailed information" is additional information about the detour location (e.g., required time, cost, overview).

[0925] This invention is a system that suggests optimal routes and detour spots when a traveler inputs their destination and mode of transportation. Based on the information entered by the user, the system calculates the optimal route to the destination, searches for tourist spots and restaurants around the route, and calculates the travel time and cost required for each. The system also selects optimal detour plans taking into account the user's past usage data and preference data, and displays them on a display inside the self-driving vehicle.

[0926] Program processing explanation

[0927] The hardware includes a server, a user terminal, and a display in the autonomous vehicle, while the software uses Python, Google Maps API, and Google Places API.

[0928] When a user inputs their destination and transportation method, the input data is sent to the server, which uses this information to calculate the optimal route to the destination using the Google Maps API. The server then retrieves tourist attractions and restaurants near the route from a database via the Google Places API, including details such as location, rating, and category.

[0929] The server then calculates the additional time and cost of each detour stop, again using the Google Maps API to calculate the route changes and travel time required for each stop, as well as any additional costs associated with each stop (entrance fees, food and drink, etc.).

[0930] To take into account the user's past usage data and preference data, the server applies machine learning algorithms that analyze the places the user has visited in the past and the categories they like, and provide the user with the most suitable detour plan.

[0931] Finally, the server generates the selected detour plan and sends it to the user terminal, which visually displays the plan on a display inside the autonomous vehicle, allowing the user to review the plan's details and approve or customize it as needed.

[0932] Specific examples

[0933] For example, consider a user traveling from Osaka to Kyoto by car. When the user inputs the destination "Kyoto" and the travel method "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches the database for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost involved for each. Based on this information, and taking into account the user's past preference for historical places, it proposes a plan that includes the Great Buddha of Nara and the teahouse. The device visually presents this to the user, who then approves it as the final plan.

[0934] Prompt Sentence Examples

[0935] Generative AI model prompt:

[0936] We are considering a system that suggests optimal routes and detour spots when a user inputs their destination and transportation method. This system uses the Google Maps API and Google Places API to obtain information on optimal routes and tourist spots.

[0937] Generate a Python program using the following information:

[0938] Departure point: "Osaka"

[0939] Destination: "Kyoto"

[0940] Transportation method: "car"

[0941] Get the best route and view nearby attractions.

[0942] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0943] Step 1:

[0944] The user inputs the destination and the mode of travel.

[0945] Input: Destination and method of travel (e.g., "Kyoto", "car").

[0946] Operation: The user enters a destination and a method of travel on the application interface and presses the "Submit" button.

[0947] Output: The input information is sent to the server.

[0948] Step 2:

[0949] The server receives the input information and calculates the optimal route.

[0950] Input: User submitted destination and travel method.

[0951] How it works: The server uses the Google Maps API to get the best route from the start point to the destination.

[0952] Output: Optimal route data.

[0953] Step 3:

[0954] The server searches for tourist attractions and restaurants around the route.

[0955] Input: Optimal route data.

[0956] How it works: The server uses the Google Places API to obtain information about tourist attractions and restaurants near the route.

[0957] Output: A list of attractions and restaurants.

[0958] Step 4:

[0959] The server calculates the additional time and cost of stopping at each detour location.

[0960] Input: List of attractions and restaurants and optimal route data.

[0961] How it works: The server again uses the Google Maps API to calculate route changes and travel times for each stop, as well as any additional costs associated with each stop (e.g., entrance fees, food and drink, etc.).

[0962] Output: Additional time and cost information for each detour location.

[0963] Step 5:

[0964] The server selects the optimal detour plan taking into consideration the user's past usage data and preference data.

[0965] Input: A list of tourist attractions and restaurants, additional time and cost information for each detour, and the user's past usage data.

[0966] How it works: The server uses machine learning algorithms to analyze the user's preferences and past behavioral history and automatically select the most suitable detour plan.

[0967] Output: The selected detour plan.

[0968] Step 6:

[0969] The server generates the selected detour plan and transmits it to the terminal.

[0970] Input: The selected detour plan.

[0971] Operation: The server generates a detour plan and sends it to the user terminal.

[0972] Output: The detour plan received by the device.

[0973] Step 7:

[0974] The terminal displays detailed information about the detour plan on a display inside the user's self-driving vehicle.

[0975] Input: Detour plan sent from the server.

[0976] How it works: The device receives detour plans and details and visually displays them on a display inside the autonomous vehicle, allowing the user to review and customize them in real time.

[0977] Output: Detailed information about the detour plan displayed on the display.

[0978] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0979] This invention is a system that suggests optimal routes and detour spots when a user inputs their destination and transportation method, and furthermore, recognizes the user's emotions to personalize the suggestions. This provides an enjoyable travel experience while traveling and also contributes to regional revitalization.

[0980] First, the user launches the application on a device such as a smartphone or tablet. The user then inputs their destination and travel method on the application interface. This input information is then sent from the device to the server.

[0981] Next, the server receives the destination and mode of travel and calculates the optimal route. The server uses a map API to generate a route from the starting point to the destination. The server then performs a database query to find tourist attractions and restaurants near the route. This database contains information on a variety of tourist attractions and restaurants.

[0982] The server then calculates the additional time and cost of each detour stop, which involves again using the map API to calculate the route changes and travel time required for each stop, as well as any additional costs associated with each stop (entrance fees, food and drink, etc.).

[0983] The server then uses an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and voice data to determine emotions such as joy, sadness, and surprise in real time. This emotion data is sent to the server and, together with the user's usage data and preference data, is reflected in the selection of detour plans.

[0984] The server considers the emotional data and selects the best detour plan for the user's current emotional state. For example, if the user is tired, it will recommend relaxing places, and if the user is excited, it will recommend active tourist spots. Using a machine learning algorithm, it generates the best detour plan that combines the user's preferences and emotional state.

[0985] The server generates a detour plan based on the selected destinations and sends it to the device. The device displays the plan on the application interface, where the user can view detailed information about the detour spots, as well as additional time and costs. The user can then approve the final plan and customize it as needed.

[0986] As a concrete example, consider a user traveling by car from Osaka to Kyoto. When the user inputs the destination "Kyoto" and the mode of travel "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches the database for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost required for each. If the emotion engine recognizes that the user has a tendency to like historical places in the past and that their current emotional state is relaxed, the server will suggest a plan that includes the Great Buddha of Nara and the teahouse. The device visually presents this to the user, who then approves it as the final plan.

[0987] In this way, the present invention improves the quality of travel by suggesting optimal routes and detour spots based on the destination and mode of travel, and also by taking the user's emotions into consideration. Furthermore, by proactively introducing tourist spots in depopulated areas, the system also contributes to regional revitalization.

[0988] The processing flow will be explained below.

[0989] Step 1:

[0990] The user launches the application. The device displays a user interface and provides a screen for inputting the destination and travel method.

[0991] Step 2:

[0992] The user inputs the destination and transportation method. Specifically, the user inputs "Kyoto" in the text box and selects "Car" from the drop-down menu.

[0993] Step 3:

[0994] The device sends the entered destination and travel method to the server. The device packages the input data as an HTTP request and sends it to the server.

[0995] Step 4:

[0996] The server receives the input information and calculates the optimal route. The server calls the map API and generates a route from the starting point (the user's current location) to the destination (Kyoto).

[0997] Step 5:

[0998] The server searches for tourist attractions and restaurants around the route. The server queries a database to obtain a list of tourist attractions (e.g., the Great Buddha of Nara) and restaurants (e.g., local teahouses) within a certain range of the route.

[0999] Step 6:

[1000] The server calculates the additional time and cost of stopping at each detour spot. The server again uses the map API to calculate the route and travel time that will be changed by stopping at each spot. It also calculates additional costs such as entrance fees and food and drink costs at each spot.

[1001] Step 7:

[1002] To recognize the user's emotions, the device's built-in emotion engine analyzes facial expressions and voice data. Specifically, the device's camera and microphone are used to capture the user's facial expressions and voice, which are then analyzed in real time by the emotion engine.

[1003] Step 8:

[1004] The emotion engine sends the analysis results (e.g., joy, sadness, surprise, etc.) to the server. The device includes the analysis results in an HTTP request and sends it.

[1005] Step 9:

[1006] The server receives the emotion data and selects the optimal detour plan based on the user's past usage data and preference data. It uses a machine learning algorithm to select corresponding detour spots based on the user's current emotion.

[1007] Step 10:

[1008] The server generates an optimal detour plan and sends it to the device. Specifically, it generates information such as detour spots, additional time, and costs in a structured data format (e.g., JSON) and sends it to the device as an HTTP response.

[1009] Step 11:

[1010] The device receives the plan from the server and displays it on the user interface. The device analyzes the received data, displays detour spots on the map with markers, and visually presents detailed information about each spot.

[1011] Step 12:

[1012] The user can review the presented detour plan and customize it as needed. The user can add or remove spots, change the order, and the device will resubmit the finalized plan to the server.

[1013] Step 13:

[1014] The user approves the final plan and begins the trip. The device saves the final plan locally and enters navigation mode. The user travels along the plan, and the device provides necessary navigation and detour information.

[1015] Example 2

[1016] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1017] Conventional travel planning systems require users to search for detour spots and manually calculate travel time and costs, which is time-consuming and does not take into account the preferences and emotional state of individual users. Furthermore, it is difficult to individually optimize the travel experience, and these systems do not contribute sufficiently to regional revitalization. The present invention aims to solve these problems by providing optimal travel plans tailored to the individual needs and emotional state of users, and by introducing tourist spots in depopulated areas, contributing to regional revitalization.

[1018] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1019] In this invention, the server includes a means for analyzing facial expressions and voice data to recognize the user's emotions, a means for selecting an optimal detour plan based on the user's past usage data and emotional data, and a means for presenting the detour plan to the user. This allows the server to propose an optimal detour plan based on the user's current emotional state, improving the quality of the trip. Furthermore, personalized plans based on the user's preferences and past data are provided, enabling a travel experience that meets individual needs. Furthermore, by actively introducing local tourist attractions and dining facilities, the server can contribute to regional revitalization.

[1020] "Destination" refers to the location where the user ultimately wants to arrive.

[1021] "Transportation method" refers to the means of transportation (e.g., car, train, walking, etc.) that a user uses to get to a destination.

[1022] "Terminal" refers to an electronic device that is directly operated by a user (e.g., smartphone, tablet, PC, etc.).

[1023] "Server" refers to a remote computer that processes information sent from a terminal and provides the necessary data.

[1024] "Route" refers to the route traveled from a starting point to a destination.

[1025] "Tourist attractions" refer to specific places that are worth visiting for tourists (e.g., historical buildings, natural landscapes, theme parks, etc.).

[1026] "Food and beverage establishment" refers to a place that serves food and drinks (e.g., restaurant, cafe, food stall, etc.).

[1027] "Information base" refers to a database system that stores information on tourist attractions and dining establishments.

[1028] A "side trip point" refers to a place where a user stops temporarily on the way to a destination.

[1029] "Emotion recognition" refers to the technology of analyzing facial expressions and voice data to determine a user's emotional state.

[1030] "Past usage data" refers to historical information about a user's use of the system.

[1031] "Emotion data" refers to information about the user's emotional state analyzed from facial expressions and voice data.

[1032] A "side trip plan" refers to a recommended route that includes stops at tourist attractions and dining facilities on the way to the destination.

[1033] "User interface" refers to the part that provides the screen and operating method for the user to operate the system.

[1034] This invention is a system that, when a user inputs a destination and a mode of transportation, suggests optimal routes and detours, and further personalizes the suggestions by recognizing the user's emotions. This system is composed of a device such as a smartphone or tablet and a server located in a remote location.

[1035] First, the user launches a dedicated application on a device such as a smartphone or tablet. The user inputs their destination and method of transportation on the application interface. This input information is sent from the device to a server. The server receives this information and uses a map API (e.g., Google Maps API) to calculate the optimal route from the starting point to the destination.

[1036] Next, the server searches for tourist attractions and restaurants near the calculated route from an information base, which contains information on various tourist attractions and restaurants (such as name, location, opening hours, category, etc.). The server executes an SQL query to retrieve the relevant spots.

[1037] The server then calculates the additional time and cost of each detour, again using the map API to calculate the route changes and travel time required to visit each stop, as well as any additional costs associated with each stop (entrance fees, food, drink, etc.).

[1038] Next, the device captures the user's facial expression and voice data and sends them to the server, which then uses an emotion engine (e.g., facial expression recognition technology or voice analysis technology) to analyze the user's emotional state. The analysis results are obtained in real time and the user's emotional state (e.g., joy, sadness, surprise, etc.) is determined.

[1039] The server combines the emotional data with the user's past usage data and uses machine learning algorithms to generate optimal detour plans, recommending quiet and relaxing places if the user is relaxed, or active tourist spots if the user is excited.

[1040] The generated detour plan is sent from the server to the device and displayed on the application interface. The user can review the plan and check detailed information (such as additional time and costs). Furthermore, the user can approve the final plan and customize it as needed.

[1041] As a concrete example, consider a user traveling by car from Osaka to Kyoto. When the user inputs the destination "Kyoto" and the mode of travel "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches the information base for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost required for each. If the emotion engine recognizes that the user has a tendency to like historical places in the past and that their current emotional state is relaxed, the server will suggest a plan that includes the Great Buddha of Nara and the teahouse. The device visually presents this information to the user, and the user ultimately approves the plan.

[1042] In this way, the present invention improves the quality of travel by suggesting optimal routes and detours based on the destination and mode of travel, and by taking into account the user's emotions. Furthermore, by proactively introducing tourist spots in depopulated areas, the system also contributes to regional revitalization.

[1043] The following can be used as an example prompt:

[1044] Using the example of a user traveling by car from Osaka to Kyoto, please explain in detail all the processing steps, from inputting the destination and mode of transportation to generating an optimal plan of detour spots. Please also mention the specific operations and APIs and algorithms used. For example, calculating the optimal route using a map API, or analyzing the user's emotions using an emotion recognition engine.

[1045]

[1046] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1047] Step 1:

[1048] The user starts the application using a smartphone or tablet device and inputs the destination "Kyoto" and the method of transportation "car." The input information is entered in text format into a form on the device. When the user has completed the input, they press the "Send" button to send the information to the server. The input data includes "Destination: Kyoto" and "Method of transportation: Car." The output is the input information sent to the server.

[1049] Step 2:

[1050] The terminal converts the "destination" and "transportation method" information entered by the user into JSON format and sends it to the server via an HTTP POST request. The input is the "destination" and "transportation method" data from the user, and the output is the JSON-formatted request data sent to the server. Specifically, the terminal sends the data to an endpoint such as "https: / / example.com / api / route".

[1051] Step 3:

[1052] The server receives the HTTP request and extracts the "destination" and "transportation method" information from the JSON data. It then uses a map API (for example, Google Maps API) to calculate the optimal route from the starting point (the user's current location) to the destination "Kyoto." The input is the user's location information, destination, and transportation method, and the output is the optimal route information. Specifically, the server sends a request to the map API and receives route data in response.

[1053] Step 4:

[1054] The server analyzes the route information returned from the map API and obtains the optimal route. Based on this route information, the server executes an SQL query to search the information base for tourist attractions and restaurants near the route. The input is the route data obtained from the map API, and the output is a list of relevant tourist attractions and restaurants. Specifically, the server executes queries such as "SELECT FROM spots WHERE location NEAR route_coordinates".

[1055] Step 5:

[1056] The server calculates the additional time and cost of stopping at each detour point. This involves using the map API again to calculate the route changes and travel time required to stop at each spot. It also calculates any associated additional costs, such as admission fees and food and beverage costs, for each spot. The input is a list of tourist attractions and restaurants, and the output is the additional time and cost data for each detour point. Specifically, the server executes functions such as "Calculate End-to-End Route Time" and "Fetch Additional Costs."

[1057] Step 6:

[1058] The device captures the user's facial expressions and voice data using input devices such as the device's camera and microphone. The captured data is sent to the server in real time. The input is the user's facial expressions and voice data, and the output is digital data sent to the server. Specifically, the device executes functions such as "Start Camera Capture" and "Record Audio."

[1059] Step 7:

[1060] The server uses an emotion recognition engine to analyze the received facial expression and voice data. Based on the analysis results, the server determines the user's emotional state in real time. The input is the facial expression and voice data sent from the device, and the output is data on the user's emotional state. Specifically, the server executes algorithms such as "Analyze Facial Expression" and "Evaluate Audio Tone."

[1061] Step 8:

[1062] The server uses a machine learning algorithm to generate an optimal detour plan based on the user's past usage data and current emotional data. For example, if the user is relaxed, it will suggest a quiet and relaxing place, and if the user is excited, it will recommend an active tourist spot. The input is the user's past data and emotional data, and the output is the optimal detour plan. Specifically, the server executes processes such as "Generate Personalized Plan" and "Optimize Travel Itinerary."

[1063] Step 9:

[1064] The server sends the generated detour plan to the device. The input is the generated detour plan, and the output is the plan data to be sent to the device. Specifically, the server executes a command such as "Send Plan to Device."

[1065] Step 10:

[1066] The terminal displays the received detour plan on the application interface. The user can check the plan details (e.g., additional time and cost). The input is the plan data sent from the server, and the output is the interface display visually presented to the user. Specifically, the terminal executes functions such as "Display Plan on Screen" and "Show Plan Details."

[1067] (Application example 2)

[1068] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1069] Conventional route guidance systems have the ability to suggest optimal routes to a destination and detour spots, but they are unable to provide personalized suggestions that take into account the user's emotions and real-time situation. This limits the user experience and poses challenges in providing a more fulfilling travel experience. Furthermore, suggestions that do not take emotions into account are often inappropriate for the user's situation, potentially reducing satisfaction.

[1070] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input a destination and a method of travel, means for transmitting the input information to the server, means for calculating an optimal route based on the destination and method of travel, means for selecting an optimal detour plan taking into account the user's past usage data and current emotional state, and means for capturing and analyzing the user's facial expressions in real time using face recognition technology with a smartphone or a head-mounted display. This makes it possible to grasp the user's emotional state in real time and make personalized suggestions that are optimal for the situation.

[1071] A "user" is a person who uses the system by inputting a destination and a mode of travel.

[1072] A "destination" is a place that a user wishes to visit.

[1073] A "mode of travel" is the means of transportation a user chooses to reach a destination.

[1074] "Means" refers to a method or apparatus by which a system performs a particular function.

[1075] A "server" is a central processing unit that performs route calculations, database searches, sentiment analysis, and the like.

[1076] "Input information" refers to data that a user provides to the system regarding a destination and a method of travel.

[1077] The "optimal route" is the route that most effectively reaches the destination entered by the user.

[1078] "Tourist attractions" are places of interest or famous places that users can visit.

[1079] A "restaurant" is a store where users can purchase food and drinks.

[1080] A "database" is a system that organizes and stores information about tourist spots and restaurants.

[1081] A "side trip spot" is a place where a user can stop by on the way to their destination.

[1082] "Time" is the travel time added by stopping at a detour spot.

[1083] "Cost" is the additional monetary cost incurred by stopping at a detour spot.

[1084] "Usage data" refers to data and behavioral history of a user's past use.

[1085] "Emotional state" is a state that indicates the user's emotions in real time.

[1086] A "side trip plan" is a list of suggested side trip spots based on the optimal route.

[1087] A "smartphone" is a type of mobile phone and is a device that a user can use to access the system.

[1088] A "head-mounted display (HMD)" is a device worn by a user to visually obtain displayed information.

[1089] "Facial recognition technology" is a technology that captures and analyzes a user's facial expressions.

[1090] "Capture" is the act of acquiring an image or data.

[1091] "Analysis" is the act of analyzing the acquired data in detail.

[1092] "Real time" is a time frame in which processing occurs immediately without delay.

[1093] This invention is a system that suggests optimal routes and detour spots when a user inputs their destination and transportation method, and further personalizes the suggestions by recognizing the user's emotions. Specifically, this system is realized using the following hardware and software.

[1094] Hardware used:

[1095] 1. Smartphone

[1096] It is used by the user to input the destination and travel method and send it to the server.

[1097] It has a camera function and is used to capture the user's facial expressions.

[1098] 2. Head-Mounted Display (HMD)

[1099] This device allows users to visually confirm information and is used to capture the user's facial expressions in real time using facial recognition technology.

[1100] 3. Server

[1101] This is the central processing unit that handles major processes such as route calculation, database search, and emotion analysis.

[1102] Software used:

[1103] 1. Facial Recognition API

[1104] Examples: Amazon Rekognition, Google Cloud Vision API

[1105] 2. Map API

[1106] Example: Google Maps API

[1107] 3. Database Management System

[1108] Example: PostgreSQL

[1109] 4. Machine Learning Algorithms

[1110] Example: TensorFlow

[1111] Process flow:

[1112] Step 1: Obtain user data

[1113] The user inputs the destination and travel method using a smartphone or HMD, and this information is sent from the device to the server.

[1114] Step 2: Route calculation and spot search by server

[1115] The server uses a map API to calculate the optimal route from the departure point to the destination and generate route information, while simultaneously searching a database for tourist spots and restaurants along the calculated route.

[1116] Step 3: Emotion Recognition

[1117] The user's face is captured using a smartphone or HMD camera, and emotions are analyzed in real time using a facial recognition API.

[1118] Step 4: Generate optimal detour plans

[1119] The server uses a machine learning algorithm to select the optimal detour spots based on the user's past usage data and analyzed emotional data, and generates a detour plan.

[1120] Step 5: Present to the user

[1121] The generated detour plan is sent from the server to the device and visually displayed to the user via a smartphone or HMD, where the user can review the proposal and customize it as needed.

[1122] Examples:

[1123] For example, imagine a user traveling from Osaka to Kyoto in an autonomous vehicle. The user uses their smartphone to input their destination "Kyoto" and their mode of travel "car." The HMD then recognizes the user's face, and the emotion engine analyzes whether they are in a relaxed state. Based on this data, the server suggests detour spots where the user can relax, such as the Great Buddha of Nara or a famous local teahouse. The user can visually confirm these suggestions and either adopt them as an optimal travel plan or customize them.

[1124] Example prompt sentence:

[1125] Suggest the best travel route and detour spots using the following information: The user's destination is "Kyoto" and the mode of transportation is "car." The user's current emotional state is relaxed. List recommended detour spots.

[1126] This allows us to provide a personalized travel experience that utilizes the user's emotions and travel information.

[1127] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1128] Step 1: Obtain user data

[1129] The user inputs their destination and travel method using a smartphone or HMD. The input information (destination and travel method) is sent to the server via the device. This allows the server to receive basic instructions about the user's trip and obtain data for the next processing step.

[1130] Step 2: Route calculation

[1131] The server calculates the optimal route using a map API (e.g., Google Maps API) based on the input information it receives. Specifically, it receives the starting point and destination as input and outputs the optimal route taking into account the shortest distance and required time. This output includes the most efficient route information for the user.

[1132] Step 3: Search for spots

[1133] Based on the calculated optimal route, the server searches for tourist attractions and restaurants near the route using a database query. The database (e.g., PostgreSQL) stores information on various tourist attractions and restaurants, and the search query effectively extracts this information. The input is the optimal route, and the output is a list of detour spots.

[1134] Step 4: Calculate additional time and costs

[1135] The server calculates the additional time and cost of stopping at each listed detour spot. It uses the map API again, taking as input the route changes and travel time required for each stop, and outputs the additional time and cost involved.

[1136] Step 5: Emotion Recognition

[1137] The user's facial expressions are captured by a smartphone or HMD camera, and emotions are analyzed in real time using a facial recognition API (e.g., Amazon Rekognition). The input is the captured facial expression data, and the output is the user's emotional state (e.g., relaxed, excited, etc.). This allows the server to understand the user's current emotional state.

[1138] Step 6: Choose a detour plan

[1139] The server uses a machine learning algorithm (e.g., TensorFlow) to select optimal detour spots based on the user's past usage data and current emotional data. The input is past usage data and the user's current emotional state, and the output is an optimal list of detour spots. The machine learning algorithm analyzes the past data and emotional data to suggest the most suitable spots for the user.

[1140] Step 7: Generate and submit your detour plan

[1141] The server sends the generated detour plan to the terminal. The input is a list of selected detour spots, and the output is the detour plan displayed on the user's terminal, allowing the user to review the final plan and customize it as needed.

[1142] Step 8: Present to the user

[1143] The device then displays the received detour plan on the user interface of the smartphone or HMD. Specifically, the interface allows the user to visually check detailed information about the spots, the additional time and cost, etc. This allows the user to understand the proposed detour plan and achieve the optimal travel experience.

[1144] Through these steps, the system can provide a personalized travel experience that utilizes the user's emotions and travel information.

[1145] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1146] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1147] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1148] [Fourth embodiment]

[1149] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1150] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1151] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1152] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1153] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1154] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1155] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1156] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1157] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1158] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1159] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1160] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1161] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1162] This invention is a system that suggests optimal routes and detour spots when travelers input their destination and transportation method, providing an enjoyable travel experience while on the move. Furthermore, it contributes to regional revitalization by introducing tourist spots in depopulated areas.

[1163] First, the user launches the application on a device such as a smartphone or tablet. The user then inputs their destination and travel method on the application interface. This input information is then sent from the device to the server.

[1164] Once the server receives the destination and mode of travel, it calculates the optimal route. Using a map API, the server generates a route from the starting point to the destination. The server then queries a database to find tourist attractions and restaurants near the route. This database contains information on a variety of tourist attractions and restaurants.

[1165] The server then calculates the additional time and cost of each detour stop, again using the map API to calculate the route changes and travel time required for each stop, as well as any additional costs associated with each stop (entrance fees, food and drink, etc.).

[1166] At this stage, the server takes into account the user's past usage and preference data. It references the user's history and analyzes their interest in specific categories (e.g., historical places, natural scenery, etc.). It then applies machine learning algorithms to select the most suitable detour plan for the user.

[1167] The server then generates a detour plan based on the selected itinerary and sends it to the device. The device then displays the plan on the application interface, allowing the user to visually confirm the detour plan. Detailed information about each stop, as well as additional time and costs, are also displayed, allowing the user to easily approve or customize the plan.

[1168] As a concrete example, consider a user traveling by car from Osaka to Kyoto. When the user inputs the destination "Kyoto" and the mode of travel "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches a database for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost required for each. Based on this information, and taking into account the user's tendency to prefer historical places in the past, it proposes a plan that includes the Great Buddha of Nara and the teahouse. The terminal visually presents this to the user, who then approves it as the final plan.

[1169] In this way, the present invention is a system that improves the quality of travel and contributes to regional revitalization by suggesting optimal routes and detour spots based on the destination and mode of travel.

[1170] The processing flow will be explained below.

[1171] Step 1:

[1172] The user launches the application. The device displays a user interface and provides a screen for inputting the destination and travel method.

[1173] Step 2:

[1174] The user inputs the destination and transportation method. Specifically, the user inputs "Kyoto" in the text box and selects "Car" from the drop-down menu.

[1175] Step 3:

[1176] The device sends the entered destination and travel method to the server. The device packages the input data as an HTTP request and sends it to the server.

[1177] Step 4:

[1178] The server receives the input information and calculates the optimal route. The server calls the map API and generates a route from the starting point (the user's current location) to the destination (Kyoto).

[1179] Step 5:

[1180] The server searches for tourist attractions and restaurants around the route. The server queries a database to obtain a list of tourist attractions (e.g., the Great Buddha of Nara) and restaurants (e.g., local teahouses) within a certain range of the route.

[1181] Step 6:

[1182] The server calculates the additional time and cost of stopping at each detour spot. The server again uses the map API to calculate the route and travel time that will be changed by stopping at each spot. It also calculates additional costs such as entrance fees and food and drink costs at each spot.

[1183] Step 7:

[1184] The server selects the optimal detour plan based on the user's past usage data and preference data. The server refers to the user's past history and analyzes their interest in specific categories using machine learning algorithms.

[1185] Step 8:

[1186] The server generates an optimal detour plan and sends it to the device. Specifically, it generates data in a structured data format (e.g., JSON) including information such as detour spots, additional time, and costs, and sends it to the device as an HTTP response.

[1187] Step 9:

[1188] The device receives the plan from the server and displays it on the user interface. The device analyzes the received data, displays detour spots on the map with markers, and visually presents detailed information about each spot.

[1189] Step 10:

[1190] The user can review the presented detour plan and customize it as needed. The user can add or remove spots, change the order, and the device will resubmit the finalized plan to the server.

[1191] Step 11:

[1192] The user approves the final plan and begins the trip. The device saves the final plan locally and enters navigation mode. The user travels along the plan, and the device provides necessary navigation and detour information.

[1193] Example 1

[1194] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1195] Conventional travel planning systems require users to manually set their destination and route, which requires a lot of time and effort when selecting detour spots and calculating travel time and additional costs. Furthermore, it is difficult to provide optimal detour plans that take into account the user's travel preferences and past usage. As a result, users' travel experiences are limited, and they are unable to fully utilize local tourist spots.

[1196] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1197] In this invention, the server includes a means for selecting an optimal detour plan taking into account the user's past usage data and preference data, a means for using a map API to calculate the optimal route, and a means for analyzing the user's preference data using a machine learning algorithm. This allows the user to automatically receive optimal routes and detour spots without having to manually create complex plans, thereby improving the quality of the travel experience.

[1198] 1. "User" refers to a person who uses the system to set a destination and specify a method of travel.

[1199] 2. "Terminal" refers to a mobile information terminal such as a smartphone or tablet, which transmits information entered by the user to a server.

[1200] 3. "Server" refers to the central processing unit that receives and analyzes information entered by the user and calculates and generates optimal routes and detour plans.

[1201] 4. "Destination" refers to the final destination set by a user as the purpose of a trip or movement.

[1202] 5. "Method of transportation" refers to the means of transportation specified by the user (e.g., car, train, bus, etc.).

[1203] 6. "Optimal route" refers to the most efficient and effective route from a starting point to a destination.

[1204] 7. "Tourist spot" refers to a famous place or tourist destination that travelers visit.

[1205] 8. "Restaurant" refers to an establishment where travelers stop to have a meal.

[1206] 9. "Database" refers to an information aggregation system in which information on tourist spots and restaurants is systematically stored.

[1207] 10. "Detour Spot" refers to a tourist spot or restaurant located on the user's travel route that can be visited along the way.

[1208] 11. "Additional time" refers to the extra travel time or time spent at a detour spot.

[1209] 12. "Additional costs" refers to fees such as entrance fees and food and drink costs incurred by stopping at detour spots.

[1210] 13. "Preference Data" refers to information regarding the categories and types of places that a User has previously visited.

[1211] 14. "Optimal detour plan" refers to a plan that combines the most suitable tourist spots and restaurants to stop at while traveling to the user's destination.

[1212] 15. "Visual display" refers to displaying information in an easily viewable manner as text or images on a user interface.

[1213] 16. "Map API" means an application program interface that provides map information and enables route calculation and geographic data retrieval.

[1214] 17. "Machine learning algorithm" refers to a computational method that learns patterns from past data and makes predictions or classifications for new data.

[1215] MODE FOR CARRYING OUT THE INVENTION

[1216] This invention is a system that suggests optimal routes and detour spots when travelers input their destination and transportation method, providing an enjoyable travel experience while on the move. Furthermore, by introducing tourist spots in depopulated areas, it contributes to regional revitalization.

[1217] Hardware and Software

[1218] 1. Terminal

[1219] Mobile information terminals such as smartphones and tablets

[1220] Any device with internet connectivity

[1221] 2. Server

[1222] A high-performance computer that acts as a central processing unit

[1223] Map APIs, database systems, and computers capable of running machine learning algorithms

[1224] software

[1225] 1. Map API

[1226] Google Maps API, etc.

[1227] 2. Database

[1228] A database that stores information about tourist attractions and restaurants and allows you to run SQL queries

[1229] 3. Machine Learning Algorithms

[1230] Analyze user preference data using algorithms such as K-means clustering

[1231] How it works

[1232] 1. Device operation

[1233] The user operates a smartphone or tablet and launches a travel assistance application.

[1234] Enter your destination and transportation method on the application interface.

[1235] 2. Sending input information

[1236] The device collects the input information and sends it to the server, using an internet connection.

[1237] 3. Calculating the optimal route

[1238] The server calls the Google Maps API or similar to calculate the optimal route from the starting point to the destination.

[1239] 4. Search for tourist attractions and restaurants

[1240] The server searches a database of tourist attractions and restaurants around the route, including categories and location information.

[1241] 5. Calculation of additional time and costs

[1242] The server calculates route changes, travel time, and additional costs for each detour, using information obtained from the map API and other databases.

[1243] 6. Considering user preferences

[1244] The server analyzes the user's past usage data and preference data, and applies machine learning algorithms to select the optimal detour plan for the user.

[1245] 7. Detour Plan Generation

[1246] The server generates a detour plan based on the optimal route, combining related detour spots and their detailed information, and sends the generated plan to the device.

[1247] 8. Visual Indications

[1248] The device receives the plan and displays it visually on the application interface with a map, where the user can review the details of each stop, the additional time and cost, and then approve or customize it.

[1249] Specific examples

[1250] Consider an example of a user traveling by car from Osaka to Kyoto. The user inputs the destination "Kyoto" and the mode of travel "car," and the server calculates the optimal route from Osaka to Kyoto. It then searches the database for detour spots, such as the Great Buddha of Nara and famous teahouses in the area, and calculates the additional time and cost involved for each. Based on this information and taking into account the user's past preference for historical places, the server proposes a plan that includes the Great Buddha of Nara and the teahouse. The terminal visually presents this to the user, who then approves it as the final plan.

[1251] Prompt Sentence Examples

[1252] "I'm traveling by car from Osaka to Kyoto. Please suggest the best route, including a stop at the Great Buddha and a teahouse in Nara."

[1253] "Please create a driving plan from Osaka to Kyoto, including tourist spots and restaurants."

[1254] In this way, the present invention is a system that suggests optimal routes and detour spots based on the destination and mode of travel, improving the quality of travel while also contributing to regional revitalization.

[1255] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1256] Step 1:

[1257] The user starts the application and inputs the destination and the method of travel.

[1258] The user launches the application on a smartphone or tablet and inputs their destination and mode of transportation on the interface. The input data is the destination (e.g., Kyoto) and mode of transportation (e.g., car). This input provides the initial data for calculating the optimal route in the next step.

[1259] Step 2:

[1260] The terminal sends the input information to the server.

[1261] The terminal sends the destination and travel method information entered by the user to the server in the form of an HTTP request. The input here is the destination and travel method from the user, and the output is the request data sent to the server.

[1262] Step 3:

[1263] The server calculates the optimal route.

[1264] Based on the received destination and travel method, the server calls a map API such as Google Maps API to calculate the optimal route from the starting point to the destination. The input for this process is the transmitted destination and travel method, and the output is the optimal route information. The server analyzes the route data returned from the API and obtains an efficient route from the starting point to the destination.

[1265] Step 4:

[1266] The server searches for tourist attractions and restaurants around the route.

[1267] The server searches a database for tourist attractions and restaurants near the calculated route. The input is the optimal route information, and the output is a list of relevant tourist attractions and restaurants. The server issues an SQL query to retrieve relevant spots from the database.

[1268] Step 5:

[1269] The server calculates the additional time and cost of stopping at the detour spot.

[1270] The server again uses the map API to calculate route changes and travel times for each detour spot. It also retrieves additional costs, such as entrance fees and food and drink costs, for each spot from a separate database. The input for this process is a list of tourist spots and restaurants and route data, and the output is additional time and cost information for each spot.

[1271] Step 6:

[1272] The server takes into account the user's past usage and preference data.

[1273] The server retrieves the user's past travel history and preference data from a database and applies machine learning algorithms to analyze it. The input is the user's history and preference data, and the output is a detour plan that is suitable for the user.

[1274] Step 7:

[1275] The server generates an optimal detour plan and transmits it to the terminal.

[1276] The server generates a detour plan that combines related detour spots and their detailed information based on optimal route information and user preference data. The generated detour plan is sent to the terminal. The input is tourist spot information obtained from the database, calculated time and cost information, and user preference data, and the output is the detour plan data.

[1277] Step 8:

[1278] The terminal visually displays the detour plan and the user approves or customizes the plan.

[1279] The terminal displays the received detour plan on the application interface. The user can check the detailed information of each spot, the additional time and cost, and approve or customize the plan. The input is the detour plan received from the server, and the output is the visual display to the user.

[1280] (Application example 1)

[1281] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1282] When travelers select their destination and mode of transportation, there is a demand for systems that can automatically suggest optimal routes and detour spots, providing an enjoyable experience while traveling. It is also expected that this will contribute to raising awareness of tourist spots in depopulated areas and revitalizing the region. Furthermore, when users use autonomous vehicles, a system is needed that allows them to easily check and customize detour plans while traveling.

[1283] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1284] In this invention, the server includes a means for a user to input a destination and a mode of transportation, a means for transmitting the input information to the server, and a means for calculating an optimal route based on the destination and mode of transportation, thereby enabling a user of an autonomous vehicle to visually check, approve, or customize a detour plan.

[1285] "User" means an individual or organization that uses the System to plan and manage a trip.

[1286] The "destination" is the final destination of the trip set by the user.

[1287] "Transportation method" refers to the means of transportation selected by the user (e.g., car, train, bus, etc.).

[1288] A "server" is a computer system that receives user input and processes the data.

[1289] "Input information" is data relating to the destination and travel method specified by the user.

[1290] An "optimal route" is the most efficient route from a starting point to a destination.

[1291] A "database" is a collection of data that stores information on tourist spots, restaurants, etc.

[1292] "Places" are tourist spots, restaurants, and other stops along the route.

[1293] "Time and cost" refers to the additional travel time and cost incurred by stopping at each detour location.

[1294] "Usage data" is information relating to a user's past behavioral history and preferences.

[1295] A "side trip plan" is a travel plan that includes suggested destinations along the optimal route.

[1296] "Transportation" refers to the means of transportation or vehicles used by the user.

[1297] A "display" is a display device that displays a user interface.

[1298] "Detailed information" is additional information about the detour location (e.g., required time, cost, overview).

[1299] This invention is a system that suggests optimal routes and detour spots when a traveler inputs their destination and mode of transportation. Based on the information entered by the user, the system calculates the optimal route to the destination, searches for tourist spots and restaurants around the route, and calculates the travel time and cost required for each. The system also selects optimal detour plans taking into account the user's past usage data and preference data, and displays them on a display inside the self-driving vehicle.

[1300] Program processing explanation

[1301] The hardware includes a server, a user terminal, and a display in the autonomous vehicle, while the software uses Python, Google Maps API, and Google Places API.

[1302] When a user inputs their destination and transportation method, the input data is sent to the server, which uses this information to calculate the optimal route to the destination using the Google Maps API. The server then retrieves tourist attractions and restaurants near the route from a database via the Google Places API, including details such as location, rating, and category.

[1303] The server then calculates the additional time and cost of each detour stop, again using the Google Maps API to calculate the route changes and travel time required for each stop, as well as any additional costs associated with each stop (entrance fees, food and drink, etc.).

[1304] To take into account the user's past usage data and preference data, the server applies machine learning algorithms that analyze the places the user has visited in the past and the categories they like, and provide the user with the most suitable detour plan.

[1305] Finally, the server generates the selected detour plan and sends it to the user terminal, which visually displays the plan on a display inside the autonomous vehicle, allowing the user to review the plan's details and approve or customize it as needed.

[1306] Specific examples

[1307] For example, consider a user traveling from Osaka to Kyoto by car. When the user inputs the destination "Kyoto" and the travel method "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches the database for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost involved for each. Based on this information, and taking into account the user's past preference for historical places, it proposes a plan that includes the Great Buddha of Nara and the teahouse. The device visually presents this to the user, who then approves it as the final plan.

[1308] Prompt Sentence Examples

[1309] Generative AI model prompt:

[1310] We are considering a system that suggests optimal routes and detour spots when a user inputs their destination and transportation method. This system uses the Google Maps API and Google Places API to obtain information on optimal routes and tourist spots.

[1311] Generate a Python program using the following information:

[1312] Departure point: "Osaka"

[1313] Destination: "Kyoto"

[1314] Transportation method: "car"

[1315] Get the best route and view nearby attractions.

[1316] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1317] Step 1:

[1318] The user inputs the destination and the mode of travel.

[1319] Input: Destination and method of travel (e.g., "Kyoto", "car").

[1320] Operation: The user enters a destination and a method of travel on the application interface and presses the "Submit" button.

[1321] Output: The input information is sent to the server.

[1322] Step 2:

[1323] The server receives the input information and calculates the optimal route.

[1324] Input: User submitted destination and travel method.

[1325] How it works: The server uses the Google Maps API to get the best route from the start point to the destination.

[1326] Output: Optimal route data.

[1327] Step 3:

[1328] The server searches for tourist attractions and restaurants around the route.

[1329] Input: Optimal route data.

[1330] How it works: The server uses the Google Places API to obtain information about tourist attractions and restaurants near the route.

[1331] Output: A list of attractions and restaurants.

[1332] Step 4:

[1333] The server calculates the additional time and cost of stopping at each detour location.

[1334] Input: List of attractions and restaurants and optimal route data.

[1335] How it works: The server again uses the Google Maps API to calculate route changes and travel times for each stop, as well as any additional costs associated with each stop (e.g., entrance fees, food and drink, etc.).

[1336] Output: Additional time and cost information for each detour location.

[1337] Step 5:

[1338] The server selects the optimal detour plan taking into consideration the user's past usage data and preference data.

[1339] Input: A list of tourist attractions and restaurants, additional time and cost information for each detour, and the user's past usage data.

[1340] How it works: The server uses machine learning algorithms to analyze the user's preferences and past behavioral history and automatically select the most suitable detour plan.

[1341] Output: The selected detour plan.

[1342] Step 6:

[1343] The server generates the selected detour plan and transmits it to the terminal.

[1344] Input: The selected detour plan.

[1345] Operation: The server generates a detour plan and sends it to the user terminal.

[1346] Output: The detour plan received by the device.

[1347] Step 7:

[1348] The terminal displays detailed information about the detour plan on a display inside the user's self-driving vehicle.

[1349] Input: Detour plan sent from the server.

[1350] How it works: The device receives detour plans and details and visually displays them on a display inside the autonomous vehicle, allowing the user to review and customize them in real time.

[1351] Output: Detailed information about the detour plan displayed on the display.

[1352] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1353] This invention is a system that suggests optimal routes and detour spots when a user inputs their destination and transportation method, and furthermore, recognizes the user's emotions to personalize the suggestions. This provides an enjoyable travel experience while traveling and also contributes to regional revitalization.

[1354] First, the user launches the application on a device such as a smartphone or tablet. The user then inputs their destination and travel method on the application interface. This input information is then sent from the device to the server.

[1355] Next, the server receives the destination and mode of travel and calculates the optimal route. The server uses a map API to generate a route from the starting point to the destination. The server then performs a database query to find tourist attractions and restaurants near the route. This database contains information on a variety of tourist attractions and restaurants.

[1356] The server then calculates the additional time and cost of each detour stop, which involves again using the map API to calculate the route changes and travel time required for each stop, as well as any additional costs associated with each stop (entrance fees, food and drink, etc.).

[1357] The server then uses an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and voice data to determine emotions such as joy, sadness, and surprise in real time. This emotion data is sent to the server and, together with the user's usage data and preference data, is reflected in the selection of detour plans.

[1358] The server considers the emotional data and selects the best detour plan for the user's current emotional state. For example, if the user is tired, it will recommend relaxing places, and if the user is excited, it will recommend active tourist spots. Using a machine learning algorithm, it generates the best detour plan that combines the user's preferences and emotional state.

[1359] The server generates a detour plan based on the selected destinations and sends it to the device. The device displays the plan on the application interface, where the user can view detailed information about the detour spots, as well as additional time and costs. The user can then approve the final plan and customize it as needed.

[1360] As a concrete example, consider a user traveling by car from Osaka to Kyoto. When the user inputs the destination "Kyoto" and the mode of travel "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches the database for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost required for each. If the emotion engine recognizes that the user has a tendency to like historical places in the past and that their current emotional state is relaxed, the server will suggest a plan that includes the Great Buddha of Nara and the teahouse. The device visually presents this to the user, who then approves it as the final plan.

[1361] In this way, the present invention improves the quality of travel by suggesting optimal routes and detour spots based on the destination and mode of travel, and also by taking the user's emotions into consideration. Furthermore, by proactively introducing tourist spots in depopulated areas, the system also contributes to regional revitalization.

[1362] The processing flow will be explained below.

[1363] Step 1:

[1364] The user launches the application. The device displays a user interface and provides a screen for inputting the destination and travel method.

[1365] Step 2:

[1366] The user inputs the destination and transportation method. Specifically, the user inputs "Kyoto" in the text box and selects "Car" from the drop-down menu.

[1367] Step 3:

[1368] The device sends the entered destination and travel method to the server. The device packages the input data as an HTTP request and sends it to the server.

[1369] Step 4:

[1370] The server receives the input information and calculates the optimal route. The server calls the map API and generates a route from the starting point (the user's current location) to the destination (Kyoto).

[1371] Step 5:

[1372] The server searches for tourist attractions and restaurants around the route. The server queries a database to obtain a list of tourist attractions (e.g., the Great Buddha of Nara) and restaurants (e.g., local teahouses) within a certain range of the route.

[1373] Step 6:

[1374] The server calculates the additional time and cost of stopping at each detour spot. The server again uses the map API to calculate the route and travel time that will be changed by stopping at each spot. It also calculates additional costs such as entrance fees and food and drink costs at each spot.

[1375] Step 7:

[1376] To recognize the user's emotions, the device's built-in emotion engine analyzes facial expressions and voice data. Specifically, the device's camera and microphone are used to capture the user's facial expressions and voice, which are then analyzed in real time by the emotion engine.

[1377] Step 8:

[1378] The emotion engine sends the analysis results (e.g., joy, sadness, surprise, etc.) to the server. The device includes the analysis results in an HTTP request and sends it.

[1379] Step 9:

[1380] The server receives the emotion data and selects the optimal detour plan based on the user's past usage data and preference data. It uses a machine learning algorithm to select corresponding detour spots based on the user's current emotion.

[1381] Step 10:

[1382] The server generates an optimal detour plan and sends it to the device. Specifically, it generates information such as detour spots, additional time, and costs in a structured data format (e.g., JSON) and sends it to the device as an HTTP response.

[1383] Step 11:

[1384] The device receives the plan from the server and displays it on the user interface. The device analyzes the received data, displays detour spots on the map with markers, and visually presents detailed information about each spot.

[1385] Step 12:

[1386] The user can review the presented detour plan and customize it as needed. The user can add or remove spots, change the order, and the device will resubmit the finalized plan to the server.

[1387] Step 13:

[1388] The user approves the final plan and begins the trip. The device saves the final plan locally and enters navigation mode. The user travels along the plan, and the device provides necessary navigation and detour information.

[1389] Example 2

[1390] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1391] Conventional travel planning systems require users to search for detour spots and manually calculate travel time and costs, which is time-consuming and does not take into account the preferences and emotional state of individual users. Furthermore, it is difficult to individually optimize the travel experience, and these systems do not contribute sufficiently to regional revitalization. The present invention aims to solve these problems by providing optimal travel plans tailored to the individual needs and emotional state of users, and by introducing tourist spots in depopulated areas, contributing to regional revitalization.

[1392] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1393] In this invention, the server includes a means for analyzing facial expressions and voice data to recognize the user's emotions, a means for selecting an optimal detour plan based on the user's past usage data and emotional data, and a means for presenting the detour plan to the user. This allows the server to propose an optimal detour plan based on the user's current emotional state, improving the quality of the trip. Furthermore, personalized plans based on the user's preferences and past data are provided, enabling a travel experience that meets individual needs. Furthermore, by actively introducing local tourist attractions and dining facilities, the server can contribute to regional revitalization.

[1394] "Destination" refers to the location where the user ultimately wants to arrive.

[1395] "Transportation method" refers to the means of transportation (e.g., car, train, walking, etc.) that a user uses to get to a destination.

[1396] "Terminal" refers to an electronic device that is directly operated by a user (e.g., smartphone, tablet, PC, etc.).

[1397] "Server" refers to a remote computer that processes information sent from a terminal and provides the necessary data.

[1398] "Route" refers to the route traveled from a starting point to a destination.

[1399] "Tourist attractions" refer to specific places that are worth visiting for tourists (e.g., historical buildings, natural landscapes, theme parks, etc.).

[1400] "Food and beverage establishment" refers to a place that serves food and drinks (e.g., restaurant, cafe, food stall, etc.).

[1401] "Information base" refers to a database system that stores information on tourist attractions and dining establishments.

[1402] A "side trip point" refers to a place where a user stops temporarily on the way to a destination.

[1403] "Emotion recognition" refers to the technology of analyzing facial expressions and voice data to determine a user's emotional state.

[1404] "Past usage data" refers to historical information about a user's use of the system.

[1405] "Emotion data" refers to information about the user's emotional state analyzed from facial expressions and voice data.

[1406] A "side trip plan" refers to a recommended route that includes stops at tourist attractions and dining facilities on the way to the destination.

[1407] "User interface" refers to the part that provides the screen and operating method for the user to operate the system.

[1408] This invention is a system that, when a user inputs a destination and a mode of transportation, suggests optimal routes and detours, and further personalizes the suggestions by recognizing the user's emotions. This system is composed of a device such as a smartphone or tablet and a server located in a remote location.

[1409] First, the user launches a dedicated application on a device such as a smartphone or tablet. The user inputs their destination and method of transportation on the application interface. This input information is sent from the device to a server. The server receives this information and uses a map API (e.g., Google Maps API) to calculate the optimal route from the starting point to the destination.

[1410] Next, the server searches for tourist attractions and restaurants near the calculated route from an information base, which contains information on various tourist attractions and restaurants (such as name, location, opening hours, category, etc.). The server executes an SQL query to retrieve the relevant spots.

[1411] The server then calculates the additional time and cost of each detour, again using the map API to calculate the route changes and travel time required to visit each stop, as well as any additional costs associated with each stop (entrance fees, food, drink, etc.).

[1412] Next, the device captures the user's facial expression and voice data and sends them to the server, which then uses an emotion engine (e.g., facial expression recognition technology or voice analysis technology) to analyze the user's emotional state. The analysis results are obtained in real time and the user's emotional state (e.g., joy, sadness, surprise, etc.) is determined.

[1413] The server combines the emotional data with the user's past usage data and uses machine learning algorithms to generate optimal detour plans, recommending quiet and relaxing places if the user is relaxed, or active tourist spots if the user is excited.

[1414] The generated detour plan is sent from the server to the device and displayed on the application interface. The user can review the plan and check detailed information (such as additional time and costs). Furthermore, the user can approve the final plan and customize it as needed.

[1415] As a concrete example, consider a user traveling by car from Osaka to Kyoto. When the user inputs the destination "Kyoto" and the mode of travel "car," the server calculates the optimal route from Osaka to Kyoto. Next, it searches the information base for detour spots such as the Great Buddha of Nara and famous local teahouses, and calculates the additional time and cost required for each. If the emotion engine recognizes that the user has a tendency to like historical places in the past and that their current emotional state is relaxed, the server will suggest a plan that includes the Great Buddha of Nara and the teahouse. The device visually presents this information to the user, and the user ultimately approves the plan.

[1416] In this way, the present invention improves the quality of travel by suggesting optimal routes and detours based on the destination and mode of travel, and by taking into account the user's emotions. Furthermore, by proactively introducing tourist spots in depopulated areas, the system also contributes to regional revitalization.

[1417] The following can be used as an example prompt:

[1418] Using the example of a user traveling by car from Osaka to Kyoto, please explain in detail all the processing steps, from inputting the destination and mode of transportation to generating an optimal plan of detour spots. Please also mention the specific operations and APIs and algorithms used. For example, calculating the optimal route using a map API, or analyzing the user's emotions using an emotion recognition engine.

[1419]

[1420] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1421] Step 1:

[1422] The user starts the application using a smartphone or tablet device and inputs the destination "Kyoto" and the method of transportation "car." The input information is entered in text format into a form on the device. When the user has completed the input, they press the "Send" button to send the information to the server. The input data includes "Destination: Kyoto" and "Method of transportation: Car." The output is the input information sent to the server.

[1423] Step 2:

[1424] The terminal converts the "destination" and "transportation method" information entered by the user into JSON format and sends it to the server via an HTTP POST request. The input is the "destination" and "transportation method" data from the user, and the output is the JSON-formatted request data sent to the server. Specifically, the terminal sends the data to an endpoint such as "https: / / example.com / api / route".

[1425] Step 3:

[1426] The server receives the HTTP request and extracts the "destination" and "transportation method" information from the JSON data. It then uses a map API (for example, Google Maps API) to calculate the optimal route from the starting point (the user's current location) to the destination "Kyoto." The input is the user's location information, destination, and transportation method, and the output is the optimal route information. Specifically, the server sends a request to the map API and receives route data in response.

[1427] Step 4:

[1428] The server analyzes the route information returned from the map API and obtains the optimal route. Based on this route information, the server executes an SQL query to search the information base for tourist attractions and restaurants near the route. The input is the route data obtained from the map API, and the output is a list of relevant tourist attractions and restaurants. Specifically, the server executes queries such as "SELECT FROM spots WHERE location NEAR route_coordinates".

[1429] Step 5:

[1430] The server calculates the additional time and cost of stopping at each detour point. This involves using the map API again to calculate the route changes and travel time required to stop at each spot. It also calculates any associated additional costs, such as admission fees and food and beverage costs, for each spot. The input is a list of tourist attractions and restaurants, and the output is the additional time and cost data for each detour point. Specifically, the server executes functions such as "Calculate End-to-End Route Time" and "Fetch Additional Costs."

[1431] Step 6:

[1432] The device captures the user's facial expressions and voice data using input devices such as the device's camera and microphone. The captured data is sent to the server in real time. The input is the user's facial expressions and voice data, and the output is digital data sent to the server. Specifically, the device executes functions such as "Start Camera Capture" and "Record Audio."

[1433] Step 7:

[1434] The server uses an emotion recognition engine to analyze the received facial expression and voice data. Based on the analysis results, the server determines the user's emotional state in real time. The input is the facial expression and voice data sent from the device, and the output is data on the user's emotional state. Specifically, the server executes algorithms such as "Analyze Facial Expression" and "Evaluate Audio Tone."

[1435] Step 8:

[1436] The server uses a machine learning algorithm to generate an optimal detour plan based on the user's past usage data and current emotional data. For example, if the user is relaxed, it will suggest a quiet and relaxing place, and if the user is excited, it will recommend an active tourist spot. The input is the user's past data and emotional data, and the output is the optimal detour plan. Specifically, the server executes processes such as "Generate Personalized Plan" and "Optimize Travel Itinerary."

[1437] Step 9:

[1438] The server sends the generated detour plan to the device. The input is the generated detour plan, and the output is the plan data to be sent to the device. Specifically, the server executes a command such as "Send Plan to Device."

[1439] Step 10:

[1440] The terminal displays the received detour plan on the application interface. The user can check the plan details (e.g., additional time and cost). The input is the plan data sent from the server, and the output is the interface display visually presented to the user. Specifically, the terminal executes functions such as "Display Plan on Screen" and "Show Plan Details."

[1441] (Application example 2)

[1442] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1443] Conventional route guidance systems have the ability to suggest optimal routes to a destination and detour spots, but they are unable to provide personalized suggestions that take into account the user's emotions and real-time situation. This limits the user experience and poses challenges in providing a more fulfilling travel experience. Furthermore, suggestions that do not take emotions into account are often inappropriate for the user's situation, potentially reducing satisfaction.

[1444] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input a destination and a method of travel, means for transmitting the input information to the server, means for calculating an optimal route based on the destination and method of travel, means for selecting an optimal detour plan taking into account the user's past usage data and current emotional state, and means for capturing and analyzing the user's facial expressions in real time using face recognition technology with a smartphone or a head-mounted display. This makes it possible to grasp the user's emotional state in real time and make personalized suggestions that are optimal for the situation.

[1445] A "user" is a person who uses the system by inputting a destination and a mode of travel.

[1446] A "destination" is a place that a user wishes to visit.

[1447] A "mode of travel" is the means of transportation a user chooses to reach a destination.

[1448] "Means" refers to a method or apparatus by which a system performs a particular function.

[1449] A "server" is a central processing unit that performs route calculations, database searches, sentiment analysis, and the like.

[1450] "Input information" refers to data that a user provides to the system regarding a destination and a method of travel.

[1451] The "optimal route" is the route that most effectively reaches the destination entered by the user.

[1452] "Tourist attractions" are places of interest or famous places that users can visit.

[1453] A "restaurant" is a store where users can purchase food and drinks.

[1454] A "database" is a system that organizes and stores information about tourist spots and restaurants.

[1455] A "side trip spot" is a place where a user can stop by on the way to their destination.

[1456] "Time" is the travel time added by stopping at a detour spot.

[1457] "Cost" is the additional monetary cost incurred by stopping at a detour spot.

[1458] "Usage data" refers to data and behavioral history of a user's past use.

[1459] "Emotional state" is a state that indicates the user's emotions in real time.

[1460] A "side trip plan" is a list of suggested side trip spots based on the optimal route.

[1461] A "smartphone" is a type of mobile phone and is a device that a user can use to access the system.

[1462] A "head-mounted display (HMD)" is a device worn by a user to visually obtain displayed information.

[1463] "Facial recognition technology" is a technology that captures and analyzes a user's facial expressions.

[1464] "Capture" is the act of acquiring an image or data.

[1465] "Analysis" is the act of analyzing the acquired data in detail.

[1466] "Real time" is a time frame in which processing occurs immediately without delay.

[1467] This invention is a system that suggests optimal routes and detour spots when a user inputs their destination and transportation method, and further personalizes the suggestions by recognizing the user's emotions. Specifically, this system is realized using the following hardware and software.

[1468] Hardware used:

[1469] 1. Smartphone

[1470] It is used by the user to input the destination and travel method and send it to the server.

[1471] It has a camera function and is used to capture the user's facial expressions.

[1472] 2. Head-Mounted Display (HMD)

[1473] This device allows users to visually confirm information and is used to capture the user's facial expressions in real time using facial recognition technology.

[1474] 3. Server

[1475] This is the central processing unit that handles major processes such as route calculation, database search, and emotion analysis.

[1476] Software used:

[1477] 1. Facial Recognition API

[1478] Examples: Amazon Rekognition, Google Cloud Vision API

[1479] 2. Map API

[1480] Example: Google Maps API

[1481] 3. Database Management System

[1482] Example: PostgreSQL

[1483] 4. Machine Learning Algorithms

[1484] Example: TensorFlow

[1485] Process flow:

[1486] Step 1: Obtain user data

[1487] The user inputs the destination and travel method using a smartphone or HMD, and this information is sent from the device to the server.

[1488] Step 2: Route calculation and spot search by server

[1489] The server uses a map API to calculate the optimal route from the departure point to the destination and generate route information, while simultaneously searching a database for tourist spots and restaurants along the calculated route.

[1490] Step 3: Emotion Recognition

[1491] The user's face is captured using a smartphone or HMD camera, and emotions are analyzed in real time using a facial recognition API.

[1492] Step 4: Generate optimal detour plans

[1493] The server uses a machine learning algorithm to select the optimal detour spots based on the user's past usage data and analyzed emotional data, and generates a detour plan.

[1494] Step 5: Present to the user

[1495] The generated detour plan is sent from the server to the device and visually displayed to the user via a smartphone or HMD, where the user can review the proposal and customize it as needed.

[1496] Examples:

[1497] For example, imagine a user traveling from Osaka to Kyoto in an autonomous vehicle. The user uses their smartphone to input their destination "Kyoto" and their mode of travel "car." The HMD then recognizes the user's face, and the emotion engine analyzes whether they are in a relaxed state. Based on this data, the server suggests detour spots where the user can relax, such as the Great Buddha of Nara or a famous local teahouse. The user can visually confirm these suggestions and either adopt them as an optimal travel plan or customize them.

[1498] Example prompt sentence:

[1499] Suggest the best travel route and detour spots using the following information: The user's destination is "Kyoto" and the mode of transportation is "car." The user's current emotional state is relaxed. List recommended detour spots.

[1500] This allows us to provide a personalized travel experience that utilizes the user's emotions and travel information.

[1501] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1502] Step 1: Obtain user data

[1503] The user inputs their destination and travel method using a smartphone or HMD. The input information (destination and travel method) is sent to the server via the device. This allows the server to receive basic instructions about the user's trip and obtain data for the next processing step.

[1504] Step 2: Route calculation

[1505] The server calculates the optimal route using a map API (e.g., Google Maps API) based on the input information it receives. Specifically, it receives the starting point and destination as input and outputs the optimal route taking into account the shortest distance and required time. This output includes the most efficient route information for the user.

[1506] Step 3: Search for spots

[1507] Based on the calculated optimal route, the server searches for tourist attractions and restaurants near the route using a database query. The database (e.g., PostgreSQL) stores information on various tourist attractions and restaurants, and the search query effectively extracts this information. The input is the optimal route, and the output is a list of detour spots.

[1508] Step 4: Calculate additional time and costs

[1509] The server calculates the additional time and cost of stopping at each listed detour spot. It uses the map API again, taking as input the route changes and travel time required for each stop, and outputs the additional time and cost involved.

[1510] Step 5: Emotion Recognition

[1511] The user's facial expressions are captured by a smartphone or HMD camera, and emotions are analyzed in real time using a facial recognition API (e.g., Amazon Rekognition). The input is the captured facial expression data, and the output is the user's emotional state (e.g., relaxed, excited, etc.). This allows the server to understand the user's current emotional state.

[1512] Step 6: Choose a detour plan

[1513] The server uses a machine learning algorithm (e.g., TensorFlow) to select optimal detour spots based on the user's past usage data and current emotional data. The input is past usage data and the user's current emotional state, and the output is an optimal list of detour spots. The machine learning algorithm analyzes the past data and emotional data to suggest the most suitable spots for the user.

[1514] Step 7: Generate and submit your detour plan

[1515] The server sends the generated detour plan to the terminal. The input is a list of selected detour spots, and the output is the detour plan displayed on the user's terminal, allowing the user to review the final plan and customize it as needed.

[1516] Step 8: Present to the user

[1517] The device then displays the received detour plan on the user interface of the smartphone or HMD. Specifically, the interface allows the user to visually check detailed information about the spots, the additional time and cost, etc. This allows the user to understand the proposed detour plan and achieve the optimal travel experience.

[1518] Through these steps, the system can provide a personalized travel experience that utilizes the user's emotions and travel information.

[1519] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1520] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1521] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1522] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1523] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1524] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1525] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1526] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1527] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1528] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1529] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1530] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1531] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1532] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1533] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1534] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1535] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1536] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1537] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1538] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1539] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1540] The following is further disclosed regarding the above embodiment.

[1541] (Claim 1)

[1542] a means for a user to input a destination and a mode of travel;

[1543] means for transmitting the input information to a server;

[1544] means for calculating an optimal route based on the destination and the mode of travel;

[1545] A means for searching a database for tourist spots and restaurants around the route;

[1546] A way to calculate the additional time and cost of stopping at each detour spot,

[1547] A means for selecting an optimal detour plan taking into consideration the user's past usage data;

[1548] means for presenting the detour plan to a user;

[1549] A system including:

[1550] (Claim 2)

[1551] 2. The system according to claim 1, wherein a server has means for generating the detour plan and transmitting it to the terminal.

[1552] (Claim 3)

[1553] 2. The system of claim 1, wherein the terminal comprises means for displaying the generated detour plan on a user interface.

[1554] "Example 1"

[1555] (Claim 1)

[1556] a means for a user to input a destination and a mode of travel;

[1557] means for transmitting the input information to a server by the terminal;

[1558] means for calculating an optimal route based on the destination and the mode of travel;

[1559] A means for searching a database for tourist spots and restaurants around the route;

[1560] A way to calculate the additional time and cost of stopping at each detour spot,

[1561] A means for selecting an optimal detour plan taking into consideration the user's past usage data and preference data;

[1562] means for visually displaying the detour plan on a user interface;

[1563] A system including:

[1564] (Claim 2)

[1565] 10. The system of claim 1, wherein the server comprises means for using a map API to calculate the optimal route.

[1566] (Claim 3)

[1567] 10. The system of claim 1, wherein the server comprises means for analyzing the user preference data using machine learning algorithms.

[1568] "Application Example 1"

[1569] (Claim 1)

[1570] a means for a user to input a destination and a mode of travel;

[1571] means for transmitting the input information to a server;

[1572] means for calculating an optimal route based on the destination and the mode of travel;

[1573] means for searching a database for locations around the route;

[1574] A means to calculate the additional time and cost of stopping at each detour location;

[1575] A means for selecting an optimal detour plan taking into consideration the user's past usage data;

[1576] means for displaying the detour plan on a display of the user's means of transportation and allowing the user to approve or customize the plan;

[1577] means for visually presenting detailed information about the detour plan on a display of the means of transportation;

[1578] A system including:

[1579] (Claim 2)

[1580] 2. The system according to claim 1, wherein a server has means for generating the detour plan and transmitting it to the terminal.

[1581] (Claim 3)

[1582] 2. The system of claim 1, wherein the terminal comprises means for displaying the generated detour plan on a user interface.

[1583] "Example 2: Combining Emotion Engines"

[1584] (Claim 1)

[1585] a means for a user to input a destination and a mode of travel;

[1586] means for transmitting the input information to a terminal;

[1587] means for calculating an optimal route based on the destination and the mode of travel;

[1588] A means for searching an information base for tourist attractions and restaurants around the route;

[1589] A means to calculate the additional time and cost of stopping at each detour point;

[1590] A means for recognizing the user's emotions by analyzing facial expressions and voice data;

[1591] A means for selecting an optimal detour plan taking into consideration the user's past usage data and emotional data;

[1592] means for presenting the detour plan to a user;

[1593] A system including:

[1594] (Claim 2)

[1595] 2. The system according to claim 1, wherein a server has means for generating the detour plan and transmitting it to the terminal.

[1596] (Claim 3)

[1597] 2. The system of claim 1, wherein the terminal comprises means for displaying the generated detour plan on a user interface.

[1598] "Application example 2 when combining emotion engines"

[1599] (Claim 1)

[1600] a means for a user to input a destination and a mode of travel;

[1601] means for transmitting the input information to a server;

[1602] means for calculating an optimal route based on the destination and the mode of travel;

[1603] A means for searching a database for tourist spots and restaurants around the route;

[1604] A way to calculate the additional time and cost of stopping at each detour spot,

[1605] A means for selecting an optimal detour plan taking into account the user's past usage data and current emotional state;

[1606] means for presenting the detour plan to a user;

[1607] A means for capturing and analyzing a user's facial expressions in real time using a smartphone or a head-mounted display with facial recognition technology;

[1608] A system including:

[1609] (Claim 2)

[1610] 2. The system according to claim 1, wherein a server has means for generating the detour plan and transmitting it to the terminal.

[1611] (Claim 3)

[1612] 2. The system of claim 1, wherein the terminal comprises means for displaying the generated detour plan on a user interface. [Explanation of symbols]

[1613] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for a user to input a destination and a mode of travel; means for transmitting the input information to a server; means for calculating an optimal route based on the destination and the mode of travel; A means for searching a database for tourist spots and restaurants around the route; A way to calculate the additional time and cost of stopping at each detour spot, A means for selecting an optimal detour plan taking into consideration the user's past usage data; means for presenting the detour plan to a user; A system including:

2. 2. The system according to claim 1, wherein the server has means for generating the detour plan and transmitting it to the terminal.

3. 2. The system of claim 1, wherein the terminal comprises means for displaying the generated detour plan on a user interface.

Images