System

Abstract

A system is provided.SOLUTION: A system includes a user input means, a generation means for processing a request for a travel plan received from a user and generating an optimal travel candidate site or plan, a display means for displaying the generated candidate site or plan to the user, and a guidance means for performing real-time guidance based on the plan selected by the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When planning a trip, users need to gather a lot of information about places to visit, food, tourist spots, and other information to determine the optimal travel plan. However, especially when traveling abroad, language and cultural differences make it difficult to gather information, and users often feel anxious without real-time guidance while traveling. This can make the user's travel experience stressful and can detract from the enjoyment of the trip. Therefore, there is a need for a system that allows users to plan trips efficiently and stress-free, and to enjoy their trip with peace of mind while they are planning. [Means for solving the problem]

[0005] The present invention provides a generation means that processes travel planning requests received from users and generates optimal candidate travel destinations and plans. The user inputs information such as places to visit, desired foods, genres of interest, and budget into an app. The generation means then generates optimal candidate destinations and plans based on this information. The present invention also provides a guidance means that displays the generated candidate destinations and plans to the user and provides real-time guidance based on the plan selected by the user. The guidance means works in conjunction with a map application to display routes to the user and further provides multilingual voice guidance using a device with voice guidance, thereby providing a system that allows users to travel with peace of mind.

[0006] "User input means" refers to an interface through which a user inputs their travel plan requests and receives information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[0007] The "generation means" is a means having the function of processing travel plan requests received from users and generating optimal travel destinations and plans based on that information.

[0008] The "display means" is a means for visually presenting the potential travel destinations and plans generated by the generation means to the user.

[0009] "Guidance means" refers to a means for providing real-time guidance based on the travel plan selected by the user, and includes functions such as displaying routes in conjunction with map applications and providing multilingual audio guidance using a device with audio guidance.

[0010] A "map application" is a software application that provides navigation between a user's current location and a destination.

[0011] A "device with voice guidance" is a device that is worn by a user and provides real-time voice guidance, and has the function of providing voice guidance in multiple languages.

[0012] "Optimal travel destinations and plans" refers to travel destinations and activities that best suit the user's preferences and conditions, as suggested by the generation means based on the information entered by the user.

[0013] "Real-time guidance" refers to instantaneous guidance information provided to users based on their current location and selected itinerary while they are traveling. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The present invention is a system that generates optimal travel destinations and plans based on travel planning requests and provides real-time guidance on the day of travel. Specific embodiments of this system are described below.

[0036] System configuration

[0037] The system mainly consists of the following components:

[0038] 1. User Input Method

[0039] 2. Generation means

[0040] 3. Display means

[0041] 4. Guidance

[0042] User Input Method

[0043] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[0044] generation means

[0045] The generation means is located on a server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[0046] Display means

[0047] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[0048] Guidance means

[0049] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with a map application and has the function of displaying routes to the user. It also provides multilingual audio guidance using a device with audio guidance. This device is expected to be a glasses-type device worn by the user.

[0050] Program processing

[0051] User information entry and submission

[0052] 1. The user launches the application and enters their travel plan requirements, such as "Tokyo" as the area they want to visit, "sushi" as the food they want to enjoy, "historical places" as their interest, and "5,000 yen per day" as their budget.

[0053] 2. The terminal sends the entered information to the server.

[0054] Generative AI generates candidate sites and plans

[0055] 1. Based on the received information, the server uses generative AI to generate optimal travel destinations and plans. For example, a plan might be generated that includes breakfast at Tsukiji Market, a visit to Sensoji Temple, lunch at a sushi restaurant in Ginza, a tour of the Imperial Palace, and dinner at a pub in Shinjuku.

[0056] Display of candidate sites and plans

[0057] 1. The server sends the generated plan to the terminal.

[0058] 2. The device will display the plan details to the user, who can then review each plan and select the one they like best.

[0059] Real-time guidance

[0060] 1. On the day of travel, the user launches the app.

[0061] 2. The device loads the final plan saved on the server and displays the route in conjunction with the map application.

[0062] 3. The terminal works with a device that supports voice guidance to provide voice guidance, such as "Turn left next and go straight to Sensoji Temple."

[0063] Specific examples

[0064] User information entry and submission

[0065] The user starts the application and enters the following information:

[0066] Area I want to visit: Tokyo

[0067] Food I want to enjoy: Sushi

[0068] Interests: Historical places

[0069] Budget: 5,000 yen per day

[0070] The terminal sends this information to the server.

[0071] Generative AI generates candidate sites and plans

[0072] The server uses the above information to generate a plan like this:

[0073] Breakfast: Enjoy sushi at Tsukiji Market

[0074] Morning: Visit to Sensoji Temple

[0075] Lunch: Eat at a famous sushi restaurant in Ginza

[0076] Afternoon: Visit the Imperial Palace

[0077] Dinner: Savor local cuisine at a Shinjuku izakaya

[0078] Display of candidate sites and plans

[0079] The server sends the generated plan to the terminal, which displays it to the user, who then checks and selects the plan.

[0080] Real-time guidance

[0081] On the day of the trip, the user launches the app. The device loads the final plan stored on the server and displays the route in conjunction with the map application. Additionally, the user wears a voice guidance device that provides real-time, multilingual guidance.

[0082] The present invention allows users to plan their trip efficiently and stress-free, and to act with peace of mind while traveling.

[0083] The processing flow will be explained below.

[0084] Step 1:

[0085] Users launch the travel app and enter information such as the area they want to visit, the food they want to enjoy, the genres they are interested in, their budget, etc. Once they have finished entering the information, they tap the "Submit" button.

[0086] Step 2:

[0087] The device sends the input information, including the place to visit, type of food, genre of interest, budget, etc., to the server.

[0088] Step 3:

[0089] Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best suit the user's preferences and conditions.

[0090] Step 4:

[0091] The server then sends the generated candidate locations and plans to the device, including information on multiple tourist spots, places to eat, and activities.

[0092] Step 5:

[0093] The device will display the details of the plans received to the user, who can then review the plans and select the one they like. At this time, they can also view detailed information, photos, and reviews for each plan.

[0094] Step 6:

[0095] The user selects the desired plan and taps the "Confirm" button. Once the selection is confirmed, the device sends the final plan to the server.

[0096] Step 7:

[0097] The server saves the selected plan as the final plan, which includes tourist spots to visit, places to eat, and routes to travel.

[0098] Step 8:

[0099] On the day of the trip, the user launches the app again and confirms their plan. The device loads the saved final plan and works with the map application to display their current location and route.

[0100] Step 9:

[0101] The user starts moving along the route specified by the user, and the device updates the map information in real time and navigates to the next destination.

[0102] Step 10:

[0103] The user wears the device with voice guidance. The device prepares the voice guidance information and starts guiding based on the user's current location and final plan. For example, instructions such as "Turn left next and go straight to Sensoji Temple" are provided.

[0104] Step 11:

[0105] The device will display additional information and guides as needed for each tourist attraction or dining spot during the trip, and its multilingual capabilities will provide guidance according to the user's language settings.

[0106] Step 12:

[0107] The user completes their travel plan. The device sends the travel history to the server, where it stores the data for later access.

[0108] By following these steps, users can plan their trip efficiently and stress-free, and enjoy their trip with peace of mind, receiving detailed real-time guidance during their trip.

[0109] Example 1

[0110] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0111] Conventional travel planning systems have difficulty generating optimal travel plans based on user requests. They also lack efficient means for providing real-time guidance. As a result, users spend a great deal of time and effort planning and are unable to receive sufficient guidance during their trip.

[0112] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0113] In this invention, the server includes a generation unit that processes travel plan requests received from users and generates optimal travel destinations and plans, a generation unit that references an existing database and inputs prompts into a generative AI model to generate a travel plan, and a unit that displays the generated travel destinations and plans to the user, thereby enabling users to efficiently and accurately plan their travels and receive detailed guidance in real time.

[0114] "User input means" refers to an interface through which a user inputs their travel planning requirements.

[0115] The "generation means" is a means for processing a travel plan request received from a user and generating optimal travel destinations and plans.

[0116] "Means of referencing existing databases" refers to means of obtaining data that meets the user's needs by using a database that holds travel-related information.

[0117] "Means for inputting prompt sentences into a generative AI model" refers to a means for inputting prompt sentences based on the user's requests into a generative AI model and generating an optimal travel plan.

[0118] The "display means" is a means for visually presenting the generated travel destinations and plans to the user.

[0119] The "guidance means" is a means for providing real-time guidance based on the plan selected by the user.

[0120] The "means for linking with a map application" refers to a means for communicating with a map application and displaying information in order to display a travel route to the user.

[0121] A "device with voice guidance" is a device that provides voice guidance and is capable of providing multilingual voice guidance in real time.

[0122] The present invention is a system that generates optimal travel destinations and plans based on travel planning requests and provides real-time guidance on the day of travel. Specific embodiments of this system are described below.

[0123] System configuration

[0124] The system mainly consists of the following components:

[0125] 1. User Input Method

[0126] 2. Generation means

[0127] 3. Display means

[0128] 4. Guidance

[0129] User Input Method

[0130] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, their interests, and their budget. For example, a user inputs detailed information such as "areas they want to visit: Kyoto," "foods they want to enjoy: Japanese sweets," "interests: gardens," and "budget: 7,000 yen per day."

[0131] generation means

[0132] The generation means is placed on the server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to an existing database (for example, the API of a travel information service) and proposes a plan that best suits the user's preferences and conditions. The server inputs a prompt statement into the generation AI model to generate a travel plan. For example, the prompt statement could be, "The user has inputted 'Kyoto' as the area they would like to visit, 'Japanese sweets' as the food they would like to enjoy, 'gardens' as the genre they are interested in, and '7,000 yen per day' as their budget. Please generate the optimal travel plan based on this information."

[0133] Display means

[0134] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[0135] Guidance means

[0136] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with map applications (such as Google Maps or Apple Maps) and has the function of displaying routes to the user. It also provides audio guidance using a device with audio guidance (such as a multilingual audio guidance device). This device is expected to be a glasses-type device worn by the user. For example, it provides specific audio guidance such as, "Turn left next and go straight to Sensoji Temple."

[0137] In this way, users can plan their trips efficiently and optimally, and receive detailed real-time guidance while traveling.

[0138] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0139] Step 1:

[0140] Users launch the mobile app and input their travel plan requirements, including the areas they want to visit, the foods they want to enjoy, their interests, and their budget. This information is then sent to the device as input data.

[0141] Step 2:

[0142] The device sends the collected user input data to the server. The data is kept safe using a secure communication protocol (e.g., HTTPS). For example, the input data might be "Area I want to visit: Kyoto," "Food I want to enjoy: Japanese sweets," "Interested genre: gardens," or "Budget: 7,000 yen per day."

[0143] Step 3:

[0144] The server processes the received input data. Based on this data, it searches for related travel spots and information from an existing database. Next, it inputs a prompt statement into the generative AI model. An example of a prompt statement is: "The user has entered 'Kyoto' as the area they would like to visit, 'Japanese sweets' as the food they would like to enjoy, 'gardens' as a genre they are interested in, and '7,000 yen per day' as their budget. Please generate the optimal travel plan based on this information."

[0145] Step 4:

[0146] The generative AI model calculates data based on the prompt text and generates an optimal travel plan. Examples of generated plans include "Enjoy Japanese sweets at a sweet shop," "Visit Kinkakuji Temple," "Have lunch in Gion," "Tour of Otawara Pond Garden," and "Stroll along the Kamo River." This plan becomes the output data.

[0147] Step 5:

[0148] The server formats the generated travel plan and sends it to the device. The output data is formatted in JSON or XML format and provided to the device.

[0149] Step 6:

[0150] The device analyzes the received travel plans and visually displays them to the user, including detailed time schedules, maps, and user reviews for each plan. The user can then review these and select the plan they want.

[0151] Step 7:

[0152] The user selects and confirms the travel plan they like, and the device notifies the server of the selected plan as the final plan.

[0153] Step 8:

[0154] On the day of the trip, the user launches the app. The device loads the final plan saved on the server and uses GPS to identify the user's current location. It then connects to a map application (e.g., Google Maps) to display the route and begin guiding the trip.

[0155] Step 9:

[0156] The terminal works in conjunction with a voice guidance device to provide real-time voice guidance, such as "Turn left next and go straight to Sensoji Temple," allowing users to carry out efficient and optimal travel planning.

[0157] (Application example 1)

[0158] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0159] Conventional travel planning systems have had many problems with planning trips and providing real-time guidance during the trip. It is difficult to easily create an optimal plan tailored to the user's needs, and guidance is particularly inadequate in self-driving vehicles. Furthermore, there is a lack of systems that provide real-time navigation and guidance on-site, leading to users often getting lost. Furthermore, voice guidance is often not multilingual, which is extremely inconvenient for foreign tourists. The present invention aims to solve these problems.

[0160] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0161] In this invention, the server includes an input means for receiving a travel plan request from a user, a means for processing the travel plan request received from the input means and generating optimal travel destinations and plans using a generative AI model, a means for displaying the generated travel destinations and plans to the user, and a means for providing real-time guidance based on the plan selected by the user. This allows the user to enjoy a comfortable trip in an autonomous vehicle and receive real-time guidance on routes and tourist spots. Furthermore, multilingual audio guidance allows users to efficiently enjoy their trip beyond language barriers.

[0162] "Travel planning requests" are information including the user's desired travel destinations, activities of interest, and budget.

[0163] The "input means" is an interface for receiving travel planning requests from users, and is an application implemented on a smartphone or in-vehicle infotainment system.

[0164] A "generative AI model" is an artificial intelligence technology that automatically generates optimal travel destinations and plans based on travel planning requests received from users.

[0165] The "means for generating" is a processing device that uses a generative AI model to create optimal travel destinations and plans based on the user's requests.

[0166] The "display means" refers to a device for visually presenting the generated travel destinations and plans to the user, such as an in-car infotainment system or a smartphone display.

[0167] The "guidance means" is a device that provides real-time route and destination guidance based on a plan selected by the user.

[0168] A "navigation system" is a system that uses GPS data and map applications to display the route from the user's current location to their destination in real time.

[0169] A "voice guidance system" is a system that uses multilingual voice synthesis technology to provide users with real-time voice guidance on travel and tourist spots.

[0170] System configuration

[0171] The system of the present invention comprises the following components:

[0172] 1. Input Method

[0173] The input means of this system is an interface for receiving travel planning requests from users. It is provided as an application installed on smartphones or in-car infotainment systems. Users input information such as the areas they want to visit, activities they want to enjoy, places of interest, and their budget.

[0174] 2. Means of generation

[0175] The generation means runs on the server. Based on requests sent from the input means, the generative AI model generates optimal travel destinations and plans. The generative AI model refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[0176] 3. Display method

[0177] The display means is used to visually present the generated travel destinations and plans to the user. It runs on the same device as the input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[0178] 4. Guidance

[0179] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with the navigation system and has the function of displaying routes to the user. It also provides real-time voice guidance using a multilingual voice guidance system. The voice guidance system effectively provides guidance through a device worn by the user (for example, a speaker built into the in-vehicle infotainment system).

[0180] Specific explanation of the system's operation

[0181] The operation of this system will be explained in detail below, showing how each element of the system works together.

[0182] 1. User information entry and submission

[0183] The user launches the application and inputs their travel plan requirements, for example, selecting "Kyoto" as the area they want to visit, "sightseeing" as the activity they want to enjoy, "historical places" as their points of interest, and "7,000 yen per day" as their budget. This information is then sent to the server by the device.

[0184] 2. Generating candidate sites and plans using a generative AI model

[0185] Based on the received information, the server uses a generative AI model to generate optimal travel destinations and plans. For example, a plan including tourist spots such as Kiyomizu-dera Temple, Kinkaku-ji Temple, and Nijo Castle may be generated.

[0186] 3. Display of candidate sites and plans

[0187] The server sends the generated plans to the device, which displays the plan details to the user, allowing the user to review each plan and select the one they like best.

[0188] 4. Real-time guidance

[0189] On the day of the trip, the user launches the app. The device loads the final plan saved on the server and displays the route in conjunction with the navigation system. It also links with the voice guidance system to provide real-time voice guidance in multiple languages. For example, the app may provide voice guidance such as, "Turn left next and go straight to Kiyomizu-dera Temple."

[0190] Specific examples

[0191] User information entry and submission

[0192] The user starts the application and enters the following information:

[0193] Area I want to visit: Kyoto

[0194] Activities to enjoy: Sightseeing

[0195] Places of interest: Historical places

[0196] Budget: 7,000 yen per day

[0197] The terminal sends this information to the server.

[0198] Generative AI model generates candidate sites and plans

[0199] The server uses the above information to generate a plan like this:

[0200] Touring sightseeing spots: Kiyomizu-dera Temple, Kinkaku-ji Temple, Nijo Castle

[0201] Lunch: Restaurant to enjoy local cuisine

[0202] Guide:Transportation to accommodation

[0203] Real-time guidance

[0204] On the day of the trip, the user launches the app in the autonomous vehicle. The device loads the final plan stored on the server and displays the route in conjunction with the navigation system. In addition, the voice guidance system provides real-time, multilingual guidance.

[0205] Prompt Sentence Examples

[0206] "Area I want to visit: Kyoto, Activities I'm interested in: Sightseeing, historical places, Budget: 7,000 yen per day"

[0207] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0208] Step 1:

[0209] Enter the user's travel planning requests

[0210] The user launches the application and inputs specific travel planning requirements (areas to visit, activities of interest, historical locations, budget, etc.) The input data is stored on the device via the smartphone or in-car infotainment system application.

[0211] input:

[0212] Area I want to visit: Kyoto

[0213] Activities to enjoy: Sightseeing

[0214] Places of interest: Historical places

[0215] Budget: 7,000 yen per day

[0216] output:

[0217] User travel planning information data

[0218] Step 2:

[0219] Send the user's travel plan information data to the server

[0220] The terminal transmits the user's travel plan information data entered in step 1 to the server. The data is transmitted using a secure communication channel.

[0221] input:

[0222] User travel planning information data

[0223] output:

[0224] User travel plan information data stored on the server

[0225] Step 3:

[0226] Generate optimal travel destinations and plans using generative AI models

[0227] The server processes the received user travel plan information data and uses a generative AI model to generate optimal travel destinations and plans. The AI ​​model references existing travel databases and reviews to create the optimal plan based on the user's requests.

[0228] input:

[0229] User travel plan information data stored on the server

[0230] output:

[0231] Generated travel destination and plan data

[0232] Step 4:

[0233] Displaying generated travel destinations and plans to users

[0234] The server sends the generated travel destination and plan data to the device, which receives it and visually displays it to the user. The user can then check the various options on the display screen and select the plan that best suits their needs.

[0235] input:

[0236] Generated travel destination and plan data

[0237] output:

[0238] A list of plan options displayed to the user

[0239] Step 5:

[0240] User selects final plan

[0241] The user selects the plan they like from the displayed list of plan candidates, and the selected plan is sent to the server by the terminal.

[0242] input:

[0243] User-selected plan data

[0244] output:

[0245] Final plan data sent to the server

[0246] Step 6:

[0247] Real-time information on the day of travel

[0248] On the day of the trip, the user launches the app, which loads the final plan stored on the server, displays the route in conjunction with the navigation system, and provides real-time voice guidance in multiple languages ​​in conjunction with the voice guidance system.

[0249] input:

[0250] Final plan data stored on the server

[0251] output:

[0252] Navigation system showing route

[0253] Real-time multilingual voice guidance

[0254] Supplementary information on specific actions

[0255] In step 1, the user enters the necessary information via a smartphone or in-car infotainment system application. In step 2, this information is sent to the server via the internet. In step 3, the server uses this data to generate prompts for the generative AI model and builds an optimal travel plan. In steps 4 and 5, the server sends the plan it has generated to the device, where the user can confirm and select. Step 6 provides real-time guidance on the day of the trip, with navigation and voice guidance based on the user's location information.

[0256] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0257] The present invention is a system that generates optimal travel destinations and plans based on travel planning requests and provides real-time guidance on the day of travel. This system incorporates an emotion engine that recognizes the user's emotions and provides feedback accordingly. Specific embodiments of this system are described below.

[0258] System configuration

[0259] The system mainly consists of the following components:

[0260] 1. User Input Method

[0261] 2. Generation means

[0262] 3. Display means

[0263] 4. Guidance

[0264] 5. Emotion Engine

[0265] User Input Method

[0266] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[0267] generation means

[0268] The generation means is located on a server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[0269] Display means

[0270] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[0271] Guidance means

[0272] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with a map application and has the function of displaying routes to the user. It also provides multilingual audio guidance using a device with audio guidance. This device is expected to be a glasses-type device worn by the user.

[0273] Emotion Engine

[0274] The emotion engine recognizes the user's emotions and adjusts the content of the plan and guidance accordingly. It works in conjunction with the user input means, generation means, display means, and guidance means, and optimizes the system's operation based on the user's emotional data.

[0275] Program processing

[0276] User information entry and submission

[0277] 1. The user launches the application and enters information such as the area they want to visit, the food they want to enjoy, their interests, and their budget. For example, the area they want to visit might be "Tokyo," the food they want to enjoy might be "sushi," their interests might be "historical places," and their budget might be "5,000 yen per day."

[0278] 2. The terminal sends the entered information to the server.

[0279] Generative AI generates candidate sites and plans

[0280] 1. Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best fit the user's preferences and conditions. For example, a plan might be generated that includes breakfast at Tsukiji Market, a visit to Sensoji Temple, lunch at a sushi restaurant in Ginza, a tour of the Imperial Palace, and dinner at a izakaya in Shinjuku.

[0281] Display of candidate sites and plans

[0282] 1. The server sends the generated plan to the terminal.

[0283] 2. The device displays the plan details to the user, who then reviews the plans and selects the one they like.

[0284] Real-time guidance

[0285] 1. On the day of travel, the user launches the app.

[0286] 2. The device loads the final plan saved on the server and works with the map application to display the current location and route.

[0287] 3. The user begins traveling along the specified route. The device updates the map information in real time and navigates to the next destination.

[0288] Emotion Engine Operation

[0289] 1. The user wears a device with voice guidance.

[0290] 2. The emotion engine recognizes the user's emotions and adjusts the tone and content of the guidance accordingly. For example, if the user is tired, the guidance will be gentler and suggest places to rest.

[0291] 3. The device prepares voice guidance information and starts guiding based on the user's current location and final plan. For example, instructions such as "Turn left next and go straight to Sensoji Temple" are provided.

[0292] Emotional Data Feedback

[0293] 1. The device collects the user's emotional data during the trip and sends it to the server.

[0294] 2. The server analyzes the collected emotional data, and the generative AI uses it to generate future plans.

[0295] Specific examples

[0296] User information entry and submission

[0297] The user starts the application and enters the following information:

[0298] Area I want to visit: Tokyo

[0299] Food I want to enjoy: Sushi

[0300] Interests: Historical places

[0301] Budget: 5,000 yen per day

[0302] The terminal sends this information to the server.

[0303] Generative AI generates candidate sites and plans

[0304] The server uses the above information to generate a plan like this:

[0305] Breakfast: Enjoy sushi at Tsukiji Market

[0306] Morning: Visit to Sensoji Temple

[0307] Lunch: Eat at a famous sushi restaurant in Ginza

[0308] Afternoon: Visit the Imperial Palace

[0309] Dinner: Savor local cuisine at a Shinjuku izakaya

[0310] Display of candidate sites and plans

[0311] The server sends the generated plan to the terminal, which displays it to the user, who then checks and selects the plan.

[0312] Real-time guidance

[0313] On the day of the trip, the user launches the app. The device loads the final plan stored on the server and displays the route in conjunction with the map application. Additionally, the user wears a voice guidance device that provides real-time, multilingual guidance.

[0314] Emotion Engine Operation

[0315] During the trip, the emotion engine will recognize the user's emotions and provide feedback accordingly. For example, if it detects that the user is tired, it will provide guidance such as "You can take a break here and refresh yourself at the cafe."

[0316] The present invention allows users to plan their trip efficiently and stress-free, and to enjoy their trip with peace of mind while receiving detailed, real-time guidance that responds to their emotions during their trip.

[0317] The processing flow will be explained below.

[0318] Step 1:

[0319] Users launch a travel app and enter information such as the area they want to visit, the food they want to enjoy, their interests, and their budget. For example, they might choose "Tokyo" as the area they want to visit, "sushi" as the food they want to enjoy, "historical places" as their interests, and "5,000 yen per day" as their budget.

[0320] Step 2:

[0321] The device sends the input information, including the place to visit, type of food, genre of interest, budget, etc., to the server.

[0322] Step 3:

[0323] Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best suit the user's preferences and conditions.

[0324] Step 4:

[0325] The server then sends the generated candidate locations and plans to the device, including information on multiple tourist spots, places to eat, and activities.

[0326] Step 5:

[0327] The device will display the details of the plans received to the user, who can then review the plans and select the one they like. At this time, they can also view detailed information, photos, and reviews for each plan.

[0328] Step 6:

[0329] The user selects the desired plan and taps the "Confirm" button. Once the selection is confirmed, the device sends the final plan to the server.

[0330] Step 7:

[0331] The server saves the selected plan as the final plan, which includes tourist spots to visit, places to eat, and routes to travel.

[0332] Step 8:

[0333] On the day of the trip, the user launches the app again and confirms their plan. The device loads the saved final plan and works with the map application to display their current location and route.

[0334] Step 9:

[0335] The user starts moving along the route specified by the user, and the device updates the map information in real time and navigates to the next destination.

[0336] Step 10:

[0337] The user wears a device with voice guidance, such as a pair of eyeglasses. The device prepares voice guidance information and begins guiding based on the user's current location and final plan. For example, instructions such as "Turn left next and go straight to Sensoji Temple" are provided.

[0338] Step 11:

[0339] The emotion engine uses a camera or microphone to recognize the user's emotions, for example, by analyzing facial expressions and tone of voice.

[0340] Step 12:

[0341] The emotion engine transmits the recognized emotion data to the device, such as whether the user is tired, happy, or anxious.

[0342] Step 13:

[0343] The device adjusts the tone and content of the guidance based on the emotional data it receives. For example, if it detects that the user is tired, it will soften the tone of the guidance and make suggestions such as, "You can take a break here and refresh yourself at the cafe."

[0344] Step 14:

[0345] The device will display additional information and guides as needed for each tourist attraction or dining spot during the trip, and its multilingual capabilities will provide guidance according to the user's language settings.

[0346] Step 15:

[0347] Once the user completes their travel plan, the device sends their travel history and emotional data to the server, where it is stored so that the user can access it again later.

[0348] Through these steps, users can plan their trip efficiently and stress-free, and enjoy their trip with peace of mind, receiving detailed, real-time, and emotionally-sensitive guidance during their trip.

[0349] Example 2

[0350] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0351] Conventional travel planning systems have difficulty reflecting a user's individual preferences and emotions in real time. As a result, they are unable to respond to changes in the user's situation or mood during the trip, resulting in a lack of flexibility in the plan. Furthermore, real-time guidance is limited to general map applications and voice guidance, making it difficult to meet the diverse needs of users.

[0352] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a user input means, a generation means for processing a travel plan request received from the user and generating optimal candidate travel destinations and plans, a display means for displaying the generated candidate destinations and plans to the user, a guidance means for providing real-time guidance based on the plan selected by the user, and an emotion engine for recognizing the user's emotions and adjusting the content of the plan and guidance accordingly. This makes it possible to provide a travel plan that reflects the user's individual preferences and emotions in real time and to provide flexible real-time guidance.

[0353] "User input means" refers to an interface through which a user inputs their travel planning requirements.

[0354] The "generation means" is a means for generating optimal travel destinations and plans based on travel plan requests received from users.

[0355] The "display means" is a means for visually presenting the generated travel destinations and plans to the user.

[0356] The "guidance means" is a means for providing real-time guidance based on the plan selected by the user.

[0357] The "emotion engine" is an engine that recognizes the user's emotions and adjusts the content of the plan or guidance accordingly.

[0358] "Map Application" means software used to display travel routes to a user.

[0359] A "voice-guided device" is a device used to provide real-time voice guidance.

[0360] This invention relates to a system that generates optimal travel destinations and plans based on a user's travel planning requests and provides real-time guidance on the day of the trip. This system incorporates an emotion engine that recognizes the user's emotions and provides feedback accordingly. Specific embodiments of this system are described below.

[0361] System configuration

[0362] The system mainly consists of the following components:

[0363] 1. User Input Method

[0364] 2. Generation means

[0365] 3. Display means

[0366] 4. Guidance

[0367] 5. Emotion Engine

[0368] User Input Method

[0369] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[0370] Specifically, the user launches the application and enters the following information:

[0371] Area I want to visit: Tokyo

[0372] Food I want to enjoy: Sushi

[0373] Interests: Historical places

[0374] Budget: 5,000 yen per day

[0375] generation means

[0376] The generation means is located on the server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions. For example, the following specific plans may be generated:

[0377] Breakfast: Enjoy sushi at Tsukiji Market

[0378] Morning: Visit to Sensoji Temple

[0379] Lunch: Eat at a famous sushi restaurant in Ginza

[0380] Afternoon: Visit the Imperial Palace

[0381] Dinner: Savor local cuisine at a Shinjuku izakaya

[0382] Display means

[0383] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[0384] Guidance means

[0385] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with a map application and has the function of displaying routes to the user. It also provides multilingual audio guidance using a device with audio guidance. This device is expected to be a glasses-type device worn by the user.

[0386] For example, on the day of a trip, when a user launches the app, the device loads the final plan saved on the server and displays the route in conjunction with the map application. As the user begins to move, the device updates the map information in real time and navigates to the next destination.

[0387] Emotion Engine

[0388] The emotion engine recognizes the user's emotions and adjusts the tone and content of the guidance based on those emotions. It works in conjunction with the user input means, generation means, display means, and guidance means, optimizing the system's operation based on the user's emotional data.

[0389] Specifically, when a user wears a device with voice guidance, the emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. For example, if the user is tired, the engine will provide guidance such as, "You can take a break here and refresh yourself at the cafe." In this way, it is possible to provide appropriate feedback according to the user's emotions.

[0390] This system allows users to plan their trip efficiently and stress-free, and they can enjoy their trip with peace of mind while receiving detailed, real-time guidance that responds to their emotions. Furthermore, the collection and analysis of emotion data will enable the system to provide even more accurate travel plans.

[0391] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0392] Step 1: Enter your user information

[0393] Users launch the application and enter information such as the area they want to visit, the food they want to enjoy, their interests, and their budget. For example, they can enter information such as "Tokyo," "sushi," "historical place," and "5,000 yen per day." The entered data is temporarily stored on the device.

[0394] Input: User's travel plan request (e.g. Tokyo, sushi, historical places, budget 5000 yen)

[0395] Output: Input data saved on the device

[0396] Step 2: Submit user information

[0397] The device sends the entered information to the server, where each item is converted into an appropriate format and a data format such as JSON is generated.

[0398] Input: Input data stored on the device

[0399] Output: User's travel plan request data sent to the server

[0400] Step 3: Receiving and analyzing information

[0401] The server receives user information sent from the device. The received data is analyzed by an analysis engine to identify the user's preferences and conditions. The analysis results are used to generate the next plan.

[0402] Input: User data sent to the server

[0403] Output: User preferences and conditions identified by the analytics engine

[0404] Step 4: Generate your travel plan

[0405] The AI ​​on the server generates optimal travel destinations and plans based on the analyzed user preferences and conditions. The AI ​​refers to existing travel databases and reviews to suggest multiple combinations of destinations and activities.

[0406] Input: User preferences and conditions identified by the analytics engine

[0407] Output: Data for generating optimal travel destinations and plans

[0408] Step 5: Submit and view your plan

[0409] The server sends the generated travel plan to the terminal, which visually displays the received plan for the user to check.

[0410] Input: Data for generating optimal travel destinations and plans

[0411] Output: Travel plan displayed on the device

[0412] Step 6: User chooses plan

[0413] The user selects the plan they like from the ones presented, and the selected plan is sent to the server via the device and saved as the final plan.

[0414] Input: Travel plan displayed on the device

[0415] Output: The final plan sent to the server

[0416] Step 7: Start real-time guidance

[0417] On the day of the trip, the user launches the app again, and the device loads the final plan saved on the server. The loaded plan is linked to the map application, and a route is displayed based on the user's current location. The user then begins traveling according to this route.

[0418] Input: Final plan saved on the server

[0419] Output: Display of travel route by linking with map application

[0420] Step 8: Emotion Engine Feedback

[0421] The user wears a device with voice guidance. The emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. Based on the recognized emotions, the tone and content of the guidance are adjusted and appropriate feedback is provided.

[0422] Input: User's facial expressions and tone of voice data

[0423] Output: Adjusted announcement tone and content

[0424] Step 9: Collect and analyze emotion data

[0425] The device continuously collects the user's emotional data while traveling. The collected data is sent to a server, where it is analyzed by an analysis engine. The analysis results are used by the AI ​​to generate travel plans for future trips.

[0426] Input: User emotion data during travel

[0427] Output: Analysis results of emotional data fed back to the generative AI

[0428] (Application example 2)

[0429] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0430] While itinerary optimization and real-time guidance have become commonplace, recognizing users' emotions and dynamically adjusting entertainment and guidance content based on that data is crucial to further enriching the travel experience. However, current systems lack a means to effectively recognize users' emotions and incorporate them into itinerary planning and real-time guidance. This poses a challenge, making it difficult for users to truly relax and enjoy their trip.

[0431] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes user input means, generation means for processing travel planning requests received from the user and generating optimal travel candidate destinations and plans, display means for displaying the generated candidate destinations and plans to the user, guidance means for providing real-time guidance based on the plan selected by the user, emotion engine means for recognizing the user's emotions using sensor means installed in the vehicle and processing the data, and entertainment control means for dynamically adjusting the content and tone of entertainment based on the recognized emotion data. This makes it possible to customize the travel experience according to the user's emotions.

[0432] The "user input means" is an interface through which a user inputs their travel planning requests, and is implemented as an application installed on a mobile device such as a smartphone or tablet.

[0433] The "generation means" has the function of processing a travel plan request received from a user input means and generating optimal travel destinations and plans.

[0434] The "display means" is a device that visually presents the travel destinations and plans generated by the generation means to the user, and operates on the same device as the user input means.

[0435] The "guidance means" provides real-time guidance based on the plan selected by the user and has the function of displaying the route in cooperation with a map application.

[0436] The "emotion engine means" is a part of the system that uses sensor means installed in the vehicle to recognize the user's emotions and process the data.

[0437] An "entertainment control means" is a means that dynamically adjusts the content and tone of entertainment based on recognized emotional data.

[0438] A "map application" is software that provides geographic location information, displays a user's travel route, and provides navigation.

[0439] A "device with voice guidance" is a device that provides real-time multilingual voice guidance.

[0440] The present invention provides a system for optimizing a user's travel plan and providing real-time guidance, particularly for autonomous vehicles. The system includes a user input means, a generation means, a display means, a guidance means, an emotion engine means, and an entertainment control means.

[0441] System configuration

[0442] User Input Method

[0443] The user input means is an interface through which users input their travel plan requests. For example, it is implemented as an application installed on a mobile device such as a smartphone or tablet. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[0444] generation means

[0445] The generation means is located on a server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[0446] Display means

[0447] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[0448] Guidance means

[0449] The guidance means provides real-time guidance based on the final plan selected by the user. For example, it works with a map application to display a route based on the current location information of the autonomous vehicle. It also provides real-time voice guidance in multiple languages ​​using a device with voice guidance.

[0450] Emotion Engine Means

[0451] The emotion engine uses sensors installed in the vehicle to recognize the user's emotions and process the data. This recognition process uses facial recognition cameras and voice analysis microphones. For example, Intel RealSense Depth Cameras and Shure SM7B microphones are considered. The emotion data is sent to a server and used to optimize the operation of the entire system.

[0452] Entertainment Controls

[0453] The entertainment control means dynamically adjusts the content and tone of entertainment based on the recognized emotion data. For example, it controls the playback of music streaming services and video content. Examples of software used include the Spotify API and YouTube API.

[0454] Specific examples

[0455] User information entry and submission

[0456] The user starts the application and enters the following information:

[0457] Area I want to visit: Tokyo

[0458] Food I want to enjoy: Sushi

[0459] Interests: Historical places

[0460] Budget: 5,000 yen per day

[0461] This information is sent to the server via the terminal.

[0462] Generative AI generates candidate sites and plans

[0463] Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best fit the user's preferences and conditions. For example, a plan might be generated that includes breakfast at Tsukiji Market, a visit to Sensoji Temple, lunch at a sushi restaurant in Ginza, a tour of the Imperial Palace, and dinner at a izakaya in Shinjuku.

[0464] Plan display and real-time guidance

[0465] The server sends the generated plan to the device, which displays it to the user. The user can review and select the plan. On the day of the trip, the device loads the final plan saved on the server and displays the route in conjunction with a map application. In addition, a voice guidance device worn by the user provides real-time multilingual guidance.

[0466] Emotion engine and entertainment tuning

[0467] During the trip, the emotion engine will recognize the user's emotions and provide feedback accordingly. For example, if it detects that the user is tired, it will provide guidance such as "You can take a break here and refresh yourself at the cafe."

[0468] Prompt Sentence Examples

[0469] An example prompt is:

[0470] The user provided the following information:

[0471] Area I want to visit: Tokyo

[0472] Food I want to enjoy: Sushi

[0473] Interests: Historical places

[0474] Budget: 5,000 yen per day

[0475] Based on this, suggest the best travel plan.

[0476] Thus, the present invention is designed to enhance a user's travel experience, providing real-time customized guidance and entertainment based on the user's emotions.

[0477] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0478] Step 1:

[0479] The user launches the application and inputs their travel plan requirements. The input information includes areas they want to visit, foods they want to enjoy, genres they are interested in, budget, etc. Input: User's travel plan requirements data. Output: User's input information stored on the device.

[0480] Step 2:

[0481] The terminal sends the information entered by the user to the server. Input: User input information stored in the terminal. Output: User travel plan request data sent to the server.

[0482] Step 3:

[0483] The server analyzes the received user information and uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans based on the user's preferences and conditions. Input: User's travel plan request data sent to the server. Output: Optimal travel destinations and plans.

[0484] Step 4:

[0485] The server sends the generated plan to the terminal. Input: Generated travel plan. Output: Optimal travel plan data sent to the terminal.

[0486] Step 5:

[0487] The terminal displays the plan sent to the user. The user selects the desired plan from the displayed candidates. Input: Optimal travel plan data sent from the server. Output: Travel plan displayed to the user and the user's selection information.

[0488] Step 6:

[0489] On the day of the trip, when the user restarts the app, the device loads the saved final plan from the server and works with the map application to display a route based on the current location. Input: Final plan data saved on the server. Output: Route displayed in the map application.

[0490] Step 7:

[0491] The emotion engine recognizes the user's emotions through sensing devices (cameras and microphones) installed in the vehicle. Input: Data on the user's facial expressions and voice. Output: Recognized emotion data.

[0492] Step 8:

[0493] The terminal processes the recognized emotion data, and the entertainment control means dynamically adjusts the content and tone of the entertainment according to the user's emotion. Input: Recognized emotion data. Output: Adjusted entertainment content.

[0494] Step 9:

[0495] The terminal voice guidance device provides multilingual voice guidance and feedback to the user in real time. Input: Travel route data and final plan selected. Output: Voice guidance and feedback provided to the user.

[0496] The system allows users to optimize their trip planning and enjoy entertainment and navigation that responds to their emotions in real time.

[0497] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0498] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0499] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0500] [Second embodiment]

[0501] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0502] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0503] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0504] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0505] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0506] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0507] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0508] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0509] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0510] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0511] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0512] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0513] The present invention is a system that generates optimal travel destinations and plans based on travel planning requests and provides real-time guidance on the day of travel. Specific embodiments of this system are described below.

[0514] System configuration

[0515] The system mainly consists of the following components:

[0516] 1. User Input Method

[0517] 2. Generation means

[0518] 3. Display means

[0519] 4. Guidance

[0520] User Input Method

[0521] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[0522] generation means

[0523] The generation means is located on a server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[0524] Display means

[0525] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[0526] Guidance means

[0527] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with a map application and has the function of displaying routes to the user. It also provides multilingual audio guidance using a device with audio guidance. This device is expected to be a glasses-type device worn by the user.

[0528] Program processing

[0529] User information entry and submission

[0530] 1. The user launches the application and enters their travel plan requirements, such as "Tokyo" as the area they want to visit, "sushi" as the food they want to enjoy, "historical places" as their interest, and "5,000 yen per day" as their budget.

[0531] 2. The terminal sends the entered information to the server.

[0532] Generative AI generates candidate sites and plans

[0533] 1. Based on the received information, the server uses generative AI to generate optimal travel destinations and plans. For example, a plan might be generated that includes breakfast at Tsukiji Market, a visit to Sensoji Temple, lunch at a sushi restaurant in Ginza, a tour of the Imperial Palace, and dinner at a pub in Shinjuku.

[0534] Display of candidate sites and plans

[0535] 1. The server sends the generated plan to the terminal.

[0536] 2. The device will display the plan details to the user, who can then review each plan and select the one they like best.

[0537] Real-time guidance

[0538] 1. On the day of travel, the user launches the app.

[0539] 2. The device loads the final plan saved on the server and displays the route in conjunction with the map application.

[0540] 3. The terminal works with a device that supports voice guidance to provide voice guidance, such as "Turn left next and go straight to Sensoji Temple."

[0541] Specific examples

[0542] User information entry and submission

[0543] The user starts the application and enters the following information:

[0544] Area I want to visit: Tokyo

[0545] Food I want to enjoy: Sushi

[0546] Interests: Historical places

[0547] Budget: 5,000 yen per day

[0548] The terminal sends this information to the server.

[0549] Generative AI generates candidate sites and plans

[0550] The server uses the above information to generate a plan like this:

[0551] Breakfast: Enjoy sushi at Tsukiji Market

[0552] Morning: Visit to Sensoji Temple

[0553] Lunch: Eat at a famous sushi restaurant in Ginza

[0554] Afternoon: Visit the Imperial Palace

[0555] Dinner: Savor local cuisine at a Shinjuku izakaya

[0556] Display of candidate sites and plans

[0557] The server sends the generated plan to the terminal, which displays it to the user, who then checks and selects the plan.

[0558] Real-time guidance

[0559] On the day of the trip, the user launches the app. The device loads the final plan stored on the server and displays the route in conjunction with the map application. Additionally, the user wears a voice guidance device that provides real-time, multilingual guidance.

[0560] The present invention allows users to plan their trip efficiently and stress-free, and to act with peace of mind while traveling.

[0561] The processing flow will be explained below.

[0562] Step 1:

[0563] Users launch the travel app and enter information such as the area they want to visit, the food they want to enjoy, the genres they are interested in, their budget, etc. Once they have finished entering the information, they tap the "Submit" button.

[0564] Step 2:

[0565] The device sends the input information, including the place to visit, type of food, genre of interest, budget, etc., to the server.

[0566] Step 3:

[0567] Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best suit the user's preferences and conditions.

[0568] Step 4:

[0569] The server then sends the generated candidate locations and plans to the device, including information on multiple tourist spots, places to eat, and activities.

[0570] Step 5:

[0571] The device will display the details of the plans received to the user, who can then review the plans and select the one they like. At this time, they can also view detailed information, photos, and reviews for each plan.

[0572] Step 6:

[0573] The user selects the desired plan and taps the "Confirm" button. Once the selection is confirmed, the device sends the final plan to the server.

[0574] Step 7:

[0575] The server saves the selected plan as the final plan, which includes tourist spots to visit, places to eat, and routes to travel.

[0576] Step 8:

[0577] On the day of the trip, the user launches the app again and confirms their plan. The device loads the saved final plan and works with the map application to display their current location and route.

[0578] Step 9:

[0579] The user starts moving along the route specified by the user, and the device updates the map information in real time and navigates to the next destination.

[0580] Step 10:

[0581] The user wears the device with voice guidance. The device prepares the voice guidance information and starts guiding based on the user's current location and final plan. For example, instructions such as "Turn left next and go straight to Sensoji Temple" are provided.

[0582] Step 11:

[0583] The device will display additional information and guides as needed for each tourist attraction or dining spot during the trip, and its multilingual capabilities will provide guidance according to the user's language settings.

[0584] Step 12:

[0585] The user completes their travel plan. The device sends the travel history to the server, where it stores the data for later access.

[0586] By following these steps, users can plan their trip efficiently and stress-free, and enjoy their trip with peace of mind, receiving detailed real-time guidance during their trip.

[0587] Example 1

[0588] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0589] Conventional travel planning systems have difficulty generating optimal travel plans based on user requests. They also lack efficient means for providing real-time guidance. As a result, users spend a great deal of time and effort planning and are unable to receive sufficient guidance during their trip.

[0590] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0591] In this invention, the server includes a generation unit that processes travel plan requests received from users and generates optimal travel destinations and plans, a generation unit that references an existing database and inputs prompts into a generative AI model to generate a travel plan, and a unit that displays the generated travel destinations and plans to the user, thereby enabling users to efficiently and accurately plan their travels and receive detailed guidance in real time.

[0592] "User input means" refers to an interface through which a user inputs their travel planning requirements.

[0593] The "generation means" is a means for processing a travel plan request received from a user and generating optimal travel destinations and plans.

[0594] "Means of referencing existing databases" refers to means of obtaining data that meets the user's needs by using a database that holds travel-related information.

[0595] "Means for inputting prompt sentences into a generative AI model" refers to a means for inputting prompt sentences based on the user's requests into a generative AI model and generating an optimal travel plan.

[0596] The "display means" is a means for visually presenting the generated travel destinations and plans to the user.

[0597] The "guidance means" is a means for providing real-time guidance based on the plan selected by the user.

[0598] The "means for linking with a map application" refers to a means for communicating with a map application and displaying information in order to display a travel route to the user.

[0599] A "device with voice guidance" is a device that provides voice guidance and is capable of providing multilingual voice guidance in real time.

[0600] The present invention is a system that generates optimal travel destinations and plans based on travel planning requests and provides real-time guidance on the day of travel. Specific embodiments of this system are described below.

[0601] System configuration

[0602] The system mainly consists of the following components:

[0603] 1. User Input Method

[0604] 2. Generation means

[0605] 3. Display means

[0606] 4. Guidance

[0607] User Input Method

[0608] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, their interests, and their budget. For example, a user inputs detailed information such as "areas they want to visit: Kyoto," "foods they want to enjoy: Japanese sweets," "interests: gardens," and "budget: 7,000 yen per day."

[0609] generation means

[0610] The generation means is placed on the server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to an existing database (for example, the API of a travel information service) and proposes a plan that best suits the user's preferences and conditions. The server inputs a prompt statement into the generation AI model to generate a travel plan. For example, the prompt statement could be, "The user has inputted 'Kyoto' as the area they would like to visit, 'Japanese sweets' as the food they would like to enjoy, 'gardens' as the genre they are interested in, and '7,000 yen per day' as their budget. Please generate the optimal travel plan based on this information."

[0611] Display means

[0612] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[0613] Guidance means

[0614] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with map applications (such as Google Maps or Apple Maps) and has the function of displaying routes to the user. It also provides audio guidance using a device with audio guidance (such as a multilingual audio guidance device). This device is expected to be a glasses-type device worn by the user. For example, it provides specific audio guidance such as, "Turn left next and go straight to Sensoji Temple."

[0615] In this way, users can plan their trips efficiently and optimally, and receive detailed real-time guidance during their trip.

[0616] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0617] Step 1:

[0618] Users launch the mobile app and input their travel plan requirements, including the areas they want to visit, the foods they want to enjoy, their interests, and their budget. This information is then sent to the device as input data.

[0619] Step 2:

[0620] The device sends the collected user input data to the server. The data is kept safe using a secure communication protocol (e.g., HTTPS). For example, the input data might be "Area I want to visit: Kyoto," "Food I want to enjoy: Japanese sweets," "Interested genre: gardens," or "Budget: 7,000 yen per day."

[0621] Step 3:

[0622] The server processes the received input data. Based on this data, it searches for related travel spots and information from an existing database. Next, it inputs a prompt statement into the generative AI model. An example of a prompt statement is: "The user has entered 'Kyoto' as the area they would like to visit, 'Japanese sweets' as the food they would like to enjoy, 'gardens' as a genre they are interested in, and '7,000 yen per day' as their budget. Please generate the optimal travel plan based on this information."

[0623] Step 4:

[0624] The generative AI model calculates data based on the prompt text and generates an optimal travel plan. Examples of generated plans include "Enjoy Japanese sweets at a sweet shop," "Visit Kinkakuji Temple," "Have lunch in Gion," "Tour of Otawara Pond Garden," and "Stroll along the Kamo River." This plan becomes the output data.

[0625] Step 5:

[0626] The server formats the generated travel plan and sends it to the device. The output data is formatted in JSON or XML format and provided to the device.

[0627] Step 6:

[0628] The device analyzes the received travel plans and visually displays them to the user, including detailed time schedules, maps, and user reviews for each plan. The user can then review these and select the plan they want.

[0629] Step 7:

[0630] The user selects and confirms the travel plan they like, and the device notifies the server of the selected plan as the final plan.

[0631] Step 8:

[0632] On the day of the trip, the user launches the app. The device loads the final plan stored on the server and uses GPS to identify the user's current location. It then connects to a map application (e.g., Google Maps) to display the route and begin guiding the trip.

[0633] Step 9:

[0634] The terminal works in conjunction with a voice guidance device to provide real-time voice guidance, such as "Turn left next and go straight to Sensoji Temple," allowing users to carry out efficient and optimal travel planning.

[0635] (Application example 1)

[0636] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0637] Conventional travel planning systems have had many problems with planning trips and providing real-time guidance during the trip. It is difficult to easily create an optimal plan tailored to the user's needs, and guidance is particularly inadequate in self-driving vehicles. Furthermore, there is a lack of systems that provide real-time navigation and guidance on-site, leading to users often getting lost. Furthermore, voice guidance is often not multilingual, which is extremely inconvenient for foreign tourists. The present invention aims to solve these problems.

[0638] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0639] In this invention, the server includes an input means for receiving a travel plan request from a user, a means for processing the travel plan request received from the input means and generating optimal travel destinations and plans using a generative AI model, a means for displaying the generated travel destinations and plans to the user, and a means for providing real-time guidance based on the plan selected by the user. This allows the user to enjoy a comfortable trip in an autonomous vehicle and receive real-time guidance on routes and tourist spots. Furthermore, multilingual audio guidance allows users to efficiently enjoy their trip beyond language barriers.

[0640] "Travel planning requests" are information including the user's desired travel destinations, activities of interest, and budget.

[0641] The "input means" is an interface for receiving travel planning requests from users, and is an application implemented on a smartphone or in-vehicle infotainment system.

[0642] A "generative AI model" is an artificial intelligence technology that automatically generates optimal travel destinations and plans based on travel planning requests received from users.

[0643] The "means for generating" is a processing device that uses a generative AI model to create optimal travel destinations and plans based on the user's requests.

[0644] The "display means" refers to a device for visually presenting the generated travel destinations and plans to the user, such as an in-car infotainment system or a smartphone display.

[0645] The "guidance means" is a device that provides real-time route and destination guidance based on a plan selected by the user.

[0646] A "navigation system" is a system that uses GPS data and map applications to display the route from the user's current location to their destination in real time.

[0647] A "voice guidance system" is a system that uses multilingual voice synthesis technology to provide users with real-time voice guidance on travel and tourist spots.

[0648] System configuration

[0649] The system of the present invention comprises the following components:

[0650] 1. Input Method

[0651] The input means of this system is an interface for receiving travel planning requests from users. It is provided as an application installed on smartphones or in-car infotainment systems. Users input information such as the areas they want to visit, activities they want to enjoy, places of interest, and their budget.

[0652] 2. Means of generation

[0653] The generation means runs on the server. Based on requests sent from the input means, the generative AI model generates optimal travel destinations and plans. The generative AI model refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[0654] 3. Display method

[0655] The display means is used to visually present the generated travel candidate destinations and plans to the user. It runs on the same device as the input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[0656] 4. Guidance

[0657] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with the navigation system and has the function of displaying routes to the user. It also provides real-time voice guidance using a multilingual voice guidance system. The voice guidance system effectively provides guidance through a device worn by the user (for example, a speaker built into the in-vehicle infotainment system).

[0658] Specific explanation of the system's operation

[0659] The operation of this system will be explained in detail below, showing how each element of the system works together.

[0660] 1. User information entry and submission

[0661] The user launches the application and inputs their travel plan requirements, for example, selecting "Kyoto" as the area they want to visit, "sightseeing" as the activity they want to enjoy, "historical places" as their points of interest, and "7,000 yen per day" as their budget. This information is then sent to the server by the device.

[0662] 2. Generating candidate sites and plans using a generative AI model

[0663] Based on the received information, the server uses a generative AI model to generate optimal travel destinations and plans. For example, a plan including tourist spots such as Kiyomizu-dera Temple, Kinkaku-ji Temple, and Nijo Castle may be generated.

[0664] 3. Display of candidate sites and plans

[0665] The server sends the generated plans to the device, which displays the plan details to the user, allowing the user to review each plan and select the one they like best.

[0666] 4. Real-time guidance

[0667] On the day of the trip, the user launches the app. The device loads the final plan saved on the server and displays the route in conjunction with the navigation system. It also links with the voice guidance system to provide real-time voice guidance in multiple languages. For example, the app may provide voice guidance such as, "Turn left next and go straight to Kiyomizu-dera Temple."

[0668] Specific examples

[0669] User information entry and submission

[0670] The user starts the application and enters the following information:

[0671] Area I want to visit: Kyoto

[0672] Activities to enjoy: Sightseeing

[0673] Places of interest: Historical places

[0674] Budget: 7,000 yen per day

[0675] The terminal sends this information to the server.

[0676] Generative AI model generates candidate sites and plans

[0677] The server uses the above information to generate a plan like this:

[0678] Touring sightseeing spots: Kiyomizu-dera Temple, Kinkaku-ji Temple, Nijo Castle

[0679] Lunch: Restaurant to enjoy local cuisine

[0680] Guide:Transportation to accommodation

[0681] Real-time guidance

[0682] On the day of the trip, the user launches the app in the autonomous vehicle. The device loads the final plan stored on the server and displays the route in conjunction with the navigation system. In addition, the voice guidance system provides real-time, multilingual guidance.

[0683] Prompt Sentence Examples

[0684] "Area I want to visit: Kyoto, Activities I'm interested in: Sightseeing, historical places, Budget: 7,000 yen per day"

[0685] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0686] Step 1:

[0687] Enter the user's travel planning requests

[0688] The user launches the application and inputs specific travel planning requirements (areas to visit, activities of interest, historical locations, budget, etc.) The input data is stored on the device via the smartphone or in-car infotainment system application.

[0689] input:

[0690] Area I want to visit: Kyoto

[0691] Activities to enjoy: Sightseeing

[0692] Places of interest: Historical places

[0693] Budget: 7,000 yen per day

[0694] output:

[0695] User travel planning information data

[0696] Step 2:

[0697] Send the user's travel plan information data to the server

[0698] The terminal transmits the user's travel plan information data entered in step 1 to the server. The data is transmitted using a secure communication channel.

[0699] input:

[0700] User travel planning information data

[0701] output:

[0702] User travel plan information data stored on the server

[0703] Step 3:

[0704] Generate optimal travel destinations and plans using generative AI models

[0705] The server processes the received user travel plan information data and uses a generative AI model to generate optimal travel destinations and plans. The AI ​​model references existing travel databases and reviews to create the optimal plan based on the user's requests.

[0706] input:

[0707] User travel plan information data stored on the server

[0708] output:

[0709] Generated travel destination and plan data

[0710] Step 4:

[0711] Displaying generated travel destinations and plans to users

[0712] The server sends the generated travel destination and plan data to the device, which receives it and visually displays it to the user. The user can then check the various options on the display screen and select the plan that best suits their needs.

[0713] input:

[0714] Generated travel destination and plan data

[0715] output:

[0716] A list of plan options displayed to the user

[0717] Step 5:

[0718] User selects final plan

[0719] The user selects the plan they like from the displayed list of plan candidates, and the selected plan is sent to the server by the terminal.

[0720] input:

[0721] User-selected plan data

[0722] output:

[0723] Final plan data sent to the server

[0724] Step 6:

[0725] Real-time information on the day of travel

[0726] On the day of the trip, the user launches the app. The device loads the final plan stored on the server, and displays the route in conjunction with the navigation system. It also works with the voice guidance system to provide real-time voice guidance in multiple languages.

[0727] input:

[0728] Final plan data stored on the server

[0729] output:

[0730] Navigation system showing route

[0731] Real-time multilingual voice guidance

[0732] Supplementary information on specific actions

[0733] In step 1, the user enters the necessary information via a smartphone or in-car infotainment system application. In step 2, this information is sent to the server via the internet. In step 3, the server uses this data to generate prompts for the generative AI model and builds an optimal travel plan. In steps 4 and 5, the server sends the plan it has generated to the device, where the user can confirm and select. Step 6 provides real-time guidance on the day of the trip, with navigation and voice guidance based on the user's location information.

[0734] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0735] The present invention is a system that generates optimal travel destinations and plans based on travel planning requests and provides real-time guidance on the day of travel. This system incorporates an emotion engine that recognizes the user's emotions and provides feedback accordingly. Specific embodiments of this system are described below.

[0736] System configuration

[0737] The system mainly consists of the following components:

[0738] 1. User Input Method

[0739] 2. Generation means

[0740] 3. Display means

[0741] 4. Guidance

[0742] 5. Emotion Engine

[0743] User Input Method

[0744] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[0745] generation means

[0746] The generation means is located on a server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[0747] Display means

[0748] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[0749] Guidance means

[0750] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with a map application and has the function of displaying routes to the user. It also provides multilingual audio guidance using a device with audio guidance. This device is expected to be a glasses-type device worn by the user.

[0751] Emotion Engine

[0752] The emotion engine recognizes the user's emotions and adjusts the content of the plan and guidance accordingly. It works in conjunction with the user input means, generation means, display means, and guidance means, and optimizes the system's operation based on the user's emotional data.

[0753] Program processing

[0754] User information entry and submission

[0755] 1. The user launches the application and enters information such as the area they want to visit, the food they want to enjoy, their interests, and their budget. For example, the area they want to visit might be "Tokyo," the food they want to enjoy might be "sushi," their interests might be "historical places," and their budget might be "5,000 yen per day."

[0756] 2. The terminal sends the entered information to the server.

[0757] Generative AI generates candidate sites and plans

[0758] 1. Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best fit the user's preferences and conditions. For example, a plan might be generated that includes breakfast at Tsukiji Market, a visit to Sensoji Temple, lunch at a sushi restaurant in Ginza, a tour of the Imperial Palace, and dinner at a izakaya in Shinjuku.

[0759] Display of candidate sites and plans

[0760] 1. The server sends the generated plan to the terminal.

[0761] 2. The device displays the plan details to the user, who then reviews the plans and selects the one they like.

[0762] Real-time guidance

[0763] 1. On the day of travel, the user launches the app.

[0764] 2. The device loads the final plan saved on the server and works with the map application to display the current location and route.

[0765] 3. The user begins traveling along the specified route. The device updates the map information in real time and navigates to the next destination.

[0766] Emotion Engine Operation

[0767] 1. The user wears a device with voice guidance.

[0768] 2. The emotion engine recognizes the user's emotions and adjusts the tone and content of the guidance accordingly. For example, if the user is tired, the guidance will be gentler and suggest places to rest.

[0769] 3. The device prepares voice guidance information and starts guiding based on the user's current location and final plan. For example, instructions such as "Turn left next and go straight to Sensoji Temple" are provided.

[0770] Emotional Data Feedback

[0771] 1. The device collects the user's emotional data during the trip and sends it to the server.

[0772] 2. The server analyzes the collected emotional data, and the generative AI uses it to generate future plans.

[0773] Specific examples

[0774] User information entry and submission

[0775] The user starts the application and enters the following information:

[0776] Area I want to visit: Tokyo

[0777] Food I want to enjoy: Sushi

[0778] Interests: Historical places

[0779] Budget: 5,000 yen per day

[0780] The terminal sends this information to the server.

[0781] Generative AI generates candidate sites and plans

[0782] The server uses the above information to generate a plan like this:

[0783] Breakfast: Enjoy sushi at Tsukiji Market

[0784] Morning: Visit to Sensoji Temple

[0785] Lunch: Eat at a famous sushi restaurant in Ginza

[0786] Afternoon: Visit the Imperial Palace

[0787] Dinner: Savor local cuisine at a Shinjuku izakaya

[0788] Display of candidate sites and plans

[0789] The server sends the generated plan to the terminal, which displays it to the user, who then checks and selects the plan.

[0790] Real-time guidance

[0791] On the day of the trip, the user launches the app. The device loads the final plan stored on the server and displays the route in conjunction with the map application. Additionally, the user wears a voice guidance device that provides real-time, multilingual guidance.

[0792] Emotion Engine Operation

[0793] During the trip, the emotion engine will recognize the user's emotions and provide feedback accordingly. For example, if it detects that the user is tired, it will provide guidance such as "You can take a break here and refresh yourself at the cafe."

[0794] The present invention allows users to plan their trip efficiently and stress-free, and to enjoy their trip with peace of mind while receiving detailed, real-time guidance that responds to their emotions during their trip.

[0795] The processing flow will be explained below.

[0796] Step 1:

[0797] Users launch a travel app and enter information such as the area they want to visit, the food they want to enjoy, their interests, and their budget. For example, they might choose "Tokyo" as the area they want to visit, "sushi" as the food they want to enjoy, "historical places" as their interests, and "5,000 yen per day" as their budget.

[0798] Step 2:

[0799] The device sends the input information, including the place to visit, type of food, genre of interest, budget, etc., to the server.

[0800] Step 3:

[0801] Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best suit the user's preferences and conditions.

[0802] Step 4:

[0803] The server then sends the generated candidate locations and plans to the device, including information on multiple tourist spots, places to eat, and activities.

[0804] Step 5:

[0805] The device will display the details of the plans received to the user, who can then review the plans and select the one they like. At this time, they can also view detailed information, photos, and reviews for each plan.

[0806] Step 6:

[0807] The user selects the desired plan and taps the "Confirm" button. Once the selection is confirmed, the device sends the final plan to the server.

[0808] Step 7:

[0809] The server saves the selected plan as the final plan, which includes tourist spots to visit, places to eat, and routes to travel.

[0810] Step 8:

[0811] On the day of the trip, the user launches the app again and confirms their plan. The device loads the saved final plan and works with the map application to display their current location and route.

[0812] Step 9:

[0813] The user starts moving along the route specified by the user, and the device updates the map information in real time and navigates to the next destination.

[0814] Step 10:

[0815] The user wears a device with voice guidance, such as a pair of eyeglasses. The device prepares voice guidance information and begins guiding based on the user's current location and final plan. For example, instructions such as "Turn left next and go straight to Sensoji Temple" are provided.

[0816] Step 11:

[0817] The emotion engine uses a camera or microphone to recognize the user's emotions, for example, by analyzing facial expressions and tone of voice.

[0818] Step 12:

[0819] The emotion engine transmits the recognized emotion data to the device, such as whether the user is tired, happy, or anxious.

[0820] Step 13:

[0821] The device adjusts the tone and content of the guidance based on the emotional data it receives. For example, if it detects that the user is tired, it will soften the tone of the guidance and make suggestions such as, "You can take a break here and refresh yourself at the cafe."

[0822] Step 14:

[0823] The device will display additional information and guides as needed for each tourist attraction or dining spot during the trip, and its multilingual capabilities will provide guidance according to the user's language settings.

[0824] Step 15:

[0825] Once the user completes their travel plan, the device sends their travel history and emotional data to the server, where it is stored so that the user can access it again later.

[0826] Through these steps, users can plan their trip efficiently and stress-free, and enjoy their trip with peace of mind, receiving detailed, real-time, and emotionally-sensitive guidance during their trip.

[0827] Example 2

[0828] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0829] Conventional travel planning systems have difficulty reflecting a user's individual preferences and emotions in real time. As a result, they are unable to respond to changes in the user's situation or mood during the trip, resulting in a lack of flexibility in the plan. Furthermore, real-time guidance is limited to general map applications and voice guidance, making it difficult to meet the diverse needs of users.

[0830] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a user input means, a generation means for processing a travel plan request received from the user and generating optimal candidate travel destinations and plans, a display means for displaying the generated candidate destinations and plans to the user, a guidance means for providing real-time guidance based on the plan selected by the user, and an emotion engine for recognizing the user's emotions and adjusting the content of the plan and guidance accordingly. This makes it possible to provide a travel plan that reflects the user's individual preferences and emotions in real time and to provide flexible real-time guidance.

[0831] "User input means" refers to an interface through which a user inputs their travel planning requirements.

[0832] The "generation means" is a means for generating optimal travel destinations and plans based on travel plan requests received from users.

[0833] The "display means" is a means for visually presenting the generated travel destinations and plans to the user.

[0834] The "guidance means" is a means for providing real-time guidance based on the plan selected by the user.

[0835] The "emotion engine" is an engine that recognizes the user's emotions and adjusts the content of the plan or guidance accordingly.

[0836] "Map Application" means software used to display travel routes to a user.

[0837] A "voice-guided device" is a device used to provide real-time voice guidance.

[0838] This invention relates to a system that generates optimal travel destinations and plans based on a user's travel planning requests and provides real-time guidance on the day of the trip. This system incorporates an emotion engine that recognizes the user's emotions and provides feedback accordingly. Specific embodiments of this system are described below.

[0839] System configuration

[0840] The system mainly consists of the following components:

[0841] 1. User Input Method

[0842] 2. Generation means

[0843] 3. Display means

[0844] 4. Guidance

[0845] 5. Emotion Engine

[0846] User Input Method

[0847] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[0848] Specifically, the user launches the application and enters the following information:

[0849] Area I want to visit: Tokyo

[0850] Food I want to enjoy: Sushi

[0851] Interests: Historical places

[0852] Budget: 5,000 yen per day

[0853] generation means

[0854] The generation means is located on the server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions. For example, the following specific plans may be generated:

[0855] Breakfast: Enjoy sushi at Tsukiji Market

[0856] Morning: Visit to Sensoji Temple

[0857] Lunch: Eat at a famous sushi restaurant in Ginza

[0858] Afternoon: Visit the Imperial Palace

[0859] Dinner: Savor local cuisine at a Shinjuku izakaya

[0860] Display means

[0861] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[0862] Guidance means

[0863] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with a map application and has the function of displaying routes to the user. It also provides multilingual audio guidance using a device with audio guidance. This device is expected to be a glasses-type device worn by the user.

[0864] For example, on the day of a trip, when a user launches the app, the device loads the final plan saved on the server and displays the route in conjunction with the map application. As the user begins to move, the device updates the map information in real time and navigates to the next destination.

[0865] Emotion Engine

[0866] The emotion engine recognizes the user's emotions and adjusts the tone and content of the guidance based on those emotions. It works in conjunction with the user input means, generation means, display means, and guidance means, optimizing the system's operation based on the user's emotional data.

[0867] Specifically, when a user wears a device with voice guidance, the emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. For example, if the user is tired, the engine will provide guidance such as, "You can take a break here and refresh yourself at the cafe." In this way, it is possible to provide appropriate feedback according to the user's emotions.

[0868] This system allows users to plan their trip efficiently and stress-free, and they can enjoy their trip with peace of mind while receiving detailed, real-time guidance that responds to their emotions. Furthermore, the collection and analysis of emotion data will enable the system to provide even more accurate travel plans.

[0869] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0870] Step 1: Enter your user information

[0871] Users launch the application and enter information such as the area they want to visit, the food they want to enjoy, their interests, and their budget. For example, they can enter information such as "Tokyo," "sushi," "historical place," and "5,000 yen per day." The entered data is temporarily stored on the device.

[0872] Input: User's travel plan request (e.g. Tokyo, sushi, historical places, budget 5000 yen)

[0873] Output: Input data saved on the device

[0874] Step 2: Submit user information

[0875] The device sends the entered information to the server, where each item is converted into an appropriate format and a data format such as JSON is generated.

[0876] Input: Input data stored on the device

[0877] Output: User's travel plan request data sent to the server

[0878] Step 3: Receiving and analyzing information

[0879] The server receives user information sent from the device. The received data is analyzed by an analysis engine to identify the user's preferences and conditions. The analysis results are used to generate the next plan.

[0880] Input: User data sent to the server

[0881] Output: User preferences and conditions identified by the analytics engine

[0882] Step 4: Generate your travel plan

[0883] The AI ​​on the server generates optimal travel destinations and plans based on the analyzed user preferences and conditions. The AI ​​refers to existing travel databases and reviews to suggest multiple combinations of destinations and activities.

[0884] Input: User preferences and conditions identified by the analytics engine

[0885] Output: Data for generating optimal travel destinations and plans

[0886] Step 5: Submit and view your plan

[0887] The server sends the generated travel plan to the terminal, which visually displays the received plan for the user to check.

[0888] Input: Data for generating optimal travel destinations and plans

[0889] Output: Travel plan displayed on the device

[0890] Step 6: User chooses plan

[0891] The user selects the plan they like from the ones presented, and the selected plan is sent to the server via the device and saved as the final plan.

[0892] Input: Travel plan displayed on the device

[0893] Output: The final plan sent to the server

[0894] Step 7: Start real-time guidance

[0895] On the day of the trip, the user launches the app again, and the device loads the final plan saved on the server. The loaded plan is linked to the map application, and a route is displayed based on the user's current location. The user then begins traveling according to this route.

[0896] Input: Final plan saved on the server

[0897] Output: Display of travel route by linking with map application

[0898] Step 8: Emotion Engine Feedback

[0899] The user wears a device with voice guidance. The emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. Based on the recognized emotion, the tone and content of the guidance are adjusted and appropriate feedback is provided.

[0900] Input: User's facial expressions and tone of voice data

[0901] Output: Adjusted announcement tone and content

[0902] Step 9: Collect and analyze emotion data

[0903] The device continuously collects emotional data from the user while traveling. The collected data is sent to a server, where it is analyzed by an analysis engine. The analysis results are used by the AI ​​to generate travel plans for future trips.

[0904] Input: User emotion data during travel

[0905] Output: Analysis results of emotional data fed back to the generative AI

[0906] (Application example 2)

[0907] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0908] While itinerary optimization and real-time guidance have become commonplace, recognizing users' emotions and dynamically adjusting entertainment and guidance content based on that data is crucial to further enriching the travel experience. However, current systems lack a means to effectively recognize users' emotions and incorporate them into itinerary planning and real-time guidance. This poses a challenge, making it difficult for users to truly relax and enjoy their trip.

[0909] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes user input means, generation means for processing travel planning requests received from the user and generating optimal candidate travel destinations and plans, display means for displaying the generated candidate destinations and plans to the user, guidance means for providing real-time guidance based on the plan selected by the user, emotion engine means for recognizing the user's emotions using sensor means installed in the vehicle and processing the data, and entertainment control means for dynamically adjusting the content and tone of entertainment based on the recognized emotion data. This makes it possible to customize the travel experience according to the user's emotions.

[0910] The "user input means" is an interface through which a user inputs their travel planning requests, and is implemented as an application installed on a mobile device such as a smartphone or tablet.

[0911] The "generation means" has the function of processing a travel plan request received from a user input means and generating optimal travel destinations and plans.

[0912] The "display means" is a device that visually presents the travel destinations and plans generated by the generation means to the user, and operates on the same device as the user input means.

[0913] The "guidance means" provides real-time guidance based on the plan selected by the user and has the function of displaying the route in cooperation with a map application.

[0914] The "emotion engine means" is a part of the system that uses sensor means installed in the vehicle to recognize the user's emotions and process the data.

[0915] An "entertainment control means" is a means that dynamically adjusts the content and tone of entertainment based on recognized emotional data.

[0916] A "map application" is software that provides geographic location information, displays a user's travel route, and provides navigation.

[0917] A "device with voice guidance" is a device that provides real-time multilingual voice guidance.

[0918] The present invention provides a system for optimizing a user's travel plan and providing real-time guidance, particularly for autonomous vehicles. The system includes a user input means, a generation means, a display means, a guidance means, an emotion engine means, and an entertainment control means.

[0919] System configuration

[0920] User Input Method

[0921] The user input means is an interface through which users input their travel plan requests. For example, it is implemented as an application installed on a mobile device such as a smartphone or tablet. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[0922] generation means

[0923] The generation means is located on a server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[0924] Display means

[0925] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[0926] Guidance means

[0927] The guidance means provides real-time guidance based on the final plan selected by the user. For example, it works with a map application to display a route based on the current location information of the autonomous vehicle. It also provides real-time voice guidance in multiple languages ​​using a device with voice guidance.

[0928] Emotion Engine Means

[0929] The emotion engine uses sensors installed in the vehicle to recognize the user's emotions and process the data. This recognition process uses facial recognition cameras and voice analysis microphones. For example, Intel RealSense Depth Cameras and Shure SM7B microphones are considered. The emotion data is sent to a server and used to optimize the operation of the entire system.

[0930] Entertainment Controls

[0931] The entertainment control means dynamically adjusts the content and tone of entertainment based on the recognized emotion data. For example, it controls the playback of music streaming services and video content. Examples of software used include the Spotify API and YouTube API.

[0932] Specific examples

[0933] User information entry and submission

[0934] The user starts the application and enters the following information:

[0935] Area I want to visit: Tokyo

[0936] Food I want to enjoy: Sushi

[0937] Interests: Historical places

[0938] Budget: 5,000 yen per day

[0939] This information is sent to the server via the terminal.

[0940] Generative AI generates candidate sites and plans

[0941] Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best fit the user's preferences and conditions. For example, a plan might be generated that includes breakfast at Tsukiji Market, a visit to Sensoji Temple, lunch at a sushi restaurant in Ginza, a tour of the Imperial Palace, and dinner at a izakaya in Shinjuku.

[0942] Plan display and real-time guidance

[0943] The server sends the generated plan to the device, which displays it to the user. The user can review and select the plan. On the day of the trip, the device loads the final plan saved on the server and displays the route in conjunction with a map application. In addition, a voice guidance device worn by the user provides real-time multilingual guidance.

[0944] Emotion engine and entertainment tuning

[0945] During the trip, the emotion engine will recognize the user's emotions and provide feedback accordingly. For example, if it detects that the user is tired, it will provide guidance such as "You can take a break here and refresh yourself at the cafe."

[0946] Prompt Sentence Examples

[0947] An example prompt is:

[0948] The user provided the following information:

[0949] Area I want to visit: Tokyo

[0950] Food I want to enjoy: Sushi

[0951] Interests: Historical places

[0952] Budget: 5,000 yen per day

[0953] Based on this, suggest the best travel plan.

[0954] Thus, the present invention is designed to enhance a user's travel experience, providing real-time customized guidance and entertainment based on the user's emotions.

[0955] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0956] Step 1:

[0957] The user launches the application and inputs their travel plan requirements. The input information includes areas they want to visit, foods they want to enjoy, genres they are interested in, budget, etc. Input: User's travel plan requirements data. Output: User's input information stored on the device.

[0958] Step 2:

[0959] The terminal sends the information entered by the user to the server. Input: User input information stored in the terminal. Output: User travel plan request data sent to the server.

[0960] Step 3:

[0961] The server analyzes the received user information and uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans based on the user's preferences and conditions. Input: User's travel plan request data sent to the server. Output: Optimal travel destinations and plans.

[0962] Step 4:

[0963] The server sends the generated plan to the terminal. Input: Generated travel plan. Output: Optimal travel plan data sent to the terminal.

[0964] Step 5:

[0965] The terminal displays the plan sent to the user. The user selects the desired plan from the displayed candidates. Input: Optimal travel plan data sent from the server. Output: Travel plan displayed to the user and the user's selection information.

[0966] Step 6:

[0967] On the day of the trip, when the user restarts the app, the device loads the saved final plan from the server and works with the map application to display a route based on the current location. Input: Final plan data saved on the server. Output: Route displayed in the map application.

[0968] Step 7:

[0969] The emotion engine recognizes the user's emotions through sensing devices (cameras and microphones) installed in the vehicle. Input: Data on the user's facial expressions and voice. Output: Recognized emotion data.

[0970] Step 8:

[0971] The terminal processes the recognized emotion data, and the entertainment control means dynamically adjusts the content and tone of the entertainment according to the user's emotion. Input: Recognized emotion data. Output: Adjusted entertainment content.

[0972] Step 9:

[0973] The terminal voice guidance device provides multilingual voice guidance and feedback to the user in real time. Input: Travel route data and final plan selected. Output: Voice guidance and feedback provided to the user.

[0974] The system allows users to optimize their trip planning and enjoy entertainment and navigation that responds to their emotions in real time.

[0975] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0976] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0977] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0978] [Third embodiment]

[0979] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0980] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0981] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0982] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0983] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0984] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0985] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0986] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0987] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0988] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0989] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0990] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0991] The present invention is a system that generates optimal travel destinations and plans based on travel planning requests and provides real-time guidance on the day of travel. Specific embodiments of this system are described below.

[0992] System configuration

[0993] The system mainly consists of the following components:

[0994] 1. User Input Method

[0995] 2. Generation means

[0996] 3. Display means

[0997] 4. Guidance

[0998] User Input Method

[0999] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[1000] generation means

[1001] The generation means is located on a server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[1002] Display means

[1003] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[1004] Guidance means

[1005] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with a map application and has the function of displaying routes to the user. It also provides multilingual audio guidance using a device with audio guidance. This device is expected to be a glasses-type device worn by the user.

[1006] Program processing

[1007] User information entry and submission

[1008] 1. The user launches the application and enters their travel plan requirements, such as "Tokyo" as the area they want to visit, "sushi" as the food they want to enjoy, "historical places" as their interest, and "5,000 yen per day" as their budget.

[1009] 2. The terminal sends the entered information to the server.

[1010] Generative AI generates candidate sites and plans

[1011] 1. Based on the received information, the server uses generative AI to generate optimal travel destinations and plans. For example, a plan might be generated that includes breakfast at Tsukiji Market, a visit to Sensoji Temple, lunch at a sushi restaurant in Ginza, a tour of the Imperial Palace, and dinner at a pub in Shinjuku.

[1012] Display of candidate sites and plans

[1013] 1. The server sends the generated plan to the terminal.

[1014] 2. The device will display the plan details to the user, who can then review each plan and select the one they like best.

[1015] Real-time guidance

[1016] 1. On the day of travel, the user launches the app.

[1017] 2. The device loads the final plan saved on the server and displays the route in conjunction with the map application.

[1018] 3. The terminal works with a device that supports voice guidance to provide voice guidance, such as "Turn left next and go straight to Sensoji Temple."

[1019] Specific examples

[1020] User information entry and submission

[1021] The user starts the application and enters the following information:

[1022] Area I want to visit: Tokyo

[1023] Food I want to enjoy: Sushi

[1024] Interests: Historical places

[1025] Budget: 5,000 yen per day

[1026] The terminal sends this information to the server.

[1027] Generative AI generates candidate sites and plans

[1028] The server uses the above information to generate a plan like this:

[1029] Breakfast: Enjoy sushi at Tsukiji Market

[1030] Morning: Visit to Sensoji Temple

[1031] Lunch: Eat at a famous sushi restaurant in Ginza

[1032] Afternoon: Visit the Imperial Palace

[1033] Dinner: Savor local cuisine at a Shinjuku izakaya

[1034] Display of candidate sites and plans

[1035] The server sends the generated plan to the terminal, which displays it to the user, who then checks and selects the plan.

[1036] Real-time guidance

[1037] On the day of the trip, the user launches the app. The device loads the final plan stored on the server and displays the route in conjunction with the map application. Additionally, the user wears a voice guidance device that provides real-time, multilingual guidance.

[1038] The present invention allows users to plan their trip efficiently and stress-free, and to act with peace of mind while traveling.

[1039] The processing flow will be explained below.

[1040] Step 1:

[1041] Users launch the travel app and enter information such as the area they want to visit, the food they want to enjoy, the genres they are interested in, their budget, etc. Once they have finished entering the information, they tap the "Submit" button.

[1042] Step 2:

[1043] The device sends the input information, including the place to visit, type of food, genre of interest, budget, etc., to the server.

[1044] Step 3:

[1045] Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best suit the user's preferences and conditions.

[1046] Step 4:

[1047] The server then sends the generated candidate locations and plans to the device, including information on multiple tourist spots, places to eat, and activities.

[1048] Step 5:

[1049] The device will display the details of the plans received to the user, who can then review the plans and select the one they like. At this time, they can also view detailed information, photos, and reviews for each plan.

[1050] Step 6:

[1051] The user selects the desired plan and taps the "Confirm" button. Once the selection is confirmed, the device sends the final plan to the server.

[1052] Step 7:

[1053] The server saves the selected plan as the final plan, which includes tourist spots to visit, places to eat, and routes to travel.

[1054] Step 8:

[1055] On the day of the trip, the user launches the app again and confirms their plan. The device loads the saved final plan and works with the map application to display their current location and route.

[1056] Step 9:

[1057] The user starts moving along the route specified by the user, and the device updates the map information in real time and navigates to the next destination.

[1058] Step 10:

[1059] The user wears the device with voice guidance. The device prepares the voice guidance information and starts guiding based on the user's current location and final plan. For example, instructions such as "Turn left next and go straight to Sensoji Temple" are provided.

[1060] Step 11:

[1061] The device will display additional information and guides as needed for each tourist attraction or dining spot during the trip, and its multilingual capabilities will provide guidance according to the user's language settings.

[1062] Step 12:

[1063] The user completes their travel plan. The device sends the travel history to the server, where it stores the data for later access.

[1064] By following these steps, users can plan their trip efficiently and stress-free, and enjoy their trip with peace of mind, receiving detailed real-time guidance during their trip.

[1065] Example 1

[1066] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1067] Conventional travel planning systems have difficulty generating optimal travel plans based on user requests. They also lack efficient means for providing real-time guidance. As a result, users spend a great deal of time and effort planning and are unable to receive sufficient guidance during their trip.

[1068] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1069] In this invention, the server includes a generation unit that processes travel plan requests received from users and generates optimal travel destinations and plans, a generation unit that references an existing database and inputs prompts into a generative AI model to generate a travel plan, and a unit that displays the generated travel destinations and plans to the user, thereby enabling users to efficiently and accurately plan their travels and receive detailed guidance in real time.

[1070] "User input means" refers to an interface through which a user inputs their travel planning requirements.

[1071] The "generation means" is a means for processing a travel plan request received from a user and generating optimal travel destinations and plans.

[1072] "Means of referencing existing databases" refers to means of obtaining data that meets the user's needs by using a database that holds travel-related information.

[1073] "Means for inputting prompt sentences into a generative AI model" refers to a means for inputting prompt sentences based on the user's requests into a generative AI model and generating an optimal travel plan.

[1074] The "display means" is a means for visually presenting the generated travel destinations and plans to the user.

[1075] The "guidance means" is a means for providing real-time guidance based on the plan selected by the user.

[1076] The "means for linking with a map application" refers to a means for communicating with a map application and displaying information in order to display a travel route to the user.

[1077] A "device with voice guidance" is a device that provides voice guidance and is capable of providing multilingual voice guidance in real time.

[1078] The present invention is a system that generates optimal travel destinations and plans based on travel planning requests and provides real-time guidance on the day of travel. Specific embodiments of this system are described below.

[1079] System configuration

[1080] The system mainly consists of the following components:

[1081] 1. User Input Method

[1082] 2. Generation means

[1083] 3. Display means

[1084] 4. Guidance

[1085] User Input Method

[1086] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, their interests, and their budget. For example, a user inputs detailed information such as "areas they want to visit: Kyoto," "foods they want to enjoy: Japanese sweets," "interests: gardens," and "budget: 7,000 yen per day."

[1087] generation means

[1088] The generation means is placed on the server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to an existing database (for example, the API of a travel information service) and proposes a plan that best suits the user's preferences and conditions. The server inputs a prompt statement into the generation AI model to generate a travel plan. For example, the prompt statement could be, "The user has inputted 'Kyoto' as the area they would like to visit, 'Japanese sweets' as the food they would like to enjoy, 'gardens' as the genre they are interested in, and '7,000 yen per day' as their budget. Please generate the optimal travel plan based on this information."

[1089] Display means

[1090] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[1091] Guidance means

[1092] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with map applications (such as Google Maps or Apple Maps) and has the function of displaying routes to the user. It also provides audio guidance using a device with audio guidance (such as a multilingual audio guidance device). This device is expected to be a glasses-type device worn by the user. For example, it provides specific audio guidance such as, "Turn left next and go straight to Sensoji Temple."

[1093] In this way, users can plan their trips efficiently and optimally, and receive detailed real-time guidance during their trip.

[1094] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1095] Step 1:

[1096] Users launch the mobile app and input their travel plan requirements, including the areas they want to visit, the foods they want to enjoy, their interests, and their budget. This information is then sent to the device as input data.

[1097] Step 2:

[1098] The device sends the collected user input data to the server. The data is kept safe using a secure communication protocol (e.g., HTTPS). For example, the input data might be "Area I want to visit: Kyoto," "Food I want to enjoy: Japanese sweets," "Interested genre: gardens," or "Budget: 7,000 yen per day."

[1099] Step 3:

[1100] The server processes the received input data. Based on this data, it searches for related travel spots and information from an existing database. Next, it inputs a prompt statement into the generative AI model. An example of a prompt statement is: "The user has entered 'Kyoto' as the area they would like to visit, 'Japanese sweets' as the food they would like to enjoy, 'gardens' as a genre they are interested in, and '7,000 yen per day' as their budget. Please generate the optimal travel plan based on this information."

[1101] Step 4:

[1102] The generative AI model calculates data based on the prompt text and generates an optimal travel plan. Examples of generated plans include "Enjoy Japanese sweets at a sweet shop," "Visit Kinkakuji Temple," "Have lunch in Gion," "Tour of Otawara Pond Garden," and "Stroll along the Kamo River." This plan becomes the output data.

[1103] Step 5:

[1104] The server formats the generated travel plan and sends it to the device. The output data is formatted in JSON or XML format and provided to the device.

[1105] Step 6:

[1106] The device analyzes the received travel plans and visually displays them to the user, including detailed time schedules, maps, and user reviews for each plan. The user can then review these and select the plan they want.

[1107] Step 7:

[1108] The user selects and confirms the travel plan they like, and the device notifies the server of the selected plan as the final plan.

[1109] Step 8:

[1110] On the day of the trip, the user launches the app. The device loads the final plan stored on the server and uses GPS to identify the user's current location. It then connects to a map application (e.g., Google Maps) to display the route and begin guiding the trip.

[1111] Step 9:

[1112] The terminal works in conjunction with a voice guidance device to provide real-time voice guidance, such as "Turn left next and go straight to Sensoji Temple," allowing users to carry out efficient and optimal travel planning.

[1113] (Application example 1)

[1114] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1115] Conventional travel planning systems have had many problems with planning trips and providing real-time guidance during the trip. It is difficult to easily create an optimal plan tailored to the user's needs, and guidance is particularly inadequate in self-driving vehicles. Furthermore, there is a lack of systems that provide real-time navigation and guidance on-site, leading to users often getting lost. Furthermore, voice guidance is often not multilingual, which is extremely inconvenient for foreign tourists. The present invention aims to solve these problems.

[1116] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1117] In this invention, the server includes an input means for receiving a travel plan request from a user, a means for processing the travel plan request received from the input means and generating optimal travel destinations and plans using a generative AI model, a means for displaying the generated travel destinations and plans to the user, and a means for providing real-time guidance based on the plan selected by the user. This allows the user to enjoy a comfortable trip in an autonomous vehicle and receive real-time guidance on routes and tourist spots. Furthermore, multilingual audio guidance allows users to efficiently enjoy their trip beyond language barriers.

[1118] "Travel planning requests" are information including the user's desired travel destinations, activities of interest, and budget.

[1119] The "input means" is an interface for receiving travel planning requests from users, and is an application implemented on a smartphone or in-vehicle infotainment system.

[1120] A "generative AI model" is an artificial intelligence technology that automatically generates optimal travel destinations and plans based on travel planning requests received from users.

[1121] The "means for generating" is a processing device that uses a generative AI model to create optimal travel destinations and plans based on the user's requests.

[1122] The "display means" refers to a device for visually presenting the generated travel destinations and plans to the user, such as an in-car infotainment system or a smartphone display.

[1123] The "guidance means" is a device that provides real-time route and destination guidance based on a plan selected by the user.

[1124] A "navigation system" is a system that uses GPS data and map applications to display the route from the user's current location to their destination in real time.

[1125] A "voice guidance system" is a system that uses multilingual voice synthesis technology to provide users with real-time voice guidance on travel and tourist spots.

[1126] System configuration

[1127] The system of the present invention comprises the following components:

[1128] 1. Input Method

[1129] The input means of this system is an interface for receiving travel planning requests from users. It is provided as an application installed on smartphones or in-car infotainment systems. Users input information such as the areas they want to visit, activities they want to enjoy, places of interest, and their budget.

[1130] 2. Means of generation

[1131] The generation means runs on the server. Based on requests sent from the input means, the generative AI model generates optimal travel destinations and plans. The generative AI model refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[1132] 3. Display method

[1133] The display means is used to visually present the generated travel candidate destinations and plans to the user. It runs on the same device as the input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[1134] 4. Guidance

[1135] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with the navigation system and has the function of displaying routes to the user. It also provides real-time voice guidance using a multilingual voice guidance system. The voice guidance system effectively provides guidance through a device worn by the user (for example, a speaker built into the in-vehicle infotainment system).

[1136] Specific explanation of the system's operation

[1137] The operation of this system will be explained in detail below, showing how each element of the system works together.

[1138] 1. User information entry and submission

[1139] The user launches the application and inputs their travel plan requirements, for example, selecting "Kyoto" as the area they want to visit, "sightseeing" as the activity they want to enjoy, "historical places" as their points of interest, and "7,000 yen per day" as their budget. This information is then sent to the server by the device.

[1140] 2. Generating candidate sites and plans using a generative AI model

[1141] Based on the received information, the server uses a generative AI model to generate optimal travel destinations and plans. For example, a plan including tourist spots such as Kiyomizu-dera Temple, Kinkaku-ji Temple, and Nijo Castle may be generated.

[1142] 3. Display of candidate sites and plans

[1143] The server sends the generated plans to the device, which displays the plan details to the user, allowing the user to review each plan and select the one they like best.

[1144] 4. Real-time guidance

[1145] On the day of the trip, the user launches the app. The device loads the final plan saved on the server and displays the route in conjunction with the navigation system. It also links with the voice guidance system to provide real-time voice guidance in multiple languages. For example, the app may provide voice guidance such as, "Turn left next and go straight to Kiyomizu-dera Temple."

[1146] Specific examples

[1147] User information entry and submission

[1148] The user starts the application and enters the following information:

[1149] Area I want to visit: Kyoto

[1150] Activities to enjoy: Sightseeing

[1151] Places of interest: Historical places

[1152] Budget: 7,000 yen per day

[1153] The terminal sends this information to the server.

[1154] Generative AI model generates candidate sites and plans

[1155] The server uses the above information to generate a plan like this:

[1156] Touring sightseeing spots: Kiyomizu-dera Temple, Kinkaku-ji Temple, Nijo Castle

[1157] Lunch: Restaurant to enjoy local cuisine

[1158] Guide:Transportation to accommodation

[1159] Real-time guidance

[1160] On the day of the trip, the user launches the app in the autonomous vehicle. The device loads the final plan stored on the server and displays the route in conjunction with the navigation system. In addition, the voice guidance system provides real-time, multilingual guidance.

[1161] Prompt Sentence Examples

[1162] "Area I want to visit: Kyoto, Activities I'm interested in: Sightseeing, historical places, Budget: 7,000 yen per day"

[1163] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1164] Step 1:

[1165] Enter the user's travel planning requests

[1166] The user launches the application and inputs specific travel planning requirements (areas to visit, activities of interest, historical locations, budget, etc.) The input data is stored on the device via the smartphone or in-car infotainment system application.

[1167] input:

[1168] Area I want to visit: Kyoto

[1169] Activities to enjoy: Sightseeing

[1170] Places of interest: Historical places

[1171] Budget: 7,000 yen per day

[1172] output:

[1173] User travel planning information data

[1174] Step 2:

[1175] Send the user's travel plan information data to the server

[1176] The terminal transmits the user's travel plan information data entered in step 1 to the server. The data is transmitted using a secure communication channel.

[1177] input:

[1178] User travel planning information data

[1179] output:

[1180] User travel plan information data stored on the server

[1181] Step 3:

[1182] Generate optimal travel destinations and plans using generative AI models

[1183] The server processes the received user travel plan information data and uses a generative AI model to generate optimal travel destinations and plans. The AI ​​model references existing travel databases and reviews to create the optimal plan based on the user's requests.

[1184] input:

[1185] User travel plan information data stored on the server

[1186] output:

[1187] Generated travel destination and plan data

[1188] Step 4:

[1189] Displaying generated travel destinations and plans to users

[1190] The server sends the generated travel destination and plan data to the device, which receives it and visually displays it to the user. The user can then check the various options on the display screen and select the plan that best suits their needs.

[1191] input:

[1192] Generated travel destination and plan data

[1193] output:

[1194] A list of plan options displayed to the user

[1195] Step 5:

[1196] User selects final plan

[1197] The user selects the plan they like from the displayed list of plan candidates, and the selected plan is sent to the server by the terminal.

[1198] input:

[1199] User-selected plan data

[1200] output:

[1201] Final plan data sent to the server

[1202] Step 6:

[1203] Real-time information on the day of travel

[1204] On the day of the trip, the user launches the app. The device loads the final plan stored on the server, and displays the route in conjunction with the navigation system. It also works with the voice guidance system to provide real-time voice guidance in multiple languages.

[1205] input:

[1206] Final plan data stored on the server

[1207] output:

[1208] Navigation system showing route

[1209] Real-time multilingual voice guidance

[1210] Supplementary information on specific actions

[1211] In step 1, the user enters the necessary information via a smartphone or in-car infotainment system application. In step 2, this information is sent to the server via the internet. In step 3, the server uses this data to generate prompts for the generative AI model and builds an optimal travel plan. In steps 4 and 5, the server sends the plan it has generated to the device, where the user can confirm and select. Step 6 provides real-time guidance on the day of the trip, with navigation and voice guidance based on the user's location information.

[1212] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1213] The present invention is a system that generates optimal travel destinations and plans based on travel planning requests and provides real-time guidance on the day of travel. This system incorporates an emotion engine that recognizes the user's emotions and provides feedback accordingly. Specific embodiments of this system are described below.

[1214] System configuration

[1215] The system mainly consists of the following components:

[1216] 1. User Input Method

[1217] 2. Generation means

[1218] 3. Display means

[1219] 4. Guidance

[1220] 5. Emotion Engine

[1221] User Input Method

[1222] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[1223] generation means

[1224] The generation means is located on a server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[1225] Display means

[1226] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[1227] Guidance means

[1228] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with a map application and has the function of displaying routes to the user. It also provides multilingual audio guidance using a device with audio guidance. This device is expected to be a glasses-type device worn by the user.

[1229] Emotion Engine

[1230] The emotion engine recognizes the user's emotions and adjusts the content of the plan and guidance accordingly. It works in conjunction with the user input means, generation means, display means, and guidance means, and optimizes the system's operation based on the user's emotional data.

[1231] Program processing

[1232] User information entry and submission

[1233] 1. The user launches the application and enters information such as the area they want to visit, the food they want to enjoy, their interests, and their budget. For example, the area they want to visit might be "Tokyo," the food they want to enjoy might be "sushi," their interests might be "historical places," and their budget might be "5,000 yen per day."

[1234] 2. The terminal sends the entered information to the server.

[1235] Generative AI generates candidate sites and plans

[1236] 1. Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best fit the user's preferences and conditions. For example, a plan might be generated that includes breakfast at Tsukiji Market, a visit to Sensoji Temple, lunch at a sushi restaurant in Ginza, a tour of the Imperial Palace, and dinner at a izakaya in Shinjuku.

[1237] Display of candidate sites and plans

[1238] 1. The server sends the generated plan to the terminal.

[1239] 2. The device displays the plan details to the user, who then reviews the plans and selects the one they like.

[1240] Real-time guidance

[1241] 1. On the day of travel, the user launches the app.

[1242] 2. The device loads the final plan saved on the server and works with the map application to display the current location and route.

[1243] 3. The user begins traveling along the specified route. The device updates the map information in real time and navigates to the next destination.

[1244] Emotion Engine Operation

[1245] 1. The user wears a device with voice guidance.

[1246] 2. The emotion engine recognizes the user's emotions and adjusts the tone and content of the guidance accordingly. For example, if the user is tired, the guidance will be gentler and suggest places to rest.

[1247] 3. The device prepares voice guidance information and starts guiding based on the user's current location and final plan. For example, instructions such as "Turn left next and go straight to Sensoji Temple" are provided.

[1248] Emotional Data Feedback

[1249] 1. The device collects the user's emotional data during the trip and sends it to the server.

[1250] 2. The server analyzes the collected emotional data, and the generative AI uses it to generate future plans.

[1251] Specific examples

[1252] User information entry and submission

[1253] The user starts the application and enters the following information:

[1254] Area I want to visit: Tokyo

[1255] Food I want to enjoy: Sushi

[1256] Interests: Historical places

[1257] Budget: 5,000 yen per day

[1258] The terminal sends this information to the server.

[1259] Generative AI generates candidate sites and plans

[1260] The server uses the above information to generate a plan like this:

[1261] Breakfast: Enjoy sushi at Tsukiji Market

[1262] Morning: Visit to Sensoji Temple

[1263] Lunch: Eat at a famous sushi restaurant in Ginza

[1264] Afternoon: Visit the Imperial Palace

[1265] Dinner: Savor local cuisine at a Shinjuku izakaya

[1266] Display of candidate sites and plans

[1267] The server sends the generated plan to the terminal, which displays it to the user, who then checks and selects the plan.

[1268] Real-time guidance

[1269] On the day of the trip, the user launches the app. The device loads the final plan stored on the server and displays the route in conjunction with the map application. Additionally, the user wears a voice guidance device that provides real-time, multilingual guidance.

[1270] Emotion Engine Operation

[1271] During the trip, the emotion engine will recognize the user's emotions and provide feedback accordingly. For example, if it detects that the user is tired, it will provide guidance such as "You can take a break here and refresh yourself at the cafe."

[1272] The present invention allows users to plan their trip efficiently and stress-free, and to enjoy their trip with peace of mind while receiving detailed, real-time guidance that responds to their emotions during their trip.

[1273] The processing flow will be explained below.

[1274] Step 1:

[1275] Users launch a travel app and enter information such as the area they want to visit, the food they want to enjoy, their interests, and their budget. For example, they might choose "Tokyo" as the area they want to visit, "sushi" as the food they want to enjoy, "historical places" as their interests, and "5,000 yen per day" as their budget.

[1276] Step 2:

[1277] The device sends the input information, including the place to visit, type of food, genre of interest, budget, etc., to the server.

[1278] Step 3:

[1279] Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best suit the user's preferences and conditions.

[1280] Step 4:

[1281] The server then sends the generated candidate locations and plans to the device, including information on multiple tourist spots, places to eat, and activities.

[1282] Step 5:

[1283] The device will display the details of the plans received to the user, who can then review the plans and select the one they like. At this time, they can also view detailed information, photos, and reviews for each plan.

[1284] Step 6:

[1285] The user selects the desired plan and taps the "Confirm" button. Once the selection is confirmed, the device sends the final plan to the server.

[1286] Step 7:

[1287] The server saves the selected plan as the final plan, which includes tourist spots to visit, places to eat, and routes to travel.

[1288] Step 8:

[1289] On the day of the trip, the user launches the app again and confirms their plan. The device loads the saved final plan and works with the map application to display their current location and route.

[1290] Step 9:

[1291] The user starts moving along the route specified by the user, and the device updates the map information in real time and navigates to the next destination.

[1292] Step 10:

[1293] The user wears a device with voice guidance, such as a pair of eyeglasses. The device prepares voice guidance information and begins guiding based on the user's current location and final plan. For example, instructions such as "Turn left next and go straight to Sensoji Temple" are provided.

[1294] Step 11:

[1295] The emotion engine uses a camera or microphone to recognize the user's emotions, for example, by analyzing facial expressions and tone of voice.

[1296] Step 12:

[1297] The emotion engine transmits the recognized emotion data to the device, such as whether the user is tired, happy, or anxious.

[1298] Step 13:

[1299] The device adjusts the tone and content of the guidance based on the emotional data it receives. For example, if it detects that the user is tired, it will soften the tone of the guidance and make suggestions such as, "You can take a break here and refresh yourself at the cafe."

[1300] Step 14:

[1301] The device will display additional information and guides as needed for each tourist attraction or dining spot during the trip, and its multilingual capabilities will provide guidance according to the user's language settings.

[1302] Step 15:

[1303] Once the user completes their travel plan, the device sends their travel history and emotional data to the server, where it is stored so that the user can access it again later.

[1304] Through these steps, users can plan their trip efficiently and stress-free, and enjoy their trip with peace of mind, receiving detailed, real-time, and emotionally-sensitive guidance during their trip.

[1305] Example 2

[1306] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1307] Conventional travel planning systems have difficulty reflecting a user's individual preferences and emotions in real time. As a result, they are unable to respond to changes in the user's situation or mood during the trip, resulting in a lack of flexibility in the plan. Furthermore, real-time guidance is limited to general map applications and voice guidance, making it difficult to meet the diverse needs of users.

[1308] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a user input means, a generation means for processing a travel plan request received from the user and generating optimal candidate travel destinations and plans, a display means for displaying the generated candidate destinations and plans to the user, a guidance means for providing real-time guidance based on the plan selected by the user, and an emotion engine for recognizing the user's emotions and adjusting the content of the plan and guidance accordingly. This makes it possible to provide a travel plan that reflects the user's individual preferences and emotions in real time and to provide flexible real-time guidance.

[1309] "User input means" refers to an interface through which a user inputs their travel planning requirements.

[1310] The "generation means" is a means for generating optimal travel destinations and plans based on travel plan requests received from users.

[1311] The "display means" is a means for visually presenting the generated travel destinations and plans to the user.

[1312] The "guidance means" is a means for providing real-time guidance based on the plan selected by the user.

[1313] The "emotion engine" is an engine that recognizes the user's emotions and adjusts the content of the plan or guidance accordingly.

[1314] "Map Application" means software used to display travel routes to a user.

[1315] A "voice-guided device" is a device used to provide real-time voice guidance.

[1316] This invention relates to a system that generates optimal travel destinations and plans based on a user's travel planning requests and provides real-time guidance on the day of the trip. This system incorporates an emotion engine that recognizes the user's emotions and provides feedback accordingly. Specific embodiments of this system are described below.

[1317] System configuration

[1318] The system mainly consists of the following components:

[1319] 1. User Input Method

[1320] 2. Generation means

[1321] 3. Display means

[1322] 4. Guidance

[1323] 5. Emotion Engine

[1324] User Input Method

[1325] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[1326] Specifically, the user launches the application and enters the following information:

[1327] Area I want to visit: Tokyo

[1328] Food I want to enjoy: Sushi

[1329] Interests: Historical places

[1330] Budget: 5,000 yen per day

[1331] generation means

[1332] The generation means is located on the server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions. For example, the following specific plans may be generated:

[1333] Breakfast: Enjoy sushi at Tsukiji Market

[1334] Morning: Visit to Sensoji Temple

[1335] Lunch: Eat at a famous sushi restaurant in Ginza

[1336] Afternoon: Visit the Imperial Palace

[1337] Dinner: Savor local cuisine at a Shinjuku izakaya

[1338] Display means

[1339] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[1340] Guidance means

[1341] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with a map application and has the function of displaying routes to the user. It also provides multilingual audio guidance using a device with audio guidance. This device is expected to be a glasses-type device worn by the user.

[1342] For example, on the day of a trip, when a user launches the app, the device loads the final plan saved on the server and displays the route in conjunction with the map application. As the user begins to move, the device updates the map information in real time and navigates to the next destination.

[1343] Emotion Engine

[1344] The emotion engine recognizes the user's emotions and adjusts the tone and content of the guidance based on those emotions. It works in conjunction with the user input means, generation means, display means, and guidance means, optimizing the system's operation based on the user's emotional data.

[1345] Specifically, when a user wears a device with voice guidance, the emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. For example, if the user is tired, the engine will provide guidance such as, "You can take a break here and refresh yourself at the cafe." In this way, it is possible to provide appropriate feedback according to the user's emotions.

[1346] This system allows users to plan their trip efficiently and stress-free, and they can enjoy their trip with peace of mind while receiving detailed, real-time guidance that responds to their emotions. Furthermore, the collection and analysis of emotion data will enable the system to provide even more accurate travel plans.

[1347] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1348] Step 1: Enter your user information

[1349] Users launch the application and enter information such as the area they want to visit, the food they want to enjoy, their interests, and their budget. For example, they can enter information such as "Tokyo," "sushi," "historical place," and "5,000 yen per day." The entered data is temporarily stored on the device.

[1350] Input: User's travel plan request (e.g. Tokyo, sushi, historical places, budget 5000 yen)

[1351] Output: Input data saved on the device

[1352] Step 2: Submit user information

[1353] The device sends the entered information to the server, where each item is converted into an appropriate format and a data format such as JSON is generated.

[1354] Input: Input data stored on the device

[1355] Output: User's travel plan request data sent to the server

[1356] Step 3: Receiving and analyzing information

[1357] The server receives user information sent from the device. The received data is analyzed by an analysis engine to identify the user's preferences and conditions. The analysis results are used to generate the next plan.

[1358] Input: User data sent to the server

[1359] Output: User preferences and conditions identified by the analytics engine

[1360] Step 4: Generate your travel plan

[1361] The AI ​​on the server generates optimal travel destinations and plans based on the analyzed user preferences and conditions. The AI ​​refers to existing travel databases and reviews to suggest multiple combinations of destinations and activities.

[1362] Input: User preferences and conditions identified by the analytics engine

[1363] Output: Data for generating optimal travel destinations and plans

[1364] Step 5: Submit and view your plan

[1365] The server sends the generated travel plan to the terminal, which visually displays the received plan for the user to check.

[1366] Input: Data for generating optimal travel destinations and plans

[1367] Output: Travel plan displayed on the device

[1368] Step 6: User chooses plan

[1369] The user selects the plan they like from the ones presented, and the selected plan is sent to the server via the device and saved as the final plan.

[1370] Input: Travel plan displayed on the device

[1371] Output: The final plan sent to the server

[1372] Step 7: Start real-time guidance

[1373] On the day of the trip, the user launches the app again, and the device loads the final plan saved on the server. The loaded plan is linked to the map application, and a route is displayed based on the user's current location. The user then begins traveling according to this route.

[1374] Input: Final plan saved on the server

[1375] Output: Display of travel route by linking with map application

[1376] Step 8: Emotion Engine Feedback

[1377] The user wears a device with voice guidance. The emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. Based on the recognized emotion, the tone and content of the guidance are adjusted and appropriate feedback is provided.

[1378] Input: User's facial expressions and tone of voice data

[1379] Output: Adjusted announcement tone and content

[1380] Step 9: Collect and analyze emotion data

[1381] The device continuously collects emotional data from the user while traveling. The collected data is sent to a server, where it is analyzed by an analysis engine. The analysis results are used by the AI ​​to generate travel plans for future trips.

[1382] Input: User emotion data during travel

[1383] Output: Analysis results of emotional data fed back to the generative AI

[1384] (Application example 2)

[1385] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1386] While itinerary optimization and real-time guidance have become commonplace, recognizing users' emotions and dynamically adjusting entertainment and guidance content based on that data is crucial to further enriching the travel experience. However, current systems lack a means to effectively recognize users' emotions and incorporate them into itinerary planning and real-time guidance. This poses a challenge, making it difficult for users to truly relax and enjoy their trip.

[1387] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes user input means, generation means for processing travel planning requests received from the user and generating optimal candidate travel destinations and plans, display means for displaying the generated candidate destinations and plans to the user, guidance means for providing real-time guidance based on the plan selected by the user, emotion engine means for recognizing the user's emotions using sensor means installed in the vehicle and processing the data, and entertainment control means for dynamically adjusting the content and tone of entertainment based on the recognized emotion data. This makes it possible to customize the travel experience according to the user's emotions.

[1388] The "user input means" is an interface through which a user inputs their travel planning requests, and is implemented as an application installed on a mobile device such as a smartphone or tablet.

[1389] The "generation means" has the function of processing a travel plan request received from a user input means and generating optimal travel destinations and plans.

[1390] The "display means" is a device that visually presents the travel destinations and plans generated by the generation means to the user, and operates on the same device as the user input means.

[1391] The "guidance means" provides real-time guidance based on the plan selected by the user and has the function of displaying the route in cooperation with a map application.

[1392] The "emotion engine means" is a part of the system that uses sensor means installed in the vehicle to recognize the user's emotions and process the data.

[1393] An "entertainment control means" is a means that dynamically adjusts the content and tone of entertainment based on recognized emotional data.

[1394] A "map application" is software that provides geographic location information, displays a user's travel route, and provides navigation.

[1395] A "device with voice guidance" is a device that provides real-time multilingual voice guidance.

[1396] The present invention provides a system for optimizing a user's travel plan and providing real-time guidance, particularly for autonomous vehicles. The system includes a user input means, a generation means, a display means, a guidance means, an emotion engine means, and an entertainment control means.

[1397] System configuration

[1398] User Input Method

[1399] The user input means is an interface through which users input their travel plan requests. For example, it is implemented as an application installed on a mobile device such as a smartphone or tablet. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[1400] generation means

[1401] The generation means is located on a server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[1402] Display means

[1403] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[1404] Guidance means

[1405] The guidance means provides real-time guidance based on the final plan selected by the user. For example, it works with a map application to display a route based on the current location information of the autonomous vehicle. It also provides real-time voice guidance in multiple languages ​​using a device with voice guidance.

[1406] Emotion Engine Means

[1407] The emotion engine uses sensors installed in the vehicle to recognize the user's emotions and process the data. This recognition process uses facial recognition cameras and voice analysis microphones. For example, Intel RealSense Depth Cameras and Shure SM7B microphones are considered. The emotion data is sent to a server and used to optimize the operation of the entire system.

[1408] Entertainment Controls

[1409] The entertainment control means dynamically adjusts the content and tone of entertainment based on the recognized emotion data. For example, it controls the playback of music streaming services and video content. Examples of software used include the Spotify API and YouTube API.

[1410] Specific examples

[1411] User information entry and submission

[1412] The user starts the application and enters the following information:

[1413] Area I want to visit: Tokyo

[1414] Food I want to enjoy: Sushi

[1415] Interests: Historical places

[1416] Budget: 5,000 yen per day

[1417] This information is sent to the server via the terminal.

[1418] Generative AI generates candidate sites and plans

[1419] Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best fit the user's preferences and conditions. For example, a plan might be generated that includes breakfast at Tsukiji Market, a visit to Sensoji Temple, lunch at a sushi restaurant in Ginza, a tour of the Imperial Palace, and dinner at a izakaya in Shinjuku.

[1420] Plan display and real-time guidance

[1421] The server sends the generated plan to the device, which displays it to the user. The user can review and select the plan. On the day of the trip, the device loads the final plan saved on the server and displays the route in conjunction with a map application. In addition, a voice guidance device worn by the user provides real-time multilingual guidance.

[1422] Emotion engine and entertainment tuning

[1423] During the trip, the emotion engine will recognize the user's emotions and provide feedback accordingly. For example, if it detects that the user is tired, it will provide guidance such as "You can take a break here and refresh yourself at the cafe."

[1424] Prompt Sentence Examples

[1425] An example prompt is:

[1426] The user provided the following information:

[1427] Area I want to visit: Tokyo

[1428] Food I want to enjoy: Sushi

[1429] Interests: Historical places

[1430] Budget: 5,000 yen per day

[1431] Based on this, suggest the best travel plan.

[1432] Thus, the present invention is designed to enhance a user's travel experience, providing real-time customized guidance and entertainment based on the user's emotions.

[1433] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1434] Step 1:

[1435] The user launches the application and inputs their travel plan requirements. The input information includes areas they want to visit, foods they want to enjoy, genres they are interested in, budget, etc. Input: User's travel plan requirements data. Output: User's input information stored on the device.

[1436] Step 2:

[1437] The terminal sends the information entered by the user to the server. Input: User input information stored in the terminal. Output: User travel plan request data sent to the server.

[1438] Step 3:

[1439] The server analyzes the received user information and uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans based on the user's preferences and conditions. Input: User's travel plan request data sent to the server. Output: Optimal travel destinations and plans.

[1440] Step 4:

[1441] The server sends the generated plan to the terminal. Input: Generated travel plan. Output: Optimal travel plan data sent to the terminal.

[1442] Step 5:

[1443] The terminal displays the plan sent to the user. The user selects the desired plan from the displayed candidates. Input: Optimal travel plan data sent from the server. Output: Travel plan displayed to the user and the user's selection information.

[1444] Step 6:

[1445] On the day of the trip, when the user restarts the app, the device loads the saved final plan from the server and works with the map application to display a route based on the current location. Input: Final plan data saved on the server. Output: Route displayed in the map application.

[1446] Step 7:

[1447] The emotion engine recognizes the user's emotions through sensing devices (cameras and microphones) installed in the vehicle. Input: Data on the user's facial expressions and voice. Output: Recognized emotion data.

[1448] Step 8:

[1449] The terminal processes the recognized emotion data, and the entertainment control means dynamically adjusts the content and tone of the entertainment according to the user's emotion. Input: Recognized emotion data. Output: Adjusted entertainment content.

[1450] Step 9:

[1451] The terminal voice guidance device provides multilingual voice guidance and feedback to the user in real time. Input: Travel route data and final plan selected. Output: Voice guidance and feedback provided to the user.

[1452] The system allows users to optimize their trip planning and enjoy entertainment and navigation that responds to their emotions in real time.

[1453] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1454] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1455] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1456] [Fourth embodiment]

[1457] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1458] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1459] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1460] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1461] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1462] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1463] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1464] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1465] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1466] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1467] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1468] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1469] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1470] The present invention is a system that generates optimal travel destinations and plans based on travel planning requests and provides real-time guidance on the day of travel. Specific embodiments of this system are described below.

[1471] System configuration

[1472] The system mainly consists of the following components:

[1473] 1. User Input Method

[1474] 2. Generation means

[1475] 3. Display means

[1476] 4. Guidance

[1477] User Input Method

[1478] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[1479] generation means

[1480] The generation means is located on a server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[1481] Display means

[1482] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[1483] Guidance means

[1484] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with a map application and has the function of displaying routes to the user. It also provides multilingual audio guidance using a device with audio guidance. This device is expected to be a glasses-type device worn by the user.

[1485] Program processing

[1486] User information entry and submission

[1487] 1. The user launches the application and enters their travel plan requirements, such as "Tokyo" as the area they want to visit, "sushi" as the food they want to enjoy, "historical places" as their interest, and "5,000 yen per day" as their budget.

[1488] 2. The terminal sends the entered information to the server.

[1489] Generative AI generates candidate sites and plans

[1490] 1. Based on the received information, the server uses generative AI to generate optimal travel destinations and plans. For example, a plan might be generated that includes breakfast at Tsukiji Market, a visit to Sensoji Temple, lunch at a sushi restaurant in Ginza, a tour of the Imperial Palace, and dinner at a pub in Shinjuku.

[1491] Display of candidate sites and plans

[1492] 1. The server sends the generated plan to the terminal.

[1493] 2. The device will display the plan details to the user, who can then review each plan and select the one they like best.

[1494] Real-time guidance

[1495] 1. On the day of travel, the user launches the app.

[1496] 2. The device loads the final plan saved on the server and displays the route in conjunction with the map application.

[1497] 3. The terminal works with a device that supports voice guidance to provide voice guidance, such as "Turn left next and go straight to Sensoji Temple."

[1498] Specific examples

[1499] User information entry and submission

[1500] The user starts the application and enters the following information:

[1501] Area I want to visit: Tokyo

[1502] Food I want to enjoy: Sushi

[1503] Interests: Historical places

[1504] Budget: 5,000 yen per day

[1505] The terminal sends this information to the server.

[1506] Generative AI generates candidate sites and plans

[1507] The server uses the above information to generate a plan like this:

[1508] Breakfast: Enjoy sushi at Tsukiji Market

[1509] Morning: Visit to Sensoji Temple

[1510] Lunch: Eat at a famous sushi restaurant in Ginza

[1511] Afternoon: Visit the Imperial Palace

[1512] Dinner: Savor local cuisine at a Shinjuku izakaya

[1513] Display of candidate sites and plans

[1514] The server sends the generated plan to the terminal, which displays it to the user, who then checks and selects the plan.

[1515] Real-time guidance

[1516] On the day of the trip, the user launches the app. The device loads the final plan stored on the server and displays the route in conjunction with the map application. Additionally, the user wears a voice guidance device that provides real-time, multilingual guidance.

[1517] The present invention allows users to plan their trip efficiently and stress-free, and to act with peace of mind while traveling.

[1518] The processing flow will be explained below.

[1519] Step 1:

[1520] Users launch the travel app and enter information such as the area they want to visit, the food they want to enjoy, the genres they are interested in, their budget, etc. Once they have finished entering the information, they tap the "Submit" button.

[1521] Step 2:

[1522] The device sends the input information, including the place to visit, type of food, genre of interest, budget, etc., to the server.

[1523] Step 3:

[1524] Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best suit the user's preferences and conditions.

[1525] Step 4:

[1526] The server then sends the generated candidate locations and plans to the device, including information on multiple tourist spots, places to eat, and activities.

[1527] Step 5:

[1528] The device will display the details of the plans received to the user, who can then review the plans and select the one they like. At this time, they can also view detailed information, photos, and reviews for each plan.

[1529] Step 6:

[1530] The user selects the desired plan and taps the "Confirm" button. Once the selection is confirmed, the device sends the final plan to the server.

[1531] Step 7:

[1532] The server saves the selected plan as the final plan, which includes tourist spots to visit, places to eat, and routes to travel.

[1533] Step 8:

[1534] On the day of the trip, the user launches the app again and confirms their plan. The device loads the saved final plan and works with the map application to display their current location and route.

[1535] Step 9:

[1536] The user starts moving along the route specified by the user, and the device updates the map information in real time and navigates to the next destination.

[1537] Step 10:

[1538] The user wears the device with voice guidance. The device prepares the voice guidance information and starts guiding based on the user's current location and final plan. For example, instructions such as "Turn left next and go straight to Sensoji Temple" are provided.

[1539] Step 11:

[1540] The device will display additional information and guides as needed for each tourist attraction or dining spot during the trip, and its multilingual capabilities will provide guidance according to the user's language settings.

[1541] Step 12:

[1542] The user completes their travel plan. The device sends the travel history to the server, where it stores the data for later access.

[1543] By following these steps, users can plan their trip efficiently and stress-free, and enjoy their trip with peace of mind, receiving detailed real-time guidance during their trip.

[1544] Example 1

[1545] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1546] Conventional travel planning systems have difficulty generating optimal travel plans based on user requests. They also lack efficient means for providing real-time guidance. As a result, users spend a great deal of time and effort planning and are unable to receive sufficient guidance during their trip.

[1547] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1548] In this invention, the server includes a generation unit that processes travel plan requests received from users and generates optimal travel destinations and plans, a generation unit that references an existing database and inputs prompts into a generative AI model to generate a travel plan, and a unit that displays the generated travel destinations and plans to the user, thereby enabling users to efficiently and accurately plan their travels and receive detailed guidance in real time.

[1549] "User input means" refers to an interface through which a user inputs their travel planning requirements.

[1550] The "generation means" is a means for processing a travel plan request received from a user and generating optimal travel destinations and plans.

[1551] "Means of referencing existing databases" refers to means of obtaining data that meets the user's needs by using a database that holds travel-related information.

[1552] "Means for inputting prompt sentences into a generative AI model" refers to a means for inputting prompt sentences based on the user's requests into a generative AI model and generating an optimal travel plan.

[1553] The "display means" is a means for visually presenting the generated travel destinations and plans to the user.

[1554] The "guidance means" is a means for providing real-time guidance based on the plan selected by the user.

[1555] The "means for linking with a map application" refers to a means for communicating with a map application and displaying information in order to display a travel route to the user.

[1556] A "device with voice guidance" is a device that provides voice guidance and is capable of providing multilingual voice guidance in real time.

[1557] The present invention is a system that generates optimal travel destinations and plans based on travel planning requests and provides real-time guidance on the day of travel. Specific embodiments of this system are described below.

[1558] System configuration

[1559] The system mainly consists of the following components:

[1560] 1. User Input Method

[1561] 2. Generation means

[1562] 3. Display means

[1563] 4. Guidance

[1564] User Input Method

[1565] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, their interests, and their budget. For example, a user inputs detailed information such as "areas they want to visit: Kyoto," "foods they want to enjoy: Japanese sweets," "interests: gardens," and "budget: 7,000 yen per day."

[1566] generation means

[1567] The generation means is placed on the server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to an existing database (for example, the API of a travel information service) and proposes a plan that best suits the user's preferences and conditions. The server inputs a prompt statement into the generation AI model to generate a travel plan. For example, the prompt statement could be, "The user has inputted 'Kyoto' as the area they would like to visit, 'Japanese sweets' as the food they would like to enjoy, 'gardens' as the genre they are interested in, and '7,000 yen per day' as their budget. Please generate the optimal travel plan based on this information."

[1568] Display means

[1569] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[1570] Guidance means

[1571] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with map applications (such as Google Maps or Apple Maps) and has the function of displaying routes to the user. It also provides audio guidance using a device with audio guidance (such as a multilingual audio guidance device). This device is expected to be a glasses-type device worn by the user. For example, it provides specific audio guidance such as, "Turn left next and go straight to Sensoji Temple."

[1572] In this way, users can plan their trips efficiently and optimally, and receive detailed real-time guidance during their trip.

[1573] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1574] Step 1:

[1575] Users launch the mobile app and input their travel plan requirements, including the areas they want to visit, the foods they want to enjoy, their interests, and their budget. This information is then sent to the device as input data.

[1576] Step 2:

[1577] The device sends the collected user input data to the server. The data is kept safe using a secure communication protocol (e.g., HTTPS). For example, the input data might be "Area I want to visit: Kyoto," "Food I want to enjoy: Japanese sweets," "Interested genre: gardens," or "Budget: 7,000 yen per day."

[1578] Step 3:

[1579] The server processes the received input data. Based on this data, it searches for related travel spots and information from an existing database. Next, it inputs a prompt statement into the generative AI model. An example of a prompt statement is: "The user has entered 'Kyoto' as the area they would like to visit, 'Japanese sweets' as the food they would like to enjoy, 'gardens' as a genre they are interested in, and '7,000 yen per day' as their budget. Please generate the optimal travel plan based on this information."

[1580] Step 4:

[1581] The generative AI model calculates data based on the prompt text and generates an optimal travel plan. Examples of generated plans include "Enjoy Japanese sweets at a sweet shop," "Visit Kinkakuji Temple," "Have lunch in Gion," "Tour of Otawara Pond Garden," and "Stroll along the Kamo River." This plan becomes the output data.

[1582] Step 5:

[1583] The server formats the generated travel plan and sends it to the device. The output data is formatted in JSON or XML format and provided to the device.

[1584] Step 6:

[1585] The device analyzes the received travel plans and visually displays them to the user, including detailed time schedules, maps, and user reviews for each plan. The user can then review these and select the plan they want.

[1586] Step 7:

[1587] The user selects and confirms the travel plan they like, and the device notifies the server of the selected plan as the final plan.

[1588] Step 8:

[1589] On the day of the trip, the user launches the app. The device loads the final plan stored on the server and uses GPS to identify the user's current location. It then connects to a map application (e.g., Google Maps) to display the route and begin guiding the trip.

[1590] Step 9:

[1591] The terminal works in conjunction with a voice guidance device to provide real-time voice guidance, such as "Turn left next and go straight to Sensoji Temple," allowing users to carry out efficient and optimal travel planning.

[1592] (Application example 1)

[1593] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1594] Conventional travel planning systems have had many problems with planning trips and providing real-time guidance during the trip. It is difficult to easily create an optimal plan tailored to the user's needs, and guidance is particularly inadequate in self-driving vehicles. Furthermore, there is a lack of systems that provide real-time navigation and guidance on-site, leading to users often getting lost. Furthermore, voice guidance is often not multilingual, which is extremely inconvenient for foreign tourists. The present invention aims to solve these problems.

[1595] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1596] In this invention, the server includes an input means for receiving a travel plan request from a user, a means for processing the travel plan request received from the input means and generating optimal travel destinations and plans using a generative AI model, a means for displaying the generated travel destinations and plans to the user, and a means for providing real-time guidance based on the plan selected by the user. This allows the user to enjoy a comfortable trip in an autonomous vehicle and receive real-time guidance on routes and tourist spots. Furthermore, multilingual audio guidance allows users to efficiently enjoy their trip beyond language barriers.

[1597] "Travel planning requests" are information including the user's desired travel destinations, activities of interest, and budget.

[1598] The "input means" is an interface for receiving travel planning requests from users, and is an application implemented on a smartphone or in-vehicle infotainment system.

[1599] A "generative AI model" is an artificial intelligence technology that automatically generates optimal travel destinations and plans based on travel planning requests received from users.

[1600] The "means for generating" is a processing device that uses a generative AI model to create optimal travel destinations and plans based on the user's requests.

[1601] The "display means" refers to a device for visually presenting the generated travel destinations and plans to the user, such as an in-car infotainment system or a smartphone display.

[1602] The "guidance means" is a device that provides real-time route and destination guidance based on a plan selected by the user.

[1603] A "navigation system" is a system that uses GPS data and map applications to display the route from the user's current location to their destination in real time.

[1604] A "voice guidance system" is a system that uses multilingual voice synthesis technology to provide users with real-time voice guidance on travel and tourist spots.

[1605] System configuration

[1606] The system of the present invention comprises the following components:

[1607] 1. Input Method

[1608] The input means of this system is an interface for receiving travel planning requests from users. It is provided as an application installed on smartphones or in-car infotainment systems. Users input information such as the areas they want to visit, activities they want to enjoy, places of interest, and their budget.

[1609] 2. Means of generation

[1610] The generation means runs on the server. Based on requests sent from the input means, the generative AI model generates optimal travel destinations and plans. The generative AI model refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[1611] 3. Display method

[1612] The display means is used to visually present the generated travel candidate destinations and plans to the user. It runs on the same device as the input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[1613] 4. Guidance

[1614] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with the navigation system and has the function of displaying routes to the user. It also provides real-time voice guidance using a multilingual voice guidance system. The voice guidance system effectively provides guidance through a device worn by the user (for example, a speaker built into the in-vehicle infotainment system).

[1615] Specific explanation of the system's operation

[1616] The operation of this system will be explained in detail below, showing how each element of the system works together.

[1617] 1. User information entry and submission

[1618] The user launches the application and inputs their travel plan requirements, for example, selecting "Kyoto" as the area they want to visit, "sightseeing" as the activity they want to enjoy, "historical places" as their points of interest, and "7,000 yen per day" as their budget. This information is then sent to the server by the device.

[1619] 2. Generating candidate sites and plans using a generative AI model

[1620] Based on the received information, the server uses a generative AI model to generate optimal travel destinations and plans. For example, a plan including tourist spots such as Kiyomizu-dera Temple, Kinkaku-ji Temple, and Nijo Castle may be generated.

[1621] 3. Display of candidate sites and plans

[1622] The server sends the generated plans to the device, which displays the plan details to the user, allowing the user to review each plan and select the one they like best.

[1623] 4. Real-time guidance

[1624] On the day of the trip, the user launches the app. The device loads the final plan saved on the server and displays the route in conjunction with the navigation system. It also links with the voice guidance system to provide real-time voice guidance in multiple languages. For example, the app may provide voice guidance such as, "Turn left next and go straight to Kiyomizu-dera Temple."

[1625] Specific examples

[1626] User information entry and submission

[1627] The user starts the application and enters the following information:

[1628] Area I want to visit: Kyoto

[1629] Activities to enjoy: Sightseeing

[1630] Places of interest: Historical places

[1631] Budget: 7,000 yen per day

[1632] The terminal sends this information to the server.

[1633] Generative AI model generates candidate sites and plans

[1634] The server uses the above information to generate a plan like this:

[1635] Touring sightseeing spots: Kiyomizu-dera Temple, Kinkaku-ji Temple, Nijo Castle

[1636] Lunch: Restaurant to enjoy local cuisine

[1637] Guide:Transportation to accommodation

[1638] Real-time guidance

[1639] On the day of the trip, the user launches the app in the autonomous vehicle. The device loads the final plan stored on the server and displays the route in conjunction with the navigation system. In addition, the voice guidance system provides real-time, multilingual guidance.

[1640] Prompt Sentence Examples

[1641] "Area I want to visit: Kyoto, Activities I'm interested in: Sightseeing, historical places, Budget: 7,000 yen per day"

[1642] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1643] Step 1:

[1644] Enter the user's travel planning requests

[1645] The user launches the application and inputs specific travel planning requirements (areas to visit, activities of interest, historical locations, budget, etc.) The input data is stored on the device via the smartphone or in-car infotainment system application.

[1646] input:

[1647] Area I want to visit: Kyoto

[1648] Activities to enjoy: Sightseeing

[1649] Places of interest: Historical places

[1650] Budget: 7,000 yen per day

[1651] output:

[1652] User travel planning information data

[1653] Step 2:

[1654] Send the user's travel plan information data to the server

[1655] The terminal transmits the user's travel plan information data entered in step 1 to the server. The data is transmitted using a secure communication channel.

[1656] input:

[1657] User travel planning information data

[1658] output:

[1659] User travel plan information data stored on the server

[1660] Step 3:

[1661] Generate optimal travel destinations and plans using generative AI models

[1662] The server processes the received user travel plan information data and uses a generative AI model to generate optimal travel destinations and plans. The AI ​​model references existing travel databases and reviews to create the optimal plan based on the user's requests.

[1663] input:

[1664] User travel plan information data stored on the server

[1665] output:

[1666] Generated travel destination and plan data

[1667] Step 4:

[1668] Displaying generated travel destinations and plans to users

[1669] The server sends the generated travel destination and plan data to the device, which receives it and visually displays it to the user. The user can then check the various options on the display screen and select the plan that best suits their needs.

[1670] input:

[1671] Generated travel destination and plan data

[1672] output:

[1673] A list of plan options displayed to the user

[1674] Step 5:

[1675] User selects final plan

[1676] The user selects the plan they like from the displayed list of plan candidates, and the selected plan is sent to the server by the terminal.

[1677] input:

[1678] User-selected plan data

[1679] output:

[1680] Final plan data sent to the server

[1681] Step 6:

[1682] Real-time information on the day of travel

[1683] On the day of the trip, the user launches the app. The device loads the final plan stored on the server, and displays the route in conjunction with the navigation system. It also works with the voice guidance system to provide real-time voice guidance in multiple languages.

[1684] input:

[1685] Final plan data stored on the server

[1686] output:

[1687] Navigation system showing route

[1688] Real-time multilingual voice guidance

[1689] Supplementary information on specific actions

[1690] In step 1, the user enters the necessary information via a smartphone or in-car infotainment system application. In step 2, this information is sent to the server via the internet. In step 3, the server uses this data to generate prompts for the generative AI model and builds an optimal travel plan. In steps 4 and 5, the server sends the plan it has generated to the device, where the user can confirm and select. Step 6 provides real-time guidance on the day of the trip, with navigation and voice guidance based on the user's location information.

[1691] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1692] The present invention is a system that generates optimal travel destinations and plans based on travel planning requests and provides real-time guidance on the day of travel. This system incorporates an emotion engine that recognizes the user's emotions and provides feedback accordingly. Specific embodiments of this system are described below.

[1693] System configuration

[1694] The system mainly consists of the following components:

[1695] 1. User Input Method

[1696] 2. Generation means

[1697] 3. Display means

[1698] 4. Guidance

[1699] 5. Emotion Engine

[1700] User Input Method

[1701] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[1702] generation means

[1703] The generation means is located on a server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[1704] Display means

[1705] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[1706] Guidance means

[1707] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with a map application and has the function of displaying routes to the user. It also provides multilingual audio guidance using a device with audio guidance. This device is expected to be a glasses-type device worn by the user.

[1708] Emotion Engine

[1709] The emotion engine recognizes the user's emotions and adjusts the content of the plan and guidance accordingly. It works in conjunction with the user input means, generation means, display means, and guidance means, and optimizes the system's operation based on the user's emotional data.

[1710] Program processing

[1711] User information entry and submission

[1712] 1. The user launches the application and enters information such as the area they want to visit, the food they want to enjoy, their interests, and their budget. For example, the area they want to visit might be "Tokyo," the food they want to enjoy might be "sushi," their interests might be "historical places," and their budget might be "5,000 yen per day."

[1713] 2. The terminal sends the entered information to the server.

[1714] Generative AI generates candidate sites and plans

[1715] 1. Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best fit the user's preferences and conditions. For example, a plan might be generated that includes breakfast at Tsukiji Market, a visit to Sensoji Temple, lunch at a sushi restaurant in Ginza, a tour of the Imperial Palace, and dinner at a izakaya in Shinjuku.

[1716] Display of candidate sites and plans

[1717] 1. The server sends the generated plan to the terminal.

[1718] 2. The device displays the plan details to the user, who then reviews the plans and selects the one they like.

[1719] Real-time guidance

[1720] 1. On the day of travel, the user launches the app.

[1721] 2. The device loads the final plan saved on the server and works with the map application to display the current location and route.

[1722] 3. The user begins traveling along the specified route. The device updates the map information in real time and navigates to the next destination.

[1723] Emotion Engine Operation

[1724] 1. The user wears a device with voice guidance.

[1725] 2. The emotion engine recognizes the user's emotions and adjusts the tone and content of the guidance accordingly. For example, if the user is tired, the guidance will be gentler and suggest places to rest.

[1726] 3. The device prepares voice guidance information and starts guiding based on the user's current location and final plan. For example, instructions such as "Turn left next and go straight to Sensoji Temple" are provided.

[1727] Emotional Data Feedback

[1728] 1. The device collects the user's emotional data during the trip and sends it to the server.

[1729] 2. The server analyzes the collected emotional data, and the generative AI uses it to generate future plans.

[1730] Specific examples

[1731] User information entry and submission

[1732] The user starts the application and enters the following information:

[1733] Area I want to visit: Tokyo

[1734] Food I want to enjoy: Sushi

[1735] Interests: Historical places

[1736] Budget: 5,000 yen per day

[1737] The terminal sends this information to the server.

[1738] Generative AI generates candidate sites and plans

[1739] The server uses the above information to generate a plan like this:

[1740] Breakfast: Enjoy sushi at Tsukiji Market

[1741] Morning: Visit to Sensoji Temple

[1742] Lunch: Eat at a famous sushi restaurant in Ginza

[1743] Afternoon: Visit the Imperial Palace

[1744] Dinner: Savor local cuisine at a Shinjuku izakaya

[1745] Display of candidate sites and plans

[1746] The server sends the generated plan to the terminal, which displays it to the user, who then checks and selects the plan.

[1747] Real-time guidance

[1748] On the day of the trip, the user launches the app. The device loads the final plan stored on the server and displays the route in conjunction with the map application. Additionally, the user wears a voice guidance device that provides real-time, multilingual guidance.

[1749] Emotion Engine Operation

[1750] During the trip, the emotion engine will recognize the user's emotions and provide feedback accordingly. For example, if it detects that the user is tired, it will provide guidance such as "You can take a break here and refresh yourself at the cafe."

[1751] The present invention allows users to plan their trip efficiently and stress-free, and to enjoy their trip with peace of mind while receiving detailed, real-time guidance that responds to their emotions during their trip.

[1752] The processing flow will be explained below.

[1753] Step 1:

[1754] Users launch a travel app and enter information such as the area they want to visit, the food they want to enjoy, their interests, and their budget. For example, they might choose "Tokyo" as the area they want to visit, "sushi" as the food they want to enjoy, "historical places" as their interests, and "5,000 yen per day" as their budget.

[1755] Step 2:

[1756] The device sends the input information, including the place to visit, type of food, genre of interest, budget, etc., to the server.

[1757] Step 3:

[1758] Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best suit the user's preferences and conditions.

[1759] Step 4:

[1760] The server then sends the generated candidate locations and plans to the device, including information on multiple tourist spots, places to eat, and activities.

[1761] Step 5:

[1762] The device will display the details of the plans received to the user, who can then review the plans and select the one they like. At this time, they can also view detailed information, photos, and reviews for each plan.

[1763] Step 6:

[1764] The user selects the desired plan and taps the "Confirm" button. Once the selection is confirmed, the device sends the final plan to the server.

[1765] Step 7:

[1766] The server saves the selected plan as the final plan, which includes tourist spots to visit, places to eat, and routes to travel.

[1767] Step 8:

[1768] On the day of the trip, the user launches the app again and confirms their plan. The device loads the saved final plan and works with the map application to display their current location and route.

[1769] Step 9:

[1770] The user starts moving along the route specified by the user, and the device updates the map information in real time and navigates to the next destination.

[1771] Step 10:

[1772] The user wears a device with voice guidance, such as a pair of eyeglasses. The device prepares voice guidance information and begins guiding based on the user's current location and final plan. For example, instructions such as "Turn left next and go straight to Sensoji Temple" are provided.

[1773] Step 11:

[1774] The emotion engine uses a camera or microphone to recognize the user's emotions, for example, by analyzing facial expressions and tone of voice.

[1775] Step 12:

[1776] The emotion engine transmits the recognized emotion data to the device, such as whether the user is tired, happy, or anxious.

[1777] Step 13:

[1778] The device adjusts the tone and content of the guidance based on the emotional data it receives. For example, if it detects that the user is tired, it will soften the tone of the guidance and make suggestions such as, "You can take a break here and refresh yourself at the cafe."

[1779] Step 14:

[1780] The device will display additional information and guides as needed for each tourist attraction or dining spot during the trip, and its multilingual capabilities will provide guidance according to the user's language settings.

[1781] Step 15:

[1782] Once the user completes their travel plan, the device sends their travel history and emotional data to the server, where it is stored so that the user can access it again later.

[1783] Through these steps, users can plan their trip efficiently and stress-free, and enjoy their trip with peace of mind, receiving detailed, real-time, and emotionally-sensitive guidance during their trip.

[1784] Example 2

[1785] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1786] Conventional travel planning systems have difficulty reflecting a user's individual preferences and emotions in real time. As a result, they are unable to respond to changes in the user's situation or mood during the trip, resulting in a lack of flexibility in the plan. Furthermore, real-time guidance is limited to general map applications and voice guidance, making it difficult to meet the diverse needs of users.

[1787] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a user input means, a generation means for processing a travel plan request received from the user and generating optimal candidate travel destinations and plans, a display means for displaying the generated candidate destinations and plans to the user, a guidance means for providing real-time guidance based on the plan selected by the user, and an emotion engine for recognizing the user's emotions and adjusting the content of the plan and guidance accordingly. This makes it possible to provide a travel plan that reflects the user's individual preferences and emotions in real time and to provide flexible real-time guidance.

[1788] "User input means" refers to an interface through which a user inputs their travel planning requirements.

[1789] The "generation means" is a means for generating optimal travel destinations and plans based on travel plan requests received from users.

[1790] The "display means" is a means for visually presenting the generated travel destinations and plans to the user.

[1791] The "guidance means" is a means for providing real-time guidance based on the plan selected by the user.

[1792] The "emotion engine" is an engine that recognizes the user's emotions and adjusts the content of the plan or guidance accordingly.

[1793] "Map Application" means software used to display travel routes to a user.

[1794] A "voice-guided device" is a device used to provide real-time voice guidance.

[1795] This invention relates to a system that generates optimal travel destinations and plans based on a user's travel planning requests and provides real-time guidance on the day of the trip. This system incorporates an emotion engine that recognizes the user's emotions and provides feedback accordingly. Specific embodiments of this system are described below.

[1796] System configuration

[1797] The system mainly consists of the following components:

[1798] 1. User Input Method

[1799] 2. Generation means

[1800] 3. Display means

[1801] 4. Guidance

[1802] 5. Emotion Engine

[1803] User Input Method

[1804] The user input means is an interface through which users input their travel plan requests. This means is implemented as an application installed on mobile devices such as smartphones and tablets. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[1805] Specifically, the user launches the application and enters the following information:

[1806] Area I want to visit: Tokyo

[1807] Food I want to enjoy: Sushi

[1808] Interests: Historical places

[1809] Budget: 5,000 yen per day

[1810] generation means

[1811] The generation means is located on the server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions. For example, the following specific plans may be generated:

[1812] Breakfast: Enjoy sushi at Tsukiji Market

[1813] Morning: Visit to Sensoji Temple

[1814] Lunch: Eat at a famous sushi restaurant in Ginza

[1815] Afternoon: Visit the Imperial Palace

[1816] Dinner: Savor local cuisine at a Shinjuku izakaya

[1817] Display means

[1818] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[1819] Guidance means

[1820] The guidance means provides real-time guidance based on the final plan selected by the user. It works in conjunction with a map application and has the function of displaying routes to the user. It also provides multilingual audio guidance using a device with audio guidance. This device is expected to be a glasses-type device worn by the user.

[1821] For example, on the day of a trip, when a user launches the app, the device loads the final plan saved on the server and displays the route in conjunction with the map application. As the user begins to move, the device updates the map information in real time and navigates to the next destination.

[1822] Emotion Engine

[1823] The emotion engine recognizes the user's emotions and adjusts the tone and content of the guidance based on those emotions. It works in conjunction with the user input means, generation means, display means, and guidance means, optimizing the system's operation based on the user's emotional data.

[1824] Specifically, when a user wears a device with voice guidance, the emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. For example, if the user is tired, the engine will provide guidance such as, "You can take a break here and refresh yourself at the cafe." In this way, it is possible to provide appropriate feedback according to the user's emotions.

[1825] This system allows users to plan their trip efficiently and stress-free, and they can enjoy their trip with peace of mind while receiving detailed, real-time guidance that responds to their emotions. Furthermore, the collection and analysis of emotion data will enable the system to provide even more accurate travel plans.

[1826] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1827] Step 1: Enter your user information

[1828] Users launch the application and enter information such as the area they want to visit, the food they want to enjoy, their interests, and their budget. For example, they can enter information such as "Tokyo," "sushi," "historical place," and "5,000 yen per day." The entered data is temporarily stored on the device.

[1829] Input: User's travel plan request (e.g. Tokyo, sushi, historical places, budget 5000 yen)

[1830] Output: Input data saved on the device

[1831] Step 2: Submit user information

[1832] The device sends the entered information to the server, where each item is converted into an appropriate format and a data format such as JSON is generated.

[1833] Input: Input data stored on the device

[1834] Output: User's travel plan request data sent to the server

[1835] Step 3: Receiving and analyzing information

[1836] The server receives user information sent from the device. The received data is analyzed by an analysis engine to identify the user's preferences and conditions. The analysis results are used to generate the next plan.

[1837] Input: User data sent to the server

[1838] Output: User preferences and conditions identified by the analytics engine

[1839] Step 4: Generate your travel plan

[1840] The AI ​​on the server generates optimal travel destinations and plans based on the analyzed user preferences and conditions. The AI ​​refers to existing travel databases and reviews to suggest multiple combinations of destinations and activities.

[1841] Input: User preferences and conditions identified by the analytics engine

[1842] Output: Data for generating optimal travel destinations and plans

[1843] Step 5: Submit and view your plan

[1844] The server sends the generated travel plan to the terminal, which visually displays the received plan for the user to check.

[1845] Input: Data for generating optimal travel destinations and plans

[1846] Output: Travel plan displayed on the device

[1847] Step 6: User chooses plan

[1848] The user selects the plan they like from the ones presented, and the selected plan is sent to the server via the device and saved as the final plan.

[1849] Input: Travel plan displayed on the device

[1850] Output: The final plan sent to the server

[1851] Step 7: Start real-time guidance

[1852] On the day of the trip, the user launches the app again, and the device loads the final plan saved on the server. The loaded plan is linked to the map application, and a route is displayed based on the user's current location. The user then begins traveling according to this route.

[1853] Input: Final plan saved on the server

[1854] Output: Display of travel route by linking with map application

[1855] Step 8: Emotion Engine Feedback

[1856] The user wears a device with voice guidance. The emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. Based on the recognized emotion, the tone and content of the guidance are adjusted and appropriate feedback is provided.

[1857] Input: User's facial expressions and tone of voice data

[1858] Output: Adjusted announcement tone and content

[1859] Step 9: Collect and analyze emotion data

[1860] The device continuously collects emotional data from the user while traveling. The collected data is sent to a server, where it is analyzed by an analysis engine. The analysis results are used by the AI ​​to generate travel plans for future trips.

[1861] Input: User emotion data during travel

[1862] Output: Analysis results of emotional data fed back to the generative AI

[1863] (Application example 2)

[1864] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1865] While itinerary optimization and real-time guidance have become commonplace, recognizing users' emotions and dynamically adjusting entertainment and guidance content based on that data is crucial to further enriching the travel experience. However, current systems lack a means to effectively recognize users' emotions and incorporate them into itinerary planning and real-time guidance. This poses a challenge, making it difficult for users to truly relax and enjoy their trip.

[1866] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes user input means, generation means for processing travel planning requests received from the user and generating optimal candidate travel destinations and plans, display means for displaying the generated candidate destinations and plans to the user, guidance means for providing real-time guidance based on the plan selected by the user, emotion engine means for recognizing the user's emotions using sensor means installed in the vehicle and processing the data, and entertainment control means for dynamically adjusting the content and tone of entertainment based on the recognized emotion data. This makes it possible to customize the travel experience according to the user's emotions.

[1867] The "user input means" is an interface through which a user inputs their travel planning requests, and is implemented as an application installed on a mobile device such as a smartphone or tablet.

[1868] The "generation means" has the function of processing a travel plan request received from a user input means and generating optimal travel destinations and plans.

[1869] The "display means" is a device that visually presents the travel destinations and plans generated by the generation means to the user, and operates on the same device as the user input means.

[1870] The "guidance means" provides real-time guidance based on the plan selected by the user and has the function of displaying the route in cooperation with a map application.

[1871] The "emotion engine means" is a part of the system that uses sensor means installed in the vehicle to recognize the user's emotions and process the data.

[1872] An "entertainment control means" is a means that dynamically adjusts the content and tone of entertainment based on recognized emotional data.

[1873] A "map application" is software that provides geographic location information, displays a user's travel route, and provides navigation.

[1874] A "device with voice guidance" is a device that provides real-time multilingual voice guidance.

[1875] The present invention provides a system for optimizing a user's travel plan and providing real-time guidance, particularly for autonomous vehicles. The system includes a user input means, a generation means, a display means, a guidance means, an emotion engine means, and an entertainment control means.

[1876] System configuration

[1877] User Input Method

[1878] The user input means is an interface through which users input their travel plan requests. For example, it is implemented as an application installed on a mobile device such as a smartphone or tablet. Users input information such as the areas they want to visit, the food they want to enjoy, the genres they are interested in, and their budget.

[1879] generation means

[1880] The generation means is located on a server and receives information sent from the user input means. Based on this information, the generation AI generates optimal travel destinations and plans. The generation AI refers to existing travel databases and word-of-mouth information to propose plans that best suit the user's preferences and conditions.

[1881] Display means

[1882] The display means visually presents the generated travel destinations and plans to the user. It runs on the same device as the user input means and displays the data sent from the generation means. The user can check the various candidates on this display screen and select the plan that best suits their needs.

[1883] Guidance means

[1884] The guidance means provides real-time guidance based on the final plan selected by the user. For example, it works with a map application to display a route based on the current location information of the autonomous vehicle. It also provides real-time voice guidance in multiple languages ​​using a device with voice guidance.

[1885] Emotion Engine Means

[1886] The emotion engine uses sensors installed in the vehicle to recognize the user's emotions and process the data. This recognition process uses facial recognition cameras and voice analysis microphones. For example, Intel RealSense Depth Cameras and Shure SM7B microphones are considered. The emotion data is sent to a server and used to optimize the operation of the entire system.

[1887] Entertainment Controls

[1888] The entertainment control means dynamically adjusts the content and tone of entertainment based on the recognized emotion data. For example, it controls the playback of music streaming services and video content. Examples of software used include the Spotify API and YouTube API.

[1889] Specific examples

[1890] User information entry and submission

[1891] The user starts the application and enters the following information:

[1892] Area I want to visit: Tokyo

[1893] Food I want to enjoy: Sushi

[1894] Interests: Historical places

[1895] Budget: 5,000 yen per day

[1896] This information is sent to the server via the terminal.

[1897] Generative AI generates candidate sites and plans

[1898] Based on the received user information, the server uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans that best fit the user's preferences and conditions. For example, a plan might be generated that includes breakfast at Tsukiji Market, a visit to Sensoji Temple, lunch at a sushi restaurant in Ginza, a tour of the Imperial Palace, and dinner at a izakaya in Shinjuku.

[1899] Plan display and real-time guidance

[1900] The server sends the generated plan to the device, which displays it to the user. The user can review and select the plan. On the day of the trip, the device loads the final plan saved on the server and displays the route in conjunction with a map application. In addition, a voice guidance device worn by the user provides real-time multilingual guidance.

[1901] Emotion engine and entertainment tuning

[1902] During the trip, the emotion engine will recognize the user's emotions and provide feedback accordingly. For example, if it detects that the user is tired, it will provide guidance such as "You can take a break here and refresh yourself at the cafe."

[1903] Prompt Sentence Examples

[1904] An example prompt is:

[1905] The user provided the following information:

[1906] Area I want to visit: Tokyo

[1907] Food I want to enjoy: Sushi

[1908] Interests: Historical places

[1909] Budget: 5,000 yen per day

[1910] Based on this, suggest the best travel plan.

[1911] Thus, the present invention is designed to enhance a user's travel experience, providing real-time customized guidance and entertainment based on the user's emotions.

[1912] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1913] Step 1:

[1914] The user launches the application and inputs their travel plan requirements. The input information includes areas they want to visit, foods they want to enjoy, genres they are interested in, budget, etc. Input: User's travel plan requirements data. Output: User's input information stored on the device.

[1915] Step 2:

[1916] The terminal sends the information entered by the user to the server. Input: User input information stored in the terminal. Output: User travel plan request data sent to the server.

[1917] Step 3:

[1918] The server analyzes the received user information and uses a generation AI to generate optimal travel destinations and plans. The generation AI refers to existing travel databases and reviews to propose plans based on the user's preferences and conditions. Input: User's travel plan request data sent to the server. Output: Optimal travel destinations and plans.

[1919] Step 4:

[1920] The server sends the generated plan to the terminal. Input: Generated travel plan. Output: Optimal travel plan data sent to the terminal.

[1921] Step 5:

[1922] The terminal displays the plan sent to the user. The user selects the desired plan from the displayed candidates. Input: Optimal travel plan data sent from the server. Output: Travel plan displayed to the user and the user's selection information.

[1923] Step 6:

[1924] On the day of the trip, when the user restarts the app, the device loads the saved final plan from the server and works with the map application to display a route based on the current location. Input: Final plan data saved on the server. Output: Route displayed in the map application.

[1925] Step 7:

[1926] The emotion engine recognizes the user's emotions through sensing devices (cameras and microphones) installed in the vehicle. Input: Data on the user's facial expressions and voice. Output: Recognized emotion data.

[1927] Step 8:

[1928] The terminal processes the recognized emotion data, and the entertainment control means dynamically adjusts the content and tone of the entertainment according to the user's emotion. Input: Recognized emotion data. Output: Adjusted entertainment content.

[1929] Step 9:

[1930] The terminal voice guidance device provides multilingual voice guidance and feedback to the user in real time. Input: Travel route data and final plan selected. Output: Voice guidance and feedback provided to the user.

[1931] The system allows users to optimize their trip planning and enjoy entertainment and navigation that responds to their emotions in real time.

[1932] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1933] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1934] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1935] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1936] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1937] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1938] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1939] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1940] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1941] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1942] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1943] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1944] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1945] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1946] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1947] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1948] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1949] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1950] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1951] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1952] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1953] The following is further disclosed regarding the above embodiment.

[1954] (Claim 1)

[1955] a user input means;

[1956] A generating means for processing a travel plan request received from a user and generating optimal travel destinations and plans;

[1957] a display means for displaying the generated candidate sites and plans to the user;

[1958] a guidance means for providing real-time guidance based on a plan selected by the user;

[1959] A system including:

[1960] (Claim 2)

[1961] The system according to claim 1 , wherein the guidance means includes means for linking with a map application and displaying a travel route to the user.

[1962] (Claim 3)

[1963] 10. The system of claim 1, wherein the means for providing guidance includes means for providing real-time multilingual voice guidance using a voice guidance device.

[1964] "Example 1"

[1965] (Claim 1)

[1966] a user input means;

[1967] A generating means for processing a travel plan request received from a user and generating optimal travel destinations and plans;

[1968] The generation means refers to an existing database and inputs a prompt sentence into the generative AI model to generate a travel plan;

[1969] a display means for displaying the generated candidate sites and plans to the user;

[1970] a guidance means for providing real-time guidance based on a plan selected by the user;

[1971] A system including:

[1972] (Claim 2)

[1973] The system according to claim 1 , wherein the guidance means includes means for linking with a map application and displaying a travel route to the user.

[1974] (Claim 3)

[1975] 10. The system of claim 1, wherein the means for providing guidance includes means for providing real-time multilingual voice guidance using a voice guidance device.

[1976] "Application Example 1"

[1977] (Claim 1)

[1978] an input means for receiving travel planning requests from a user;

[1979] A means for processing travel plan requests received from an input means and generating optimal travel destinations and plans using a generative AI model;

[1980] A means for displaying the generated travel destinations and plans to the user;

[1981] a means for providing real-time guidance based on the user's selected plan;

[1982] A system including:

[1983] (Claim 2)

[1984] The system according to claim 1 , wherein the guidance means includes means for displaying a travel route to a user in cooperation with a navigation system.

[1985] (Claim 3)

[1986] 2. The system according to claim 1, wherein the guidance means includes means for providing multilingual voice guidance in real time using a voice guidance system.

[1987] "Example 2: Combining Emotion Engines"

[1988] (Claim 1)

[1989] a user input means;

[1990] A generating means for processing a travel plan request received from a user and generating optimal travel destinations and plans;

[1991] a display means for displaying the generated candidate sites and plans to the user;

[1992] a guidance means for providing real-time guidance based on a plan selected by the user;

[1993] An emotion engine that recognizes the user's emotions and adjusts the plan and guidance accordingly.

[1994] A system including:

[1995] (Claim 2)

[1996] The system according to claim 1 , wherein the guidance means includes means for linking with a map application and displaying a travel route to the user.

[1997] (Claim 3)

[1998] 10. The system of claim 1, wherein the means for providing guidance includes means for providing real-time multilingual voice guidance using a voice guidance device.

[1999] "Application example 2 when combining emotion engines"

[2000] (Claim 1)

[2001] a user input means;

[2002] A generating means for processing a travel plan request received from a user and generating optimal travel destinations and plans;

[2003] a display means for displaying the generated candidate sites and plans to the user;

[2004] a guidance means for providing real-time guidance based on a plan selected by the user;

[2005] emotion engine means for recognizing the emotion of a user using sensor means installed in the vehicle and processing the data;

[2006] entertainment control means for dynamically adjusting the content and tone of entertainment based on the recognized emotion data;

[2007] A system including:

[2008] (Claim 2)

[2009] The system according to claim 1 , wherein the guidance means includes means for linking with a map application and displaying a travel route to the user.

[2010] (Claim 3)

[2011] 10. The system of claim 1, wherein the means for providing guidance includes means for providing real-time multilingual voice guidance using a voice guidance device. [Explanation of symbols]

[2012] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a user input means; A generating means for processing a travel plan request received from a user and generating optimal travel destinations and plans; a display means for displaying the generated candidate sites and plans to the user; a guidance means for providing real-time guidance based on a plan selected by the user; A system including:

2. The system according to claim 1 , wherein the guidance means includes means for displaying a travel route to the user in cooperation with a map application.

3. The system of claim 1 , wherein the means for providing guidance includes means for providing real-time multilingual voice guidance using a voice guidance device.

Images