System

Abstract

A system is provided.SOLUTION: A system comprising: input means for allowing a user to input a question or a request related to a travel; generation means for generating a travel plan on the basis of the question or the request of the user input through the input means; presentation means for presenting the generated travel plan to the user; customization means for customizing the travel plan through a dialogue with the user; reservation means for supporting a reservation procedure on the basis of the customized travel plan; and management means for managing a travel history of the user and making the travel history browsable.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the modern travel industry, users often spend a lot of time and effort creating efficient and personalized travel plans. It can be difficult for users with limited experience or knowledge to obtain appropriate information. Furthermore, existing travel planning systems are unable to flexibly accommodate users' specific needs and constraints, often resulting in unsatisfactory travel plans. Furthermore, travel plans often cannot be easily modified or customized, forcing users to rely on external information sources. The present invention aims to solve these problems and provide users with quick and accurate travel plans. [Means for solving the problem]

[0005] The present invention provides a system that includes an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the user's questions and requests input through the input means, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through interaction with the user, a reservation means for supporting reservation procedures based on the customized travel plan, and a management means for managing and making the user's travel history available for viewing. This allows users to easily create efficient and personalized travel plans using their smartphones or computers. Furthermore, travel plan suggestions using a generative AI model can quickly provide detailed and reliable information. Furthermore, travel plans can be flexibly customized through interaction with a chatbot, thereby increasing user satisfaction.

[0006] "User" refers to an individual or entity that inputs travel questions and requests and proposes and customizes travel plans.

[0007] "Input Means" means a device or interface through which a User inputs travel questions or requests into the System.

[0008] The "generation means" refers to a function that generates a travel plan based on the user's questions and requests input through the input means.

[0009] "Presentation means" refers to the function of providing the generated travel plan to the user visually or audibly.

[0010] "Customization" refers to functionality that provides the ability to modify or change a travel plan through user interaction.

[0011] "Reservation tool" refers to a function that supports the booking process for hotels, flights, and activities based on confirmed travel plans.

[0012] "Management means" refers to the function that stores and manages the user's travel history and makes that information available for viewing as needed.

[0013] A "generative AI model" refers to artificial intelligence technology that automatically generates optimal travel plans based on information entered by the user.

[0014] A "chatbot" is an automated dialogue system that provides information and customizes travel plans through dialogue with users. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention provides a system that includes an input means for users to input travel-related questions and requests, a generation means that generates a travel plan using a generative AI model based on the input information, a presentation means that presents the generated plan to the user, a customization means that customizes the travel plan through dialogue with the user, a reservation means that supports the reservation procedure based on the customized travel plan, and a management means that manages the user's travel history and makes it viewable. Below, the program processing of this system is explained in natural language. Specific examples are also provided.

[0037] Program processing

[0038] 1. Users enter their travel questions and requests

[0039] Users access the screen of their smartphone or computer and enter desired conditions such as travel destination, theme, budget, etc. into an input form. For example, they might enter, "I'd like to plan a family trip to Hawaii this summer, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[0040] The terminal (smartphone or computer) sends the entered information to the server.

[0041] 2. The server analyzes the user information and generates an initial travel plan

[0042] The server analyzes the received user information and retrieves tourist attractions, hotels, and transportation information from the database based on the user's request. Specifically, it retrieves a list of "tourist attractions in Hawaii, resort hotels available within a budget, and family activities."

[0043] The server uses a generative AI model to generate an initial itinerary based on this data, such as "A 7-day trip to Hawaii: Day 1 - Beach, Day 2 - Shopping, Day 3 - Snorkeling..."

[0044] The server presents the generated travel plan to the user in the form of a chatbot.

[0045] 3. Users interact with the chatbot to get more information

[0046] The user reviews the chatbot's suggestions and types, "Tell me more about this tourist spot."

[0047] The device sends the user's question to the server, which retrieves relevant information from a database, providing detailed information such as "You can enjoy beach volleyball and kayaking at this tourist spot."

[0048] 4. Customize your travel plans based on the information you provide

[0049] Users can input instructions to modify their plans based on the chatbot's suggestions, for example, "Instead of this hotel, find another one closer to the beach."

[0050] The server regenerates the itinerary based on the new request and presents the updated itinerary to the user, such as "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and Beach Volleyball..."

[0051] 5. The user finalizes their travel plans and completes the booking process

[0052] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[0053] The device sends this request to the server, which then connects with the reservation system to check availability of the hotel and flight the user desires. If the reservation is successful, the user is given confirmation information, such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX."

[0054] 6. Managing and viewing your travel history

[0055] To check their travel history, users type "view past travel history."

[0056] The device sends this request to the server, which retrieves the user's past travel data from the database. It formats and displays information such as "Retrieve details of past Hawaii trips and European tours." For example, it displays information such as "August 2022 Hawaii trip: 5 nights and 7 days, tourist spots visited, and hotel information."

[0057] Specific examples

[0058] When a user uses their smartphone to input "I want to take a family trip to Hawaii this summer," the device sends this information to a server. The server analyzes the received information and retrieves the most suitable information from its database using the keywords "Hawaii, family-friendly, budget 300,000 yen." It then uses a generative AI model to generate a travel plan suited to the user and presents it to the user through a chatbot. If the user inputs "Tell me more about places where I can snorkel," the server searches for related information and provides more detailed information. The user can then reconstruct their plan based on the information, ultimately booking hotels and flights, and review their detailed travel history at a later date. This series of actions results in efficient and personalized travel planning.

[0059] The processing flow will be explained below.

[0060] Step 1:

[0061] Users enter their travel questions and requests.

[0062] Users use their smartphones or computers to enter their desired travel destination, theme, budget, and other preferences into an input form.

[0063] For example, you might enter information such as, "I want to plan a trip to Hawaii with my family during summer vacation, my budget is 300,000 yen, and I want to stay at a resort hotel."

[0064] The terminal transmits this input information to the server.

[0065] Step 2:

[0066] The server analyzes the user information.

[0067] The server analyzes the data received from the user and extracts keywords such as travel destination, theme, and budget.

[0068] For example, extract keywords such as "Hawaii," "family," and "budget 300,000 yen."

[0069] The server then searches a database to retrieve suggested attractions, hotels, and transportation options that match these keywords.

[0070] Step 3:

[0071] The server generates an initial itinerary.

[0072] The server uses a generative AI model to generate an initial itinerary based on the acquired data.

[0073] For example, generate a plan such as "7-day Hawaii travel plan. Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling..."

[0074] The generated travel plan is formatted in a user-friendly format.

[0075] Step 4:

[0076] The server presents the generated travel plan to the user.

[0077] The server presents an initial travel plan to the user through a chatbot.

[0078] Users can check the details of their travel plans through the chatbot.

[0079] Step 5:

[0080] Users interact with the chatbot to get more information.

[0081] Users can enter specific questions about the plan presented by the chatbot.

[0082] For example, send a question such as "Tell me more about this tourist spot."

[0083] The terminal sends this question to the server.

[0084] Step 6:

[0085] The server provides information based on the user's question.

[0086] The server analyzes the user's question and retrieves relevant details from the database.

[0087] For example, detailed information such as "At this tourist spot, you can enjoy beach volleyball and kayaking" is sent to the chatbot.

[0088] The user can view this information through their device.

[0089] Step 7:

[0090] Customize your travel plans based on the information you provide.

[0091] Users can modify or add to their travel plans through interactions with the chatbot.

[0092] For example, "Instead of this hotel, find another hotel closer to the beach."

[0093] The terminal sends this request to the server.

[0094] Step 8:

[0095] The server regenerates the itinerary and presents it to the user.

[0096] The server regenerates the travel plan based on the user's new requirements and presents the customized plan to the user through the chatbot.

[0097] For example, it presents a new plan such as "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and beach volleyball..."

[0098] Step 9:

[0099] The user finalizes their travel plans and completes the booking process.

[0100] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[0101] The terminal sends this request to the server.

[0102] Step 10:

[0103] The server supports the reservation process.

[0104] The server connects to the reservation system to check availability of the hotel or flight the user desires.

[0105] If the reservation is successful, confirmation information is sent to the user via the terminal.

[0106] For example, information such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX" may be displayed.

[0107] Step 11:

[0108] Manage and view your trip history.

[0109] To check their travel history, users type "view past travel history."

[0110] The terminal sends this request to the server.

[0111] Step 12:

[0112] The server provides the user's travel history.

[0113] The server retrieves the user's past travel data from the database, formats it, and sends it to the terminal.

[0114] For example, it will display "Trip to Hawaii in August 2022: 5 nights and 7 days, information on tourist spots visited and hotels stayed."

[0115] Example 1

[0116] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0117] Conventional travel plan creation systems have had difficulty automatically generating specific and personalized travel plans based on users' requests. Furthermore, they lacked a means for users to interactively customize the generated travel plans, or a function to support the reservation process based on the customized plans. This has resulted in problems that make it difficult for users to efficiently create travel plans and smoothly complete the necessary reservation procedures.

[0118] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0119] In this invention, the server includes an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the user's questions and requests input through the input means, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting the reservation procedure based on the customized travel plan, a management means for managing the user's travel history and making it viewable, a means for the generation means to generate a travel plan using a generative AI model, a means for presenting the generated travel plan to the user in chatbot format, a means for the user to obtain detailed information by dialogue with the chatbot, and a means for retrieving travel plan data from a database based on the user's input. This enables efficient generation, customization, and support for the reservation procedure of a personalized travel plan based on the user's requests.

[0120] "User" means any person or entity that uses the System to create and customize a travel plan.

[0121] "Input means" refers to the interface through which a user inputs travel-related questions or requests into the system, such as a form or text entry field displayed on a computer or smartphone screen.

[0122] "Generation Means" refers to the functionality for generating a travel plan based on information provided by the user through the Input Means, including retrieving information from a generative AI model or database.

[0123] The "presentation means" refers to a mechanism for displaying the generated travel plan to the user. Specifically, it includes a chatbot-style user interface.

[0124] "Customization means" refers to the function of modifying and adjusting the travel plan generated through interaction with the user.

[0125] "Booking Method" refers to functionality that supports booking accommodation and transportation based on customized travel plans.

[0126] "Management means" refers to the function that manages a user's past travel history and allows the user to view it.

[0127] "Generative AI model" refers to an artificial intelligence model that generates travel plans based on information provided by users, including models that utilize natural language processing and machine learning.

[0128] A "chatbot" is an automated response program that interacts with users to exchange information.

[0129] "Database" refers to a collection of electronic data for storing information (such as tourist attractions, hotels, and transportation options) required to generate a travel plan.

[0130] A "prompt sentence" refers to a sentence that is input to a generative AI model to generate a specific travel plan.

[0131] The present invention is a system that allows users to input travel-related questions and requests, and then generate, customize, book, and manage travel plans based on that information. This system uses the following hardware and software.

[0132] Users access an application on their smartphone or computer screen and input information such as travel destination, theme, budget, etc. For example, they might input conditions such as "I want to plan a family trip to Hawaii this summer, with a budget of 300,000 yen, and I want to stay at a resort hotel." This input information is then sent from the device to the server.

[0133] The server uses natural language processing tools to analyze the information received from users. Specifically, it uses NLP tools such as Google Cloud Natural Language API to analyze the input text and extract important keywords. It also collects data on tourist attractions, hotels, and transportation options from databases and external APIs (e.g., Google Places API, Expedia API).

[0134] Based on the collected data, the server generates a travel plan using a generative AI model (e.g., OpenAI GPT-4). The server inputs a prompt such as "Generate a family travel plan in Hawaii within a budget of 300,000 yen" into the generative AI model, which then generates an appropriate travel plan. The generated travel plan is converted into a chatbot-style UI and presented to the user via the device.

[0135] Users use the chatbot to get more information about the proposed itinerary. For example, they can type "Tell me more about this tourist spot," and the device will send this request to the server. The server will then query a database or external API to gather relevant details and present them to the user, providing them with specific information based on the proposed itinerary.

[0136] If users want to customize their travel plans, they can input instructions through the chatbot. The server will retrieve the data again based on the new request and generate an updated plan using the generative AI model. For example, a user could instruct the chatbot, "Instead of this hotel, find another hotel closer to the beach."

[0137] Finally, when the user has finalized their travel plans and wants to proceed with the reservation, they enter, "I'd like to confirm this plan and book a hotel and flight." The device sends this request to the server, which then connects with the reservation system to check the availability of the accommodation and transportation options desired by the user. If the reservation is successful, confirmation information is provided to the user. Information such as "Hotel reservation completed. Confirmation number is XXXXXX" is displayed.

[0138] Additionally, users can check their past travel history. By inputting the command "View past travel history," the device sends this request to the server, which retrieves and displays the user's past travel data from the database. For example, "A trip to Hawaii in August 2022: 5 nights and 7 days, tourist spots visited, and hotel information."

[0139] The system allows users to easily create, customize, book and manage efficient and personalized travel plans.

[0140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0141] Program processing

[0142] Step 1:

[0143] A user accesses a travel planning application on a smartphone or computer screen and inputs information such as the desired travel destination, theme, and budget. Specifically, the user writes in the input form, "I'd like to plan a trip to Hawaii for my family this summer. My budget is 300,000 yen. I'd like to stay at a resort hotel." The device then sends the input information to the server using an HTTP request. The input data is the user's request (travel destination, theme, budget, etc.).

[0144] Step 2:

[0145] The server uses natural language processing (NLP) tools to process the unstructured text information received from the user. Specifically, it uses the Google Cloud Natural Language API to analyze the text, tokenize it, and extract keywords. The input data is the unstructured text based on the user's request, and the output data is important keywords (e.g., "Hawaii," "family-friendly," "300,000 yen").

[0146] Step 3:

[0147] The server queries its own database or external APIs (e.g., Google Places API, Expedia API) based on the extracted keywords to collect data on related tourist attractions, hotels, and transportation options. For example, it obtains information such as "tourist attractions in Hawaii, resort hotels available within a budget, and family-friendly activities." The input data is a set of keywords, and the output data is related travel information (tourist attractions, hotels, and transportation options).

[0148] Step 4:

[0149] The server uses the collected data as input to generate an initial travel plan using a generative AI model (e.g., OpenAI GPT-4). The model is given a prompt, "Generate a family trip to Hawaii within a budget of 300,000 yen," and a specific travel plan is generated. The generated travel plan includes itineraries, activities, accommodations, etc. The input data is the travel prompt and travel-related data from an external API, and the output data is the generated travel plan.

[0150] Step 5:

[0151] The server formats the generated travel plan in chatbot format and sends it to the terminal to present to the user. Specifically, it converts the travel plan into text format and displays it on the user interface. The terminal displays the received travel plan information on the screen. The input data is the generated travel plan, and the output data is the travel plan text displayed on the screen.

[0152] Step 6:

[0153] The user inputs detailed information and customization instructions for the travel plan presented in chatbot format. For example, the user may give instructions such as "Tell me more about this tourist spot" or "Instead of this hotel, find another hotel closer to the beach." The device then sends these instructions to the server. The input data is the user's questions and customization instructions, and the output data is the detailed information and customization request.

[0154] Step 7:

[0155] Based on the user's questions and customization instructions, the server queries the database or external API again to collect and generate detailed information and a new travel plan. Specifically, the updated conditions are input into the generative AI model to obtain a revised plan. It also collects detailed information about tourist spots and re-suggested hotels and presents them to the user in the form of a chatbot. The input data is the detailed information and new conditions, and the output data is the updated travel plan and detailed information.

[0156] Step 8:

[0157] The user finalizes the travel plan and inputs instructions to complete the reservation process. For example, the user might say, "I'd like to finalize this plan and book a hotel and flight." The device then sends this instruction to the server. The input data is the final confirmation instruction from the user, and the output data is the reservation request.

[0158] Step 9:

[0159] The server connects with a reservation system (e.g., hotel reservation system, airline reservation system) to check the availability of the specified accommodation or transportation. If the reservation is successful, it generates confirmation information and sends it to the terminal. For example, the user is provided with information such as "Your hotel reservation has been completed. The confirmation number is XXXXXX." The input data is the reservation request, and the output data is the reservation confirmation information.

[0160] Step 10:

[0161] A user can make a request to view past travel history. When the user inputs the instruction "Show past travel history," the terminal sends this request to the server. The input data is a request for past travel history, and the output data is a request to display travel history.

[0162] Step 11:

[0163] The server retrieves the user's past travel data from the database, formats it, and sends it to the terminal. For example, it provides data to display "A trip to Hawaii in August 2022: 5 nights and 7 days, information on tourist spots visited, and hotels stayed." The input data is a past travel history request, and the output data is the formatted past travel data.

[0164] (Application example 1)

[0165] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0166] Conventional travel planning systems offer limited means for users to visually grasp details about travel destinations and accommodations, leaving a need for improved user experience. There is also a demand for intuitive operation via voice input and gestures when customizing travel plans. Furthermore, there is a growing need to provide a realistic travel experience in advance by generating and presenting travel plans in a virtual environment. New technologies are needed to address these challenges.

[0167] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0168] In this invention, the server includes an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the user's questions and requests input through the input means, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting reservation procedures based on the customized travel plan, a management means for managing and making the user's travel history viewable, a means for the user to wear a head-mounted display and create a travel plan in a virtual environment, a means for displaying 3D views of tourist attractions and accommodations in the virtual environment so the user can visually confirm them, and a means for the user to input questions and requests by voice input or gestures. This allows the user to check and customize the travel plan in real time in the virtual environment, enabling visual and intuitive travel planning.

[0169] "User" means any individual or entity that uses the Travel Planning System.

[0170] "Input Means" refers to a device or interface through which a User inputs travel-related questions or requests into the System.

[0171] "Generation means" refers to a combination of software or hardware for generating a travel plan based on information input by a user.

[0172] "Presentation means" refers to a device or interface for displaying the generated travel plan to the user.

[0173] "Customization means" refers to a device or interface for modifying or changing a travel plan through interaction with the user.

[0174] "Reservation Instrument" means a device or interface that supports the reservation process based on a customized travel plan.

[0175] "Management means" refers to databases and software used to store and make available users' travel history.

[0176] A "head-mounted display" refers to a device worn by a user that displays visual information in three dimensions.

[0177] "Virtual environment" refers to a computer-generated 3D space constructed to allow users to experience virtual reality.

[0178] "3D views of tourist attractions and accommodation facilities" refers to images and videos that visually display details of tourist attractions and accommodation facilities in three dimensions.

[0179] "Voice input" refers to a means by which a user can give instructions or information to a system through voice.

[0180] "Gesture input" refers to a means by which a user gives instructions or information to a system through hand or body movements.

[0181] To realize this invention, it is necessary to build a system using the following steps: First, provide an input means for users to input their travel-related questions and requests. This input means can include a smartphone, a computer touch panel, a voice input device, etc.

[0182] Hardware and Software Use

[0183] Hardware

[0184] Head-mounted display (HMD): A device worn by the user to provide a virtual reality environment, such as Oculus Rift or HTC Vive.

[0185] software

[0186] Unity: Builds a virtual reality environment to display travel plans within a virtual environment.

[0187] Azure Chatbot Service: Provides a conversational interface that enables interaction with users.

[0188] GCP AI Platform: Hosts generative AI models and generates itineraries based on user questions and requests.

[0189] Firebase: Manages user travel history and booking data.

[0190] Program processing

[0191] Input Method

[0192] Users put on the HMD and log in to the virtual environment. They then input their travel questions and requests into the system using voice and gestures. For example, they might say, "A fun trip to Europe for the whole family." This voice input is then converted to text using the Google Cloud Speech-to-Text API.

[0193] generation means

[0194] The server receives user input data and generates a travel plan using a generative AI model hosted on the GCP AI Platform, for example, a list of recommended tourist spots and accommodations for a family trip to Europe.

[0195] Presentation means

[0196] The generated itinerary is presented to the user in a virtual environment using Unity, where the user can visually see 3D views of tourist attractions and accommodations.

[0197] Customization methods

[0198] Users can interact with the Azure Chatbot Service to customize the details of their travel plans, such as "Tell me more resort hotels."

[0199] Reservation procedure

[0200] The system automatically processes reservations based on the plan customized by the user, manages reservation data using Firebase, and provides confirmation information to the user once the reservation is complete.

[0201] management measures

[0202] Users' travel history is managed by Firebase and can be viewed by users. Past travel history and detailed information can be checked at any time.

[0203] Specific examples

[0204] The following example prompt sentences will be used to explain how the system works:

[0205] When a user types "Tell me about your trip to Hawaii," the generative AI model responds as follows:

[0206] "Prompt to generate a travel plan:

[0207] User request: I'm planning a summer vacation to Hawaii with my family. My budget is under 300,000 yen, and I'd like to stay at a resort hotel. I enjoy snorkeling and shopping. Based on the input criteria, please generate a 7-day itinerary that includes the following elements:

[0208] Required elements:

[0209] List of tourist attractions

[0210] Family Activities

[0211] Budget-friendly resort hotels

[0212] Dining options

[0213] Shopping spot

[0214] Based on the above prompts, the generative AI model can construct a specific travel plan and provide it to the user through the virtual assistant.

[0215] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0216] Step 1:

[0217] The user puts on a head-mounted display and logs into the virtual environment.

[0218] Input: User ID and authentication information

[0219] Output: Home screen in the virtual environment

[0220] Specific operation: The user receives visual information through the HMD and enters authentication information on the login screen. The device sends this information to the server, and if authentication is successful, the home screen is displayed.

[0221] Step 2:

[0222] Users input travel questions and requests using voice input or gestures.

[0223] Input: Voice and gesture data

[0224] Output: Text data (questions and requests)

[0225] What it does: The user says, "I want to plan a fun family trip to Europe." The device converts this into text using the Google Cloud Speech-to-Text API and sends it to the server.

[0226] Step 3:

[0227] The server analyzes user information and generates an initial travel plan using a generative AI model.

[0228] Input: User's question or request (text data)

[0229] Output: Initial travel plan (text data)

[0230] Specific operation: The server receives the text data and calls the generative AI model on the GCP AI Platform. The generative AI model generates a "European travel plan for families" and returns it to the server.

[0231] Step 4:

[0232] The server generates a travel plan and presents it in a virtual environment.

[0233] Input: Initial travel plan (text data)

[0234] Output: 3D view of the trip plan

[0235] What it does: The initial travel plan is passed to Unity as text data, and tourist attractions and accommodations are displayed in a 3D view, which the user can view through the HMD.

[0236] Step 5:

[0237] Users interact with chatbots to customize their travel plans.

[0238] Input: User's new question or request (text data)

[0239] Output: Customized travel plan (text data)

[0240] How it works: When a user types "find other resort hotels instead of this one," the device sends this to the server, which then invokes the generative AI model again to generate a customized itinerary based on the new request.

[0241] Step 6:

[0242] Users make reservations based on customized travel plans.

[0243] Input: Customized travel plan (text data)

[0244] Output: Reservation confirmation information

[0245] Specific operation: The user inputs "I would like to confirm this plan and make a reservation." The device sends this to the server, which then connects to the reservation system to check availability. If the reservation is successful, the confirmation information is provided to the user.

[0246] Step 7:

[0247] Users can manage and view their travel history.

[0248] Input: User ID and authentication information

[0249] Output: List of trip history

[0250] Specific operation: The user types "Show me my past travel history." The device sends this to the server, which retrieves past travel data from Firebase, formats it, and displays it to the user.

[0251] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0252] The present invention provides a system including an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the questions and requests input through the input means, a presentation means for presenting the generated travel plan, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting the reservation procedure based on the customized travel plan, a management means for managing the user's travel history and making it viewable, and an emotion engine for recognizing the user's emotions. Furthermore, the emotion engine is used to analyze the user's emotional state, and the travel plan is generated and customized based on that information. Specific embodiments of the present invention are described below.

[0253] Program processing

[0254] 1. Users enter their travel questions and requests

[0255] Users access the screen of their smartphone or computer and enter desired conditions such as travel destination, theme, budget, etc. into an input form. For example, they might enter, "I'd like to plan a family trip to Hawaii this summer, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[0256] The terminal transmits this input information to the server.

[0257] 2. The server analyzes the user information

[0258] The server analyzes the data received from the user and extracts keywords such as travel destination, theme, budget, etc. For example, keywords such as "Hawaii," "family," and "budget of 300,000 yen" are extracted.

[0259] The server then searches a database to retrieve suggested attractions, hotels, and transportation options that match these keywords.

[0260] 3. The emotion engine analyzes the user's emotional state

[0261] The emotion engine analyzes the user's emotional state from input and voice data, for example, identifying emotions such as "happiness," "anxiety," and "expectation" from input text and voice.

[0262] The emotion engine transmits the user's emotional state to the server.

[0263] 4. The server generates the initial itinerary

[0264] The server uses a generative AI model to generate an initial travel plan based on the acquired data and emotional information, such as "A 7-day trip to Hawaii. Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling..."

[0265] This initial itinerary is adjusted based on the user's emotional information, for example, "if you're looking to relax, we'll suggest a relaxed itinerary."

[0266] 5. The server presents the generated itinerary to the user.

[0267] The server presents the initial travel plan to the user through the chatbot, and the user can confirm the details of the travel plan through the chatbot.

[0268] 6. Users interact with chatbots to get more information

[0269] Users can input specific questions about the plan presented by the chatbot, such as "Tell me more about this tourist spot."

[0270] The terminal sends this question to the server.

[0271] 7. The server provides information based on the user's question.

[0272] The server analyzes the user's question and retrieves relevant details from the database, such as "You can enjoy beach volleyball and kayaking at this tourist spot," and sends the details to the chatbot.

[0273] The user can view this information through their device.

[0274] 8. Customize your travel plans based on the information you provide

[0275] Users can interact with the chatbot to modify or add to their travel plans, for example by typing, "Instead of this hotel, find another hotel closer to the beach."

[0276] The emotion engine monitors the user's emotional changes in real time during the conversation and transmits the emotional information to the server. For example, if the user expresses dissatisfaction, this is analyzed.

[0277] 9. The server regenerates the itinerary and presents it to the user.

[0278] The server regenerates the travel plan based on the user's new requests and emotional information, and presents a customized plan to the user through the chatbot, for example, "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and beach volleyball..."

[0279] 10. The user finalizes their travel plans and completes the booking process

[0280] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[0281] The terminal sends this request to the server.

[0282] 11. The server supports the reservation process

[0283] The server connects to the reservation system to check availability of the hotel and flight desired by the user. If the reservation is successful, the server sends a confirmation to the user via the terminal. For example, the server displays information such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX."

[0284] 12. Managing and Viewing Your Trip History

[0285] To check their travel history, users type "view past travel history."

[0286] The terminal sends this request to the server.

[0287] 13. The server provides the user's travel history

[0288] The server retrieves the user's past travel data from the database, formats it, and sends it to the device. For example, it displays "A trip to Hawaii in August 2022: 5 nights and 7 days, information on the tourist spots visited, and the hotel where the user stayed."

[0289] Specific examples

[0290] When a user uses their smartphone to type, "I want to take a family trip to Hawaii this summer," the device sends this information to the server. The server analyzes the received information and retrieves the most appropriate information from its database using keywords like "Hawaii," "family-friendly," and "budget of ¥300,000." The emotion engine then analyzes emotional information from the user's input and voice data, identifying, for example, "I'm looking for relaxation." The server then generates an initial travel plan based on this information and presents it to the user through a chatbot. If the user types, "Tell me more about snorkeling spots," the server searches for related information and provides more detailed information. During this process, the emotion engine monitors the user's emotions in real time and adjusts the plan as needed. Finally, the user can confirm the travel plan, complete the booking process, and easily check their past travel history. This series of operations results in efficient and personalized travel planning.

[0291] The processing flow will be explained below.

[0292] Step 1:

[0293] Users enter their travel questions and requests.

[0294] Users access the screen of their smartphone or computer and enter, "I would like to plan a trip to Hawaii with my family this summer vacation, my budget is 300,000 yen, and I would like to stay at a resort hotel."

[0295] The terminal transmits this input information to the server.

[0296] Step 2:

[0297] The server analyzes the user information.

[0298] The server analyzes the data received from the user and extracts keywords such as "Hawaii," "family," and "budget of 300,000 yen."

[0299] The server then searches a database to retrieve suggested attractions, hotels, and transportation options that match these keywords.

[0300] Step 3:

[0301] The emotion engine analyzes the user's emotional state.

[0302] The emotion engine analyzes the user's emotional state from their input and voice data, identifying emotions such as "joy," "anxiety," and "expectation."

[0303] The emotion engine transmits the user's emotional state to the server.

[0304] Step 4:

[0305] The server generates an initial itinerary.

[0306] The server uses a generative AI model to generate an initial travel plan based on the acquired data and emotional information.

[0307] For example, generate a detailed plan such as "7-day travel plan for Hawaii. Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling..."

[0308] The plan will be adjusted to take into account the user's emotional state, adding relaxing elements and so on.

[0309] Step 5:

[0310] The server presents the generated travel plan to the user.

[0311] The server presents the initial travel plan to the user via the chatbot and asks for confirmation.

[0312] Users can check the details of their travel plans through the chatbot via their device.

[0313] Step 6:

[0314] Users interact with the chatbot to get more information.

[0315] Users can then input questions about the plan presented by the chatbot, such as "Tell me more about this tourist spot."

[0316] The terminal sends this question to the server.

[0317] Step 7:

[0318] The server provides information based on the user's question.

[0319] The server analyzes the user's question and retrieves relevant information from a database.

[0320] For example, the chatbot provides users with detailed information such as, "At this tourist spot, you can enjoy beach volleyball and kayaking."

[0321] Step 8:

[0322] The emotion engine monitors the user's emotional changes in real time.

[0323] The emotion engine recognizes the emotions expressed by the user while interacting with the chatbot and sends this information to the server.

[0324] For example, if the user expresses an emotion of satisfaction, capture that.

[0325] Step 9:

[0326] Customize your travel plans based on the information you provide.

[0327] Users interact with the chatbot to make specific changes, such as "find another hotel closer to the beach."

[0328] The terminal sends this request to the server.

[0329] Step 10:

[0330] The server regenerates the itinerary.

[0331] The server regenerates the travel plan based on the user's new instructions and emotional information.

[0332] For example, it generates a new plan such as "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and beach volleyball..." and presents it to the user.

[0333] Step 11:

[0334] The user finalizes their travel plans and completes the booking process.

[0335] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[0336] The terminal sends this request to the server.

[0337] Step 12:

[0338] The server supports the reservation process.

[0339] The server connects to the reservation system to check availability of the hotel or flight the user desires.

[0340] If the reservation is successful, confirmation information is sent to the user via the terminal.

[0341] For example, information such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX" may be displayed.

[0342] Step 13:

[0343] Manage and view your travel history.

[0344] To check their travel history, users type "view past travel history."

[0345] The terminal sends this request to the server.

[0346] Step 14:

[0347] The server provides the user's travel history.

[0348] The server retrieves the user's past travel data from the database, formats it, and sends it to the terminal.

[0349] For example, it will display "Trip to Hawaii in August 2022: 5 nights and 7 days, information on tourist spots visited and hotels stayed."

[0350] Example 2

[0351] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0352] Conventional travel plan generation systems only provide standard plans without considering the user's emotional state, and therefore are unable to respond to individual user needs and emotions. This makes it difficult to provide a travel plan that satisfies the user, resulting in a lot of effort required for customization and a decrease in satisfaction.

[0353] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the user's questions and requests input through the input means, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting the reservation procedure based on the customized travel plan, a management means for managing the user's travel history and making it viewable, an emotion recognition means for analyzing the user's emotional state, and an adjustment means for adjusting the travel plan based on the emotion information analyzed by the emotion recognition means. This makes it possible to provide a personalized travel plan that takes the user's emotional state into consideration.

[0354] "Input means" refers to the means by which a user inputs questions or requests regarding a trip.

[0355] The "generation means" is a means for generating a travel plan based on the user's questions and requests input through the input means.

[0356] The "presentation means" is a means for presenting the generated travel plan to the user.

[0357] "Customization means" refers to a means for customizing a travel plan through interaction with the user.

[0358] A "reservation tool" is a tool that supports the reservation process based on a customized travel plan.

[0359] "Management means" refers to the means for managing and making available the user's travel history.

[0360] "Emotion recognition means" is a means for analyzing the emotional state of a user.

[0361] The "adjustment means" is a means for adjusting the travel plan based on the emotion information analyzed by the emotion recognition means.

[0362] The present invention is a system for planning and managing travel, which aims to provide a personalized travel plan based on user input. The system includes an input means for a user to input travel questions and requests, a generation means for generating a travel plan based on the input information, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue, a reservation means for supporting a reservation procedure based on the customized plan, a management means for managing the user's travel history and making it viewable, an emotion recognition means for analyzing the user's emotional state, and an adjustment means for adjusting the plan based on the emotion recognition means.

[0363] System Overview

[0364] 1. Input Method

[0365] Users use their smartphones or computers to input desired conditions such as travel destination, theme, budget, etc. For example, they might input, "I'd like to plan a family trip to Hawaii during the summer vacation, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[0366] 2. Generation means

[0367] The server analyzes the input sent by the user and extracts the necessary keywords, such as "Hawaii," "family," and "budget of 300,000 yen."

[0368] The server uses a generative AI model to retrieve data on relevant tourist attractions, accommodations, and transportation from a database and generate a travel plan.

[0369] 3. Presentation means

[0370] The server then presents the generated travel plan to the user through an interface such as a chatbot, and the user can review this information on their device.

[0371] 4. Customization Methods

[0372] Users can use the chatbot to input specific questions or requests for modifications to the proposed itinerary, such as "Tell me more about this tourist spot" or "Instead of this hotel, find another hotel closer to the beach."

[0373] 5. Reservation Methods

[0374] Based on the plan confirmed by the user, the server checks the availability of hotels and flights in conjunction with the reservation system and completes the reservation process. If the reservation is successful, the server provides the user with this information. For example, a message such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX" is displayed on the terminal.

[0375] 6. Control measures

[0376] Users can submit requests to the system to view their past travel history, for example, by typing "Show me my past travel history," and the information will be displayed.

[0377] 7. Emotion recognition means

[0378] The emotion engine analyzes the text and voice data entered by the user to identify the user's emotional state, which allows it to understand what the user wants and reflect that in its plans.

[0379] 8. Adjustment means

[0380] The server dynamically adjusts the travel plan based on the emotional information obtained by the emotion engine, providing a plan that matches the user's emotions, such as suggesting a more relaxed schedule for a user looking to relax.

[0381] Specific examples

[0382] To explain the specific operation in more detail, some specific examples will be given.

[0383] Example prompt sentence:

[0384] "I want to plan a relaxing trip abroad with my family during summer vacation."

[0385] "Tell me about tourist spots in Europe"

[0386] "How much do you need to budget for your trip?"

[0387] When a user uses their smartphone to type, "I want to take my family on a summer vacation to Hawaii," the device sends this information to the server. The server analyzes the received information and retrieves the most relevant information from its database using keywords like "Hawaii," "family-friendly," and "budget of ¥300,000." The emotion engine then analyzes the user's input and voice data to identify emotional information, such as "I'm looking for relaxation." The server then generates an initial travel plan based on this information and presents it to the user through a chatbot. If the user types, "Tell me more about snorkeling spots," the server searches for related information and provides more information. During this process, the emotion engine monitors the user's emotions in real time and adjusts the plan as needed. Finally, the user can confirm the travel plan and proceed with the booking process. Users can also easily check their past travel history. This series of actions results in efficient and personalized travel planning.

[0388] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0389] Step 1:

[0390] Users enter travel questions and requests

[0391] Users use their smartphones or computers to input conditions such as desired travel destination, theme, budget, etc. Specifically, they might write in the input form, "I'd like to plan a family trip to Hawaii during the summer vacation, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[0392] Input: User's travel question or request

[0393] Output: User-entered criteria (travel destination, theme, budget, etc.)

[0394] Step 2:

[0395] The device sends the input information to the server

[0396] The device sends the user-entered conditions to the server in real time, with the data being sent via HTTP requests.

[0397] Input: Conditions entered by the user on the device

[0398] Output: User criteria sent to server

[0399] Step 3:

[0400] The server analyzes the user information and searches the database

[0401] The server analyzes the received user conditions and extracts important keywords (travel destination, budget, theme, etc.). For example, keywords such as "Hawaii," "family," and "budget of 300,000 yen" are extracted.

[0402] The server then uses these keywords to search a database to retrieve relevant tourist spots, hotels, and transportation options.

[0403] Input: User criteria sent to the server

[0404] Output: Important keywords, travel data retrieved from the database

[0405] Step 4:

[0406] Emotion engine analyzes the user's emotional state

[0407] The emotion engine analyzes user input and voice data to identify emotional states, such as joy, anxiety, and anticipation, for example.

[0408] The emotion engine transmits the analyzed emotion information to the server.

[0409] Input: User input, voice data

[0410] Output: Parsed emotion information

[0411] Step 5:

[0412] The server generates the initial itinerary

[0413] The server uses a generative AI model to generate an initial itinerary based on the acquired data and emotional information, such as "7-day Hawaii itinerary: Day 1: Beach visit, Day 2: Shopping, Day 3: Snorkeling."

[0414] The system takes into account the user's emotional information and optimizes the plan, such as adjusting to a more relaxed schedule if the user is looking to relax.

[0415] Input: Travel data obtained from the database, analyzed emotion information

[0416] Output: Initial itinerary

[0417] Step 6:

[0418] The server presents the generated itinerary to the user.

[0419] The server presents the initial travel plan to the user through the chatbot, and the user can review the plan on their device.

[0420] Input: Initial travel plan

[0421] Output: The itinerary presented to the user

[0422] Step 7:

[0423] Users interact with the chatbot to get more information

[0424] Users can then use the chatbot to ask specific questions about the proposed plan, such as "Tell me more about this tourist spot."

[0425] The terminal sends this question to the server.

[0426] Input: User's specific question

[0427] Output: The question sent to the server

[0428] Step 8:

[0429] The server provides information based on the user's question.

[0430] The server analyzes the user's question and retrieves appropriate details from the database, such as "You can enjoy beach volleyball and kayaking at this tourist spot," and sends this information to the chatbot.

[0431] The user can view this information through their device.

[0432] Input: User question, database details

[0433] Output: Answer with detailed information provided

[0434] Step 9:

[0435] Customize your travel plans based on the information you provide

[0436] Users can then use the details they have acquired to modify or add to their travel plans, for example, by saying, "Instead of this hotel, find another hotel closer to the beach."

[0437] The emotion engine monitors the user's emotional changes in real time through this dialogue and transmits the emotional information to the server.

[0438] Input: User's correction instructions, emotional information

[0439] Output: revised plan, emotional information

[0440] Step 10:

[0441] The server regenerates the itinerary and presents it to the user.

[0442] The server regenerates the travel plan based on the user's new requests and emotional information, and presents a customized plan through the chatbot, for example, "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and Beach Volleyball."

[0443] Input: User's new request, emotion information

[0444] Output: Regenerated itinerary

[0445] Step 11:

[0446] The user confirms their travel plans and completes the booking process

[0447] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[0448] The terminal sends this request to the server.

[0449] Input: User confirmation request

[0450] Output: Confirm request sent to server

[0451] Step 12:

[0452] The server handles the reservation process

[0453] The server connects to the reservation system to check availability of the hotel and flight desired by the user. If the reservation is successful, the server sends a confirmation to the user via the terminal. For example, the server displays information such as "Your hotel reservation has been completed. The confirmation number is XXXXXX."

[0454] Input: User's reservation request information, availability in the reservation system

[0455] Output: Reservation confirmation information

[0456] Step 13:

[0457] Users can manage and view their travel history

[0458] If a user wants to check their past travel history, they type "View past travel history."

[0459] The terminal sends this request to the server.

[0460] Input: User's request to view travel history

[0461] Output: View request sent to the server

[0462] Step 14:

[0463] The server provides the user's travel history

[0464] The server retrieves the user's past travel data from the database, formats it, and sends it to the device. For example, it displays "A trip to Hawaii in August 2022: 5 nights and 7 days, information on the tourist spots visited, and the hotel where the user stayed."

[0465] Input: Travel history data from the database

[0466] Output: Formatted travel history information

[0467] (Application example 2)

[0468] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0469] The problem to be solved by the present invention is to provide a system that efficiently generates a personalized meal plan taking into account the user's preferences and emotional state when the user inputs their meal requests and questions, and smoothly supports the delivery procedure, including the customization process.

[0470] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an input means for the user to input their wishes and requests regarding their request, a generation means for generating a plan based on the user's wishes and requests input through the input means, a presentation means for presenting the generated plan to the user, a customization means for customizing the plan through dialogue with the user, a reservation means for supporting procedures based on the customized plan, and a management means for managing the user's history and making it viewable. This enables the generation, customization, and reservation procedures of personalized meal plans.

[0471] "Requests" refer to the conditions desired by the user and the content of the services they would like to receive.

[0472] "Input means" refers to an interface that allows a user to input information into a device or system.

[0473] "Generation means" refers to the process or technical mechanism for automatically generating a specific plan or proposal based on input information.

[0474] "Presentation means" refers to the method or interface for displaying the generated plan or information to the user.

[0475] "Customization means" refers to a method or system that allows users to modify or change the plans or information presented to them to suit their preferences.

[0476] "Reservation Method" refers to the method or system for carrying out formal procedures based on the plan selected or modified by the User.

[0477] "Management means" refers to the method or system for storing and organizing users' past usage history and information, and making them available for viewing as needed.

[0478] An "automatic response system" refers to technology or mechanisms that allow a system to automatically communicate with users.

[0479] An "information repository" refers to a system or database for storing and managing various data and information.

[0480] A "generative AI model" refers to an algorithm or model that uses machine learning or artificial intelligence to generate new plans or information based on specific patterns or information.

[0481] The present invention is a system that includes an input means for a user to input meal preferences and questions, a generation means for generating a meal plan based on the preferences and questions input through the input means, a presentation means for presenting the generated plan to the user, a customization means for customizing the plan through dialogue with the user, a reservation means for supporting delivery procedures based on the customized plan, and a management means for managing the user's meal history and making it viewable.

[0482] Detailed System Description

[0483] 1. Input method:

[0484] Users use their smartphones to input their meal preferences, budget, delivery time, and other requests into the application's input form. If a user inputs, "I want to order a healthy, stress-reducing meal," this information is sent from the smartphone to the server.

[0485] 2. Generation means:

[0486] The server analyzes the received user information and performs sentiment analysis using the Emotion API. The analyzed data is then used to generate an appropriate meal plan using a generative AI model (e.g., GPT-4). This generative AI model uses the input data and the extracted sentiment information as prompts to suggest the optimal meal plan.

[0487] 3. Means of presentation:

[0488] The generated meal plan is presented to the user through a chatbot, which provides the user with the meal plan details in a conversational format.

[0489] 4. Customization methods:

[0490] Users can modify or add to the generated meal plan through dialogue with the chatbot. For example, if they input a request such as "I'd like to add avocado to my healthy bowl," the server will again use the generative AI model to generate a new plan.

[0491] 5. Reservation Method:

[0492] Once the user is satisfied with the customized meal plan and selects "I would like to order this plan," the server connects with the delivery system to process the reservation. If the reservation is successful, a confirmation is sent to the user.

[0493] 6. Control measures:

[0494] The user's past meal history is stored in a database and can be viewed as needed. When a user types "I want to check my past order history," the relevant information is displayed.

[0495] Hardware and software used

[0496] Device: Smartphone (iOS / Android)

[0497] Emotion engine: Emotion API (e.g. Microsoft Azure Cognitive Services)

[0498] Generative AI model: GPT-4 (OpenAI API)

[0499] Database: PostgreSQL (Amazon RDS)

[0500] Frontend: React Native

[0501] Backend: Node.js (Express)

[0502] Example prompt sentence

[0503] "The user wants to order a meal. The user's wishes are as follows: Travel destination: Please suggest a menu with the theme of 'healthy and relaxing meals', within a budget of 2000 yen, and that can be delivered within 30 minutes. Furthermore, please consider 'stress,' 'relaxation,' and 'fatigue' based on the results of the sentiment analysis."

[0504] By implementing this invention, users can select, customize, and reserve a personalized meal plan based on their emotional state and preferences, resulting in a more satisfying delivery service.

[0505] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0506] Step 1:

[0507] A user accesses the smartphone application and inputs information such as meal preferences, questions, budget, and delivery time. For example, the user might input, "I would like to order a healthy and relaxing meal." The smartphone then sends this input information to the server.

[0508] Input: User's wishes, questions, budget, delivery time

[0509] Output: User information sent to the server

[0510] Step 2:

[0511] The server analyzes the information received from the user and extracts the input requests and questions. The analyzed data is subjected to emotion analysis using the Emotion API to identify the user's emotional state (e.g., "stressed," "relaxed," "fatigued").

[0512] Input: User information sent to the server

[0513] Data processing / data calculation: Extraction of wishes and questions, sentiment analysis

[0514] Output: Extracted data and sentiment information

[0515] Step 3:

[0516] The server uses a generative AI model (e.g., GPT-4) to generate an appropriate meal plan based on the extracted data and emotional information. At this time, the server passes prompts to the generative AI model, which then proposes a plan that reflects the user's preferences and emotional state.

[0517] Input: Extracted data and sentiment information

[0518] Data processing / data calculation: Prompt sentence generation, plan generation using generative AI models

[0519] Output: Generated meal plan

[0520] Step 4:

[0521] The server presents the generated meal plan to the user through the chatbot, and the user can check the details of the plan via their smartphone.

[0522] Input: Generated meal plan

[0523] Output: The plan presented to the user through the chatbot

[0524] Step 5:

[0525] Users can change or add to their meal plan through interactions with the chatbot, for example by typing, "I'd like to add avocado to my healthy bowl." The smartphone then sends this request to the server.

[0526] Input: User request

[0527] Output: New request sent to the server

[0528] Step 6:

[0529] The server regenerates the meal plan using the generative AI model again based on the new request and emotional information received. It generates a new prompt sentence and passes it to the generative AI model to suggest a new plan.

[0530] Input: New request, emotion information

[0531] Data processing / data calculation: Regeneration of prompt sentences, regeneration by generative AI model

[0532] Output: Regenerated meal plan

[0533] Step 7:

[0534] The server then presents the regenerated meal plan to the user via the chatbot, and the user can check the plan details again via their smartphone.

[0535] Input: Regenerated meal plan

[0536] Output: Plan re-presented through the chatbot

[0537] Step 8:

[0538] When the user is finally satisfied with the plan and types "I would like to order this plan," the smartphone sends this request to the server, which then coordinates with the delivery system to complete the reservation process and sends reservation confirmation information to the smartphone.

[0539] Input: Final confirmation request by user

[0540] Output: Final confirmation request sent to the server, booking confirmation information

[0541] Step 9:

[0542] The user's meal history is stored in a database. When the user types "I want to check my past order history," the smartphone sends this request to the server, which retrieves the relevant information from the database and displays it to the user.

[0543] Input: User history check request

[0544] Output: History information retrieved from the database, history information displayed to the user

[0545] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0546] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0547] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0548] [Second embodiment]

[0549] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0550] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0551] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0552] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0553] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0554] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0555] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0556] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0557] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0558] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0559] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0560] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0561] The present invention provides a system that includes an input means for users to input travel-related questions and requests, a generation means that generates a travel plan using a generative AI model based on the input information, a presentation means that presents the generated plan to the user, a customization means that customizes the travel plan through dialogue with the user, a reservation means that supports the reservation procedure based on the customized travel plan, and a management means that manages the user's travel history and makes it viewable. Below, the program processing of this system is explained in natural language. Specific examples are also provided.

[0562] Program processing

[0563] 1. Users enter their travel questions and requests

[0564] Users access the screen of their smartphone or computer and enter desired conditions such as travel destination, theme, budget, etc. into an input form. For example, they might enter, "I'd like to plan a family trip to Hawaii this summer, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[0565] The terminal (smartphone or computer) sends the entered information to the server.

[0566] 2. The server analyzes the user information and generates an initial travel plan

[0567] The server analyzes the received user information and retrieves tourist attractions, hotels, and transportation information from the database based on the user's request. Specifically, it retrieves a list of "tourist attractions in Hawaii, resort hotels available within a budget, and family activities."

[0568] The server uses a generative AI model to generate an initial itinerary based on this data, such as "A 7-day trip to Hawaii: Day 1 - Beach, Day 2 - Shopping, Day 3 - Snorkeling..."

[0569] The server presents the generated travel plan to the user in the form of a chatbot.

[0570] 3. Users interact with the chatbot to get more information

[0571] The user reviews the chatbot's suggestions and types, "Tell me more about this tourist spot."

[0572] The device sends the user's question to the server, which retrieves relevant information from a database, providing detailed information such as "You can enjoy beach volleyball and kayaking at this tourist spot."

[0573] 4. Customize your travel plans based on the information you provide

[0574] Users can input instructions to modify their plans based on the chatbot's suggestions, for example, "Instead of this hotel, find another one closer to the beach."

[0575] The server regenerates the itinerary based on the new request and presents the updated itinerary to the user, such as "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and Beach Volleyball..."

[0576] 5. The user finalizes their travel plans and completes the booking process

[0577] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[0578] The device sends this request to the server, which then connects with the reservation system to check availability of the hotel and flight the user desires. If the reservation is successful, the user is given confirmation information, such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX."

[0579] 6. Managing and viewing your travel history

[0580] To check their travel history, users type "view past travel history."

[0581] The device sends this request to the server, which retrieves the user's past travel data from the database. It formats and displays information such as "Retrieve details of past Hawaii trips and European tours." For example, it displays information such as "August 2022 Hawaii trip: 5 nights and 7 days, tourist spots visited, and hotel information."

[0582] Specific examples

[0583] When a user uses their smartphone to input "I want to take a family trip to Hawaii this summer," the device sends this information to a server. The server analyzes the received information and retrieves the most suitable information from its database using the keywords "Hawaii, family-friendly, budget 300,000 yen." It then uses a generative AI model to generate a travel plan suited to the user and presents it to the user through a chatbot. If the user inputs "Tell me more about places where I can snorkel," the server searches for related information and provides more detailed information. The user can then reconstruct their plan based on the information, ultimately booking hotels and flights, and review their detailed travel history at a later date. This series of actions results in efficient and personalized travel planning.

[0584] The processing flow will be explained below.

[0585] Step 1:

[0586] Users enter their travel questions and requests.

[0587] Users use their smartphones or computers to enter their desired travel destination, theme, budget, and other preferences into an input form.

[0588] For example, you might enter information such as, "I want to plan a trip to Hawaii with my family during summer vacation, my budget is 300,000 yen, and I want to stay at a resort hotel."

[0589] The terminal transmits this input information to the server.

[0590] Step 2:

[0591] The server analyzes the user information.

[0592] The server analyzes the data received from the user and extracts keywords such as travel destination, theme, and budget.

[0593] For example, extract keywords such as "Hawaii," "family," and "budget 300,000 yen."

[0594] The server then searches a database to retrieve suggested attractions, hotels, and transportation options that match these keywords.

[0595] Step 3:

[0596] The server generates an initial itinerary.

[0597] The server uses a generative AI model to generate an initial itinerary based on the acquired data.

[0598] For example, generate a plan such as "7-day Hawaii travel plan. Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling..."

[0599] The generated travel plan is formatted in a user-friendly format.

[0600] Step 4:

[0601] The server presents the generated travel plan to the user.

[0602] The server presents an initial travel plan to the user through a chatbot.

[0603] Users can check the details of their travel plans through the chatbot.

[0604] Step 5:

[0605] Users interact with the chatbot to get more information.

[0606] Users can enter specific questions about the plan presented by the chatbot.

[0607] For example, send a question such as "Tell me more about this tourist spot."

[0608] The terminal sends this question to the server.

[0609] Step 6:

[0610] The server provides information based on the user's question.

[0611] The server analyzes the user's question and retrieves relevant details from the database.

[0612] For example, detailed information such as "At this tourist spot, you can enjoy beach volleyball and kayaking" is sent to the chatbot.

[0613] The user can view this information through their device.

[0614] Step 7:

[0615] Customize your travel plans based on the information you provide.

[0616] Users can modify or add to their travel plans through interactions with the chatbot.

[0617] For example, "Instead of this hotel, find another hotel closer to the beach."

[0618] The terminal sends this request to the server.

[0619] Step 8:

[0620] The server regenerates the itinerary and presents it to the user.

[0621] The server regenerates the travel plan based on the user's new requirements and presents the customized plan to the user through the chatbot.

[0622] For example, it presents a new plan such as "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and beach volleyball..."

[0623] Step 9:

[0624] The user finalizes their travel plans and completes the booking process.

[0625] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[0626] The terminal sends this request to the server.

[0627] Step 10:

[0628] The server supports the reservation process.

[0629] The server connects to the reservation system to check availability of the hotel or flight the user desires.

[0630] If the reservation is successful, confirmation information is sent to the user via the terminal.

[0631] For example, information such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX" may be displayed.

[0632] Step 11:

[0633] Manage and view your trip history.

[0634] To check their travel history, users type "view past travel history."

[0635] The terminal sends this request to the server.

[0636] Step 12:

[0637] The server provides the user's travel history.

[0638] The server retrieves the user's past travel data from the database, formats it, and sends it to the terminal.

[0639] For example, it will display "Trip to Hawaii in August 2022: 5 nights and 7 days, information on tourist spots visited and hotels stayed."

[0640] Example 1

[0641] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0642] Conventional travel plan creation systems have had difficulty automatically generating specific and personalized travel plans based on users' requests. Furthermore, they lacked a means for users to interactively customize the generated travel plans, or a function to support the reservation process based on the customized plans. This has resulted in problems that make it difficult for users to efficiently create travel plans and smoothly complete the necessary reservation procedures.

[0643] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0644] In this invention, the server includes an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the user's questions and requests input through the input means, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting the reservation procedure based on the customized travel plan, a management means for managing the user's travel history and making it viewable, a means for the generation means to generate a travel plan using a generative AI model, a means for presenting the generated travel plan to the user in chatbot format, a means for the user to obtain detailed information by dialogue with the chatbot, and a means for retrieving travel plan data from a database based on the user's input. This enables efficient generation, customization, and support for the reservation procedure of a personalized travel plan based on the user's requests.

[0645] "User" means any person or entity that uses the System to create and customize a travel plan.

[0646] "Input means" refers to the interface through which a user inputs travel-related questions or requests into the system, such as a form or text entry field displayed on a computer or smartphone screen.

[0647] "Generation Means" refers to the functionality for generating a travel plan based on information provided by the user through the Input Means, including retrieving information from a generative AI model or database.

[0648] The "presentation means" refers to a mechanism for displaying the generated travel plan to the user. Specifically, it includes a chatbot-style user interface.

[0649] "Customization means" refers to the function of modifying and adjusting the travel plan generated through interaction with the user.

[0650] "Booking Method" refers to functionality that supports booking accommodation and transportation based on customized travel plans.

[0651] "Management means" refers to the function that manages a user's past travel history and allows the user to view it.

[0652] "Generative AI model" refers to an artificial intelligence model that generates travel plans based on information provided by users, including models that utilize natural language processing and machine learning.

[0653] A "chatbot" is an automated response program that interacts with users to exchange information.

[0654] "Database" refers to a collection of electronic data for storing information (such as tourist attractions, hotels, and transportation options) required to generate a travel plan.

[0655] A "prompt sentence" refers to a sentence that is input to a generative AI model to generate a specific travel plan.

[0656] The present invention is a system that allows users to input travel-related questions and requests, and then generate, customize, book, and manage travel plans based on that information. This system uses the following hardware and software.

[0657] Users access an application on their smartphone or computer screen and input information such as travel destination, theme, budget, etc. For example, they might input conditions such as "I want to plan a family trip to Hawaii this summer, with a budget of 300,000 yen, and I want to stay at a resort hotel." This input information is then sent from the device to the server.

[0658] The server uses natural language processing tools to analyze the information received from users. Specifically, it uses NLP tools such as Google Cloud Natural Language API to analyze the input text and extract important keywords. It also collects data on tourist attractions, hotels, and transportation options from databases and external APIs (e.g., Google Places API, Expedia API).

[0659] Based on the collected data, the server generates a travel plan using a generative AI model (e.g., OpenAI GPT-4). The server inputs a prompt such as "Generate a family travel plan in Hawaii within a budget of 300,000 yen" into the generative AI model, which then generates an appropriate travel plan. The generated travel plan is converted into a chatbot-style UI and presented to the user via the device.

[0660] Users use the chatbot to get more information about the proposed itinerary. For example, they can type "Tell me more about this tourist spot," and the device will send this request to the server. The server will then query a database or external API to gather relevant details and present them to the user, providing them with specific information based on the proposed itinerary.

[0661] If users want to customize their travel plans, they can input instructions through the chatbot. The server will retrieve the data again based on the new request and generate an updated plan using the generative AI model. For example, a user could instruct the chatbot, "Instead of this hotel, find another hotel closer to the beach."

[0662] Finally, when the user has finalized their travel plans and wants to proceed with the reservation, they enter, "I'd like to confirm this plan and book a hotel and flight." The device sends this request to the server, which then connects with the reservation system to check the availability of the accommodation and transportation options desired by the user. If the reservation is successful, confirmation information is provided to the user. Information such as "Hotel reservation completed. Confirmation number is XXXXXX" is displayed.

[0663] Additionally, users can check their past travel history. By inputting the command "View past travel history," the device sends this request to the server, which retrieves and displays the user's past travel data from the database. For example, "A trip to Hawaii in August 2022: 5 nights and 7 days, tourist spots visited, and hotel information."

[0664] The system allows users to easily create, customize, book and manage efficient and personalized travel plans.

[0665] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0666] Program processing

[0667] Step 1:

[0668] A user accesses a travel planning application on a smartphone or computer screen and inputs information such as the desired travel destination, theme, and budget. Specifically, the user writes in the input form, "I'd like to plan a trip to Hawaii for my family this summer. My budget is 300,000 yen. I'd like to stay at a resort hotel." The device then sends the input information to the server using an HTTP request. The input data is the user's request (travel destination, theme, budget, etc.).

[0669] Step 2:

[0670] The server uses natural language processing (NLP) tools to process the unstructured text information received from the user. Specifically, it uses the Google Cloud Natural Language API to analyze the text, tokenize it, and extract keywords. The input data is the unstructured text based on the user's request, and the output data is important keywords (e.g., "Hawaii," "family-friendly," "300,000 yen").

[0671] Step 3:

[0672] The server queries its own database or external APIs (e.g., Google Places API, Expedia API) based on the extracted keywords to collect data on related tourist attractions, hotels, and transportation options. For example, it obtains information such as "tourist attractions in Hawaii, resort hotels available within a budget, and family-friendly activities." The input data is a set of keywords, and the output data is related travel information (tourist attractions, hotels, and transportation options).

[0673] Step 4:

[0674] The server uses the collected data as input to generate an initial travel plan using a generative AI model (e.g., OpenAI GPT-4). The model is given a prompt, "Generate a family trip to Hawaii within a budget of 300,000 yen," and a specific travel plan is generated. The generated travel plan includes itineraries, activities, accommodations, etc. The input data is the travel prompt and travel-related data from an external API, and the output data is the generated travel plan.

[0675] Step 5:

[0676] The server formats the generated travel plan in chatbot format and sends it to the terminal to present to the user. Specifically, it converts the travel plan into text format and displays it on the user interface. The terminal displays the received travel plan information on the screen. The input data is the generated travel plan, and the output data is the travel plan text displayed on the screen.

[0677] Step 6:

[0678] The user inputs detailed information and customization instructions for the travel plan presented in chatbot format. For example, the user may give instructions such as "Tell me more about this tourist spot" or "Instead of this hotel, find another hotel closer to the beach." The device then sends these instructions to the server. The input data is the user's questions and customization instructions, and the output data is the detailed information and customization request.

[0679] Step 7:

[0680] Based on the user's questions and customization instructions, the server queries the database or external API again to collect and generate detailed information and a new travel plan. Specifically, the updated conditions are input into the generative AI model to obtain a revised plan. It also collects detailed information about tourist spots and re-suggested hotels and presents them to the user in the form of a chatbot. The input data is the detailed information and new conditions, and the output data is the updated travel plan and detailed information.

[0681] Step 8:

[0682] The user finalizes the travel plan and inputs instructions to complete the reservation process. For example, the user might say, "I'd like to finalize this plan and book a hotel and flight." The device then sends this instruction to the server. The input data is the final confirmation instruction from the user, and the output data is the reservation request.

[0683] Step 9:

[0684] The server connects with a reservation system (e.g., hotel reservation system, airline reservation system) to check the availability of the specified accommodation or transportation. If the reservation is successful, it generates confirmation information and sends it to the terminal. For example, the user is provided with information such as "Your hotel reservation has been completed. The confirmation number is XXXXXX." The input data is the reservation request, and the output data is the reservation confirmation information.

[0685] Step 10:

[0686] A user can make a request to view past travel history. When the user inputs the instruction "Show past travel history," the terminal sends this request to the server. The input data is a request for past travel history, and the output data is a request to display travel history.

[0687] Step 11:

[0688] The server retrieves the user's past travel data from the database, formats it, and sends it to the terminal. For example, it provides data to display "A trip to Hawaii in August 2022: 5 nights and 7 days, information on tourist spots visited, and hotels stayed." The input data is a past travel history request, and the output data is the formatted past travel data.

[0689] (Application example 1)

[0690] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0691] Conventional travel planning systems offer limited means for users to visually grasp details about travel destinations and accommodations, leaving a need for improved user experience. There is also a demand for intuitive operation via voice input and gestures when customizing travel plans. Furthermore, there is a growing need to provide a realistic travel experience in advance by generating and presenting travel plans in a virtual environment. New technologies are needed to address these challenges.

[0692] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0693] In this invention, the server includes an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the user's questions and requests input through the input means, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting reservation procedures based on the customized travel plan, a management means for managing and making the user's travel history viewable, a means for the user to wear a head-mounted display and create a travel plan in a virtual environment, a means for displaying 3D views of tourist attractions and accommodations in the virtual environment so the user can visually confirm them, and a means for the user to input questions and requests by voice input or gestures. This allows the user to check and customize the travel plan in real time in the virtual environment, enabling visual and intuitive travel planning.

[0694] "User" means any individual or entity that uses the Travel Planning System.

[0695] "Input Means" refers to a device or interface through which a User inputs travel-related questions or requests into the System.

[0696] "Generation means" refers to a combination of software or hardware for generating a travel plan based on information input by a user.

[0697] "Presentation means" refers to a device or interface for displaying the generated travel plan to the user.

[0698] "Customization means" refers to a device or interface for modifying or changing a travel plan through interaction with the user.

[0699] "Reservation Instrument" means a device or interface that supports the reservation process based on a customized travel plan.

[0700] "Management means" refers to databases and software used to store and make available users' travel history.

[0701] A "head-mounted display" refers to a device worn by a user that displays visual information in three dimensions.

[0702] "Virtual environment" refers to a computer-generated 3D space constructed to allow users to experience virtual reality.

[0703] "3D views of tourist attractions and accommodation facilities" refers to images and videos that visually display details of tourist attractions and accommodation facilities in three dimensions.

[0704] "Voice input" refers to a means by which a user can give instructions or information to a system through voice.

[0705] "Gesture input" refers to a means by which a user gives instructions or information to a system through hand or body movements.

[0706] To realize this invention, it is necessary to build a system using the following steps: First, provide an input means for users to input their travel-related questions and requests. This input means can include a smartphone, a computer touch panel, a voice input device, etc.

[0707] Hardware and Software Use

[0708] Hardware

[0709] Head-mounted display (HMD): A device worn by the user to provide a virtual reality environment, such as Oculus Rift or HTC Vive.

[0710] software

[0711] Unity: Builds a virtual reality environment to display travel plans within a virtual environment.

[0712] Azure Chatbot Service: Provides a conversational interface that enables interaction with users.

[0713] GCP AI Platform: Hosts generative AI models and generates itineraries based on user questions and requests.

[0714] Firebase: Manages user travel history and booking data.

[0715] Program processing

[0716] Input Method

[0717] Users put on the HMD and log in to the virtual environment. They then input their travel questions and requests into the system using voice and gestures. For example, they might say, "A fun trip to Europe for the whole family." This voice input is then converted to text using the Google Cloud Speech-to-Text API.

[0718] generation means

[0719] The server receives user input data and generates a travel plan using a generative AI model hosted on the GCP AI Platform, for example, a list of recommended tourist spots and accommodations for a family trip to Europe.

[0720] Presentation means

[0721] The generated itinerary is presented to the user in a virtual environment using Unity, where the user can visually see 3D views of tourist attractions and accommodations.

[0722] Customization methods

[0723] Users can interact with the Azure Chatbot Service to customize the details of their travel plans, such as "Tell me more resort hotels."

[0724] Reservation procedure

[0725] The system automatically processes reservations based on the plan customized by the user, manages reservation data using Firebase, and provides confirmation information to the user once the reservation is complete.

[0726] management measures

[0727] Users' travel history is managed by Firebase and can be viewed by users. Past travel history and detailed information can be checked at any time.

[0728] Specific examples

[0729] The following example prompt sentences will be used to explain how the system works:

[0730] When a user types "Tell me about your trip to Hawaii," the generative AI model responds as follows:

[0731] "Prompt to generate a travel plan:

[0732] User request: I'm planning a summer vacation to Hawaii with my family. My budget is under 300,000 yen, and I'd like to stay at a resort hotel. I enjoy snorkeling and shopping. Based on the input criteria, please generate a 7-day itinerary that includes the following elements:

[0733] Required elements:

[0734] List of tourist attractions

[0735] Family Activities

[0736] Budget-friendly resort hotels

[0737] Dining options

[0738] Shopping spot

[0739] Based on the above prompts, the generative AI model can construct a specific travel plan and provide it to the user through the virtual assistant.

[0740] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0741] Step 1:

[0742] The user puts on a head-mounted display and logs into the virtual environment.

[0743] Input: User ID and authentication information

[0744] Output: Home screen in the virtual environment

[0745] Specific operation: The user receives visual information through the HMD and enters authentication information on the login screen. The device sends this information to the server, and if authentication is successful, the home screen is displayed.

[0746] Step 2:

[0747] Users input travel questions and requests using voice input or gestures.

[0748] Input: Voice and gesture data

[0749] Output: Text data (questions and requests)

[0750] What it does: The user says, "I want to plan a fun family trip to Europe." The device converts this into text using the Google Cloud Speech-to-Text API and sends it to the server.

[0751] Step 3:

[0752] The server analyzes user information and generates an initial travel plan using a generative AI model.

[0753] Input: User's question or request (text data)

[0754] Output: Initial travel plan (text data)

[0755] Specific operation: The server receives the text data and calls the generative AI model on the GCP AI Platform. The generative AI model generates a "European travel plan for families" and returns it to the server.

[0756] Step 4:

[0757] The server generates a travel plan and presents it in a virtual environment.

[0758] Input: Initial travel plan (text data)

[0759] Output: 3D view of the trip plan

[0760] What it does: The initial travel plan is passed to Unity as text data, and tourist attractions and accommodations are displayed in a 3D view, which the user can view through the HMD.

[0761] Step 5:

[0762] Users interact with chatbots to customize their travel plans.

[0763] Input: User's new question or request (text data)

[0764] Output: Customized travel plan (text data)

[0765] How it works: When a user types "find other resort hotels instead of this one," the device sends this to the server, which then invokes the generative AI model again to generate a customized itinerary based on the new request.

[0766] Step 6:

[0767] Users make reservations based on customized travel plans.

[0768] Input: Customized travel plan (text data)

[0769] Output: Reservation confirmation information

[0770] Specific operation: The user inputs "I would like to confirm this plan and make a reservation." The device sends this to the server, which then connects to the reservation system to check availability. If the reservation is successful, the confirmation information is provided to the user.

[0771] Step 7:

[0772] Users can manage and view their travel history.

[0773] Input: User ID and authentication information

[0774] Output: List of trip history

[0775] Specific operation: The user types "Show me my past travel history." The device sends this to the server, which retrieves past travel data from Firebase, formats it, and displays it to the user.

[0776] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0777] The present invention provides a system including an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the questions and requests input through the input means, a presentation means for presenting the generated travel plan, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting the reservation procedure based on the customized travel plan, a management means for managing the user's travel history and making it viewable, and an emotion engine for recognizing the user's emotions. Furthermore, the emotion engine is used to analyze the user's emotional state, and the travel plan is generated and customized based on that information. Specific embodiments of the present invention are described below.

[0778] Program processing

[0779] 1. Users enter their travel questions and requests

[0780] Users access the screen of their smartphone or computer and enter desired conditions such as travel destination, theme, budget, etc. into an input form. For example, they might enter, "I'd like to plan a family trip to Hawaii this summer, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[0781] The terminal transmits this input information to the server.

[0782] 2. The server analyzes the user information

[0783] The server analyzes the data received from the user and extracts keywords such as travel destination, theme, budget, etc. For example, keywords such as "Hawaii," "family," and "budget of 300,000 yen" are extracted.

[0784] The server then searches a database to retrieve suggested attractions, hotels, and transportation options that match these keywords.

[0785] 3. The emotion engine analyzes the user's emotional state

[0786] The emotion engine analyzes the user's emotional state from input and voice data, for example, identifying emotions such as "happiness," "anxiety," and "expectation" from input text and voice.

[0787] The emotion engine transmits the user's emotional state to the server.

[0788] 4. The server generates the initial itinerary

[0789] The server uses a generative AI model to generate an initial travel plan based on the acquired data and emotional information, such as "A 7-day trip to Hawaii. Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling..."

[0790] This initial itinerary is adjusted based on the user's emotional information, for example, "if you're looking to relax, we'll suggest a relaxed itinerary."

[0791] 5. The server presents the generated itinerary to the user.

[0792] The server presents the initial travel plan to the user through the chatbot, and the user can confirm the details of the travel plan through the chatbot.

[0793] 6. Users interact with chatbots to get more information

[0794] Users can input specific questions about the plan presented by the chatbot, such as "Tell me more about this tourist spot."

[0795] The terminal sends this question to the server.

[0796] 7. The server provides information based on the user's question.

[0797] The server analyzes the user's question and retrieves relevant details from the database, such as "You can enjoy beach volleyball and kayaking at this tourist spot," and sends the details to the chatbot.

[0798] The user can view this information through their device.

[0799] 8. Customize your travel plans based on the information you provide

[0800] Users can interact with the chatbot to modify or add to their travel plans, for example by typing, "Instead of this hotel, find another hotel closer to the beach."

[0801] The emotion engine monitors the user's emotional changes in real time during the conversation and transmits the emotional information to the server. For example, if the user expresses dissatisfaction, this is analyzed.

[0802] 9. The server regenerates the itinerary and presents it to the user.

[0803] The server regenerates the travel plan based on the user's new requests and emotional information, and presents a customized plan to the user through the chatbot, for example, "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and beach volleyball..."

[0804] 10. The user finalizes their travel plans and completes the booking process

[0805] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[0806] The terminal sends this request to the server.

[0807] 11. The server supports the reservation process

[0808] The server connects to the reservation system to check availability of the hotel and flight desired by the user. If the reservation is successful, the server sends a confirmation to the user via the terminal. For example, the server displays information such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX."

[0809] 12. Managing and Viewing Your Trip History

[0810] To check their travel history, users type "view past travel history."

[0811] The terminal sends this request to the server.

[0812] 13. The server provides the user's travel history

[0813] The server retrieves the user's past travel data from the database, formats it, and sends it to the device. For example, it displays "A trip to Hawaii in August 2022: 5 nights and 7 days, information on the tourist spots visited, and the hotel where the user stayed."

[0814] Specific examples

[0815] When a user uses their smartphone to type, "I want to take a family trip to Hawaii this summer," the device sends this information to the server. The server analyzes the received information and retrieves the most appropriate information from its database using keywords like "Hawaii," "family-friendly," and "budget of ¥300,000." The emotion engine then analyzes emotional information from the user's input and voice data, identifying, for example, "I'm looking for relaxation." The server then generates an initial travel plan based on this information and presents it to the user through a chatbot. If the user types, "Tell me more about snorkeling spots," the server searches for related information and provides more detailed information. During this process, the emotion engine monitors the user's emotions in real time and adjusts the plan as needed. Finally, the user can confirm the travel plan, complete the booking process, and easily check their past travel history. This series of operations results in efficient and personalized travel planning.

[0816] The processing flow will be explained below.

[0817] Step 1:

[0818] Users enter their travel questions and requests.

[0819] Users access the screen of their smartphone or computer and enter, "I would like to plan a trip to Hawaii with my family this summer vacation, my budget is 300,000 yen, and I would like to stay at a resort hotel."

[0820] The terminal transmits this input information to the server.

[0821] Step 2:

[0822] The server analyzes the user information.

[0823] The server analyzes the data received from the user and extracts keywords such as "Hawaii," "family," and "budget of 300,000 yen."

[0824] The server then searches a database to retrieve suggested attractions, hotels, and transportation options that match these keywords.

[0825] Step 3:

[0826] The emotion engine analyzes the user's emotional state.

[0827] The emotion engine analyzes the user's emotional state from their input and voice data, identifying emotions such as "joy," "anxiety," and "expectation."

[0828] The emotion engine transmits the user's emotional state to the server.

[0829] Step 4:

[0830] The server generates an initial itinerary.

[0831] The server uses a generative AI model to generate an initial travel plan based on the acquired data and emotional information.

[0832] For example, generate a detailed plan such as "7-day travel plan for Hawaii. Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling..."

[0833] The plan will be adjusted to take into account the user's emotional state, adding relaxing elements and so on.

[0834] Step 5:

[0835] The server presents the generated travel plan to the user.

[0836] The server presents the initial travel plan to the user via the chatbot and asks for confirmation.

[0837] Users can check the details of their travel plans through the chatbot via their device.

[0838] Step 6:

[0839] Users interact with the chatbot to get more information.

[0840] Users can then input questions about the plan presented by the chatbot, such as "Tell me more about this tourist spot."

[0841] The terminal sends this question to the server.

[0842] Step 7:

[0843] The server provides information based on the user's question.

[0844] The server analyzes the user's question and retrieves relevant information from a database.

[0845] For example, the chatbot provides users with detailed information such as, "At this tourist spot, you can enjoy beach volleyball and kayaking."

[0846] Step 8:

[0847] The emotion engine monitors the user's emotional changes in real time.

[0848] The emotion engine recognizes the emotions expressed by the user while interacting with the chatbot and sends this information to the server.

[0849] For example, if the user expresses an emotion of satisfaction, capture that.

[0850] Step 9:

[0851] Customize your travel plans based on the information you provide.

[0852] Users interact with the chatbot to make specific changes, such as "find another hotel closer to the beach."

[0853] The terminal sends this request to the server.

[0854] Step 10:

[0855] The server regenerates the itinerary.

[0856] The server regenerates the travel plan based on the user's new instructions and emotional information.

[0857] For example, it generates a new plan such as "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and beach volleyball..." and presents it to the user.

[0858] Step 11:

[0859] The user finalizes their travel plans and completes the booking process.

[0860] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[0861] The terminal sends this request to the server.

[0862] Step 12:

[0863] The server supports the reservation process.

[0864] The server connects to the reservation system to check availability of the hotel or flight the user desires.

[0865] If the reservation is successful, confirmation information is sent to the user via the terminal.

[0866] For example, information such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX" may be displayed.

[0867] Step 13:

[0868] Manage and view your travel history.

[0869] To check their travel history, users type "view past travel history."

[0870] The terminal sends this request to the server.

[0871] Step 14:

[0872] The server provides the user's travel history.

[0873] The server retrieves the user's past travel data from the database, formats it, and sends it to the terminal.

[0874] For example, it will display "Trip to Hawaii in August 2022: 5 nights and 7 days, information on tourist spots visited and hotels stayed."

[0875] Example 2

[0876] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0877] Conventional travel plan generation systems only provide standard plans without considering the user's emotional state, and therefore are unable to respond to individual user needs and emotions. This makes it difficult to provide a travel plan that satisfies the user, resulting in a lot of effort required for customization and a decrease in satisfaction.

[0878] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the user's questions and requests input through the input means, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting the reservation procedure based on the customized travel plan, a management means for managing the user's travel history and making it viewable, an emotion recognition means for analyzing the user's emotional state, and an adjustment means for adjusting the travel plan based on the emotion information analyzed by the emotion recognition means. This makes it possible to provide a personalized travel plan that takes the user's emotional state into consideration.

[0879] "Input means" refers to the means by which a user inputs questions or requests regarding a trip.

[0880] The "generation means" is a means for generating a travel plan based on the user's questions and requests input through the input means.

[0881] The "presentation means" is a means for presenting the generated travel plan to the user.

[0882] "Customization means" refers to a means for customizing a travel plan through interaction with the user.

[0883] A "reservation tool" is a tool that supports the reservation process based on a customized travel plan.

[0884] "Management means" refers to the means for managing and making available the user's travel history.

[0885] "Emotion recognition means" is a means for analyzing the emotional state of a user.

[0886] The "adjustment means" is a means for adjusting the travel plan based on the emotion information analyzed by the emotion recognition means.

[0887] The present invention is a system for planning and managing travel, which aims to provide a personalized travel plan based on user input. The system includes an input means for a user to input travel questions and requests, a generation means for generating a travel plan based on the input information, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue, a reservation means for supporting a reservation procedure based on the customized plan, a management means for managing the user's travel history and making it viewable, an emotion recognition means for analyzing the user's emotional state, and an adjustment means for adjusting the plan based on the emotion recognition means.

[0888] System Overview

[0889] 1. Input Method

[0890] Users use their smartphones or computers to input desired conditions such as travel destination, theme, budget, etc. For example, they might input, "I'd like to plan a family trip to Hawaii during the summer vacation, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[0891] 2. Generation means

[0892] The server analyzes the input sent by the user and extracts the necessary keywords, such as "Hawaii," "family," and "budget of 300,000 yen."

[0893] The server uses a generative AI model to retrieve data on relevant tourist attractions, accommodations, and transportation from a database and generate a travel plan.

[0894] 3. Presentation means

[0895] The server then presents the generated travel plan to the user through an interface such as a chatbot, and the user can review this information on their device.

[0896] 4. Customization Methods

[0897] Users can use the chatbot to input specific questions or requests for modifications to the proposed itinerary, such as "Tell me more about this tourist spot" or "Instead of this hotel, find another hotel closer to the beach."

[0898] 5. Reservation Methods

[0899] Based on the plan confirmed by the user, the server checks the availability of hotels and flights in conjunction with the reservation system and completes the reservation process. If the reservation is successful, the server provides the user with this information. For example, a message such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX" is displayed on the terminal.

[0900] 6. Control measures

[0901] Users can submit requests to the system to view their past travel history, for example, by typing "Show me my past travel history," and the information will be displayed.

[0902] 7. Emotion recognition means

[0903] The emotion engine analyzes the text and voice data entered by the user to identify the user's emotional state, which allows it to understand what the user wants and reflect that in its plans.

[0904] 8. Adjustment means

[0905] The server dynamically adjusts the travel plan based on the emotional information obtained by the emotion engine, providing a plan that matches the user's emotions, such as suggesting a more relaxed schedule for a user looking to relax.

[0906] Specific examples

[0907] To explain the specific operation in more detail, some specific examples will be given.

[0908] Example prompt sentence:

[0909] "I want to plan a relaxing trip abroad with my family during summer vacation."

[0910] "Tell me about tourist spots in Europe"

[0911] "How much do you need to budget for your trip?"

[0912] When a user uses their smartphone to type, "I want to take my family on a summer vacation to Hawaii," the device sends this information to the server. The server analyzes the received information and retrieves the most relevant information from its database using keywords like "Hawaii," "family-friendly," and "budget of ¥300,000." The emotion engine then analyzes the user's input and voice data to identify emotional information, such as "I'm looking for relaxation." The server then generates an initial travel plan based on this information and presents it to the user through a chatbot. If the user types, "Tell me more about snorkeling spots," the server searches for related information and provides more information. During this process, the emotion engine monitors the user's emotions in real time and adjusts the plan as needed. Finally, the user can confirm the travel plan and proceed with the booking process. Users can also easily check their past travel history. This series of actions results in efficient and personalized travel planning.

[0913] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0914] Step 1:

[0915] Users enter travel questions and requests

[0916] Users use their smartphones or computers to input conditions such as desired travel destination, theme, budget, etc. Specifically, they might write in the input form, "I'd like to plan a family trip to Hawaii during the summer vacation, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[0917] Input: User's travel question or request

[0918] Output: User-entered criteria (travel destination, theme, budget, etc.)

[0919] Step 2:

[0920] The device sends the input information to the server

[0921] The device sends the user-entered conditions to the server in real time, with the data being sent via HTTP requests.

[0922] Input: Conditions entered by the user on the device

[0923] Output: User criteria sent to server

[0924] Step 3:

[0925] The server analyzes the user information and searches the database

[0926] The server analyzes the received user conditions and extracts important keywords (travel destination, budget, theme, etc.). For example, keywords such as "Hawaii," "family," and "budget of 300,000 yen" are extracted.

[0927] The server then uses these keywords to search a database to retrieve relevant tourist spots, hotels, and transportation options.

[0928] Input: User criteria sent to the server

[0929] Output: Important keywords, travel data retrieved from the database

[0930] Step 4:

[0931] Emotion engine analyzes the user's emotional state

[0932] The emotion engine analyzes user input and voice data to identify emotional states, such as joy, anxiety, and anticipation, for example.

[0933] The emotion engine transmits the analyzed emotion information to the server.

[0934] Input: User input, voice data

[0935] Output: Parsed emotion information

[0936] Step 5:

[0937] The server generates the initial itinerary

[0938] The server uses a generative AI model to generate an initial itinerary based on the acquired data and emotional information, such as "7-day Hawaii itinerary: Day 1: Beach visit, Day 2: Shopping, Day 3: Snorkeling."

[0939] The system takes into account the user's emotional information and optimizes the plan, such as adjusting to a more relaxed schedule if the user is looking to relax.

[0940] Input: Travel data obtained from the database, analyzed emotion information

[0941] Output: Initial itinerary

[0942] Step 6:

[0943] The server presents the generated itinerary to the user.

[0944] The server presents the initial travel plan to the user through the chatbot, and the user can review the plan on their device.

[0945] Input: Initial travel plan

[0946] Output: The itinerary presented to the user

[0947] Step 7:

[0948] Users interact with the chatbot to get more information

[0949] Users can then use the chatbot to ask specific questions about the proposed plan, such as "Tell me more about this tourist spot."

[0950] The terminal sends this question to the server.

[0951] Input: User's specific question

[0952] Output: The question sent to the server

[0953] Step 8:

[0954] The server provides information based on the user's question.

[0955] The server analyzes the user's question and retrieves appropriate details from the database, such as "You can enjoy beach volleyball and kayaking at this tourist spot," and sends this information to the chatbot.

[0956] The user can view this information through their device.

[0957] Input: User question, database details

[0958] Output: Answer with detailed information provided

[0959] Step 9:

[0960] Customize your travel plans based on the information you provide

[0961] Users can then use the details they have acquired to modify or add to their travel plans, for example, by saying, "Instead of this hotel, find another hotel closer to the beach."

[0962] The emotion engine monitors the user's emotional changes in real time through this dialogue and transmits the emotional information to the server.

[0963] Input: User's correction instructions, emotional information

[0964] Output: revised plan, emotional information

[0965] Step 10:

[0966] The server regenerates the itinerary and presents it to the user.

[0967] The server regenerates the travel plan based on the user's new requests and emotional information, and presents a customized plan through the chatbot, for example, "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and Beach Volleyball."

[0968] Input: User's new request, emotion information

[0969] Output: Regenerated itinerary

[0970] Step 11:

[0971] The user confirms their travel plans and completes the booking process

[0972] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[0973] The terminal sends this request to the server.

[0974] Input: User confirmation request

[0975] Output: Confirm request sent to server

[0976] Step 12:

[0977] The server handles the reservation process

[0978] The server connects to the reservation system to check availability of the hotel and flight desired by the user. If the reservation is successful, the server sends a confirmation to the user via the terminal. For example, the server displays information such as "Your hotel reservation has been completed. The confirmation number is XXXXXX."

[0979] Input: User's reservation request information, availability in the reservation system

[0980] Output: Reservation confirmation information

[0981] Step 13:

[0982] Users can manage and view their travel history

[0983] If a user wants to check their past travel history, they type "View past travel history."

[0984] The terminal sends this request to the server.

[0985] Input: User's request to view travel history

[0986] Output: View request sent to the server

[0987] Step 14:

[0988] The server provides the user's travel history

[0989] The server retrieves the user's past travel data from the database, formats it, and sends it to the device. For example, it displays "A trip to Hawaii in August 2022: 5 nights and 7 days, information on the tourist spots visited, and the hotel where the user stayed."

[0990] Input: Travel history data from the database

[0991] Output: Formatted travel history information

[0992] (Application example 2)

[0993] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0994] The problem to be solved by the present invention is to provide a system that efficiently generates a personalized meal plan taking into account the user's preferences and emotional state when the user inputs their meal requests and questions, and smoothly supports the delivery procedure, including the customization process.

[0995] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an input means for the user to input their wishes and requests regarding their request, a generation means for generating a plan based on the user's wishes and requests input through the input means, a presentation means for presenting the generated plan to the user, a customization means for customizing the plan through dialogue with the user, a reservation means for supporting procedures based on the customized plan, and a management means for managing the user's history and making it viewable. This enables the generation, customization, and reservation procedures of personalized meal plans.

[0996] "Requests" refer to the conditions desired by the user and the content of the services they would like to receive.

[0997] "Input means" refers to an interface that allows a user to input information into a device or system.

[0998] "Generation means" refers to the process or technical mechanism for automatically generating a specific plan or proposal based on input information.

[0999] "Presentation means" refers to the method or interface for displaying the generated plan or information to the user.

[1000] "Customization means" refers to a method or system that allows users to modify or change the plans or information presented to them to suit their preferences.

[1001] "Reservation Method" refers to the method or system for carrying out formal procedures based on the plan selected or modified by the User.

[1002] "Management means" refers to the method or system for storing and organizing users' past usage history and information, and making them available for viewing as needed.

[1003] An "automatic response system" refers to technology or mechanisms that allow a system to automatically communicate with users.

[1004] An "information repository" refers to a system or database for storing and managing various data and information.

[1005] A "generative AI model" refers to an algorithm or model that uses machine learning or artificial intelligence to generate new plans or information based on specific patterns or information.

[1006] The present invention is a system that includes an input means for a user to input meal preferences and questions, a generation means for generating a meal plan based on the preferences and questions input through the input means, a presentation means for presenting the generated plan to the user, a customization means for customizing the plan through dialogue with the user, a reservation means for supporting delivery procedures based on the customized plan, and a management means for managing the user's meal history and making it viewable.

[1007] Detailed System Description

[1008] 1. Input method:

[1009] Users use their smartphones to input their meal preferences, budget, delivery time, and other requests into the application's input form. If a user inputs, "I want to order a healthy, stress-reducing meal," this information is sent from the smartphone to the server.

[1010] 2. Generation means:

[1011] The server analyzes the received user information and performs sentiment analysis using the Emotion API. The analyzed data is then used to generate an appropriate meal plan using a generative AI model (e.g., GPT-4). This generative AI model uses the input data and the extracted sentiment information as prompts to suggest the optimal meal plan.

[1012] 3. Means of presentation:

[1013] The generated meal plan is presented to the user through a chatbot, which provides the user with the meal plan details in a conversational format.

[1014] 4. Customization methods:

[1015] Users can modify or add to the generated meal plan through dialogue with the chatbot. For example, if they input a request such as "I'd like to add avocado to my healthy bowl," the server will again use the generative AI model to generate a new plan.

[1016] 5. Reservation Method:

[1017] Once the user is satisfied with the customized meal plan and selects "I would like to order this plan," the server connects with the delivery system to process the reservation. If the reservation is successful, a confirmation is sent to the user.

[1018] 6. Control measures:

[1019] The user's past meal history is stored in a database and can be viewed as needed. When a user types "I want to check my past order history," the relevant information is displayed.

[1020] Hardware and software used

[1021] Device: Smartphone (iOS / Android)

[1022] Emotion engine: Emotion API (e.g. Microsoft Azure Cognitive Services)

[1023] Generative AI model: GPT-4 (OpenAI API)

[1024] Database: PostgreSQL (Amazon RDS)

[1025] Frontend: React Native

[1026] Backend: Node.js (Express)

[1027] Example prompt sentence

[1028] "The user wants to order a meal. The user's wishes are as follows: Travel destination: Please suggest a menu with the theme of 'healthy and relaxing meals', within a budget of 2000 yen, and that can be delivered within 30 minutes. Furthermore, please consider 'stress,' 'relaxation,' and 'fatigue' based on the results of the sentiment analysis."

[1029] By implementing this invention, users can select, customize, and reserve a personalized meal plan based on their emotional state and preferences, resulting in a more satisfying delivery service.

[1030] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1031] Step 1:

[1032] A user accesses the smartphone application and inputs information such as meal preferences, questions, budget, and delivery time. For example, the user might input, "I would like to order a healthy and relaxing meal." The smartphone then sends this input information to the server.

[1033] Input: User's wishes, questions, budget, delivery time

[1034] Output: User information sent to the server

[1035] Step 2:

[1036] The server analyzes the information received from the user and extracts the input requests and questions. The analyzed data is subjected to emotion analysis using the Emotion API to identify the user's emotional state (e.g., "stressed," "relaxed," "fatigued").

[1037] Input: User information sent to the server

[1038] Data processing / data calculation: Extraction of wishes and questions, sentiment analysis

[1039] Output: Extracted data and sentiment information

[1040] Step 3:

[1041] The server uses a generative AI model (e.g., GPT-4) to generate an appropriate meal plan based on the extracted data and emotional information. At this time, the server passes prompts to the generative AI model, which then proposes a plan that reflects the user's preferences and emotional state.

[1042] Input: Extracted data and sentiment information

[1043] Data processing / data calculation: Prompt sentence generation, plan generation using generative AI models

[1044] Output: Generated meal plan

[1045] Step 4:

[1046] The server presents the generated meal plan to the user through the chatbot, and the user can check the details of the plan via their smartphone.

[1047] Input: Generated meal plan

[1048] Output: The plan presented to the user through the chatbot

[1049] Step 5:

[1050] Users can change or add to their meal plan through interactions with the chatbot, for example by typing, "I'd like to add avocado to my healthy bowl." The smartphone then sends this request to the server.

[1051] Input: User request

[1052] Output: New request sent to the server

[1053] Step 6:

[1054] The server regenerates the meal plan using the generative AI model again based on the new request and emotional information received. It generates a new prompt sentence and passes it to the generative AI model to suggest a new plan.

[1055] Input: New request, emotion information

[1056] Data processing / data calculation: Regeneration of prompt sentences, regeneration by generative AI model

[1057] Output: Regenerated meal plan

[1058] Step 7:

[1059] The server then presents the regenerated meal plan to the user via the chatbot, and the user can check the plan details again via their smartphone.

[1060] Input: Regenerated meal plan

[1061] Output: Plan re-presented through the chatbot

[1062] Step 8:

[1063] When the user is finally satisfied with the plan and types "I would like to order this plan," the smartphone sends this request to the server, which then coordinates with the delivery system to complete the reservation process and sends reservation confirmation information to the smartphone.

[1064] Input: Final confirmation request by user

[1065] Output: Final confirmation request sent to the server, booking confirmation information

[1066] Step 9:

[1067] The user's meal history is stored in a database. When the user types "I want to check my past order history," the smartphone sends this request to the server, which retrieves the relevant information from the database and displays it to the user.

[1068] Input: User history check request

[1069] Output: History information retrieved from the database, history information displayed to the user

[1070] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1071] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1072] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1073] [Third embodiment]

[1074] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1075] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1076] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1077] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1078] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1079] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1080] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1081] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1082] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1083] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1084] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1085] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1086] The present invention provides a system that includes an input means for users to input travel-related questions and requests, a generation means that generates a travel plan using a generative AI model based on the input information, a presentation means that presents the generated plan to the user, a customization means that customizes the travel plan through dialogue with the user, a reservation means that supports the reservation procedure based on the customized travel plan, and a management means that manages the user's travel history and makes it viewable. Below, the program processing of this system is explained in natural language. Specific examples are also provided.

[1087] Program processing

[1088] 1. Users enter their travel questions and requests

[1089] Users access the screen of their smartphone or computer and enter desired conditions such as travel destination, theme, budget, etc. into an input form. For example, they might enter, "I'd like to plan a family trip to Hawaii this summer, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[1090] The terminal (smartphone or computer) sends the entered information to the server.

[1091] 2. The server analyzes the user information and generates an initial travel plan

[1092] The server analyzes the received user information and retrieves tourist attractions, hotels, and transportation information from the database based on the user's request. Specifically, it retrieves a list of "tourist attractions in Hawaii, resort hotels available within a budget, and family activities."

[1093] The server uses a generative AI model to generate an initial itinerary based on this data, such as "A 7-day trip to Hawaii: Day 1 - Beach, Day 2 - Shopping, Day 3 - Snorkeling..."

[1094] The server presents the generated travel plan to the user in the form of a chatbot.

[1095] 3. Users interact with the chatbot to get more information

[1096] The user reviews the chatbot's suggestions and types, "Tell me more about this tourist spot."

[1097] The device sends the user's question to the server, which retrieves relevant information from a database, providing detailed information such as "You can enjoy beach volleyball and kayaking at this tourist spot."

[1098] 4. Customize your travel plans based on the information you provide

[1099] Users can input instructions to modify their plans based on the chatbot's suggestions, for example, "Instead of this hotel, find another one closer to the beach."

[1100] The server regenerates the itinerary based on the new request and presents the updated itinerary to the user, such as "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and Beach Volleyball..."

[1101] 5. The user finalizes their travel plans and completes the booking process

[1102] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[1103] The device sends this request to the server, which then connects with the reservation system to check availability of the hotel and flight the user desires. If the reservation is successful, the user is given confirmation information, such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX."

[1104] 6. Managing and viewing your travel history

[1105] To check their travel history, users type "view past travel history."

[1106] The device sends this request to the server, which retrieves the user's past travel data from the database. It formats and displays information such as "Retrieve details of past Hawaii trips and European tours." For example, it displays information such as "August 2022 Hawaii trip: 5 nights and 7 days, tourist spots visited, and hotel information."

[1107] Specific examples

[1108] When a user uses their smartphone to input "I want to take a family trip to Hawaii this summer," the device sends this information to a server. The server analyzes the received information and retrieves the most suitable information from its database using the keywords "Hawaii, family-friendly, budget 300,000 yen." It then uses a generative AI model to generate a travel plan suited to the user and presents it to the user through a chatbot. If the user inputs "Tell me more about places where I can snorkel," the server searches for related information and provides more detailed information. The user can then reconstruct their plan based on the information, ultimately booking hotels and flights, and review their detailed travel history at a later date. This series of actions results in efficient and personalized travel planning.

[1109] The processing flow will be explained below.

[1110] Step 1:

[1111] Users enter their travel questions and requests.

[1112] Users use their smartphones or computers to enter their desired travel destination, theme, budget, and other preferences into an input form.

[1113] For example, you might enter information such as, "I want to plan a trip to Hawaii with my family during summer vacation, my budget is 300,000 yen, and I want to stay at a resort hotel."

[1114] The terminal transmits this input information to the server.

[1115] Step 2:

[1116] The server analyzes the user information.

[1117] The server analyzes the data received from the user and extracts keywords such as travel destination, theme, and budget.

[1118] For example, extract keywords such as "Hawaii," "family," and "budget 300,000 yen."

[1119] The server then searches a database to retrieve suggested attractions, hotels, and transportation options that match these keywords.

[1120] Step 3:

[1121] The server generates an initial itinerary.

[1122] The server uses a generative AI model to generate an initial itinerary based on the acquired data.

[1123] For example, generate a plan such as "7-day Hawaii travel plan. Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling..."

[1124] The generated travel plan is formatted in a user-friendly format.

[1125] Step 4:

[1126] The server presents the generated travel plan to the user.

[1127] The server presents an initial travel plan to the user through a chatbot.

[1128] Users can check the details of their travel plans through the chatbot.

[1129] Step 5:

[1130] Users interact with the chatbot to get more information.

[1131] Users can enter specific questions about the plan presented by the chatbot.

[1132] For example, send a question such as "Tell me more about this tourist spot."

[1133] The terminal sends this question to the server.

[1134] Step 6:

[1135] The server provides information based on the user's question.

[1136] The server analyzes the user's question and retrieves relevant details from the database.

[1137] For example, detailed information such as "At this tourist spot, you can enjoy beach volleyball and kayaking" is sent to the chatbot.

[1138] The user can view this information through their device.

[1139] Step 7:

[1140] Customize your travel plans based on the information you provide.

[1141] Users can modify or add to their travel plans through interactions with the chatbot.

[1142] For example, "Instead of this hotel, find another hotel closer to the beach."

[1143] The terminal sends this request to the server.

[1144] Step 8:

[1145] The server regenerates the itinerary and presents it to the user.

[1146] The server regenerates the travel plan based on the user's new requirements and presents the customized plan to the user through the chatbot.

[1147] For example, it presents a new plan such as "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and beach volleyball..."

[1148] Step 9:

[1149] The user finalizes their travel plans and completes the booking process.

[1150] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[1151] The terminal sends this request to the server.

[1152] Step 10:

[1153] The server supports the reservation process.

[1154] The server connects to the reservation system to check availability of the hotel or flight the user desires.

[1155] If the reservation is successful, confirmation information is sent to the user via the terminal.

[1156] For example, information such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX" may be displayed.

[1157] Step 11:

[1158] Manage and view your trip history.

[1159] To check their travel history, users type "view past travel history."

[1160] The terminal sends this request to the server.

[1161] Step 12:

[1162] The server provides the user's travel history.

[1163] The server retrieves the user's past travel data from the database, formats it, and sends it to the terminal.

[1164] For example, it will display "Trip to Hawaii in August 2022: 5 nights and 7 days, information on tourist spots visited and hotels stayed."

[1165] Example 1

[1166] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1167] Conventional travel plan creation systems have had difficulty automatically generating specific and personalized travel plans based on users' requests. Furthermore, they lacked a means for users to interactively customize the generated travel plans, or a function to support the reservation process based on the customized plans. This has resulted in problems that make it difficult for users to efficiently create travel plans and smoothly complete the necessary reservation procedures.

[1168] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1169] In this invention, the server includes an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the user's questions and requests input through the input means, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting the reservation procedure based on the customized travel plan, a management means for managing the user's travel history and making it viewable, a means for the generation means to generate a travel plan using a generative AI model, a means for presenting the generated travel plan to the user in chatbot format, a means for the user to obtain detailed information by dialogue with the chatbot, and a means for retrieving travel plan data from a database based on the user's input. This enables efficient generation, customization, and support for the reservation procedure of a personalized travel plan based on the user's requests.

[1170] "User" means any person or entity that uses the System to create and customize a travel plan.

[1171] "Input means" refers to the interface through which a user inputs travel-related questions or requests into the system, such as a form or text entry field displayed on a computer or smartphone screen.

[1172] "Generation Means" refers to the functionality for generating a travel plan based on information provided by the user through the Input Means, including retrieving information from a generative AI model or database.

[1173] The "presentation means" refers to a mechanism for displaying the generated travel plan to the user. Specifically, it includes a chatbot-style user interface.

[1174] "Customization means" refers to the function of modifying and adjusting the travel plan generated through interaction with the user.

[1175] "Booking Method" refers to functionality that supports booking accommodation and transportation based on customized travel plans.

[1176] "Management means" refers to the function that manages a user's past travel history and allows the user to view it.

[1177] "Generative AI model" refers to an artificial intelligence model that generates travel plans based on information provided by users, including models that utilize natural language processing and machine learning.

[1178] A "chatbot" is an automated response program that interacts with users to exchange information.

[1179] "Database" refers to a collection of electronic data for storing information (such as tourist attractions, hotels, and transportation options) required to generate a travel plan.

[1180] A "prompt sentence" refers to a sentence that is input to a generative AI model to generate a specific travel plan.

[1181] The present invention is a system that allows users to input travel-related questions and requests, and then generate, customize, book, and manage travel plans based on that information. This system uses the following hardware and software.

[1182] Users access an application on their smartphone or computer screen and input information such as travel destination, theme, budget, etc. For example, they might input conditions such as "I want to plan a family trip to Hawaii this summer, with a budget of 300,000 yen, and I want to stay at a resort hotel." This input information is then sent from the device to the server.

[1183] The server uses natural language processing tools to analyze the information received from users. Specifically, it uses NLP tools such as Google Cloud Natural Language API to analyze the input text and extract important keywords. It also collects data on tourist attractions, hotels, and transportation options from databases and external APIs (e.g., Google Places API, Expedia API).

[1184] Based on the collected data, the server generates a travel plan using a generative AI model (e.g., OpenAI GPT-4). The server inputs a prompt such as "Generate a family travel plan in Hawaii within a budget of 300,000 yen" into the generative AI model, which then generates an appropriate travel plan. The generated travel plan is converted into a chatbot-style UI and presented to the user via the device.

[1185] Users use the chatbot to get more information about the proposed itinerary. For example, they can type "Tell me more about this tourist spot," and the device will send this request to the server. The server will then query a database or external API to gather relevant details and present them to the user, providing them with specific information based on the proposed itinerary.

[1186] If users want to customize their travel plans, they can input instructions through the chatbot. The server will retrieve the data again based on the new request and generate an updated plan using the generative AI model. For example, a user could instruct the chatbot, "Instead of this hotel, find another hotel closer to the beach."

[1187] Finally, when the user has finalized their travel plans and wants to proceed with the reservation, they enter, "I'd like to confirm this plan and book a hotel and flight." The device sends this request to the server, which then connects with the reservation system to check the availability of the accommodation and transportation options desired by the user. If the reservation is successful, confirmation information is provided to the user. Information such as "Hotel reservation completed. Confirmation number is XXXXXX" is displayed.

[1188] Additionally, users can check their past travel history. By inputting the command "View past travel history," the device sends this request to the server, which retrieves and displays the user's past travel data from the database. For example, "A trip to Hawaii in August 2022: 5 nights and 7 days, tourist spots visited, and hotel information."

[1189] The system allows users to easily create, customize, book and manage efficient and personalized travel plans.

[1190] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1191] Program processing

[1192] Step 1:

[1193] A user accesses a travel planning application on a smartphone or computer screen and inputs information such as the desired travel destination, theme, and budget. Specifically, the user writes in the input form, "I'd like to plan a trip to Hawaii for my family this summer. My budget is 300,000 yen. I'd like to stay at a resort hotel." The device then sends the input information to the server using an HTTP request. The input data is the user's request (travel destination, theme, budget, etc.).

[1194] Step 2:

[1195] The server uses natural language processing (NLP) tools to process the unstructured text information received from the user. Specifically, it uses the Google Cloud Natural Language API to analyze the text, tokenize it, and extract keywords. The input data is the unstructured text based on the user's request, and the output data is important keywords (e.g., "Hawaii," "family-friendly," "300,000 yen").

[1196] Step 3:

[1197] The server queries its own database or external APIs (e.g., Google Places API, Expedia API) based on the extracted keywords to collect data on related tourist attractions, hotels, and transportation options. For example, it obtains information such as "tourist attractions in Hawaii, resort hotels available within a budget, and family-friendly activities." The input data is a set of keywords, and the output data is related travel information (tourist attractions, hotels, and transportation options).

[1198] Step 4:

[1199] The server uses the collected data as input to generate an initial travel plan using a generative AI model (e.g., OpenAI GPT-4). The model is given a prompt, "Generate a family trip to Hawaii within a budget of 300,000 yen," and a specific travel plan is generated. The generated travel plan includes itineraries, activities, accommodations, etc. The input data is the travel prompt and travel-related data from an external API, and the output data is the generated travel plan.

[1200] Step 5:

[1201] The server formats the generated travel plan in chatbot format and sends it to the terminal to present to the user. Specifically, it converts the travel plan into text format and displays it on the user interface. The terminal displays the received travel plan information on the screen. The input data is the generated travel plan, and the output data is the travel plan text displayed on the screen.

[1202] Step 6:

[1203] The user inputs detailed information and customization instructions for the travel plan presented in chatbot format. For example, the user may give instructions such as "Tell me more about this tourist spot" or "Instead of this hotel, find another hotel closer to the beach." The device then sends these instructions to the server. The input data is the user's questions and customization instructions, and the output data is the detailed information and customization request.

[1204] Step 7:

[1205] Based on the user's questions and customization instructions, the server queries the database or external API again to collect and generate detailed information and a new travel plan. Specifically, the updated conditions are input into the generative AI model to obtain a revised plan. It also collects detailed information about tourist spots and re-suggested hotels and presents them to the user in the form of a chatbot. The input data is the detailed information and new conditions, and the output data is the updated travel plan and detailed information.

[1206] Step 8:

[1207] The user finalizes the travel plan and inputs instructions to complete the reservation process. For example, the user might say, "I'd like to finalize this plan and book a hotel and flight." The device then sends this instruction to the server. The input data is the final confirmation instruction from the user, and the output data is the reservation request.

[1208] Step 9:

[1209] The server connects with a reservation system (e.g., hotel reservation system, airline reservation system) to check the availability of the specified accommodation or transportation. If the reservation is successful, it generates confirmation information and sends it to the terminal. For example, the user is provided with information such as "Your hotel reservation has been completed. The confirmation number is XXXXXX." The input data is the reservation request, and the output data is the reservation confirmation information.

[1210] Step 10:

[1211] A user can make a request to view past travel history. When the user inputs the instruction "Show past travel history," the terminal sends this request to the server. The input data is a request for past travel history, and the output data is a request to display travel history.

[1212] Step 11:

[1213] The server retrieves the user's past travel data from the database, formats it, and sends it to the terminal. For example, it provides data to display "A trip to Hawaii in August 2022: 5 nights and 7 days, information on tourist spots visited, and hotels stayed." The input data is a past travel history request, and the output data is the formatted past travel data.

[1214] (Application example 1)

[1215] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1216] Conventional travel planning systems offer limited means for users to visually grasp details about travel destinations and accommodations, leaving a need for improved user experience. There is also a demand for intuitive operation via voice input and gestures when customizing travel plans. Furthermore, there is a growing need to provide a realistic travel experience in advance by generating and presenting travel plans in a virtual environment. New technologies are needed to address these challenges.

[1217] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1218] In this invention, the server includes an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the user's questions and requests input through the input means, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting reservation procedures based on the customized travel plan, a management means for managing and making the user's travel history viewable, a means for the user to wear a head-mounted display and create a travel plan in a virtual environment, a means for displaying 3D views of tourist attractions and accommodations in the virtual environment so the user can visually confirm them, and a means for the user to input questions and requests by voice input or gestures. This allows the user to check and customize the travel plan in real time in the virtual environment, enabling visual and intuitive travel planning.

[1219] "User" means any individual or entity that uses the Travel Planning System.

[1220] "Input Means" refers to a device or interface through which a User inputs travel-related questions or requests into the System.

[1221] "Generation means" refers to a combination of software or hardware for generating a travel plan based on information input by a user.

[1222] "Presentation means" refers to a device or interface for displaying the generated travel plan to the user.

[1223] "Customization means" refers to a device or interface for modifying or changing a travel plan through interaction with the user.

[1224] "Reservation Instrument" means a device or interface that supports the reservation process based on a customized travel plan.

[1225] "Management means" refers to databases and software used to store and make available users' travel history.

[1226] A "head-mounted display" refers to a device worn by a user that displays visual information in three dimensions.

[1227] "Virtual environment" refers to a computer-generated 3D space constructed to allow users to experience virtual reality.

[1228] "3D views of tourist attractions and accommodation facilities" refers to images and videos that visually display details of tourist attractions and accommodation facilities in three dimensions.

[1229] "Voice input" refers to a means by which a user can give instructions or information to a system through voice.

[1230] "Gesture input" refers to a means by which a user gives instructions or information to a system through hand or body movements.

[1231] To realize this invention, it is necessary to build a system using the following steps: First, provide an input means for users to input their travel-related questions and requests. This input means can include a smartphone, a computer touch panel, a voice input device, etc.

[1232] Hardware and Software Use

[1233] Hardware

[1234] Head-mounted display (HMD): A device worn by the user to provide a virtual reality environment, such as Oculus Rift or HTC Vive.

[1235] software

[1236] Unity: Builds a virtual reality environment to display travel plans within a virtual environment.

[1237] Azure Chatbot Service: Provides a conversational interface that enables interaction with users.

[1238] GCP AI Platform: Hosts generative AI models and generates itineraries based on user questions and requests.

[1239] Firebase: Manages user travel history and booking data.

[1240] Program processing

[1241] Input Method

[1242] Users put on the HMD and log in to the virtual environment. They then input their travel questions and requests into the system using voice and gestures. For example, they might say, "A fun trip to Europe for the whole family." This voice input is then converted to text using the Google Cloud Speech-to-Text API.

[1243] generation means

[1244] The server receives user input data and generates a travel plan using a generative AI model hosted on the GCP AI Platform, for example, a list of recommended tourist spots and accommodations for a family trip to Europe.

[1245] Presentation means

[1246] The generated itinerary is presented to the user in a virtual environment using Unity, where the user can visually see 3D views of tourist attractions and accommodations.

[1247] Customization methods

[1248] Users can interact with the Azure Chatbot Service to customize the details of their travel plans, such as "Tell me more resort hotels."

[1249] Reservation procedure

[1250] The system automatically processes reservations based on the plan customized by the user, manages reservation data using Firebase, and provides confirmation information to the user once the reservation is complete.

[1251] management measures

[1252] Users' travel history is managed by Firebase and can be viewed by users. Past travel history and detailed information can be checked at any time.

[1253] Specific examples

[1254] The following example prompt sentences will be used to explain how the system works:

[1255] When a user types "Tell me about your trip to Hawaii," the generative AI model responds as follows:

[1256] "Prompt to generate a travel plan:

[1257] User request: I'm planning a summer vacation to Hawaii with my family. My budget is under 300,000 yen, and I'd like to stay at a resort hotel. I enjoy snorkeling and shopping. Based on the input criteria, please generate a 7-day itinerary that includes the following elements:

[1258] Required elements:

[1259] List of tourist attractions

[1260] Family Activities

[1261] Budget-friendly resort hotels

[1262] Dining options

[1263] Shopping spot

[1264] Based on the above prompts, the generative AI model can construct a specific travel plan and provide it to the user through the virtual assistant.

[1265] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1266] Step 1:

[1267] The user puts on a head-mounted display and logs into the virtual environment.

[1268] Input: User ID and authentication information

[1269] Output: Home screen in the virtual environment

[1270] Specific operation: The user receives visual information through the HMD and enters authentication information on the login screen. The device sends this information to the server, and if authentication is successful, the home screen is displayed.

[1271] Step 2:

[1272] Users input travel questions and requests using voice input or gestures.

[1273] Input: Voice and gesture data

[1274] Output: Text data (questions and requests)

[1275] What it does: The user says, "I want to plan a fun family trip to Europe." The device converts this into text using the Google Cloud Speech-to-Text API and sends it to the server.

[1276] Step 3:

[1277] The server analyzes user information and generates an initial travel plan using a generative AI model.

[1278] Input: User's question or request (text data)

[1279] Output: Initial travel plan (text data)

[1280] Specific operation: The server receives the text data and calls the generative AI model on the GCP AI Platform. The generative AI model generates a "European travel plan for families" and returns it to the server.

[1281] Step 4:

[1282] The server generates a travel plan and presents it in a virtual environment.

[1283] Input: Initial travel plan (text data)

[1284] Output: 3D view of the trip plan

[1285] What it does: The initial travel plan is passed to Unity as text data, and tourist attractions and accommodations are displayed in a 3D view, which the user can view through the HMD.

[1286] Step 5:

[1287] Users interact with chatbots to customize their travel plans.

[1288] Input: User's new question or request (text data)

[1289] Output: Customized travel plan (text data)

[1290] How it works: When a user types "find other resort hotels instead of this one," the device sends this to the server, which then invokes the generative AI model again to generate a customized itinerary based on the new request.

[1291] Step 6:

[1292] Users make reservations based on customized travel plans.

[1293] Input: Customized travel plan (text data)

[1294] Output: Reservation confirmation information

[1295] Specific operation: The user inputs "I would like to confirm this plan and make a reservation." The device sends this to the server, which then connects to the reservation system to check availability. If the reservation is successful, the confirmation information is provided to the user.

[1296] Step 7:

[1297] Users can manage and view their travel history.

[1298] Input: User ID and authentication information

[1299] Output: List of trip history

[1300] Specific operation: The user types "Show me my past travel history." The device sends this to the server, which retrieves past travel data from Firebase, formats it, and displays it to the user.

[1301] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1302] The present invention provides a system including an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the questions and requests input through the input means, a presentation means for presenting the generated travel plan, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting the reservation procedure based on the customized travel plan, a management means for managing the user's travel history and making it viewable, and an emotion engine for recognizing the user's emotions. Furthermore, the emotion engine is used to analyze the user's emotional state, and the travel plan is generated and customized based on that information. Specific embodiments of the present invention are described below.

[1303] Program processing

[1304] 1. Users enter their travel questions and requests

[1305] Users access the screen of their smartphone or computer and enter desired conditions such as travel destination, theme, budget, etc. into an input form. For example, they might enter, "I'd like to plan a family trip to Hawaii this summer, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[1306] The terminal transmits this input information to the server.

[1307] 2. The server analyzes the user information

[1308] The server analyzes the data received from the user and extracts keywords such as travel destination, theme, budget, etc. For example, keywords such as "Hawaii," "family," and "budget of 300,000 yen" are extracted.

[1309] The server then searches a database to retrieve suggested attractions, hotels, and transportation options that match these keywords.

[1310] 3. The emotion engine analyzes the user's emotional state

[1311] The emotion engine analyzes the user's emotional state from input and voice data, for example, identifying emotions such as "happiness," "anxiety," and "expectation" from input text and voice.

[1312] The emotion engine transmits the user's emotional state to the server.

[1313] 4. The server generates the initial itinerary

[1314] The server uses a generative AI model to generate an initial travel plan based on the acquired data and emotional information, such as "A 7-day trip to Hawaii. Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling..."

[1315] This initial itinerary is adjusted based on the user's emotional information, for example, "if you're looking to relax, we'll suggest a relaxed itinerary."

[1316] 5. The server presents the generated itinerary to the user.

[1317] The server presents the initial travel plan to the user through the chatbot, and the user can confirm the details of the travel plan through the chatbot.

[1318] 6. Users interact with chatbots to get more information

[1319] Users can input specific questions about the plan presented by the chatbot, such as "Tell me more about this tourist spot."

[1320] The terminal sends this question to the server.

[1321] 7. The server provides information based on the user's question.

[1322] The server analyzes the user's question and retrieves relevant details from the database, such as "You can enjoy beach volleyball and kayaking at this tourist spot," and sends the details to the chatbot.

[1323] The user can view this information through their device.

[1324] 8. Customize your travel plans based on the information you provide

[1325] Users can interact with the chatbot to modify or add to their travel plans, for example by typing, "Instead of this hotel, find another hotel closer to the beach."

[1326] The emotion engine monitors the user's emotional changes in real time during the conversation and transmits the emotional information to the server. For example, if the user expresses dissatisfaction, this is analyzed.

[1327] 9. The server regenerates the itinerary and presents it to the user.

[1328] The server regenerates the travel plan based on the user's new requests and emotional information, and presents a customized plan to the user through the chatbot, for example, "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and beach volleyball..."

[1329] 10. The user finalizes their travel plans and completes the booking process

[1330] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[1331] The terminal sends this request to the server.

[1332] 11. The server supports the reservation process

[1333] The server connects to the reservation system to check availability of the hotel and flight desired by the user. If the reservation is successful, the server sends a confirmation to the user via the terminal. For example, the server displays information such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX."

[1334] 12. Managing and Viewing Your Trip History

[1335] To check their travel history, users type "view past travel history."

[1336] The terminal sends this request to the server.

[1337] 13. The server provides the user's travel history

[1338] The server retrieves the user's past travel data from the database, formats it, and sends it to the device. For example, it displays "A trip to Hawaii in August 2022: 5 nights and 7 days, information on the tourist spots visited, and the hotel where the user stayed."

[1339] Specific examples

[1340] When a user uses their smartphone to type, "I want to take a family trip to Hawaii this summer," the device sends this information to the server. The server analyzes the received information and retrieves the most appropriate information from its database using keywords like "Hawaii," "family-friendly," and "budget of ¥300,000." The emotion engine then analyzes emotional information from the user's input and voice data, identifying, for example, "I'm looking for relaxation." The server then generates an initial travel plan based on this information and presents it to the user through a chatbot. If the user types, "Tell me more about snorkeling spots," the server searches for related information and provides more detailed information. During this process, the emotion engine monitors the user's emotions in real time and adjusts the plan as needed. Finally, the user can confirm the travel plan, complete the booking process, and easily check their past travel history. This series of operations results in efficient and personalized travel planning.

[1341] The processing flow will be explained below.

[1342] Step 1:

[1343] Users enter their travel questions and requests.

[1344] Users access the screen of their smartphone or computer and enter, "I would like to plan a trip to Hawaii with my family this summer vacation, my budget is 300,000 yen, and I would like to stay at a resort hotel."

[1345] The terminal transmits this input information to the server.

[1346] Step 2:

[1347] The server analyzes the user information.

[1348] The server analyzes the data received from the user and extracts keywords such as "Hawaii," "family," and "budget of 300,000 yen."

[1349] The server then searches a database to retrieve suggested attractions, hotels, and transportation options that match these keywords.

[1350] Step 3:

[1351] The emotion engine analyzes the user's emotional state.

[1352] The emotion engine analyzes the user's emotional state from their input and voice data, identifying emotions such as "joy," "anxiety," and "expectation."

[1353] The emotion engine transmits the user's emotional state to the server.

[1354] Step 4:

[1355] The server generates an initial itinerary.

[1356] The server uses a generative AI model to generate an initial travel plan based on the acquired data and emotional information.

[1357] For example, generate a detailed plan such as "7-day travel plan for Hawaii. Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling..."

[1358] The plan will be adjusted to take into account the user's emotional state, adding relaxing elements and so on.

[1359] Step 5:

[1360] The server presents the generated travel plan to the user.

[1361] The server presents the initial travel plan to the user via the chatbot and asks for confirmation.

[1362] Users can check the details of their travel plans through the chatbot via their device.

[1363] Step 6:

[1364] Users interact with the chatbot to get more information.

[1365] Users can then input questions about the plan presented by the chatbot, such as "Tell me more about this tourist spot."

[1366] The terminal sends this question to the server.

[1367] Step 7:

[1368] The server provides information based on the user's question.

[1369] The server analyzes the user's question and retrieves relevant information from a database.

[1370] For example, the chatbot provides users with detailed information such as, "At this tourist spot, you can enjoy beach volleyball and kayaking."

[1371] Step 8:

[1372] The emotion engine monitors the user's emotional changes in real time.

[1373] The emotion engine recognizes the emotions expressed by the user while interacting with the chatbot and sends this information to the server.

[1374] For example, if the user expresses an emotion of satisfaction, capture that.

[1375] Step 9:

[1376] Customize your travel plans based on the information you provide.

[1377] Users interact with the chatbot to make specific changes, such as "find another hotel closer to the beach."

[1378] The terminal sends this request to the server.

[1379] Step 10:

[1380] The server regenerates the itinerary.

[1381] The server regenerates the travel plan based on the user's new instructions and emotional information.

[1382] For example, it generates a new plan such as "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and beach volleyball..." and presents it to the user.

[1383] Step 11:

[1384] The user finalizes their travel plans and completes the booking process.

[1385] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[1386] The terminal sends this request to the server.

[1387] Step 12:

[1388] The server supports the reservation process.

[1389] The server connects to the reservation system to check availability of the hotel or flight the user desires.

[1390] If the reservation is successful, confirmation information is sent to the user via the terminal.

[1391] For example, information such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX" may be displayed.

[1392] Step 13:

[1393] Manage and view your travel history.

[1394] To check their travel history, users type "view past travel history."

[1395] The terminal sends this request to the server.

[1396] Step 14:

[1397] The server provides the user's travel history.

[1398] The server retrieves the user's past travel data from the database, formats it, and sends it to the terminal.

[1399] For example, it will display "Trip to Hawaii in August 2022: 5 nights and 7 days, information on tourist spots visited and hotels stayed."

[1400] Example 2

[1401] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1402] Conventional travel plan generation systems only provide standard plans without considering the user's emotional state, and therefore are unable to respond to individual user needs and emotions. This makes it difficult to provide a travel plan that satisfies the user, resulting in a lot of effort required for customization and a decrease in satisfaction.

[1403] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the user's questions and requests input through the input means, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting the reservation procedure based on the customized travel plan, a management means for managing the user's travel history and making it viewable, an emotion recognition means for analyzing the user's emotional state, and an adjustment means for adjusting the travel plan based on the emotion information analyzed by the emotion recognition means. This makes it possible to provide a personalized travel plan that takes the user's emotional state into consideration.

[1404] "Input means" refers to the means by which a user inputs questions or requests regarding a trip.

[1405] The "generation means" is a means for generating a travel plan based on the user's questions and requests input through the input means.

[1406] The "presentation means" is a means for presenting the generated travel plan to the user.

[1407] "Customization means" refers to a means for customizing a travel plan through interaction with the user.

[1408] A "reservation tool" is a tool that supports the reservation process based on a customized travel plan.

[1409] "Management means" refers to the means for managing and making available the user's travel history.

[1410] "Emotion recognition means" is a means for analyzing the emotional state of a user.

[1411] The "adjustment means" is a means for adjusting the travel plan based on the emotion information analyzed by the emotion recognition means.

[1412] The present invention is a system for planning and managing travel, which aims to provide a personalized travel plan based on user input. The system includes an input means for a user to input travel questions and requests, a generation means for generating a travel plan based on the input information, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue, a reservation means for supporting a reservation procedure based on the customized plan, a management means for managing the user's travel history and making it viewable, an emotion recognition means for analyzing the user's emotional state, and an adjustment means for adjusting the plan based on the emotion recognition means.

[1413] System Overview

[1414] 1. Input Method

[1415] Users use their smartphones or computers to input desired conditions such as travel destination, theme, budget, etc. For example, they might input, "I'd like to plan a family trip to Hawaii during the summer vacation, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[1416] 2. Generation means

[1417] The server analyzes the input sent by the user and extracts the necessary keywords, such as "Hawaii," "family," and "budget of 300,000 yen."

[1418] The server uses a generative AI model to retrieve data on relevant tourist attractions, accommodations, and transportation from a database and generate a travel plan.

[1419] 3. Presentation means

[1420] The server then presents the generated travel plan to the user through an interface such as a chatbot, and the user can review this information on their device.

[1421] 4. Customization Methods

[1422] Users can use the chatbot to input specific questions or requests for modifications to the proposed itinerary, such as "Tell me more about this tourist spot" or "Instead of this hotel, find another hotel closer to the beach."

[1423] 5. Reservation Methods

[1424] Based on the plan confirmed by the user, the server checks the availability of hotels and flights in conjunction with the reservation system and completes the reservation process. If the reservation is successful, the server provides the user with this information. For example, a message such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX" is displayed on the terminal.

[1425] 6. Control measures

[1426] Users can submit requests to the system to view their past travel history, for example, by typing "Show me my past travel history," and the information will be displayed.

[1427] 7. Emotion recognition means

[1428] The emotion engine analyzes the text and voice data entered by the user to identify the user's emotional state, which allows it to understand what the user wants and reflect that in its plans.

[1429] 8. Adjustment means

[1430] The server dynamically adjusts the travel plan based on the emotional information obtained by the emotion engine, providing a plan that matches the user's emotions, such as suggesting a more relaxed schedule for a user looking to relax.

[1431] Specific examples

[1432] To explain the specific operation in more detail, some specific examples will be given.

[1433] Example prompt sentence:

[1434] "I want to plan a relaxing trip abroad with my family during summer vacation."

[1435] "Tell me about tourist spots in Europe"

[1436] "How much do you need to budget for your trip?"

[1437] When a user uses their smartphone to type, "I want to take my family on a summer vacation to Hawaii," the device sends this information to the server. The server analyzes the received information and retrieves the most relevant information from its database using keywords like "Hawaii," "family-friendly," and "budget of ¥300,000." The emotion engine then analyzes the user's input and voice data to identify emotional information, such as "I'm looking for relaxation." The server then generates an initial travel plan based on this information and presents it to the user through a chatbot. If the user types, "Tell me more about snorkeling spots," the server searches for related information and provides more information. During this process, the emotion engine monitors the user's emotions in real time and adjusts the plan as needed. Finally, the user can confirm the travel plan and proceed with the booking process. Users can also easily check their past travel history. This series of actions results in efficient and personalized travel planning.

[1438] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1439] Step 1:

[1440] Users enter travel questions and requests

[1441] Users use their smartphones or computers to input conditions such as desired travel destination, theme, budget, etc. Specifically, they might write in the input form, "I'd like to plan a family trip to Hawaii during the summer vacation, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[1442] Input: User's travel question or request

[1443] Output: User-entered criteria (travel destination, theme, budget, etc.)

[1444] Step 2:

[1445] The device sends the input information to the server

[1446] The device sends the user-entered conditions to the server in real time, with the data being sent via HTTP requests.

[1447] Input: Conditions entered by the user on the device

[1448] Output: User criteria sent to server

[1449] Step 3:

[1450] The server analyzes the user information and searches the database

[1451] The server analyzes the received user conditions and extracts important keywords (travel destination, budget, theme, etc.). For example, keywords such as "Hawaii," "family," and "budget of 300,000 yen" are extracted.

[1452] The server then uses these keywords to search a database to retrieve relevant tourist spots, hotels, and transportation options.

[1453] Input: User criteria sent to the server

[1454] Output: Important keywords, travel data retrieved from the database

[1455] Step 4:

[1456] Emotion engine analyzes the user's emotional state

[1457] The emotion engine analyzes user input and voice data to identify emotional states, such as joy, anxiety, and anticipation, for example.

[1458] The emotion engine transmits the analyzed emotion information to the server.

[1459] Input: User input, voice data

[1460] Output: Parsed emotion information

[1461] Step 5:

[1462] The server generates the initial itinerary

[1463] The server uses a generative AI model to generate an initial itinerary based on the acquired data and emotional information, such as "7-day Hawaii itinerary: Day 1: Beach visit, Day 2: Shopping, Day 3: Snorkeling."

[1464] The system takes into account the user's emotional information and optimizes the plan, such as adjusting to a more relaxed schedule if the user is looking to relax.

[1465] Input: Travel data obtained from the database, analyzed emotion information

[1466] Output: Initial itinerary

[1467] Step 6:

[1468] The server presents the generated itinerary to the user.

[1469] The server presents the initial travel plan to the user through the chatbot, and the user can review the plan on their device.

[1470] Input: Initial travel plan

[1471] Output: The itinerary presented to the user

[1472] Step 7:

[1473] Users interact with the chatbot to get more information

[1474] Users can then use the chatbot to ask specific questions about the proposed plan, such as "Tell me more about this tourist spot."

[1475] The terminal sends this question to the server.

[1476] Input: User's specific question

[1477] Output: The question sent to the server

[1478] Step 8:

[1479] The server provides information based on the user's question.

[1480] The server analyzes the user's question and retrieves appropriate details from the database, such as "You can enjoy beach volleyball and kayaking at this tourist spot," and sends this information to the chatbot.

[1481] The user can view this information through their device.

[1482] Input: User question, database details

[1483] Output: Answer with detailed information provided

[1484] Step 9:

[1485] Customize your travel plans based on the information you provide

[1486] Users can then use the details they have acquired to modify or add to their travel plans, for example, by saying, "Instead of this hotel, find another hotel closer to the beach."

[1487] The emotion engine monitors the user's emotional changes in real time through this dialogue and transmits the emotional information to the server.

[1488] Input: User's correction instructions, emotional information

[1489] Output: revised plan, emotional information

[1490] Step 10:

[1491] The server regenerates the itinerary and presents it to the user.

[1492] The server regenerates the travel plan based on the user's new requests and emotional information, and presents a customized plan through the chatbot, for example, "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and Beach Volleyball."

[1493] Input: User's new request, emotion information

[1494] Output: Regenerated itinerary

[1495] Step 11:

[1496] The user confirms their travel plans and completes the booking process

[1497] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[1498] The terminal sends this request to the server.

[1499] Input: User confirmation request

[1500] Output: Confirm request sent to server

[1501] Step 12:

[1502] The server handles the reservation process

[1503] The server connects to the reservation system to check availability of the hotel and flight desired by the user. If the reservation is successful, the server sends a confirmation to the user via the terminal. For example, the server displays information such as "Your hotel reservation has been completed. The confirmation number is XXXXXX."

[1504] Input: User's reservation request information, availability in the reservation system

[1505] Output: Reservation confirmation information

[1506] Step 13:

[1507] Users can manage and view their travel history

[1508] If a user wants to check their past travel history, they type "View past travel history."

[1509] The terminal sends this request to the server.

[1510] Input: User's request to view travel history

[1511] Output: View request sent to the server

[1512] Step 14:

[1513] The server provides the user's travel history

[1514] The server retrieves the user's past travel data from the database, formats it, and sends it to the device. For example, it displays "A trip to Hawaii in August 2022: 5 nights and 7 days, information on the tourist spots visited, and the hotel where the user stayed."

[1515] Input: Travel history data from the database

[1516] Output: Formatted travel history information

[1517] (Application example 2)

[1518] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1519] The problem to be solved by the present invention is to provide a system that efficiently generates a personalized meal plan taking into account the user's preferences and emotional state when the user inputs their meal requests and questions, and smoothly supports the delivery procedure, including the customization process.

[1520] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an input means for the user to input their wishes and requests regarding their request, a generation means for generating a plan based on the user's wishes and requests input through the input means, a presentation means for presenting the generated plan to the user, a customization means for customizing the plan through dialogue with the user, a reservation means for supporting procedures based on the customized plan, and a management means for managing the user's history and making it viewable. This enables the generation, customization, and reservation procedures of personalized meal plans.

[1521] "Requests" refer to the conditions desired by the user and the content of the services they would like to receive.

[1522] "Input means" refers to an interface that allows a user to input information into a device or system.

[1523] "Generation means" refers to the process or technical mechanism for automatically generating a specific plan or proposal based on input information.

[1524] "Presentation means" refers to the method or interface for displaying the generated plan or information to the user.

[1525] "Customization means" refers to a method or system that allows users to modify or change the plans or information presented to them to suit their preferences.

[1526] "Reservation Method" refers to the method or system for carrying out formal procedures based on the plan selected or modified by the User.

[1527] "Management means" refers to the method or system for storing and organizing users' past usage history and information, and making them available for viewing as needed.

[1528] An "automatic response system" refers to technology or mechanisms that allow a system to automatically communicate with users.

[1529] An "information repository" refers to a system or database for storing and managing various data and information.

[1530] A "generative AI model" refers to an algorithm or model that uses machine learning or artificial intelligence to generate new plans or information based on specific patterns or information.

[1531] The present invention is a system that includes an input means for a user to input meal preferences and questions, a generation means for generating a meal plan based on the preferences and questions input through the input means, a presentation means for presenting the generated plan to the user, a customization means for customizing the plan through dialogue with the user, a reservation means for supporting delivery procedures based on the customized plan, and a management means for managing the user's meal history and making it viewable.

[1532] Detailed System Description

[1533] 1. Input method:

[1534] Users use their smartphones to input their meal preferences, budget, delivery time, and other requests into the application's input form. If a user inputs, "I want to order a healthy, stress-reducing meal," this information is sent from the smartphone to the server.

[1535] 2. Generation means:

[1536] The server analyzes the received user information and performs sentiment analysis using the Emotion API. The analyzed data is then used to generate an appropriate meal plan using a generative AI model (e.g., GPT-4). This generative AI model uses the input data and the extracted sentiment information as prompts to suggest the optimal meal plan.

[1537] 3. Means of presentation:

[1538] The generated meal plan is presented to the user through a chatbot, which provides the user with the meal plan details in a conversational format.

[1539] 4. Customization methods:

[1540] Users can modify or add to the generated meal plan through dialogue with the chatbot. For example, if they input a request such as "I'd like to add avocado to my healthy bowl," the server will again use the generative AI model to generate a new plan.

[1541] 5. Reservation Method:

[1542] Once the user is satisfied with the customized meal plan and selects "I would like to order this plan," the server connects with the delivery system to process the reservation. If the reservation is successful, a confirmation is sent to the user.

[1543] 6. Control measures:

[1544] The user's past meal history is stored in a database and can be viewed as needed. When a user types "I want to check my past order history," the relevant information is displayed.

[1545] Hardware and software used

[1546] Device: Smartphone (iOS / Android)

[1547] Emotion engine: Emotion API (e.g. Microsoft Azure Cognitive Services)

[1548] Generative AI model: GPT-4 (OpenAI API)

[1549] Database: PostgreSQL (Amazon RDS)

[1550] Frontend: React Native

[1551] Backend: Node.js (Express)

[1552] Example prompt sentence

[1553] "The user wants to order a meal. The user's wishes are as follows: Travel destination: Please suggest a menu with the theme of 'healthy and relaxing meals', within a budget of 2000 yen, and that can be delivered within 30 minutes. Furthermore, please consider 'stress,' 'relaxation,' and 'fatigue' based on the results of the sentiment analysis."

[1554] By implementing this invention, users can select, customize, and reserve a personalized meal plan based on their emotional state and preferences, resulting in a more satisfying delivery service.

[1555] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1556] Step 1:

[1557] A user accesses the smartphone application and inputs information such as meal preferences, questions, budget, and delivery time. For example, the user might input, "I would like to order a healthy and relaxing meal." The smartphone then sends this input information to the server.

[1558] Input: User's wishes, questions, budget, delivery time

[1559] Output: User information sent to the server

[1560] Step 2:

[1561] The server analyzes the information received from the user and extracts the input requests and questions. The analyzed data is subjected to emotion analysis using the Emotion API to identify the user's emotional state (e.g., "stressed," "relaxed," "fatigued").

[1562] Input: User information sent to the server

[1563] Data processing / data calculation: Extraction of wishes and questions, sentiment analysis

[1564] Output: Extracted data and sentiment information

[1565] Step 3:

[1566] The server uses a generative AI model (e.g., GPT-4) to generate an appropriate meal plan based on the extracted data and emotional information. At this time, the server passes prompts to the generative AI model, which then proposes a plan that reflects the user's preferences and emotional state.

[1567] Input: Extracted data and sentiment information

[1568] Data processing / data calculation: Prompt sentence generation, plan generation using generative AI models

[1569] Output: Generated meal plan

[1570] Step 4:

[1571] The server presents the generated meal plan to the user through the chatbot, and the user can check the details of the plan via their smartphone.

[1572] Input: Generated meal plan

[1573] Output: The plan presented to the user through the chatbot

[1574] Step 5:

[1575] Users can change or add to their meal plan through interactions with the chatbot, for example by typing, "I'd like to add avocado to my healthy bowl." The smartphone then sends this request to the server.

[1576] Input: User request

[1577] Output: New request sent to the server

[1578] Step 6:

[1579] The server regenerates the meal plan using the generative AI model again based on the new request and emotional information received. It generates a new prompt sentence and passes it to the generative AI model to suggest a new plan.

[1580] Input: New request, emotion information

[1581] Data processing / data calculation: Regeneration of prompt sentences, regeneration by generative AI model

[1582] Output: Regenerated meal plan

[1583] Step 7:

[1584] The server then presents the regenerated meal plan to the user via the chatbot, and the user can check the plan details again via their smartphone.

[1585] Input: Regenerated meal plan

[1586] Output: Plan re-presented through the chatbot

[1587] Step 8:

[1588] When the user is finally satisfied with the plan and types "I would like to order this plan," the smartphone sends this request to the server, which then coordinates with the delivery system to complete the reservation process and sends reservation confirmation information to the smartphone.

[1589] Input: Final confirmation request by user

[1590] Output: Final confirmation request sent to the server, booking confirmation information

[1591] Step 9:

[1592] The user's meal history is stored in a database. When the user types "I want to check my past order history," the smartphone sends this request to the server, which retrieves the relevant information from the database and displays it to the user.

[1593] Input: User history check request

[1594] Output: History information retrieved from the database, history information displayed to the user

[1595] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1596] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1597] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1598] [Fourth embodiment]

[1599] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1600] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1601] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1602] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1603] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1604] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1605] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1606] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1607] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1608] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1609] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1610] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1611] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1612] The present invention provides a system that includes an input means for users to input travel-related questions and requests, a generation means that generates a travel plan using a generative AI model based on the input information, a presentation means that presents the generated plan to the user, a customization means that customizes the travel plan through dialogue with the user, a reservation means that supports the reservation procedure based on the customized travel plan, and a management means that manages the user's travel history and makes it viewable. Below, the program processing of this system is explained in natural language. Specific examples are also provided.

[1613] Program processing

[1614] 1. Users enter their travel questions and requests

[1615] Users access the screen of their smartphone or computer and enter desired conditions such as travel destination, theme, budget, etc. into an input form. For example, they might enter, "I'd like to plan a family trip to Hawaii this summer, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[1616] The terminal (smartphone or computer) sends the entered information to the server.

[1617] 2. The server analyzes the user information and generates an initial travel plan

[1618] The server analyzes the received user information and retrieves tourist attractions, hotels, and transportation information from the database based on the user's request. Specifically, it retrieves a list of "tourist attractions in Hawaii, resort hotels available within a budget, and family activities."

[1619] The server uses a generative AI model to generate an initial itinerary based on this data, such as "A 7-day trip to Hawaii: Day 1 - Beach, Day 2 - Shopping, Day 3 - Snorkeling..."

[1620] The server presents the generated travel plan to the user in the form of a chatbot.

[1621] 3. Users interact with the chatbot to get more information

[1622] The user reviews the chatbot's suggestions and types, "Tell me more about this tourist spot."

[1623] The device sends the user's question to the server, which retrieves relevant information from a database, providing detailed information such as "You can enjoy beach volleyball and kayaking at this tourist spot."

[1624] 4. Customize your travel plans based on the information you provide

[1625] Users can input instructions to modify their plans based on the chatbot's suggestions, for example, "Instead of this hotel, find another one closer to the beach."

[1626] The server regenerates the itinerary based on the new request and presents the updated itinerary to the user, such as "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and Beach Volleyball..."

[1627] 5. The user finalizes their travel plans and completes the booking process

[1628] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[1629] The device sends this request to the server, which then connects with the reservation system to check availability of the hotel and flight the user desires. If the reservation is successful, the user is given confirmation information, such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX."

[1630] 6. Managing and viewing your travel history

[1631] To check their travel history, users type "view past travel history."

[1632] The device sends this request to the server, which retrieves the user's past travel data from the database. It formats and displays information such as "Retrieve details of past Hawaii trips and European tours." For example, it displays information such as "August 2022 Hawaii trip: 5 nights and 7 days, tourist spots visited, and hotel information."

[1633] Specific examples

[1634] When a user uses their smartphone to input "I want to take a family trip to Hawaii this summer," the device sends this information to a server. The server analyzes the received information and retrieves the most suitable information from its database using the keywords "Hawaii, family-friendly, budget 300,000 yen." It then uses a generative AI model to generate a travel plan suited to the user and presents it to the user through a chatbot. If the user inputs "Tell me more about places where I can snorkel," the server searches for related information and provides more detailed information. The user can then reconstruct their plan based on the information, ultimately booking hotels and flights, and review their detailed travel history at a later date. This series of actions results in efficient and personalized travel planning.

[1635] The processing flow will be explained below.

[1636] Step 1:

[1637] Users enter their travel questions and requests.

[1638] Users use their smartphones or computers to enter their desired travel destination, theme, budget, and other preferences into an input form.

[1639] For example, you might enter information such as, "I want to plan a trip to Hawaii with my family during summer vacation, my budget is 300,000 yen, and I want to stay at a resort hotel."

[1640] The terminal transmits this input information to the server.

[1641] Step 2:

[1642] The server analyzes the user information.

[1643] The server analyzes the data received from the user and extracts keywords such as travel destination, theme, and budget.

[1644] For example, extract keywords such as "Hawaii," "family," and "budget 300,000 yen."

[1645] The server then searches a database to retrieve suggested attractions, hotels, and transportation options that match these keywords.

[1646] Step 3:

[1647] The server generates an initial itinerary.

[1648] The server uses a generative AI model to generate an initial itinerary based on the acquired data.

[1649] For example, generate a plan such as "7-day Hawaii travel plan. Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling..."

[1650] The generated travel plan is formatted in a user-friendly format.

[1651] Step 4:

[1652] The server presents the generated travel plan to the user.

[1653] The server presents an initial travel plan to the user through a chatbot.

[1654] Users can check the details of their travel plans through the chatbot.

[1655] Step 5:

[1656] Users interact with the chatbot to get more information.

[1657] Users can enter specific questions about the plan presented by the chatbot.

[1658] For example, send a question such as "Tell me more about this tourist spot."

[1659] The terminal sends this question to the server.

[1660] Step 6:

[1661] The server provides information based on the user's question.

[1662] The server analyzes the user's question and retrieves relevant details from the database.

[1663] For example, detailed information such as "At this tourist spot, you can enjoy beach volleyball and kayaking" is sent to the chatbot.

[1664] The user can view this information through their device.

[1665] Step 7:

[1666] Customize your travel plans based on the information you provide.

[1667] Users can modify or add to their travel plans through interactions with the chatbot.

[1668] For example, "Instead of this hotel, find another hotel closer to the beach."

[1669] The terminal sends this request to the server.

[1670] Step 8:

[1671] The server regenerates the itinerary and presents it to the user.

[1672] The server regenerates the travel plan based on the user's new requirements and presents the customized plan to the user through the chatbot.

[1673] For example, it presents a new plan such as "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and beach volleyball..."

[1674] Step 9:

[1675] The user finalizes their travel plans and completes the booking process.

[1676] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[1677] The terminal sends this request to the server.

[1678] Step 10:

[1679] The server supports the reservation process.

[1680] The server connects to the reservation system to check availability of the hotel or flight the user desires.

[1681] If the reservation is successful, confirmation information is sent to the user via the terminal.

[1682] For example, information such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX" may be displayed.

[1683] Step 11:

[1684] Manage and view your trip history.

[1685] To check their travel history, users type "view past travel history."

[1686] The terminal sends this request to the server.

[1687] Step 12:

[1688] The server provides the user's travel history.

[1689] The server retrieves the user's past travel data from the database, formats it, and sends it to the terminal.

[1690] For example, it will display "Trip to Hawaii in August 2022: 5 nights and 7 days, information on tourist spots visited and hotels stayed."

[1691] Example 1

[1692] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1693] Conventional travel plan creation systems have had difficulty automatically generating specific and personalized travel plans based on users' requests. Furthermore, they lacked a means for users to interactively customize the generated travel plans, or a function to support the reservation process based on the customized plans. This has resulted in problems that make it difficult for users to efficiently create travel plans and smoothly complete the necessary reservation procedures.

[1694] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1695] In this invention, the server includes an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the user's questions and requests input through the input means, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting the reservation procedure based on the customized travel plan, a management means for managing the user's travel history and making it viewable, a means for the generation means to generate a travel plan using a generative AI model, a means for presenting the generated travel plan to the user in chatbot format, a means for the user to obtain detailed information by dialogue with the chatbot, and a means for retrieving travel plan data from a database based on the user's input. This enables efficient generation, customization, and support for the reservation procedure of a personalized travel plan based on the user's requests.

[1696] "User" means any person or entity that uses the System to create and customize a travel plan.

[1697] "Input means" refers to the interface through which a user inputs travel-related questions or requests into the system, such as a form or text entry field displayed on a computer or smartphone screen.

[1698] "Generation Means" refers to the functionality for generating a travel plan based on information provided by the user through the Input Means, including retrieving information from a generative AI model or database.

[1699] The "presentation means" refers to a mechanism for displaying the generated travel plan to the user. Specifically, it includes a chatbot-style user interface.

[1700] "Customization means" refers to the function of modifying and adjusting the travel plan generated through interaction with the user.

[1701] "Booking Method" refers to functionality that supports booking accommodation and transportation based on customized travel plans.

[1702] "Management means" refers to the function that manages a user's past travel history and allows the user to view it.

[1703] "Generative AI model" refers to an artificial intelligence model that generates travel plans based on information provided by users, including models that utilize natural language processing and machine learning.

[1704] A "chatbot" is an automated response program that interacts with users to exchange information.

[1705] "Database" refers to a collection of electronic data for storing information (such as tourist attractions, hotels, and transportation options) required to generate a travel plan.

[1706] A "prompt sentence" refers to a sentence that is input to a generative AI model to generate a specific travel plan.

[1707] The present invention is a system that allows users to input travel-related questions and requests, and then generate, customize, book, and manage travel plans based on that information. This system uses the following hardware and software.

[1708] Users access an application on their smartphone or computer screen and input information such as travel destination, theme, budget, etc. For example, they might input conditions such as "I want to plan a family trip to Hawaii this summer, with a budget of 300,000 yen, and I want to stay at a resort hotel." This input information is then sent from the device to the server.

[1709] The server uses natural language processing tools to analyze the information received from users. Specifically, it uses NLP tools such as Google Cloud Natural Language API to analyze the input text and extract important keywords. It also collects data on tourist attractions, hotels, and transportation options from databases and external APIs (e.g., Google Places API, Expedia API).

[1710] Based on the collected data, the server generates a travel plan using a generative AI model (e.g., OpenAI GPT-4). The server inputs a prompt such as "Generate a family travel plan in Hawaii within a budget of 300,000 yen" into the generative AI model, which then generates an appropriate travel plan. The generated travel plan is converted into a chatbot-style UI and presented to the user via the device.

[1711] Users use the chatbot to get more information about the proposed itinerary. For example, they can type "Tell me more about this tourist spot," and the device will send this request to the server. The server will then query a database or external API to gather relevant details and present them to the user, providing them with specific information based on the proposed itinerary.

[1712] If users want to customize their travel plans, they can input instructions through the chatbot. The server will retrieve the data again based on the new request and generate an updated plan using the generative AI model. For example, a user could instruct the chatbot, "Instead of this hotel, find another hotel closer to the beach."

[1713] Finally, when the user has finalized their travel plans and wants to proceed with the reservation, they enter, "I'd like to confirm this plan and book a hotel and flight." The device sends this request to the server, which then connects with the reservation system to check the availability of the accommodation and transportation options desired by the user. If the reservation is successful, confirmation information is provided to the user. Information such as "Hotel reservation completed. Confirmation number is XXXXXX" is displayed.

[1714] Additionally, users can check their past travel history. By inputting the command "View past travel history," the device sends this request to the server, which retrieves and displays the user's past travel data from the database. For example, "A trip to Hawaii in August 2022: 5 nights and 7 days, tourist spots visited, and hotel information."

[1715] The system allows users to easily create, customize, book and manage efficient and personalized travel plans.

[1716] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1717] Program processing

[1718] Step 1:

[1719] A user accesses a travel planning application on a smartphone or computer screen and inputs information such as the desired travel destination, theme, and budget. Specifically, the user writes in the input form, "I'd like to plan a trip to Hawaii for my family this summer. My budget is 300,000 yen. I'd like to stay at a resort hotel." The device then sends the input information to the server using an HTTP request. The input data is the user's request (travel destination, theme, budget, etc.).

[1720] Step 2:

[1721] The server uses natural language processing (NLP) tools to process the unstructured text information received from the user. Specifically, it uses the Google Cloud Natural Language API to analyze the text, tokenize it, and extract keywords. The input data is the unstructured text based on the user's request, and the output data is important keywords (e.g., "Hawaii," "family-friendly," "300,000 yen").

[1722] Step 3:

[1723] The server queries its own database or external APIs (e.g., Google Places API, Expedia API) based on the extracted keywords to collect data on related tourist attractions, hotels, and transportation options. For example, it obtains information such as "tourist attractions in Hawaii, resort hotels available within a budget, and family-friendly activities." The input data is a set of keywords, and the output data is related travel information (tourist attractions, hotels, and transportation options).

[1724] Step 4:

[1725] The server uses the collected data as input to generate an initial travel plan using a generative AI model (e.g., OpenAI GPT-4). The model is given a prompt, "Generate a family trip to Hawaii within a budget of 300,000 yen," and a specific travel plan is generated. The generated travel plan includes itineraries, activities, accommodations, etc. The input data is the travel prompt and travel-related data from an external API, and the output data is the generated travel plan.

[1726] Step 5:

[1727] The server formats the generated travel plan in chatbot format and sends it to the terminal to present to the user. Specifically, it converts the travel plan into text format and displays it on the user interface. The terminal displays the received travel plan information on the screen. The input data is the generated travel plan, and the output data is the travel plan text displayed on the screen.

[1728] Step 6:

[1729] The user inputs detailed information and customization instructions for the travel plan presented in chatbot format. For example, the user may give instructions such as "Tell me more about this tourist spot" or "Instead of this hotel, find another hotel closer to the beach." The device then sends these instructions to the server. The input data is the user's questions and customization instructions, and the output data is the detailed information and customization request.

[1730] Step 7:

[1731] Based on the user's questions and customization instructions, the server queries the database or external API again to collect and generate detailed information and a new travel plan. Specifically, the updated conditions are input into the generative AI model to obtain a revised plan. It also collects detailed information about tourist spots and re-suggested hotels and presents them to the user in the form of a chatbot. The input data is the detailed information and new conditions, and the output data is the updated travel plan and detailed information.

[1732] Step 8:

[1733] The user finalizes the travel plan and inputs instructions to complete the reservation process. For example, the user might say, "I'd like to finalize this plan and book a hotel and flight." The device then sends this instruction to the server. The input data is the final confirmation instruction from the user, and the output data is the reservation request.

[1734] Step 9:

[1735] The server connects with a reservation system (e.g., hotel reservation system, airline reservation system) to check the availability of the specified accommodation or transportation. If the reservation is successful, it generates confirmation information and sends it to the terminal. For example, the user is provided with information such as "Your hotel reservation has been completed. The confirmation number is XXXXXX." The input data is the reservation request, and the output data is the reservation confirmation information.

[1736] Step 10:

[1737] A user can make a request to view past travel history. When the user inputs the instruction "Show past travel history," the terminal sends this request to the server. The input data is a request for past travel history, and the output data is a request to display travel history.

[1738] Step 11:

[1739] The server retrieves the user's past travel data from the database, formats it, and sends it to the terminal. For example, it provides data to display "A trip to Hawaii in August 2022: 5 nights and 7 days, information on tourist spots visited, and hotels stayed." The input data is a past travel history request, and the output data is the formatted past travel data.

[1740] (Application example 1)

[1741] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1742] Conventional travel planning systems offer limited means for users to visually grasp details about travel destinations and accommodations, leaving a need for improved user experience. There is also a demand for intuitive operation via voice input and gestures when customizing travel plans. Furthermore, there is a growing need to provide a realistic travel experience in advance by generating and presenting travel plans in a virtual environment. New technologies are needed to address these challenges.

[1743] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1744] In this invention, the server includes an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the user's questions and requests input through the input means, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting reservation procedures based on the customized travel plan, a management means for managing and making the user's travel history viewable, a means for the user to wear a head-mounted display and create a travel plan in a virtual environment, a means for displaying 3D views of tourist attractions and accommodations in the virtual environment so the user can visually confirm them, and a means for the user to input questions and requests by voice input or gestures. This allows the user to check and customize the travel plan in real time in the virtual environment, enabling visual and intuitive travel planning.

[1745] "User" means any individual or entity that uses the Travel Planning System.

[1746] "Input Means" refers to a device or interface through which a User inputs travel-related questions or requests into the System.

[1747] "Generation means" refers to a combination of software or hardware for generating a travel plan based on information input by a user.

[1748] "Presentation means" refers to a device or interface for displaying the generated travel plan to the user.

[1749] "Customization means" refers to a device or interface for modifying or changing a travel plan through interaction with the user.

[1750] "Reservation Instrument" means a device or interface that supports the reservation process based on a customized travel plan.

[1751] "Management means" refers to databases and software used to store and make available users' travel history.

[1752] A "head-mounted display" refers to a device worn by a user that displays visual information in three dimensions.

[1753] "Virtual environment" refers to a computer-generated 3D space constructed to allow users to experience virtual reality.

[1754] "3D views of tourist attractions and accommodation facilities" refers to images and videos that visually display details of tourist attractions and accommodation facilities in three dimensions.

[1755] "Voice input" refers to a means by which a user can give instructions or information to a system through voice.

[1756] "Gesture input" refers to a means by which a user gives instructions or information to a system through hand or body movements.

[1757] To realize this invention, it is necessary to build a system using the following steps: First, provide an input means for users to input their travel-related questions and requests. This input means can include a smartphone, a computer touch panel, a voice input device, etc.

[1758] Hardware and Software Use

[1759] Hardware

[1760] Head-mounted display (HMD): A device worn by the user to provide a virtual reality environment, such as Oculus Rift or HTC Vive.

[1761] software

[1762] Unity: Builds a virtual reality environment to display travel plans within a virtual environment.

[1763] Azure Chatbot Service: Provides a conversational interface that enables interaction with users.

[1764] GCP AI Platform: Hosts generative AI models and generates itineraries based on user questions and requests.

[1765] Firebase: Manages user travel history and booking data.

[1766] Program processing

[1767] Input Method

[1768] Users put on the HMD and log in to the virtual environment. They then input their travel questions and requests into the system using voice and gestures. For example, they might say, "A fun trip to Europe for the whole family." This voice input is then converted to text using the Google Cloud Speech-to-Text API.

[1769] generation means

[1770] The server receives user input data and generates a travel plan using a generative AI model hosted on the GCP AI Platform, for example, a list of recommended tourist spots and accommodations for a family trip to Europe.

[1771] Presentation means

[1772] The generated itinerary is presented to the user in a virtual environment using Unity, where the user can visually see 3D views of tourist attractions and accommodations.

[1773] Customization methods

[1774] Users can interact with the Azure Chatbot Service to customize the details of their travel plans, such as "Tell me more resort hotels."

[1775] Reservation procedure

[1776] The system automatically processes reservations based on the plan customized by the user, manages reservation data using Firebase, and provides confirmation information to the user once the reservation is complete.

[1777] management measures

[1778] Users' travel history is managed by Firebase and can be viewed by users. Past travel history and detailed information can be checked at any time.

[1779] Specific examples

[1780] The following example prompt sentences will be used to explain how the system works:

[1781] When a user types "Tell me about your trip to Hawaii," the generative AI model responds as follows:

[1782] "Prompt to generate a travel plan:

[1783] User request: I'm planning a summer vacation to Hawaii with my family. My budget is under 300,000 yen, and I'd like to stay at a resort hotel. I enjoy snorkeling and shopping. Based on the input criteria, please generate a 7-day itinerary that includes the following elements:

[1784] Required elements:

[1785] List of tourist attractions

[1786] Family Activities

[1787] Budget-friendly resort hotels

[1788] Dining options

[1789] Shopping spot

[1790] Based on the above prompts, the generative AI model can construct a specific travel plan and provide it to the user through the virtual assistant.

[1791] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1792] Step 1:

[1793] The user puts on a head-mounted display and logs into the virtual environment.

[1794] Input: User ID and authentication information

[1795] Output: Home screen in the virtual environment

[1796] Specific operation: The user receives visual information through the HMD and enters authentication information on the login screen. The device sends this information to the server, and if authentication is successful, the home screen is displayed.

[1797] Step 2:

[1798] Users input travel questions and requests using voice input or gestures.

[1799] Input: Voice and gesture data

[1800] Output: Text data (questions and requests)

[1801] What it does: The user says, "I want to plan a fun family trip to Europe." The device converts this into text using the Google Cloud Speech-to-Text API and sends it to the server.

[1802] Step 3:

[1803] The server analyzes user information and generates an initial travel plan using a generative AI model.

[1804] Input: User's question or request (text data)

[1805] Output: Initial travel plan (text data)

[1806] Specific operation: The server receives the text data and calls the generative AI model on the GCP AI Platform. The generative AI model generates a "European travel plan for families" and returns it to the server.

[1807] Step 4:

[1808] The server generates a travel plan and presents it in a virtual environment.

[1809] Input: Initial travel plan (text data)

[1810] Output: 3D view of the trip plan

[1811] What it does: The initial travel plan is passed to Unity as text data, and tourist attractions and accommodations are displayed in a 3D view, which the user can view through the HMD.

[1812] Step 5:

[1813] Users interact with chatbots to customize their travel plans.

[1814] Input: User's new question or request (text data)

[1815] Output: Customized travel plan (text data)

[1816] How it works: When a user types "find other resort hotels instead of this one," the device sends this to the server, which then invokes the generative AI model again to generate a customized itinerary based on the new request.

[1817] Step 6:

[1818] Users make reservations based on customized travel plans.

[1819] Input: Customized travel plan (text data)

[1820] Output: Reservation confirmation information

[1821] Specific operation: The user inputs "I would like to confirm this plan and make a reservation." The device sends this to the server, which then connects to the reservation system to check availability. If the reservation is successful, the confirmation information is provided to the user.

[1822] Step 7:

[1823] Users can manage and view their travel history.

[1824] Input: User ID and authentication information

[1825] Output: List of trip history

[1826] Specific operation: The user types "Show me my past travel history." The device sends this to the server, which retrieves past travel data from Firebase, formats it, and displays it to the user.

[1827] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1828] The present invention provides a system including an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the questions and requests input through the input means, a presentation means for presenting the generated travel plan, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting the reservation procedure based on the customized travel plan, a management means for managing the user's travel history and making it viewable, and an emotion engine for recognizing the user's emotions. Furthermore, the emotion engine is used to analyze the user's emotional state, and the travel plan is generated and customized based on that information. Specific embodiments of the present invention are described below.

[1829] Program processing

[1830] 1. Users enter their travel questions and requests

[1831] Users access the screen of their smartphone or computer and enter desired conditions such as travel destination, theme, budget, etc. into an input form. For example, they might enter, "I'd like to plan a family trip to Hawaii this summer, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[1832] The terminal transmits this input information to the server.

[1833] 2. The server analyzes the user information

[1834] The server analyzes the data received from the user and extracts keywords such as travel destination, theme, budget, etc. For example, keywords such as "Hawaii," "family," and "budget of 300,000 yen" are extracted.

[1835] The server then searches a database to retrieve suggested attractions, hotels, and transportation options that match these keywords.

[1836] 3. The emotion engine analyzes the user's emotional state

[1837] The emotion engine analyzes the user's emotional state from input and voice data, for example, identifying emotions such as "happiness," "anxiety," and "expectation" from input text and voice.

[1838] The emotion engine transmits the user's emotional state to the server.

[1839] 4. The server generates the initial itinerary

[1840] The server uses a generative AI model to generate an initial travel plan based on the acquired data and emotional information, such as "A 7-day trip to Hawaii. Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling..."

[1841] This initial itinerary is adjusted based on the user's emotional information, for example, "if you're looking to relax, we'll suggest a relaxed itinerary."

[1842] 5. The server presents the generated itinerary to the user.

[1843] The server presents the initial travel plan to the user through the chatbot, and the user can confirm the details of the travel plan through the chatbot.

[1844] 6. Users interact with chatbots to get more information

[1845] Users can input specific questions about the plan presented by the chatbot, such as "Tell me more about this tourist spot."

[1846] The terminal sends this question to the server.

[1847] 7. The server provides information based on the user's question.

[1848] The server analyzes the user's question and retrieves relevant details from the database, such as "You can enjoy beach volleyball and kayaking at this tourist spot," and sends the details to the chatbot.

[1849] The user can view this information through their device.

[1850] 8. Customize your travel plans based on the information you provide

[1851] Users can interact with the chatbot to modify or add to their travel plans, for example by typing, "Instead of this hotel, find another hotel closer to the beach."

[1852] The emotion engine monitors the user's emotional changes in real time during the conversation and transmits the emotional information to the server. For example, if the user expresses dissatisfaction, this is analyzed.

[1853] 9. The server regenerates the itinerary and presents it to the user.

[1854] The server regenerates the travel plan based on the user's new requests and emotional information, and presents a customized plan to the user through the chatbot, for example, "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and beach volleyball..."

[1855] 10. The user finalizes their travel plans and completes the booking process

[1856] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[1857] The terminal sends this request to the server.

[1858] 11. The server supports the reservation process

[1859] The server connects to the reservation system to check availability of the hotel and flight desired by the user. If the reservation is successful, the server sends a confirmation to the user via the terminal. For example, the server displays information such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX."

[1860] 12. Managing and Viewing Your Trip History

[1861] To check their travel history, users type "view past travel history."

[1862] The terminal sends this request to the server.

[1863] 13. The server provides the user's travel history

[1864] The server retrieves the user's past travel data from the database, formats it, and sends it to the device. For example, it displays "A trip to Hawaii in August 2022: 5 nights and 7 days, information on the tourist spots visited, and the hotel where the user stayed."

[1865] Specific examples

[1866] When a user uses their smartphone to type, "I want to take a family trip to Hawaii this summer," the device sends this information to the server. The server analyzes the received information and retrieves the most appropriate information from its database using keywords like "Hawaii," "family-friendly," and "budget of ¥300,000." The emotion engine then analyzes emotional information from the user's input and voice data, identifying, for example, "I'm looking for relaxation." The server then generates an initial travel plan based on this information and presents it to the user through a chatbot. If the user types, "Tell me more about snorkeling spots," the server searches for related information and provides more detailed information. During this process, the emotion engine monitors the user's emotions in real time and adjusts the plan as needed. Finally, the user can confirm the travel plan, complete the booking process, and easily check their past travel history. This series of operations results in efficient and personalized travel planning.

[1867] The processing flow will be explained below.

[1868] Step 1:

[1869] Users enter their travel questions and requests.

[1870] Users access the screen of their smartphone or computer and enter, "I would like to plan a trip to Hawaii with my family this summer vacation, my budget is 300,000 yen, and I would like to stay at a resort hotel."

[1871] The terminal transmits this input information to the server.

[1872] Step 2:

[1873] The server analyzes the user information.

[1874] The server analyzes the data received from the user and extracts keywords such as "Hawaii," "family," and "budget of 300,000 yen."

[1875] The server then searches a database to retrieve suggested attractions, hotels, and transportation options that match these keywords.

[1876] Step 3:

[1877] The emotion engine analyzes the user's emotional state.

[1878] The emotion engine analyzes the user's emotional state from their input and voice data, identifying emotions such as "joy," "anxiety," and "expectation."

[1879] The emotion engine transmits the user's emotional state to the server.

[1880] Step 4:

[1881] The server generates an initial itinerary.

[1882] The server uses a generative AI model to generate an initial travel plan based on the acquired data and emotional information.

[1883] For example, generate a detailed plan such as "7-day travel plan for Hawaii. Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling..."

[1884] The plan will be adjusted to take into account the user's emotional state, adding relaxing elements and so on.

[1885] Step 5:

[1886] The server presents the generated travel plan to the user.

[1887] The server presents the initial travel plan to the user via the chatbot and asks for confirmation.

[1888] Users can check the details of their travel plans through the chatbot via their device.

[1889] Step 6:

[1890] Users interact with the chatbot to get more information.

[1891] Users can then input questions about the plan presented by the chatbot, such as "Tell me more about this tourist spot."

[1892] The terminal sends this question to the server.

[1893] Step 7:

[1894] The server provides information based on the user's question.

[1895] The server analyzes the user's question and retrieves relevant information from a database.

[1896] For example, the chatbot provides users with detailed information such as, "At this tourist spot, you can enjoy beach volleyball and kayaking."

[1897] Step 8:

[1898] The emotion engine monitors the user's emotional changes in real time.

[1899] The emotion engine recognizes the emotions expressed by the user while interacting with the chatbot and sends this information to the server.

[1900] For example, if the user expresses an emotion of satisfaction, capture that.

[1901] Step 9:

[1902] Customize your travel plans based on the information you provide.

[1903] Users interact with the chatbot to make specific changes, such as "find another hotel closer to the beach."

[1904] The terminal sends this request to the server.

[1905] Step 10:

[1906] The server regenerates the itinerary.

[1907] The server regenerates the travel plan based on the user's new instructions and emotional information.

[1908] For example, it generates a new plan such as "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and beach volleyball..." and presents it to the user.

[1909] Step 11:

[1910] The user finalizes their travel plans and completes the booking process.

[1911] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[1912] The terminal sends this request to the server.

[1913] Step 12:

[1914] The server supports the reservation process.

[1915] The server connects to the reservation system to check availability of the hotel or flight the user desires.

[1916] If the reservation is successful, confirmation information is sent to the user via the terminal.

[1917] For example, information such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX" may be displayed.

[1918] Step 13:

[1919] Manage and view your travel history.

[1920] To check their travel history, users type "view past travel history."

[1921] The terminal sends this request to the server.

[1922] Step 14:

[1923] The server provides the user's travel history.

[1924] The server retrieves the user's past travel data from the database, formats it, and sends it to the terminal.

[1925] For example, it will display "Trip to Hawaii in August 2022: 5 nights and 7 days, information on tourist spots visited and hotels stayed."

[1926] Example 2

[1927] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1928] Conventional travel plan generation systems only provide standard plans without considering the user's emotional state, and therefore are unable to respond to individual user needs and emotions. This makes it difficult to provide a travel plan that satisfies the user, resulting in a lot of effort required for customization and a decrease in satisfaction.

[1929] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an input means for a user to input travel-related questions and requests, a generation means for generating a travel plan based on the user's questions and requests input through the input means, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue with the user, a reservation means for supporting the reservation procedure based on the customized travel plan, a management means for managing the user's travel history and making it viewable, an emotion recognition means for analyzing the user's emotional state, and an adjustment means for adjusting the travel plan based on the emotion information analyzed by the emotion recognition means. This makes it possible to provide a personalized travel plan that takes the user's emotional state into consideration.

[1930] "Input means" refers to the means by which a user inputs questions or requests regarding a trip.

[1931] The "generation means" is a means for generating a travel plan based on the user's questions and requests input through the input means.

[1932] The "presentation means" is a means for presenting the generated travel plan to the user.

[1933] "Customization means" refers to a means for customizing a travel plan through interaction with the user.

[1934] A "reservation tool" is a tool that supports the reservation process based on a customized travel plan.

[1935] "Management means" refers to the means for managing and making available the user's travel history.

[1936] "Emotion recognition means" is a means for analyzing the emotional state of a user.

[1937] The "adjustment means" is a means for adjusting the travel plan based on the emotion information analyzed by the emotion recognition means.

[1938] The present invention is a system for planning and managing travel, which aims to provide a personalized travel plan based on user input. The system includes an input means for a user to input travel questions and requests, a generation means for generating a travel plan based on the input information, a presentation means for presenting the generated travel plan to the user, a customization means for customizing the travel plan through dialogue, a reservation means for supporting a reservation procedure based on the customized plan, a management means for managing the user's travel history and making it viewable, an emotion recognition means for analyzing the user's emotional state, and an adjustment means for adjusting the plan based on the emotion recognition means.

[1939] System Overview

[1940] 1. Input Method

[1941] Users use their smartphones or computers to input desired conditions such as travel destination, theme, budget, etc. For example, they might input, "I'd like to plan a family trip to Hawaii during the summer vacation, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[1942] 2. Generation means

[1943] The server analyzes the input sent by the user and extracts the necessary keywords, such as "Hawaii," "family," and "budget of 300,000 yen."

[1944] The server uses a generative AI model to retrieve data on relevant tourist attractions, accommodations, and transportation from a database and generate a travel plan.

[1945] 3. Presentation means

[1946] The server then presents the generated travel plan to the user through an interface such as a chatbot, and the user can review this information on their device.

[1947] 4. Customization Methods

[1948] Users can use the chatbot to input specific questions or requests for modifications to the proposed itinerary, such as "Tell me more about this tourist spot" or "Instead of this hotel, find another hotel closer to the beach."

[1949] 5. Reservation Methods

[1950] Based on the plan confirmed by the user, the server checks the availability of hotels and flights in conjunction with the reservation system and completes the reservation process. If the reservation is successful, the server provides the user with this information. For example, a message such as "Your hotel reservation has been completed. Your confirmation number is XXXXXX" is displayed on the terminal.

[1951] 6. Control measures

[1952] Users can submit requests to the system to view their past travel history, for example, by typing "Show me my past travel history," and the information will be displayed.

[1953] 7. Emotion recognition means

[1954] The emotion engine analyzes the text and voice data entered by the user to identify the user's emotional state, which allows it to understand what the user wants and reflect that in its plans.

[1955] 8. Adjustment means

[1956] The server dynamically adjusts the travel plan based on the emotional information obtained by the emotion engine, providing a plan that matches the user's emotions, such as suggesting a more relaxed schedule for a user looking to relax.

[1957] Specific examples

[1958] To explain the specific operation in more detail, some specific examples will be given.

[1959] Example prompt sentence:

[1960] "I want to plan a relaxing trip abroad with my family during summer vacation."

[1961] "Tell me about tourist spots in Europe"

[1962] "How much do you need to budget for your trip?"

[1963] When a user uses their smartphone to type, "I want to take my family on a summer vacation to Hawaii," the device sends this information to the server. The server analyzes the received information and retrieves the most relevant information from its database using keywords like "Hawaii," "family-friendly," and "budget of ¥300,000." The emotion engine then analyzes the user's input and voice data to identify emotional information, such as "I'm looking for relaxation." The server then generates an initial travel plan based on this information and presents it to the user through a chatbot. If the user types, "Tell me more about snorkeling spots," the server searches for related information and provides more information. During this process, the emotion engine monitors the user's emotions in real time and adjusts the plan as needed. Finally, the user can confirm the travel plan and proceed with the booking process. Users can also easily check their past travel history. This series of actions results in efficient and personalized travel planning.

[1964] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1965] Step 1:

[1966] Users enter travel questions and requests

[1967] Users use their smartphones or computers to input conditions such as desired travel destination, theme, budget, etc. Specifically, they might write in the input form, "I'd like to plan a family trip to Hawaii during the summer vacation, with a budget of 300,000 yen, and would like to stay at a resort hotel."

[1968] Input: User's travel question or request

[1969] Output: User-entered criteria (travel destination, theme, budget, etc.)

[1970] Step 2:

[1971] The device sends the input information to the server

[1972] The device sends the user-entered conditions to the server in real time, with the data being sent via HTTP requests.

[1973] Input: Conditions entered by the user on the device

[1974] Output: User criteria sent to server

[1975] Step 3:

[1976] The server analyzes the user information and searches the database

[1977] The server analyzes the received user conditions and extracts important keywords (travel destination, budget, theme, etc.). For example, keywords such as "Hawaii," "family," and "budget of 300,000 yen" are extracted.

[1978] The server then uses these keywords to search a database to retrieve relevant tourist spots, hotels, and transportation options.

[1979] Input: User criteria sent to the server

[1980] Output: Important keywords, travel data retrieved from the database

[1981] Step 4:

[1982] Emotion engine analyzes the user's emotional state

[1983] The emotion engine analyzes user input and voice data to identify emotional states, such as joy, anxiety, and anticipation, for example.

[1984] The emotion engine transmits the analyzed emotion information to the server.

[1985] Input: User input, voice data

[1986] Output: Parsed emotion information

[1987] Step 5:

[1988] The server generates the initial itinerary

[1989] The server uses a generative AI model to generate an initial itinerary based on the acquired data and emotional information, such as "7-day Hawaii itinerary: Day 1: Beach visit, Day 2: Shopping, Day 3: Snorkeling."

[1990] The system takes into account the user's emotional information and optimizes the plan, such as adjusting to a more relaxed schedule if the user is looking to relax.

[1991] Input: Travel data obtained from the database, analyzed emotion information

[1992] Output: Initial itinerary

[1993] Step 6:

[1994] The server presents the generated itinerary to the user.

[1995] The server presents the initial travel plan to the user through the chatbot, and the user can review the plan on their device.

[1996] Input: Initial travel plan

[1997] Output: The itinerary presented to the user

[1998] Step 7:

[1999] Users interact with the chatbot to get more information

[2000] Users can then use the chatbot to ask specific questions about the proposed plan, such as "Tell me more about this tourist spot."

[2001] The terminal sends this question to the server.

[2002] Input: User's specific question

[2003] Output: The question sent to the server

[2004] Step 8:

[2005] The server provides information based on the user's question.

[2006] The server analyzes the user's question and retrieves appropriate details from the database, such as "You can enjoy beach volleyball and kayaking at this tourist spot," and sends this information to the chatbot.

[2007] The user can view this information through their device.

[2008] Input: User question, database details

[2009] Output: Answer with detailed information provided

[2010] Step 9:

[2011] Customize your travel plans based on the information you provide

[2012] Users can then use the details they have acquired to modify or add to their travel plans, for example, by saying, "Instead of this hotel, find another hotel closer to the beach."

[2013] The emotion engine monitors the user's emotional changes in real time through this dialogue and transmits the emotional information to the server.

[2014] Input: User's correction instructions, emotional information

[2015] Output: revised plan, emotional information

[2016] Step 10:

[2017] The server regenerates the itinerary and presents it to the user.

[2018] The server regenerates the travel plan based on the user's new requests and emotional information, and presents a customized plan through the chatbot, for example, "Day 1: Beach, Day 2: Shopping, Day 3: Snorkeling and Beach Volleyball."

[2019] Input: User's new request, emotion information

[2020] Output: Regenerated itinerary

[2021] Step 11:

[2022] The user confirms their travel plans and completes the booking process

[2023] The user enters, "I'd like to confirm this plan and book a hotel and flight."

[2024] The terminal sends this request to the server.

[2025] Input: User confirmation request

[2026] Output: Confirm request sent to server

[2027] Step 12:

[2028] The server handles the reservation process

[2029] The server connects to the reservation system to check availability of the hotel and flight desired by the user. If the reservation is successful, the server sends a confirmation to the user via the terminal. For example, the server displays information such as "Your hotel reservation has been completed. The confirmation number is XXXXXX."

[2030] Input: User's reservation request information, availability in the reservation system

[2031] Output: Reservation confirmation information

[2032] Step 13:

[2033] Users can manage and view their travel history

[2034] If a user wants to check their past travel history, they type "View past travel history."

[2035] The terminal sends this request to the server.

[2036] Input: User's request to view travel history

[2037] Output: View request sent to the server

[2038] Step 14:

[2039] The server provides the user's travel history

[2040] The server retrieves the user's past travel data from the database, formats it, and sends it to the device. For example, it displays "A trip to Hawaii in August 2022: 5 nights and 7 days, information on the tourist spots visited, and the hotel where the user stayed."

[2041] Input: Travel history data from the database

[2042] Output: Formatted travel history information

[2043] (Application example 2)

[2044] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2045] The problem to be solved by the present invention is to provide a system that efficiently generates a personalized meal plan taking into account the user's preferences and emotional state when the user inputs their meal requests and questions, and smoothly supports the delivery procedure, including the customization process.

[2046] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an input means for the user to input their wishes and requests regarding their request, a generation means for generating a plan based on the user's wishes and requests input through the input means, a presentation means for presenting the generated plan to the user, a customization means for customizing the plan through dialogue with the user, a reservation means for supporting procedures based on the customized plan, and a management means for managing the user's history and making it viewable. This enables the generation, customization, and reservation procedures of personalized meal plans.

[2047] "Requests" refer to the conditions desired by the user and the content of the services they would like to receive.

[2048] "Input means" refers to an interface that allows a user to input information into a device or system.

[2049] "Generation means" refers to the process or technical mechanism for automatically generating a specific plan or proposal based on input information.

[2050] "Presentation means" refers to the method or interface for displaying the generated plan or information to the user.

[2051] "Customization means" refers to a method or system that allows users to modify or change the plans or information presented to them to suit their preferences.

[2052] "Reservation Method" refers to the method or system for carrying out formal procedures based on the plan selected or modified by the User.

[2053] "Management means" refers to the method or system for storing and organizing users' past usage history and information, and making them available for viewing as needed.

[2054] An "automatic response system" refers to technology or mechanisms that allow a system to automatically communicate with users.

[2055] An "information repository" refers to a system or database for storing and managing various data and information.

[2056] A "generative AI model" refers to an algorithm or model that uses machine learning or artificial intelligence to generate new plans or information based on specific patterns or information.

[2057] The present invention is a system that includes an input means for a user to input meal preferences and questions, a generation means for generating a meal plan based on the preferences and questions input through the input means, a presentation means for presenting the generated plan to the user, a customization means for customizing the plan through dialogue with the user, a reservation means for supporting delivery procedures based on the customized plan, and a management means for managing the user's meal history and making it viewable.

[2058] Detailed System Description

[2059] 1. Input method:

[2060] Users use their smartphones to input their meal preferences, budget, delivery time, and other requests into the application's input form. If a user inputs, "I want to order a healthy, stress-reducing meal," this information is sent from the smartphone to the server.

[2061] 2. Generation means:

[2062] The server analyzes the received user information and performs sentiment analysis using the Emotion API. The analyzed data is then used to generate an appropriate meal plan using a generative AI model (e.g., GPT-4). This generative AI model uses the input data and the extracted sentiment information as prompts to suggest the optimal meal plan.

[2063] 3. Means of presentation:

[2064] The generated meal plan is presented to the user through a chatbot, which provides the user with the meal plan details in a conversational format.

[2065] 4. Customization methods:

[2066] Users can modify or add to the generated meal plan through dialogue with the chatbot. For example, if they input a request such as "I'd like to add avocado to my healthy bowl," the server will again use the generative AI model to generate a new plan.

[2067] 5. Reservation Method:

[2068] Once the user is satisfied with the customized meal plan and selects "I would like to order this plan," the server connects with the delivery system to process the reservation. If the reservation is successful, a confirmation is sent to the user.

[2069] 6. Control measures:

[2070] The user's past meal history is stored in a database and can be viewed as needed. When a user types "I want to check my past order history," the relevant information is displayed.

[2071] Hardware and software used

[2072] Device: Smartphone (iOS / Android)

[2073] Emotion engine: Emotion API (e.g. Microsoft Azure Cognitive Services)

[2074] Generative AI model: GPT-4 (OpenAI API)

[2075] Database: PostgreSQL (Amazon RDS)

[2076] Frontend: React Native

[2077] Backend: Node.js (Express)

[2078] Example prompt sentence

[2079] "The user wants to order a meal. The user's wishes are as follows: Travel destination: Please suggest a menu with the theme of 'healthy and relaxing meals', within a budget of 2000 yen, and that can be delivered within 30 minutes. Furthermore, please consider 'stress,' 'relaxation,' and 'fatigue' based on the results of the sentiment analysis."

[2080] By implementing this invention, users can select, customize, and reserve a personalized m...

Claims

1. an input means for a user to input travel-related questions or requests; a generation means for generating a travel plan based on the user's questions and requests input through the input means; a presentation means for presenting the generated travel plan to a user; A customization method for customizing travel plans through dialogue with users; A booking method that supports the booking process based on a customized travel plan; A means of managing and making available the user's travel history; A system including:

2. 10. The system of claim 1, further comprising means for a user to use a chatbot to customize a travel plan.

3. The system of claim 1 , wherein the generating means includes means for obtaining travel data from a database based on user input and generating a travel plan using a generative AI model.

Images