system

Abstract

To provide a system that allows users to easily and effectively plan their trips. [Solution] A system including means for receiving travel destination information entered by the user, means for selecting a mode of transportation based on the received information, means for presenting the selected mode of transportation information to the user, means for searching for accommodation based on the travel destination information entered by the user, means for presenting the searched accommodation information to the user, means for generating a list of tourist spots and activities based on the user's preferences, means for presenting the generated list of tourist spots and activities to the user, means for receiving the user's request for changes and optimizing the plan again, means for presenting the optimized travel plan to the user, means for receiving final booking confirmation information and confirming the bookings for transportation, accommodation, tourist spots / activities, and activities, and means for notifying the user of the booking confirmation information.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Making a travel plan requires many individual tasks (such as investigating transportation means, reserving accommodation facilities, selecting tourist spots, etc.), which are time-consuming and laborious for users. Furthermore, since various services are used separately, there is a problem that it is difficult to properly adjust the overall schedule. Also, it is not easy to construct an optimal travel plan according to the user's preferences. Therefore, there is a need for a system that allows users to easily and effectively make travel plans.

Means for Solving the Problems

[0005] This invention provides a travel plan generation system that can automatically generate an optimal travel plan based on information entered by the user and readjust it according to the user's requests. It consists of the following means:

[0006] A means of receiving travel destination information entered by the user,

[0007] A means of selecting a mode of transportation based on the received information,

[0008] A means of presenting selected transportation information to the user,

[0009] A method for searching for accommodations based on travel destination information entered by the user,

[0010] A means of presenting searched accommodation information to the user,

[0011] A means of generating a list of tourist spots and activities based on user preferences,

[0012] A means of presenting a generated list of tourist spots and activities to the user,

[0013] A means of receiving user change requests and re-optimizing the plan,

[0014] A means of presenting users with optimized travel plans,

[0015] You will receive final booking confirmation information and a means to confirm your bookings for transportation, accommodation, and sightseeing / activity.

[0016] A means of notifying users of reservation confirmation information,

[0017] This allows users to plan their trips in one place, significantly reducing the time and effort required.

[0018] "User information" refers to data on travel destinations, travel schedules, budgets, and preferences provided by users.

[0019] "Transportation means" refers to means of transportation such as airplanes, bullet trains, buses, cars, etc. that users utilize to reach their travel destinations.

[0020] "Accommodation facilities" refers to facilities such as hotels, ryokans, and minshukus where users stay during their travels.

[0021] "Tourist attractions" refers to famous places and sights to be visited at the travel destination.

[0022] "Activities" refers to specific experiences and events that can be carried out during travel.

[0023] "Reservation confirmation information" refers to data indicating that the reservations for transportation means, accommodation facilities, and activities selected by the user have been successful.

[0024] "System" refers to the overall combination of hardware and software that provides specific services to users by combining multiple means.

[0025] "Plan optimization" refers to the process of improving and readjusting the initial travel plan in response to the user's desired changes.

[0026] "Means" refers to methods, processes, and modules for realizing specific functions.

[0027] [[ID= (36)]]"Presentation" refers to the act of the system displaying or notifying information to the user.

[0028] "Receiving" refers to the act of the system acquiring input data and requests from the user.

[0029] "Generation" refers to the act of the system creating new plans and information based on the user's input data. <000-0103>

[0030] "Accommodation search" refers to the act of a system researching and retrieving a list of accommodations based on specific criteria. [Brief explanation of the drawing]

[0031] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0032] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0033] First, let's explain the terminology used in the following explanation.

[0034] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0035] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0036] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0037] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0038] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0039] [First Embodiment]

[0040] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0041] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0042] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0043] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0044] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0045] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0046] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0047] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0048] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0049] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0050] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0051] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0052] This invention is a system that allows users to input travel destination information and handle everything from selecting transportation, accommodations, sightseeing spots, and activities, to making reservations, all in one place. This system can generate and continuously optimize optimal travel plans based on the user's preferences and requirements.

[0053] Entering user information and sending it to the server.

[0054] The user uses a terminal to input travel planning information. Specifically, they input the destination, travel dates, number of nights, budget, and purpose of the trip (e.g., historical site visits or gourmet tours). The terminal collects this information as digital data and sends it to the server.

[0055] Travel plan generation

[0056] When the server receives the above data, it first selects a mode of transportation. The server searches a database of transportation options such as flights, trains, and buses to find the best choice and selects the most suitable mode of transport based on the user's budget and preferences. Next, it searches for accommodations near the travel destination and selects facilities that are within the budget and meet the user's preferences.

[0057] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, it suggests information on historical sites and landmarks for history buffs, and popular local restaurants for foodies. In this way, a comprehensive travel plan is initially generated from multiple perspectives.

[0058] User presentation of the generated plan

[0059] The generated initial travel plan is sent from the server to the terminal and displayed to the user. The user can review the details of the presented plan and enter any desired changes or additions. For example, they can add specific tourist attractions or change accommodations.

[0060] Optimizing travel plans

[0061] When the user enters their requested changes, the device sends that information back to the server. The server then searches and selects again based on the user's requests, optimizing the travel plan. The updated plan is then sent back to the device and presented to the user.

[0062] Reservation Confirmation

[0063] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button to finalize their travel plans. The device sends this information to the server, which then makes all the necessary bookings for transportation, accommodation, and activities. Once the bookings are complete, the server sends confirmation information (such as ticket booking numbers or accommodation confirmation numbers) to the device. The device then notifies the user of this confirmation information, informing them that their travel plans have been finalized.

[0064] Specific example

[0065] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[0066] Transportation: Shinkansen ticket from Tokyo to Kyoto

[0067] Accommodation: 3-star hotels near Gion

[0068] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[0069] Gourmet spot: A famous local Japanese restaurant

[0070] If a user presented with this plan adds "I'd also like to visit Arashiyama," the server recalculates and generates a new plan that includes "Arashiyama." Once the user is satisfied with the optimized plan and confirms their booking, transportation, accommodation, and activities are all booked together.

[0071] As described above, this system allows users to easily plan their trips and flexibly respond to their individual needs.

[0072] The following describes the processing flow.

[0073] Step 1:

[0074] The user enters travel destination information.

[0075] The user enters their travel destination, travel dates, number of nights, budget, and purpose of travel into their device.

[0076] The terminal collects the input information as data packets.

[0077] Step 2:

[0078] The terminal sends the input information to the server.

[0079] The terminal sends the collected data packets to the server.

[0080] The terminal waits for confirmation of receipt from the server.

[0081] Step 3:

[0082] Server receives data

[0083] The server receives data packets sent from the terminal and analyzes their contents.

[0084] The server establishes a database connection.

[0085] Step 4:

[0086] The server selects the mode of transport.

[0087] Based on the received travel destination information, the server consults a database of transportation options to select the most suitable mode of transport.

[0088] The server retrieves detailed information about the selected mode of transport (e.g., flight number, time, price).

[0089] Step 5:

[0090] Server searches for accommodations

[0091] The server references a database of accommodations in the travel destination to search for accommodations that fit the user's budget.

[0092] We obtain real-time information on available rooms and select the most suitable accommodation.

[0093] Step 6:

[0094] The server suggests tourist spots and activities.

[0095] The server searches for tourist attractions and activities in the user's travel destination based on their preferences.

[0096] The server prioritizes the list of selected tourist spots and activities.

[0097] Step 7:

[0098] The server generates an initial travel plan.

[0099] The server generates an initial travel plan that combines information on transportation, accommodation, tourist attractions, and activities.

[0100] Step 8:

[0101] The server sends the generated plan to the terminal.

[0102] The server sends the generated travel plan to the device.

[0103] Step 9:

[0104] The device displays the plan to the user.

[0105] The device displays the user details of the travel plan.

[0106] Users can review the plan details and enter any requested modifications on their device.

[0107] Step 10:

[0108] The user enters their request to change their plan.

[0109] Users enter their requests for plan changes (e.g., adding or changing specific tourist spots).

[0110] Step 11:

[0111] The device sends the change request to the server.

[0112] The device sends the user's change request to the server.

[0113] Step 12:

[0114] The server will regenerate to reflect the requested changes.

[0115] The server regenerates the travel plan based on the user's requested changes.

[0116] Obtain information on new transportation options, accommodations, and tourist attractions as needed.

[0117] Step 13:

[0118] The server sends the optimized plan to the device.

[0119] The server sends an optimized travel plan to the device.

[0120] Step 14:

[0121] The device displays the optimized plan to the user.

[0122] The device displays an optimized travel plan to the user.

[0123] Step 15:

[0124] The user confirms the final plan.

[0125] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button on their device.

[0126] Step 16:

[0127] The terminal sends confirmed information to the server.

[0128] The terminal sends the final reservation confirmation information to the server.

[0129] Step 17:

[0130] The server executes the reservation.

[0131] The server handles reservations for transportation, accommodation, and sightseeing / activity.

[0132] The server obtains confirmation information from each service provider.

[0133] Step 18:

[0134] The server sends reservation confirmation information to the terminal.

[0135] The server sends the confirmed reservation details to the terminal.

[0136] Step 19:

[0137] The device notifies the user that the reservation is complete.

[0138] The device notifies the user that the reservation is complete and displays the necessary information (e.g., reservation number, confirmation documents).

[0139] (Example 1)

[0140] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0141] Traditional travel planning systems required users to individually research and book transportation, accommodations, tourist attractions, and activities, which was a time-consuming and laborious process. Furthermore, the systems lacked sufficient automated suggestions to match users' preferences and budgets, making it difficult to optimize plans as users had to manually select and choose from options.

[0142] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0143] In this invention, the server includes means for receiving travel destination information entered by the user, means for collecting the received information as digital data and transmitting it to the server, means for selecting a mode of transportation based on the received information, means for presenting the selected mode of transportation information to the user, means for searching for accommodation based on the travel destination information entered by the user, means for presenting the searched accommodation information to the user, means for generating a list of tourist spots and activities based on the user's preferences, means for presenting the generated list of tourist spots and activities to the user, means for receiving the user's requests for changes and optimizing the plan again, means for presenting the optimized travel plan to the user, means for receiving final booking confirmation information and confirming the bookings for transportation, accommodation, tourist spots, and activities, and means for notifying the user of the booking confirmation information. This enables the user to plan their trip all at once, optimize that plan, and make reservations based on that plan with confidence.

[0144] A "user" is an individual or group that inputs travel planning information and uses the system.

[0145] A "device" is an electronic device used by a user to input travel planning information and receive notifications such as plan presentations and booking confirmations. Specific examples include smartphones and personal computers.

[0146] A "server" is a central processing unit that receives travel plan information sent by users and uses that information to select and optimize transportation, accommodation, tourist attractions, and activities.

[0147] "Travel destination information" refers to detailed travel information entered by the user, specifically including the travel destination, travel itinerary, number of nights of stay, budget, and purpose of travel.

[0148] "Transportation" refers to the means of getting around for travel, and specific examples include airplanes, bullet trains, railways, and buses.

[0149] "Accommodation facilities" refer to places where users stay during their travels, and specific examples include hotels, inns, and guesthouses.

[0150] A "tourist spot" is a place or landmark that should be visited at a travel destination, and specific examples include historical buildings and natural landscapes.

[0151] An "activity" refers to an activity or experience that a user can have at their travel destination, and specific examples include sightseeing tours and leisure activities.

[0152] "Digital data" refers to the electronic representation of travel plan information entered by a user using a device.

[0153] A "list" is a list containing a series of tourist attractions or activity items generated based on specific criteria.

[0154] A "plan" is the overall outline of a travel plan generated based on the user's input information and preferences.

[0155] "Booking confirmation information" refers to confirmation information regarding reservations for transportation, accommodation, sightseeing spots, and activities that is sent from the server when the user finalizes their travel plans.

[0156] "Validation" is the process of verifying that user input is accurate and complete.

[0157] An "algorithm" is a set of calculations or processing steps to optimize a travel plan again based on the user's requested changes.

[0158] This invention is a system that allows users to input travel destination information and handle everything from selecting transportation, accommodations, sightseeing spots, and activities, to making reservations, all in one go. This system can generate and continuously optimize optimal travel plans based on the user's preferences and circumstances.

[0159] First, the user enters travel plan information using a device, such as a smartphone or computer. The user enters detailed information such as the travel destination, itinerary, number of nights, budget, and purpose of travel. The device collects this information as digital data and sends it to the server. During this process, the device validates the user's input in real time and displays error messages as needed.

[0160] When the server receives user information, it first selects a mode of transportation. The server queries a database of transportation options such as air travel, trains, and buses, and selects the most suitable mode of transport based on the user's budget and preferences. Specifically, it uses a Python library (e.g., pandas) to analyze data and select the appropriate mode of transport. This process utilizes a specific algorithm to efficiently search for transportation options that meet the user's criteria.

[0161] Next, the server selects accommodations. It searches the database for the most suitable accommodations based on the user's budget and requirements. The server uses SQL queries to extract properties that match the criteria and ranks the accommodations considering reviews and ratings. Finally, the most suitable property is selected and presented to the user.

[0162] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, if the user is a history buff, the server will list spots related to historical landmarks and ruins. This process uses a specific search engine (e.g., ElasticSearch®) to quickly search for relevant data and generate it as a list.

[0163] The generated initial travel plan is sent to the device and presented to the user. The user reviews the proposed plan through the device's interface and enters any desired changes. For example, they can add specific tourist attractions or change accommodations.

[0164] When a user submits a request for changes, the server searches and selects the most suitable plan again, optimizing the travel plan. This process can utilize a generative AI model to provide the user with the most appropriate plan. The recalculated plan is then sent back to the device and presented to the user.

[0165] If the user is satisfied with the optimized plan, they can confirm their travel plans by pressing the "Confirm Booking" button. The device sends this information to the server, which confirms the bookings for transportation, accommodation, sightseeing, and activities. Once the booking is complete, the server generates confirmation information (e.g., ticket booking number or accommodation confirmation number) and sends it to the device. The device notifies the user of this confirmation information, informing them that their travel plans have been confirmed.

[0166] Specific example

[0167] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[0168] Transportation: Shinkansen ticket from Tokyo to Kyoto

[0169] Accommodation: 3-star hotels near Gion

[0170] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[0171] Gourmet spot: A famous local Japanese restaurant

[0172] If a user presented with this plan adds "I'd also like to visit Arashiyama," the server recalculates and generates a new plan that includes "Arashiyama." Once the user is satisfied with the optimized plan and confirms their booking, transportation, accommodation, and activities are all booked together.

[0173] This allows the system to easily plan trips and flexibly accommodate the individual needs of each user.

[0174] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0175] The processing flow of this system's program

[0176] Step 1:

[0177] The user enters travel planning information using a device. Specifically, the user enters the destination, travel dates, number of nights, budget, and purpose of travel on the interface screen. The device collects the information entered by the user as digital data. It also validates the input data, displays error messages, and prompts the user to make corrections as needed. The input data is saved in a structured format (e.g., JSON) and sent to the server.

[0178] Step 2:

[0179] The server receives travel plan information sent from the terminal. The server stores this information in a database. Specifically, it uses a relational database management system (RDBMS) such as MySQL® to organize and store the information appropriately in each table. The received data includes travel destination, travel itinerary, number of nights of stay, budget, and purpose of travel.

[0180] Step 3:

[0181] The server selects transportation options based on the user's travel plan information. The server queries a database of transportation options to find the best option based on the user's budget and preferences. Specifically, it uses Python libraries (e.g., pandas) to analyze the data and suggest appropriate options. The output includes detailed information on the best transportation option (e.g., air travel, bullet train, bus, etc.).

[0182] Step 4:

[0183] The server presents the user with information on the selected mode of transport. The selected transport is then sent to the user's terminal and displayed. The user can review the information and proceed to the next step. The output includes detailed information about the transport option displayed on the screen.

[0184] Step 5:

[0185] The server searches for accommodations based on the user's travel plan information. The server queries a database of accommodations to find properties that meet the user's budget and preferences. Specifically, it uses SQL queries to extract properties that match the criteria and selects the best accommodation based on reviews and ratings. The output includes detailed information about the best-suited accommodation.

[0186] Step 6:

[0187] The server presents the user with searched accommodation information. The selected accommodations are sent to the terminal and displayed to the user. The user can review the presented accommodation information and proceed to the next step. The output displays detailed information about the accommodation on the screen.

[0188] Step 7:

[0189] The server generates a list of tourist attractions and activities based on the user's preferences. Specifically, it lists suggested tourist attractions and activities based on the travel purpose and preferences entered by the user. For example, it might suggest historical sites to a history enthusiast and famous local restaurants to a foodie. The output is a list of tourist attractions and activities.

[0190] Step 8:

[0191] The server generates a list of tourist attractions and activities and presents it to the user. The list is sent to the user's device and displayed. The user can review the list and enter any desired changes or additions. The output is a list of tourist attractions and activities displayed on the screen.

[0192] Step 9:

[0193] The system receives user requests for changes and resends them to the server. When a user enters corrections or additions, the terminal sends that information to the server and requests a recalculation. Specific input data includes tourist spots to be added and accommodation information to be changed.

[0194] Step 10:

[0195] The server re-optimizes the travel plan based on the user's requested changes. It queries the databases of transportation, accommodation, and attractions again using the new information, generating the optimal plan based on the latest conditions. The output includes the updated travel plan.

[0196] Step 11:

[0197] The optimized travel plan is sent to the user's device and presented to them. The recalculated plan is sent to the device for the user to review and display. The user can review the optimized plan and proceed to confirm their booking. The output displays detailed information about the optimized travel plan on the screen.

[0198] Step 12:

[0199] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button. The device sends this information to the server, which then confirms each booking. The specific input data includes the final booking confirmation information.

[0200] Step 13:

[0201] The server confirms reservations for transportation, accommodation, and tourist attractions / activities. It updates the relevant database based on each reservation to verify its success. The output includes confirmation information (e.g., ticket number and accommodation confirmation number).

[0202] Step 14:

[0203] The server sends confirmation information to the terminal and notifies the user. Final confirmation information is sent to the terminal and notified the user. The user receives the confirmation information and knows that their travel plan is confirmed. The output is the confirmation information displayed on the user's terminal.

[0204] The above outlines the specific processing steps of this system's program. This process allows users to easily plan their trips, obtain the optimal plan, and make secure reservations. A key feature of this system is its ability to automatically generate and provide optimized travel plans based on user input.

[0205] (Application Example 1)

[0206] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0207] The process of planning a trip is complex due to the many factors to consider, and it is not easy to create a plan that perfectly matches the user's desires in one go. Furthermore, users need to individually search for and book transportation, accommodations, attractions, and activities, which is also time-consuming. Additionally, when users plan and book trips in a virtual space, the lack of a suitable interface prevents them from providing the same experience as using a real travel agency.

[0208] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0209] In this invention, the server includes means for receiving travel destination information entered by the user, means for selecting a mode of transportation based on the received information, and means for providing an interface for the user to plan and book a trip in a virtual space. This allows the user to search and book transportation, accommodation, sightseeing spots, and activities all at once, and to intuitively plan, modify, and finalize a trip through an interactive experience in a virtual space.

[0210] "User" refers to the person who plans the trip.

[0211] "Travel destination information" refers to information about places that users want to visit.

[0212] "Transportation" refers to the means that users use to travel to their destination.

[0213] "Accommodation" refers to the place where a user stays during their trip.

[0214] "Tourist spots" refer to points of interest and tourist attractions at the destination the user visits.

[0215] "Activities" refer to the activities and events that users undertake while traveling.

[0216] "Preferences" refers to information about a user's interests and preferences.

[0217] "Virtual space" refers to a computer-generated three-dimensional or two-dimensional environment that users can experience through sight and sound.

[0218] An "interface" refers to the means or methods by which a user interacts with a system.

[0219] A "generative AI model" refers to an algorithm that uses artificial intelligence to create travel plans from data.

[0220] A "prompt statement" refers to an input statement used to give instructions to a generative AI model.

[0221] "Booking confirmation information" refers to information indicating that the travel plan has been finalized and all bookings have been completed.

[0222] This invention provides a travel planning system that allows users to intuitively plan, modify, and finalize travel plans in a virtual space. This system utilizes servers, terminals, and a generative AI model to streamline the entire process from travel planning to booking, providing users with an interactive experience.

[0223] Hardware and software

[0224] The hardware includes user terminals (smartphones, head-mounted displays) and servers. The software includes generative AI models (OpenAI® API) and travel planning system programs.

[0225] Data processing and data calculation

[0226] 1. User Input Collection: Users input information such as travel destination, itinerary, budget, and preferences from their devices. This information is sent to the server, which processes the received data as digital data.

[0227] 2. Plan Generation: The server generates the optimal travel plan using a generation AI model based on travel destination information, dates, budget, and preferences. The following prompts are used during the generation process.

[0228] Travel destination: Kyoto

[0229] Duration: 4 days from December 1, 2023

[0230] Budget: 100,000 yen

[0231] Interests: History and gourmet food

[0232] Please generate the optimal travel plan.

[0233] The generated plan will include transportation, accommodation, sightseeing spots, and activities.

[0234] 3. Presentation and modification of the plan: The initial travel plan is sent from the server to the terminal and presented to the user. If the user enters a request for changes, that information is sent back to the server, which uses a generative AI model to further optimize the plan.

[0235] 4. Virtual Experience: The device provides users with a virtual interface, allowing them to intuitively plan, modify, and finalize their travel plans. This interface provides users with an interactive experience that feels as if they are visiting a real travel agency.

[0236] 5. Booking Confirmation and Notification: If the user is satisfied with the final travel plan, they press the "Confirm Booking" button to confirm the booking. The server makes all bookings for transportation, accommodation, and sightseeing / activity attractions at once and sends the booking confirmation information to the device. The device then notifies the user of this confirmation information.

[0237] Specific example

[0238] For example, if a user selects "Kyoto" as their travel destination in a virtual space, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[0239] Transportation: Shinkansen ticket from Tokyo to Kyoto

[0240] Accommodation: 3-star hotels near tourist attractions

[0241] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[0242] Gourmet spot: A famous local Japanese restaurant

[0243] If a user requests to visit Arashiyama, the server recalculates and generates a new plan that includes Arashiyama. Once the user is satisfied with the optimized plan and confirms their booking, transportation, accommodation, and activities are all booked together.

[0244] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0245] Step 1:

[0246] The user enters travel destination information. The user uses a terminal to enter travel plan information (destination, itinerary, number of nights, budget, purpose of travel, preferences). The terminal collects this information as digital data and sends it to the server. Input: Travel plan information. Output: Digital data transmission to the server.

[0247] Step 2:

[0248] The server selects a mode of transportation based on the information it receives. The server chooses the most suitable mode of transportation from a database of options such as air travel, trains, and buses, based on the user's budget and preferences. The selection is made using a generated AI model based on prompt messages. Input: Travel plan information. Output: Selected mode of transportation information.

[0249] Step 3:

[0250] The server transmits the selected transportation information to the terminal and presents it to the user. The user can then view the details of the presented transportation. Input: Selected transportation information. Output: Presentation of transportation information to the user.

[0251] Step 4:

[0252] The server searches for accommodations based on the travel destination information entered by the user. It selects and presents accommodations within the user's budget and preferences from a database of accommodations near the travel destination. The search is performed using a generated AI model based on prompt messages. Input: Travel plan information. Output: Searched accommodation information.

[0253] Step 5:

[0254] The server displays the searched accommodation information to the user. The user can view the accommodation details through their device. Input: Searched accommodation information. Output: Presentation of accommodation information to the user.

[0255] Step 6:

[0256] The server generates a list of tourist attractions and activities based on the user's preferences. It creates the most suitable list from a database of tourist attractions and activities related to those preferences. Input: Travel plan information and preference information. Output: Generated list of tourist attractions and activities.

[0257] Step 7:

[0258] The server presents the user with a list of generated tourist spots and activities. The user can view the list through their device. Input: List of generated tourist spots and activities. Output: Presentation of the list to the user.

[0259] Step 8:

[0260] The server receives the user's change requests and optimizes the plan again. The user enters their change requests via their device and sends them to the server. The server uses a generated AI model to optimize the plan again. Input: Change request information. Output: Optimized travel plan.

[0261] Step 9:

[0262] The server presents the user with an optimized travel plan. The user can view the details of the optimized plan through their device. Input: Optimized travel plan. Output: Presentation of the optimized plan to the user.

[0263] Step 10:

[0264] The server receives the final booking confirmation information and confirms reservations for transportation, accommodation, and sightseeing / activity. If the user is satisfied with the optimized plan, they press the "Confirm Booking" button and send the information to the server. The server receives this and makes the bookings in bulk. Input: Final booking confirmation information. Output: Generation of booking confirmation information.

[0265] Step 11:

[0266] The server notifies the user of the reservation confirmation information. The user can receive the confirmation information via their device. Input: Reservation confirmation information. Output: Notification to the user.

[0267] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0268] This invention is a system in which a user inputs travel destination information, the system automatically generates an optimal travel plan, and further optimizes the plan based on the user's emotions using an emotion engine.

[0269] Entering user information and sending it to the server.

[0270] Users input travel planning information using a device. Specifically, they enter information such as the destination, travel dates, number of nights, budget, and purpose of travel. The device collects this information as digital data and sends it to the server.

[0271] Travel plan generation

[0272] When the server receives the above data, it first selects a mode of transportation. The server searches its database of transportation options, such as flights, trains, and buses, and selects the most suitable mode of transport based on the user's budget and preferences. Next, it searches for accommodations near the travel destination and selects facilities that are within the budget and meet the user's preferences.

[0273] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, it suggests information on historical sites and landmarks for history buffs, and popular local restaurants for foodies. In this way, a comprehensive travel plan is initially generated from multiple perspectives.

[0274] User presentation of the generated plan

[0275] The generated initial travel plan is sent from the server to the terminal and displayed to the user. The user can review the details of the presented plan and enter any desired changes or additions. For example, they can add specific tourist attractions or change accommodations.

[0276] Using an Emotion Engine

[0277] This system incorporates an emotion engine that collects and analyzes user emotional data (e.g., facial expression analysis and voice analysis). The emotion engine understands the user's emotional state (e.g., joy, surprise, fatigue) in real time and optimizes the travel plan based on the results.

[0278] For example, if the emotion engine determines that the user is "tired," the server can add rest spots or relaxing activities to the plan. Similarly, if the engine determines that the user is "excited," it can suggest more active spots or exciting activities.

[0279] Optimization and Regeneration of Travel Plans

[0280] When the user inputs a desired change, the terminal sends it to the server, along with the information of the emotion engine. Based on this information, the server optimizes the plan again. If necessary, it searches for new transportation means, accommodation facilities, and tourist spot information, and generates an optimized plan.

[0281] Confirmation of Reservation

[0282] If the user is satisfied with the optimized plan, they press the "Confirm Reservation" button to finalize the travel plan. The terminal sends this information to the server, and the server makes reservations for transportation means, accommodation facilities, and activities in one go. When the reservation is completed, the server sends confirmation information (such as the reservation number of the ticket and the confirmation number of the accommodation facility) to the terminal. The terminal notifies the user of this confirmation information and conveys to the user that the travel plan has been finalized.

[0283] Specific Example

[0284] For example, when the user selects "Kyoto" as the travel destination, enters the travel schedule as "3 nights and 4 days from December 1, 2023", the budget as "100,000 yen", and the preference as "history and gourmet", the server may generate a plan as follows.

[0285] Transportation means: Shinkansen ticket from Tokyo to Kyoto

[0286] Accommodation facility: Three-star hotel around Gion

[0287] Tourist spots: Fushimi Inari Taisha, Kinkaku-ji, Kiyomizu-dera

[0288] [[ID=3G]]Gourmet spots: Famous local Japanese cuisine restaurants

[0289] If the emotion engine detects that the user is "tired," the server can add a rest cafe or a relaxing hot spring facility to the plan. In this way, a plan adapted to the user's state can be provided.

[0290] As described above, this system allows users to easily create travel plans that are perfectly tailored to their circumstances and preferences. The introduction of an emotion engine enables the provision of more personalized services and enhances user satisfaction.

[0291] The following describes the processing flow.

[0292] Step 1:

[0293] The user enters travel destination information.

[0294] Users use their device to enter their travel destination, itinerary, number of nights, budget, and purpose of travel.

[0295] The terminal collects the input information as data packets.

[0296] Step 2:

[0297] The terminal sends the input information to the server.

[0298] The terminal sends the collected data packets to the server.

[0299] The terminal monitors the transmission status and waits for confirmation of receipt from the server.

[0300] Step 3:

[0301] Server receives data

[0302] The server receives data packets sent from the terminal and analyzes their contents.

[0303] Establish a connection to the necessary database.

[0304] Step 4:

[0305] The server selects a means of transportation

[0306] Based on the received destination information, the server selects the optimal means of transportation from the database of means of transportation.

[0307] Obtain details of the selected means of transportation (e.g., flight name, time, price).

[0308] Step 5:

[0309] The server searches for accommodation facilities

[0310] The server uses the database to search for accommodation facilities around the destination.

[0311] Select facilities that meet the user's budget and preferences and obtain vacancy information.

[0312] Step 6:

[0313] The server proposes tourist spots and activities

[0314] The server analyzes the user's preference information and searches for tourist spots and activities at the destination.

[0315] Generate a list of selected spots and activities. <\

[0316] Step 7:

[0317] The server generates an initial travel plan

[0318] The server integrates the selected means of transportation, accommodation facilities, tourist spots, and activities to generate an initial travel plan.

[0319] Step 8:

[0320] The server sends the generated plan to the terminal.

[0321] The server sends the generated travel plan to the device.

[0322] Step 9:

[0323] The device displays the plan to the user.

[0324] The device displays the travel plan details to the user.

[0325] Users review the plan details and enter any requested modifications.

[0326] Step 10:

[0327] The emotion engine recognizes the user's emotions.

[0328] The emotion engine uses sensor data from the device (e.g., camera and microphone) to analyze the user's emotions.

[0329] The detected emotion data is sent to the server.

[0330] Step 11:

[0331] The user enters their request to change their plan.

[0332] Users enter their desired changes to the plan (e.g., adding or changing specific tourist spots) into the device.

[0333] Step 12:

[0334] The device sends change requests and sentiment data to the server.

[0335] The device sends user change requests and sentiment data to the server.

[0336] Step 13:

[0337] The server will regenerate to reflect requested changes and sentiment data.

[0338] The server regenerates the travel plan based on the user's requested changes and sentiment data.

[0339] Obtain information on new transportation options, accommodations, and tourist attractions as needed.

[0340] Step 14:

[0341] The server sends the optimized plan to the device.

[0342] The server sends an optimized travel plan to the device.

[0343] Step 15:

[0344] The device displays the optimized plan to the user.

[0345] The device displays an optimized travel plan to the user.

[0346] Step 16:

[0347] The user confirms the final plan.

[0348] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button on their device.

[0349] Step 17:

[0350] The terminal sends confirmed information to the server.

[0351] The terminal sends the final reservation confirmation information to the server.

[0352] Step 18:

[0353] The server executes the reservation.

[0354] The server confirms reservations for transportation, accommodation, and sightseeing / activity.

[0355] The server obtains confirmation information from each service provider.

[0356] Step 19:

[0357] The server sends reservation confirmation information to the terminal.

[0358] The server sends the confirmed reservation details to the terminal.

[0359] Step 20:

[0360] The device notifies the user that the reservation is complete.

[0361] The device notifies the user that the reservation is complete and displays the necessary information (e.g., reservation number, confirmation documents).

[0362] (Example 2)

[0363] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0364] Conventional travel plan generation systems only generate basic travel plans based on user input, making it difficult to provide optimal plans that reflect the user's emotions and individual preferences. Furthermore, it is difficult to quickly reflect user requests for changes to the travel plan. In addition, the lack of a function to dynamically optimize the plan based on the user's emotions prevents increasing user satisfaction during the trip. To solve these problems, a system is needed that analyzes the user's emotional state in real time and optimizes and presents travel plans based on that analysis.

[0365] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0366] In this invention, the server includes means for receiving travel destination information entered by the user, means for selecting a mode of transportation based on the received information, means for presenting the mode of transportation information to the user, means for searching for accommodation based on the travel destination information entered by the user, means for presenting the accommodation information to the user, means for generating a list of tourist spots and activities based on the user's preferences, means for presenting the generated list of tourist spots and activities to the user, means for collecting and analyzing the user's emotional data, means for optimizing the travel plan based on the emotional data, means for receiving the user's request for changes and optimizing the plan again, means for presenting the optimized travel plan to the user, means for receiving final booking confirmation information and confirming the booking of transportation, accommodation, tourist spots / activities, and the booking confirmation information to the user. This enables the generation of an optimal travel plan based on the user's emotions and preferences, and real-time optimization of the plan in response to the user's request for changes.

[0367] A "user" is someone who uses this system to input travel planning information and generate and optimize travel plans.

[0368] A "terminal" is a device (such as a personal computer, smartphone, or tablet) that a user uses to input information.

[0369] A "server" is a central processing unit that receives, analyzes, optimizes, and transmits information sent by users.

[0370] "Travel destination information" refers to data entered by the user, such as the travel destination, travel itinerary, number of nights, budget, and purpose of travel.

[0371] "Transportation" refers to the means of getting around during a trip (air, train, bus, etc.).

[0372] "Accommodation facilities" refer to places of stay during travel (such as hotels and inns).

[0373] A "tourist spot" refers to famous places or attractions that users can visit at their travel destination.

[0374] "Activities" refer to activities that can be participated in during the trip (such as sports, cultural experiences, and relaxation).

[0375] "Preferences" refer to a user's personal likes and interests (such as history, food, nature, etc.).

[0376] "Emotional data" refers to data that indicates the user's emotional state (obtained through methods such as facial expression analysis and voice analysis).

[0377] A "travel plan" is a comprehensive travel plan that includes selected transportation, accommodation, sightseeing spots, activities, and other details.

[0378] "Optimization" refers to adjusting travel plans to best suit the user based on their input information and emotional data.

[0379] "Booking confirmation information" refers to the final booking confirmation data for transportation, accommodation, sightseeing spots, and activities.

[0380] This invention is a system in which a user inputs travel destination information, the system automatically generates an optimal travel plan, and further optimizes the plan based on the user's emotions using an emotion engine.

[0381] Entering user information and sending it to the server.

[0382] Users input travel planning information using their devices. Specifically, they enter information such as the destination, travel dates, number of nights, budget, and purpose of travel. The device collects this information as digital data and sends it to the server. Devices such as personal computers and smartphones are used.

[0383] Travel plan generation

[0384] When the server receives data sent by the user, it first selects a mode of transportation. The server searches a database of transportation options such as flights, trains, and buses to find the best choice and selects the most suitable mode of transport based on the user's budget and preferences. Next, it searches for accommodations near the travel destination and selects facilities that are within the budget and meet the user's preferences.

[0385] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, it suggests information on historical sites and landmarks for history buffs, and popular local restaurants for foodies. In this way, a comprehensive travel plan is initially generated from multiple perspectives.

[0386] User presentation of the generated plan

[0387] The generated initial travel plan is sent from the server to the terminal and displayed to the user. The user can review the details of the presented plan and enter any desired changes or additions. For example, they can add specific tourist attractions or change accommodations.

[0388] Using an Emotion Engine

[0389] This system incorporates an emotion engine. It collects and analyzes user emotional data (e.g., facial expression analysis and voice analysis). The emotion engine understands the user's emotional state (e.g., joy, surprise, fatigue) in real time and optimizes the travel plan based on the results.

[0390] For example, if the emotion engine determines that the user is "tired," the server will add rest spots and relaxing activities to the plan. Similarly, if the user is determined to be "excited," it will suggest more active spots and exciting activities.

[0391] Optimizing and regenerating travel plans

[0392] When a user enters a change request, the device sends it to the server, along with information from the sentiment engine. The server uses this information to optimize the plan again, searching for new transportation options, accommodations, and tourist attractions, and generating an optimized plan.

[0393] Reservation Confirmation

[0394] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button to finalize their travel plans. The device sends this information to the server, which then makes all the necessary bookings for transportation, accommodation, and activities. Once the bookings are complete, the server sends confirmation information (such as ticket booking numbers or accommodation confirmation numbers) to the device. The device then notifies the user of this confirmation information, informing them that their travel plans have been finalized.

[0395] Specific example

[0396] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, "100,000 yen" as their budget, and "history and gourmet food" as their preferences, the server might generate a plan like the following.

[0397] Transportation: Shinkansen ticket from Tokyo to Kyoto

[0398] Accommodation: 3-star hotels near Gion

[0399] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[0400] Gourmet spot: A famous local Japanese restaurant

[0401] If the emotion engine detects that the user is "tired," the server can add a rest cafe or a relaxing hot spring facility to the plan. In this way, a plan adapted to the user's state can be provided.

[0402] Example of a prompt

[0403] Please automatically generate a travel plan for "Kyoto, 3 nights / 4 days from December 1st, 2023, 100,000 yen, focusing on history and gourmet food." Please also consider the emotional data "tired" and add relaxing activities.

[0404] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0405] Step 1:

[0406] Users enter travel plan information using their devices. This includes destination, travel dates, number of nights, budget, and preferences (history, food, etc.).

[0407] Input: User's travel plan information (destination, dates, number of nights, budget, preferences)

[0408] Output: Saved to the device as JSON data.

[0409] Specific actions:

[0410] When a user enters information into a form and presses the "Submit" button, the entered information is saved to the device as digital data in JSON format. At this point, a list of the entered information is displayed on the interface.

[0411] Step 2:

[0412] The terminal sends the travel plan information entered by the user to the server.

[0413] Input: JSON data of travel plan information saved on the device

[0414] Output: Sent to the server as an HTTP POST request

[0415] Specific actions:

[0416] The JSON data saved on the device is sent as an HTTP POST request, and when the send button is pressed, a "Sending data" message is displayed. Once the data has been sent, a notification of successful transmission is displayed.

[0417] Step 3:

[0418] The server receives travel plan information sent from the terminal and stores it in the database.

[0419] Input: JSON data sent as an HTTP request

[0420] Output: Travel plan information stored in the database

[0421] Specific actions:

[0422] The server receives the HTTP request and saves the data to its database. At this time, it returns a status code 200 (OK) as confirmation of receipt.

[0423] Step 4:

[0424] The server selects the mode of transportation based on the information it receives.

[0425] Input: Travel plan information stored in the database

[0426] Output: List of optimal modes of transport

[0427] Specific actions:

[0428] A program on the server executes an SQL query to search for transportation options (air, rail, bus, etc.) that match the specified criteria. The results are stored in memory as a list of candidates.

[0429] Step 5:

[0430] The server searches for accommodations that match the user's budget and preferences based on the selected mode of transportation.

[0431] Input: List of optimal transportation options, user's budget and preferences

[0432] Output: List of optimal accommodations

[0433] Specific actions:

[0434] A similar SQL query is executed from the accommodation database to search for accommodations that match the criteria. The results are added as a list of candidates.

[0435] Step 6:

[0436] The server generates a list of tourist spots and activities based on the user's preferences.

[0437] Input: User preferences (history, food, etc.)

[0438] Output: List of tourist attractions and activities

[0439] Specific actions:

[0440] The system filters based on preference data to list tourist spots and activities. It extracts necessary data from the database and generates the list.

[0441] Step 7:

[0442] The server sends the generated travel plan to the terminal and displays it to the user.

[0443] Input: Information on the best transportation options, accommodations, attractions, and activities.

[0444] Output: Initial travel plan

[0445] Specific actions:

[0446] The generated travel plan information is constructed in JSON format and returned to the device as an HTTP response. The device then parses the received JSON data and displays it on the screen in a user-friendly format.

[0447] Step 8:

[0448] The device collects user emotional data.

[0449] Input: Real-time user emotion data (facial expression analysis, voice analysis, etc.)

[0450] Output: Analyzed emotional state data

[0451] Specific actions:

[0452] The device uses its built-in camera and microphone to collect emotional data in real time, which is then analyzed by an emotional analysis algorithm. The analysis results are stored in memory as digital data.

[0453] Step 9:

[0454] The device sends the analyzed emotion data to the server.

[0455] Input: Analyzed emotional state data

[0456] Output: Sent to the server as an HTTP POST request

[0457] Specific actions:

[0458] The analyzed emotional state data is converted into JSON format and sent to the server as an HTTP POST request.

[0459] Step 10:

[0460] The server optimizes and regenerates travel plans based on emotional data.

[0461] Input: User's emotional state data, initial travel plan

[0462] Output: Optimized travel plan

[0463] Specific actions:

[0464] The plan is adjusted by running an optimization algorithm based on the user's emotional state. For example, if the user is "tired," a rest spot is added; if they are "excited," an active spot is added. The optimized plan information is then reconstructed in JSON format.

[0465] Step 11:

[0466] The server then sends the optimized travel plan back to the device and presents it to the user.

[0467] Input: Optimized travel plan

[0468] Output: Display of the adjusted travel plan

[0469] Specific actions:

[0470] The optimized travel plan information is sent to the device in JSON format, and the new plan is displayed on the device.

[0471] Step 12:

[0472] If the user is satisfied with the optimized plan, they will confirm their reservation.

[0473] Input: User's reservation confirmation information

[0474] Output: Booking confirmation information (ticket booking number, accommodation confirmation number, etc.)

[0475] Specific actions:

[0476] When a user presses the "Confirm Reservation" button, their device sends that information to the server. The server then processes the reservations for transportation, accommodation, tourist attractions, and activities all at once and retrieves the reservation completion information.

[0477] Step 13:

[0478] The server sends reservation completion information to the terminal and notifies the user.

[0479] Input: Reservation confirmation information

[0480] Output: Display of confirmation information

[0481] Specific actions:

[0482] The acquired reservation confirmation information is sent to the device in JSON format, and the user's device displays the confirmation information. This finalizes the travel plan.

[0483] (Application Example 2)

[0484] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0485] In recent years, the use of autonomous vehicles has been rapidly increasing, but travelers are increasingly expecting real-time plan changes and optimizations based on their emotional state. However, conventional travel plan generation systems have been unable to take into account the user's emotional state, resulting in an insufficient user experience. Furthermore, the lack of real-time plan changes and optimizations has led to a decrease in convenience during travel.

[0486] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving travel destination information entered by the user, means for selecting a means of transportation based on the received information, means for presenting the selected means of transportation information to the user, means for searching for accommodation based on the travel destination information entered by the user, means for presenting the searched accommodation information to the user, means for generating a list of tourist spots and activities based on the user's preferences, means for presenting the generated list of tourist spots and activities to the user, means for receiving the user's request for changes and re-optimizing the plan, means for presenting the optimized travel plan to the user, means for receiving final reservation confirmation information and confirming reservations for transportation, accommodation, tourist spots, and activities, means for notifying the user of the reservation confirmation information, and means for collecting and analyzing the user's emotional data using the equipment of an autonomous vehicle and re-optimizing the travel plan in real time based on the emotional state. As a result, the user is continuously provided with a plan that is optimal for their state even while traveling, improving the quality of their experience.

[0487] "User-entered travel destination information" refers to data such as the destination, travel dates, budget, and purpose specified by the user planning the trip.

[0488] "Method of selecting transportation" refers to the process of selecting the most suitable mode of transport (air, train, bus, etc.) based on the entered travel destination information.

[0489] "Methods for searching for accommodations" refers to the process of finding potential places to stay based on the entered travel destination information.

[0490] "Methods for generating lists of tourist attractions and activities" refers to the process of listing places of interest and experiential activities based on the user's preferences.

[0491] "User emotion data" refers to data related to emotions that is collected using cameras and microphones and analyzed from the user's facial expressions and voice.

[0492] "Real-time re-optimization" refers to a process that instantly re-optimizes the travel plan based on the user's current emotional state and new input data.

[0493] An "infotainment system" is an information and entertainment system installed inside an autonomous vehicle that allows the user to input and confirm information.

[0494] "Confirmed booking information" refers to the detailed booking information for transportation, accommodation, and sightseeing / activity that has been finalized in the travel plan.

[0495] An "emotion engine" is an algorithm or system that analyzes a user's facial expressions and voice to determine their emotional state.

[0496] "Means for receiving change requests" refers to the process by which the system receives information about changes and additions that users wish to make to their existing travel plans.

[0497] This invention is a system that receives travel destination information entered by a user within an autonomous vehicle, generates an optimal travel plan based on that information, and further optimizes the plan in real time based on the user's emotional state using an emotion engine. The system includes the following means:

[0498] Program generation

[0499] Hardware and software to use

[0500] Hardware: Infotainment system, cameras, and microphones for autonomous vehicles.

[0501] software:

[0502] Language: Python

[0503] Frameworks: Flask (web application frontend), OpenCV (for facial expression analysis), NLP library (for speech analysis), TENSORFLOW® (emotion engine)

[0504] Explanation in natural language

[0505] The basic configuration of the system is as follows: First, the user enters travel information through the infotainment system of the autonomous vehicle. This includes the destination, travel itinerary, budget, and purpose of the trip. The entered information is collected as digital data and transmitted to the server.

[0506] Data collection and transmission

[0507] The server selects transportation methods based on the received information. It searches a database of transportation options such as flights, trains, and buses to find the best choice and selects the most suitable mode of transport based on the user's budget and preferences. It also searches for accommodations near the travel destination and selects facilities that are within the budget and meet the user's preferences. Furthermore, it generates a list of tourist attractions and activities based on the user's preferences.

[0508] Plan generation and display

[0509] The generated initial travel plan is sent from the server to the vehicle's infotainment system and displayed to the user. The user can review the details of the presented plan and enter any desired changes or additions through the infotainment system. The infotainment system then sends these requested changes to the server.

[0510] Emotional Engine

[0511] This system incorporates an emotion engine that collects and analyzes user emotion data (facial expression analysis and voice analysis). Specifically, facial expression data is acquired in real time using cameras mounted on autonomous vehicles and analyzed using OpenCV. Voice data is also collected using microphones and analyzed with an NLP library. TensorFlow is used to implement the emotion engine.

[0512] Real-time optimization

[0513] The emotion engine monitors the user's emotional state (joy, surprise, fatigue, etc.) in real time. Based on this, the server optimizes the travel plan. For example, if the system determines the user is "tired," it adds rest spots and relaxing activities to the plan. If the system determines the user is "excited," it suggests more active spots and exciting activities.

[0514] Reservation Confirmation

[0515] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button to finalize their travel plans. The device sends this information to the server, which then makes all the necessary bookings for transportation, accommodation, and activities. Once the booking is complete, the server sends confirmation information (ticket booking number and accommodation confirmation number) to the device and notifies the user.

[0516] Specific example

[0517] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, "100,000 yen" as their budget, and "history and gourmet food" as their preferences, the server might generate a plan like the following.

[0518] Transportation: Shinkansen (bullet train) ticket from a major city to your destination.

[0519] Accommodation: Standard class hotels in the city center area

[0520] Tourist attractions: Historical sites, historical temples

[0521] Foodie spots: Famous local eateries

[0522] Additionally, if the emotion engine detects that the user is "tired," the server will add a rest cafe or a relaxing hot spring facility to the plan. An example of a prompt message is as follows:

[0523] "We're starting the Kyoto travel plan. Please consider the user's needs and add the most suitable relaxation points."

[0524] In this way, users can continuously receive plans that are best suited to their needs throughout their trip, thereby improving the quality of their experience.

[0525] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0526] Step 1:

[0527] The user inputs travel destination information through the infotainment system of the autonomous vehicle. The user enters details such as the destination, travel dates, budget, and purpose of the trip. This input data is transmitted digitally from the terminal (the vehicle's infotainment system) to the server, allowing the server to receive the necessary information.

[0528] Input: Travel information (destination, itinerary, budget, purpose of travel, etc.)

[0529] Output: Travel destination information sent to the server

[0530] Step 2:

[0531] The server selects a mode of transportation based on the received travel destination information. It searches a database of transportation options such as flights, trains, and buses, and selects the most suitable mode of transport based on the user's budget and preferences. The selected mode of transport information is then sent from the server to the terminal.

[0532] Input: Travel destination information

[0533] Data processing and computation: Database search and optimization algorithms for transportation methods.

[0534] Output: Selected transportation information

[0535] Step 3:

[0536] The server queries a database of accommodations based on travel destination information and selects facilities that meet the user's budget and preferences. The selected accommodation information is then transmitted from the server to the terminal and presented to the user.

[0537] Input: Travel destination information

[0538] Data processing and calculation: Database querying and selection algorithms for accommodation facilities.

[0539] Output: Selected accommodation information

[0540] Step 4:

[0541] The server generates a list of tourist attractions and activities based on the user's preferences. It retrieves detailed information about tourist attractions and activities from a database and lists those that match the user's preferences. The generated list is sent from the server to the terminal and presented to the user.

[0542] Input: Travel destination information, user preference information

[0543] Data processing and calculation: Database search and filtering of tourist spot and activity information.

[0544] Output: Listed information on tourist attractions and activities

[0545] Step 5:

[0546] Users review the details of the presented travel plan and enter any desired changes or additions through the infotainment system. These requested changes are sent to the server via the terminal, allowing the server to receive user feedback.

[0547] Input: User's request for change

[0548] Output: Requested changes sent to the server

[0549] Step 6:

[0550] The server receives the user's request for changes and optimizes the plan again. It searches for new transportation, accommodation, and sightseeing information as needed and generates an optimized plan. The regenerated plan is sent from the server to the terminal and presented to the user again.

[0551] Input: User's request for change

[0552] Data processing and calculation: Plan re-optimization algorithms, new data search.

[0553] Output: Optimized travel plan

[0554] Step 7:

[0555] The server uses an emotion engine to collect and analyze user emotion data. Cameras and microphones in the autonomous vehicle are used to acquire user facial and voice data in real time, which is then analyzed using OpenCV and NLP libraries. The analysis results are used by an emotion engine based on TensorFlow to understand the emotional state.

[0556] Input: Facial expression data, audio data

[0557] Data processing and calculation: Facial expression analysis (OpenCV), speech analysis (NLP library), emotion analysis (TensorFlow)

[0558] Output: User's emotional state

[0559] Step 8:

[0560] The server re-optimizes the travel plan based on the emotional state analyzed by the emotion engine. For example, if the server determines that the user is "tired," it adds rest spots and relaxing activities to the plan. If the server determines that the user is "excited," it suggests more active spots and exciting activities. The optimized plan is sent from the server to the device and presented to the user.

[0561] Input: User's emotional state

[0562] Data processing and calculation: Plan re-optimization algorithms, sentiment data feedback

[0563] Output: Re-optimized travel plan

[0564] Step 9:

[0565] If the user is satisfied with the optimized plan, they press the "Confirm Reservation" button through the infotainment system. This information is then sent from the device to the server.

[0566] Input: User confirms booking

[0567] Output: Reservation confirmation information sent to the server

[0568] Step 10:

[0569] The server receives the final booking confirmation information and makes all reservations for transportation, accommodation, and sightseeing / activity in one place. Once the reservation is complete, the server sends confirmation information (such as ticket booking numbers and accommodation confirmation numbers) to the user's device and notifies them.

[0570] Input: Booking confirmation information

[0571] Data processing and calculation: Sending data to each reservation system, receiving reservation numbers.

[0572] Output: Confirmation information (ticket reservation number, accommodation confirmation number, etc.)

[0573] This allows users to continuously receive plans that are best suited to their needs throughout their trip, thereby improving the quality of their experience.

[0574] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0575] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0576] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0577] [Second Embodiment]

[0578] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0579] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0580] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0581] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0582] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0583] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0584] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0585] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0586] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0587] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0588] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0589] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0590] This invention is a system that allows users to input travel destination information and handle everything from selecting transportation, accommodations, sightseeing spots, and activities, to making reservations, all in one place. This system can generate and continuously optimize optimal travel plans based on the user's preferences and requirements.

[0591] Entering user information and sending it to the server.

[0592] The user uses a terminal to input travel planning information. Specifically, they input the destination, travel dates, number of nights, budget, and purpose of the trip (e.g., historical site visits or gourmet tours). The terminal collects this information as digital data and sends it to the server.

[0593] Travel plan generation

[0594] When the server receives the above data, it first selects a mode of transportation. The server searches a database of transportation options such as flights, trains, and buses to find the best choice and selects the most suitable mode of transport based on the user's budget and preferences. Next, it searches for accommodations near the travel destination and selects facilities that are within the budget and meet the user's preferences.

[0595] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, it suggests information on historical sites and landmarks for history buffs, and popular local restaurants for foodies. In this way, a comprehensive travel plan is initially generated from multiple perspectives.

[0596] User presentation of the generated plan

[0597] The generated initial travel plan is sent from the server to the terminal and displayed to the user. The user can review the details of the presented plan and enter any desired changes or additions. For example, they can add specific tourist attractions or change accommodations.

[0598] Optimizing travel plans

[0599] When the user enters their requested changes, the device sends that information back to the server. The server then searches and selects again based on the user's requests, optimizing the travel plan. The updated plan is then sent back to the device and presented to the user.

[0600] Reservation Confirmation

[0601] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button to finalize their travel plans. The device sends this information to the server, which then makes all the necessary bookings for transportation, accommodation, and activities. Once the bookings are complete, the server sends confirmation information (such as ticket booking numbers or accommodation confirmation numbers) to the device. The device then notifies the user of this confirmation information, informing them that their travel plans have been finalized.

[0602] Specific example

[0603] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[0604] Transportation: Shinkansen ticket from Tokyo to Kyoto

[0605] Accommodation: 3-star hotels near Gion

[0606] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[0607] Gourmet spot: A famous local Japanese restaurant

[0608] If a user presented with this plan adds "I'd also like to visit Arashiyama," the server recalculates and generates a new plan that includes "Arashiyama." Once the user is satisfied with the optimized plan and confirms their booking, transportation, accommodation, and activities are all booked together.

[0609] As described above, this system allows users to easily plan their trips and flexibly respond to their individual needs.

[0610] The following describes the processing flow.

[0611] Step 1:

[0612] The user enters travel destination information.

[0613] The user enters their travel destination, travel dates, number of nights, budget, and purpose of travel into their device.

[0614] The terminal collects the input information as data packets.

[0615] Step 2:

[0616] The terminal sends the input information to the server.

[0617] The terminal sends the collected data packets to the server.

[0618] The terminal waits for confirmation of receipt from the server.

[0619] Step 3:

[0620] Server receives data

[0621] The server receives data packets sent from the terminal and analyzes their contents.

[0622] The server establishes a database connection.

[0623] Step 4:

[0624] The server selects the mode of transport.

[0625] Based on the received travel destination information, the server consults a database of transportation options to select the most suitable mode of transport.

[0626] The server retrieves detailed information about the selected mode of transport (e.g., flight number, time, price).

[0627] Step 5:

[0628] Server searches for accommodations

[0629] The server references a database of accommodations in the travel destination to search for accommodations that fit the user's budget.

[0630] We obtain real-time information on available rooms and select the most suitable accommodation.

[0631] Step 6:

[0632] The server suggests tourist spots and activities.

[0633] The server searches for tourist attractions and activities in the user's travel destination based on their preferences.

[0634] The server prioritizes the list of selected tourist spots and activities.

[0635] Step 7:

[0636] The server generates an initial travel plan.

[0637] The server generates an initial travel plan that combines information on transportation, accommodation, tourist attractions, and activities.

[0638] Step 8:

[0639] The server sends the generated plan to the terminal.

[0640] The server sends the generated travel plan to the device.

[0641] Step 9:

[0642] The device displays the plan to the user.

[0643] The device displays the user details of the travel plan.

[0644] Users can review the plan details and enter any requested modifications on their device.

[0645] Step 10:

[0646] The user enters their request to change their plan.

[0647] Users enter their requests for plan changes (e.g., adding or changing specific tourist spots).

[0648] Step 11:

[0649] The device sends the change request to the server.

[0650] The device sends the user's change request to the server.

[0651] Step 12:

[0652] The server will regenerate to reflect the requested changes.

[0653] The server regenerates the travel plan based on the user's requested changes.

[0654] Obtain information on new transportation options, accommodations, and tourist attractions as needed.

[0655] Step 13:

[0656] The server sends the optimized plan to the device.

[0657] The server sends an optimized travel plan to the device.

[0658] Step 14:

[0659] The device displays the optimized plan to the user.

[0660] The device displays an optimized travel plan to the user.

[0661] Step 15:

[0662] The user confirms the final plan.

[0663] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button on their device.

[0664] Step 16:

[0665] The terminal sends confirmed information to the server.

[0666] The terminal sends the final reservation confirmation information to the server.

[0667] Step 17:

[0668] The server executes the reservation.

[0669] The server handles reservations for transportation, accommodation, and sightseeing / activity.

[0670] The server obtains confirmation information from each service provider.

[0671] Step 18:

[0672] The server sends reservation confirmation information to the terminal.

[0673] The server sends the confirmed reservation details to the terminal.

[0674] Step 19:

[0675] The device notifies the user that the reservation is complete.

[0676] The device notifies the user that the reservation is complete and displays the necessary information (e.g., reservation number, confirmation documents).

[0677] (Example 1)

[0678] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0679] Traditional travel planning systems required users to individually research and book transportation, accommodations, tourist attractions, and activities, which was a time-consuming and laborious process. Furthermore, the systems lacked sufficient automated suggestions to match users' preferences and budgets, making it difficult to optimize plans as users had to manually select and choose from options.

[0680] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0681] In this invention, the server includes means for receiving travel destination information entered by the user, means for collecting the received information as digital data and transmitting it to the server, means for selecting a mode of transportation based on the received information, means for presenting the selected mode of transportation information to the user, means for searching for accommodation based on the travel destination information entered by the user, means for presenting the searched accommodation information to the user, means for generating a list of tourist spots and activities based on the user's preferences, means for presenting the generated list of tourist spots and activities to the user, means for receiving the user's requests for changes and optimizing the plan again, means for presenting the optimized travel plan to the user, means for receiving final booking confirmation information and confirming the bookings for transportation, accommodation, tourist spots, and activities, and means for notifying the user of the booking confirmation information. This enables the user to plan their trip all at once, optimize that plan, and make reservations based on that plan with confidence.

[0682] A "user" is an individual or group that inputs travel planning information and uses the system.

[0683] A "device" is an electronic device used by a user to input travel planning information and receive notifications such as plan presentations and booking confirmations. Specific examples include smartphones and personal computers.

[0684] A "server" is a central processing unit that receives travel plan information sent by users and uses that information to select and optimize transportation, accommodation, tourist attractions, and activities.

[0685] "Travel destination information" refers to detailed travel information entered by the user, specifically including the travel destination, travel itinerary, number of nights of stay, budget, and purpose of travel.

[0686] "Transportation" refers to the means of getting around for travel, and specific examples include airplanes, bullet trains, railways, and buses.

[0687] "Accommodation facilities" refer to places where users stay during their travels, and specific examples include hotels, inns, and guesthouses.

[0688] A "tourist spot" is a place or landmark that should be visited at a travel destination, and specific examples include historical buildings and natural landscapes.

[0689] An "activity" refers to an activity or experience that a user can have at their travel destination, and specific examples include sightseeing tours and leisure activities.

[0690] "Digital data" refers to the electronic representation of travel plan information entered by a user using a device.

[0691] A "list" is a list containing a series of tourist attractions or activity items generated based on specific criteria.

[0692] A "plan" is the overall outline of a travel plan generated based on the user's input information and preferences.

[0693] "Booking confirmation information" refers to confirmation information regarding reservations for transportation, accommodation, sightseeing spots, and activities that is sent from the server when the user finalizes their travel plans.

[0694] "Validation" is the process of verifying that user input is accurate and complete.

[0695] An "algorithm" is a set of calculations or processing steps to optimize a travel plan again based on the user's requested changes.

[0696] This invention is a system that allows users to input travel destination information and handle everything from selecting transportation, accommodations, sightseeing spots, and activities, to making reservations, all in one go. This system can generate and continuously optimize optimal travel plans based on the user's preferences and circumstances.

[0697] First, the user enters travel plan information using a device, such as a smartphone or computer. The user enters detailed information such as the travel destination, itinerary, number of nights, budget, and purpose of travel. The device collects this information as digital data and sends it to the server. During this process, the device validates the user's input in real time and displays error messages as needed.

[0698] When the server receives user information, it first selects a mode of transportation. The server queries a database of transportation options such as air travel, trains, and buses, and selects the most suitable mode of transport based on the user's budget and preferences. Specifically, it uses a Python library (e.g., pandas) to analyze data and select the appropriate mode of transport. This process utilizes a specific algorithm to efficiently search for transportation options that meet the user's criteria.

[0699] Next, the server selects accommodations. It searches the database for the most suitable accommodations based on the user's budget and requirements. The server uses SQL queries to extract properties that match the criteria and ranks the accommodations considering reviews and ratings. Finally, the most suitable property is selected and presented to the user.

[0700] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, if the user is a history buff, the server will list spots related to historical landmarks and ruins. This process uses a specific search engine (e.g., Elasticsearch) to quickly find relevant data and generate it as a list.

[0701] The generated initial travel plan is sent to the device and presented to the user. The user reviews the proposed plan through the device's interface and enters any desired changes. For example, they can add specific tourist attractions or change accommodations.

[0702] When a user submits a request for changes, the server searches and selects the most suitable plan again, optimizing the travel plan. This process can utilize a generative AI model to provide the user with the most appropriate plan. The recalculated plan is then sent back to the device and presented to the user.

[0703] If the user is satisfied with the optimized plan, they can confirm their travel plans by pressing the "Confirm Booking" button. The device sends this information to the server, which confirms the bookings for transportation, accommodation, sightseeing, and activities. Once the booking is complete, the server generates confirmation information (e.g., ticket booking number or accommodation confirmation number) and sends it to the device. The device notifies the user of this confirmation information, informing them that their travel plans have been confirmed.

[0704] Specific example

[0705] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[0706] Transportation: Shinkansen ticket from Tokyo to Kyoto

[0707] Accommodation: 3-star hotels near Gion

[0708] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[0709] Gourmet spot: A famous local Japanese restaurant

[0710] If a user presented with this plan adds "I'd also like to visit Arashiyama," the server recalculates and generates a new plan that includes "Arashiyama." Once the user is satisfied with the optimized plan and confirms their booking, transportation, accommodation, and activities are all booked together.

[0711] This allows the system to easily plan trips and flexibly accommodate the individual needs of each user.

[0712] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0713] The processing flow of this system's program

[0714] Step 1:

[0715] The user enters travel planning information using a device. Specifically, the user enters the destination, travel dates, number of nights, budget, and purpose of travel on the interface screen. The device collects the information entered by the user as digital data. It also validates the input data, displays error messages, and prompts the user to make corrections as needed. The input data is saved in a structured format (e.g., JSON) and sent to the server.

[0716] Step 2:

[0717] The server receives travel plan information sent from the terminal. The server stores this information in a database. Specifically, it uses a relational database management system (RDBMS) such as MySQL to organize and store the information appropriately in each table. The received data includes travel destination, travel itinerary, number of nights of stay, budget, and purpose of travel.

[0718] Step 3:

[0719] The server selects transportation options based on the user's travel plan information. The server queries a database of transportation options to find the best option based on the user's budget and preferences. Specifically, it uses Python libraries (e.g., pandas) to analyze the data and suggest appropriate options. The output includes detailed information on the best transportation option (e.g., air travel, bullet train, bus, etc.).

[0720] Step 4:

[0721] The server presents the user with information on the selected mode of transport. The selected transport is then sent to the user's terminal and displayed. The user can review the information and proceed to the next step. The output includes detailed information about the transport option displayed on the screen.

[0722] Step 5:

[0723] The server searches for accommodations based on the user's travel plan information. The server queries a database of accommodations to find properties that meet the user's budget and preferences. Specifically, it uses SQL queries to extract properties that match the criteria and selects the best accommodation based on reviews and ratings. The output includes detailed information about the best-suited accommodation.

[0724] Step 6:

[0725] The server presents the user with searched accommodation information. The selected accommodations are sent to the terminal and displayed to the user. The user can review the presented accommodation information and proceed to the next step. The output displays detailed information about the accommodation on the screen.

[0726] Step 7:

[0727] The server generates a list of tourist attractions and activities based on the user's preferences. Specifically, it lists suggested tourist attractions and activities based on the travel purpose and preferences entered by the user. For example, it might suggest historical sites to a history enthusiast and famous local restaurants to a foodie. The output is a list of tourist attractions and activities.

[0728] Step 8:

[0729] The server generates a list of tourist attractions and activities and presents it to the user. The list is sent to the user's device and displayed. The user can review the list and enter any desired changes or additions. The output is a list of tourist attractions and activities displayed on the screen.

[0730] Step 9:

[0731] The system receives user requests for changes and resends them to the server. When a user enters corrections or additions, the terminal sends that information to the server and requests a recalculation. Specific input data includes tourist spots to be added and accommodation information to be changed.

[0732] Step 10:

[0733] The server re-optimizes the travel plan based on the user's requested changes. It queries the databases of transportation, accommodation, and attractions again using the new information, generating the optimal plan based on the latest conditions. The output includes the updated travel plan.

[0734] Step 11:

[0735] The optimized travel plan is sent to the user's device and presented to them. The recalculated plan is sent to the device for the user to review and display. The user can review the optimized plan and proceed to confirm their booking. The output displays detailed information about the optimized travel plan on the screen.

[0736] Step 12:

[0737] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button. The device sends this information to the server, which then confirms each booking. The specific input data includes the final booking confirmation information.

[0738] Step 13:

[0739] The server confirms reservations for transportation, accommodation, and tourist attractions / activities. It updates the relevant database based on each reservation to verify its success. The output includes confirmation information (e.g., ticket number and accommodation confirmation number).

[0740] Step 14:

[0741] The server sends confirmation information to the terminal and notifies the user. Final confirmation information is sent to the terminal and notified the user. The user receives the confirmation information and knows that their travel plan is confirmed. The output is the confirmation information displayed on the user's terminal.

[0742] The above outlines the specific processing steps of this system's program. This process allows users to easily plan their trips, obtain the optimal plan, and make secure reservations. A key feature of this system is its ability to automatically generate and provide optimized travel plans based on user input.

[0743] (Application Example 1)

[0744] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0745] The process of planning a trip is complex due to the many factors to consider, and it is not easy to create a plan that perfectly matches the user's desires in one go. Furthermore, users need to individually search for and book transportation, accommodations, attractions, and activities, which is also time-consuming. Additionally, when users plan and book trips in a virtual space, the lack of a suitable interface prevents them from providing the same experience as using a real travel agency.

[0746] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0747] In this invention, the server includes means for receiving travel destination information entered by the user, means for selecting a mode of transportation based on the received information, and means for providing an interface for the user to plan and book a trip in a virtual space. This allows the user to search and book transportation, accommodation, sightseeing spots, and activities all at once, and to intuitively plan, modify, and finalize a trip through an interactive experience in a virtual space.

[0748] "User" refers to the person who plans the trip.

[0749] "Travel destination information" refers to information about places that users want to visit.

[0750] "Transportation" refers to the means that users use to travel to their destination.

[0751] "Accommodation" refers to the place where a user stays during their trip.

[0752] "Tourist spots" refer to points of interest and tourist attractions at the destination the user visits.

[0753] "Activities" refer to the activities and events that users undertake while traveling.

[0754] "Preferences" refers to information about a user's interests and preferences.

[0755] "Virtual space" refers to a computer-generated three-dimensional or two-dimensional environment that users can experience through sight and sound.

[0756] An "interface" refers to the means or methods by which a user interacts with a system.

[0757] A "generative AI model" refers to an algorithm that uses artificial intelligence to create travel plans from data.

[0758] A "prompt statement" refers to an input statement used to give instructions to a generative AI model.

[0759] "Booking confirmation information" refers to information indicating that the travel plan has been finalized and all bookings have been completed.

[0760] This invention provides a travel planning system that allows users to intuitively plan, modify, and finalize travel plans in a virtual space. This system utilizes servers, terminals, and a generative AI model to streamline the entire process from travel planning to booking, providing users with an interactive experience.

[0761] Hardware and software

[0762] The hardware includes user terminals (smartphones, head-mounted displays) and servers. The software includes generative AI models (OpenAI APIs) and travel planning system programs.

[0763] Data processing and data calculation

[0764] 1. User Input Collection: Users input information such as travel destination, itinerary, budget, and preferences from their devices. This information is sent to the server, which processes the received data as digital data.

[0765] 2. Plan Generation: The server generates the optimal travel plan using a generation AI model based on travel destination information, dates, budget, and preferences. The following prompts are used during the generation process.

[0766] Travel destination: Kyoto

[0767] Duration: 4 days from December 1, 2023

[0768] Budget: 100,000 yen

[0769] Interests: History and gourmet food

[0770] Please generate the optimal travel plan.

[0771] The generated plan will include transportation, accommodation, sightseeing spots, and activities.

[0772] 3. Presentation and modification of the plan: The initial travel plan is sent from the server to the terminal and presented to the user. If the user enters a request for changes, that information is sent back to the server, which uses a generative AI model to further optimize the plan.

[0773] 4. Virtual Experience: The device provides users with a virtual interface, allowing them to intuitively plan, modify, and finalize their travel plans. This interface provides users with an interactive experience that feels as if they are visiting a real travel agency.

[0774] 5. Booking Confirmation and Notification: If the user is satisfied with the final travel plan, they press the "Confirm Booking" button to confirm the booking. The server makes all bookings for transportation, accommodation, and sightseeing / activity attractions at once and sends the booking confirmation information to the device. The device then notifies the user of this confirmation information.

[0775] Specific example

[0776] For example, if a user selects "Kyoto" as their travel destination in a virtual space, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[0777] Transportation: Shinkansen ticket from Tokyo to Kyoto

[0778] Accommodation: 3-star hotels near tourist attractions

[0779] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[0780] Gourmet spot: A famous local Japanese restaurant

[0781] If a user requests to visit Arashiyama, the server recalculates and generates a new plan that includes Arashiyama. Once the user is satisfied with the optimized plan and confirms their booking, transportation, accommodation, and activities are all booked together.

[0782] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0783] Step 1:

[0784] The user enters travel destination information. The user uses a terminal to enter travel plan information (destination, itinerary, number of nights, budget, purpose of travel, preferences). The terminal collects this information as digital data and sends it to the server. Input: Travel plan information. Output: Digital data transmission to the server.

[0785] Step 2:

[0786] The server selects a mode of transportation based on the information it receives. The server chooses the most suitable mode of transportation from a database of options such as air travel, trains, and buses, based on the user's budget and preferences. The selection is made using a generated AI model based on prompt messages. Input: Travel plan information. Output: Selected mode of transportation information.

[0787] Step 3:

[0788] The server transmits the selected transportation information to the terminal and presents it to the user. The user can then view the details of the presented transportation. Input: Selected transportation information. Output: Presentation of transportation information to the user.

[0789] Step 4:

[0790] The server searches for accommodations based on the travel destination information entered by the user. It selects and presents accommodations within the user's budget and preferences from a database of accommodations near the travel destination. The search is performed using a generated AI model based on prompt messages. Input: Travel plan information. Output: Searched accommodation information.

[0791] Step 5:

[0792] The server displays the searched accommodation information to the user. The user can view the accommodation details through their device. Input: Searched accommodation information. Output: Presentation of accommodation information to the user.

[0793] Step 6:

[0794] The server generates a list of tourist attractions and activities based on the user's preferences. It creates the most suitable list from a database of tourist attractions and activities related to those preferences. Input: Travel plan information and preference information. Output: Generated list of tourist attractions and activities.

[0795] Step 7:

[0796] The server presents the user with a list of generated tourist spots and activities. The user can view the list through their device. Input: List of generated tourist spots and activities. Output: Presentation of the list to the user.

[0797] Step 8:

[0798] The server receives the user's change requests and optimizes the plan again. The user enters their change requests via their device and sends them to the server. The server uses a generated AI model to optimize the plan again. Input: Change request information. Output: Optimized travel plan.

[0799] Step 9:

[0800] The server presents the user with an optimized travel plan. The user can view the details of the optimized plan through their device. Input: Optimized travel plan. Output: Presentation of the optimized plan to the user.

[0801] Step 10:

[0802] The server receives the final booking confirmation information and confirms reservations for transportation, accommodation, and sightseeing / activity. If the user is satisfied with the optimized plan, they press the "Confirm Booking" button and send the information to the server. The server receives this and makes the bookings in bulk. Input: Final booking confirmation information. Output: Generation of booking confirmation information.

[0803] Step 11:

[0804] The server notifies the user of the reservation confirmation information. The user can receive the confirmation information via their device. Input: Reservation confirmation information. Output: Notification to the user.

[0805] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0806] This invention is a system in which a user inputs travel destination information, the system automatically generates an optimal travel plan, and further optimizes the plan based on the user's emotions using an emotion engine.

[0807] Entering user information and sending it to the server.

[0808] Users input travel planning information using a device. Specifically, they enter information such as the destination, travel dates, number of nights, budget, and purpose of travel. The device collects this information as digital data and sends it to the server.

[0809] Travel plan generation

[0810] When the server receives the above data, it first selects a mode of transportation. The server searches its database of transportation options, such as flights, trains, and buses, and selects the most suitable mode of transport based on the user's budget and preferences. Next, it searches for accommodations near the travel destination and selects facilities that are within the budget and meet the user's preferences.

[0811] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, it suggests information on historical sites and landmarks for history buffs, and popular local restaurants for foodies. In this way, a comprehensive travel plan is initially generated from multiple perspectives.

[0812] User presentation of the generated plan

[0813] The generated initial travel plan is sent from the server to the terminal and displayed to the user. The user can review the details of the presented plan and enter any desired changes or additions. For example, they can add specific tourist attractions or change accommodations.

[0814] Using an Emotion Engine

[0815] This system incorporates an emotion engine that collects and analyzes user emotional data (e.g., facial expression analysis and voice analysis). The emotion engine understands the user's emotional state (e.g., joy, surprise, fatigue) in real time and optimizes the travel plan based on the results.

[0816] For example, if the emotion engine determines that the user is "tired," the server can add rest spots or relaxing activities to the plan. Similarly, if the engine determines that the user is "excited," it can suggest more active spots or exciting activities.

[0817] Optimizing and regenerating travel plans

[0818] When a user enters a change request, the device sends it to the server, along with information from the sentiment engine. The server uses this information to optimize the plan again. If necessary, it searches for new transportation, accommodation, and sightseeing information and generates an optimized plan.

[0819] Reservation Confirmation

[0820] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button to finalize their travel plans. The device sends this information to the server, which then makes all the necessary bookings for transportation, accommodation, and activities. Once the bookings are complete, the server sends confirmation information (such as ticket booking numbers or accommodation confirmation numbers) to the device. The device then notifies the user of this confirmation information, informing them that their travel plans have been finalized.

[0821] Specific example

[0822] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[0823] Transportation: Shinkansen ticket from Tokyo to Kyoto

[0824] Accommodation: 3-star hotels near Gion

[0825] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[0826] Gourmet spot: A famous local Japanese restaurant

[0827] If the emotion engine detects that the user is "tired," the server can add a rest cafe or a relaxing hot spring facility to the plan. In this way, a plan adapted to the user's state can be provided.

[0828] As described above, this system allows users to easily create travel plans that are perfectly tailored to their circumstances and preferences. The introduction of an emotion engine enables the provision of more personalized services and enhances user satisfaction.

[0829] The following describes the processing flow.

[0830] Step 1:

[0831] The user enters travel destination information.

[0832] Users use their device to enter their travel destination, itinerary, number of nights, budget, and purpose of travel.

[0833] The terminal collects the input information as data packets.

[0834] Step 2:

[0835] The terminal sends the input information to the server.

[0836] The terminal sends the collected data packets to the server.

[0837] The terminal monitors the transmission status and waits for confirmation of receipt from the server.

[0838] Step 3:

[0839] Server receives data

[0840] The server receives data packets sent from the terminal and analyzes their contents.

[0841] Establish a connection to the required database.

[0842] Step 4:

[0843] The server selects the mode of transport.

[0844] Based on the received travel destination information, the server selects the most suitable mode of transportation from its database of transportation options.

[0845] Obtain details of the selected mode of transport (e.g., flight number, time, price).

[0846] Step 5:

[0847] Server searches for accommodations

[0848] The server uses a database to search for accommodations near the travel destination.

[0849] We select facilities that match the user's budget and preferences, and obtain information on available rooms.

[0850] Step 6:

[0851] The server suggests tourist spots and activities.

[0852] The server analyzes the user's preferences and searches for tourist attractions and activities in their travel destination.

[0853] Generate a list of selected spots and activities.

[0854] Step 7:

[0855] The server generates an initial travel plan.

[0856] The server integrates selected transportation, accommodation, tourist attractions, and activities to generate an initial travel plan.

[0857] Step 8:

[0858] The server sends the generated plan to the terminal.

[0859] The server sends the generated travel plan to the device.

[0860] Step 9:

[0861] The device displays the plan to the user.

[0862] The device displays the travel plan details to the user.

[0863] Users review the plan details and enter any requested modifications.

[0864] Step 10:

[0865] The emotion engine recognizes the user's emotions.

[0866] The emotion engine uses sensor data from the device (e.g., camera and microphone) to analyze the user's emotions.

[0867] The detected emotion data is sent to the server.

[0868] Step 11:

[0869] The user enters their request to change their plan.

[0870] Users enter their desired changes to the plan (e.g., adding or changing specific tourist spots) into the device.

[0871] Step 12:

[0872] The device sends change requests and sentiment data to the server.

[0873] The device sends user change requests and sentiment data to the server.

[0874] Step 13:

[0875] The server will regenerate to reflect requested changes and sentiment data.

[0876] The server regenerates the travel plan based on the user's requested changes and sentiment data.

[0877] Obtain information on new transportation options, accommodations, and tourist attractions as needed.

[0878] Step 14:

[0879] The server sends the optimized plan to the device.

[0880] The server sends an optimized travel plan to the device.

[0881] Step 15:

[0882] The device displays the optimized plan to the user.

[0883] The device displays an optimized travel plan to the user.

[0884] Step 16:

[0885] The user confirms the final plan.

[0886] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button on their device.

[0887] Step 17:

[0888] The terminal sends confirmed information to the server.

[0889] The terminal sends the final reservation confirmation information to the server.

[0890] Step 18:

[0891] The server executes the reservation.

[0892] The server confirms reservations for transportation, accommodation, and sightseeing / activity.

[0893] The server obtains confirmation information from each service provider.

[0894] Step 19:

[0895] The server sends reservation confirmation information to the terminal.

[0896] The server sends the confirmed reservation details to the terminal.

[0897] Step 20:

[0898] The device notifies the user that the reservation is complete.

[0899] The device notifies the user that the reservation is complete and displays the necessary information (e.g., reservation number, confirmation documents).

[0900] (Example 2)

[0901] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0902] Conventional travel plan generation systems only generate basic travel plans based on user input, making it difficult to provide optimal plans that reflect the user's emotions and individual preferences. Furthermore, it is difficult to quickly reflect user requests for changes to the travel plan. In addition, the lack of a function to dynamically optimize the plan based on the user's emotions prevents increasing user satisfaction during the trip. To solve these problems, a system is needed that analyzes the user's emotional state in real time and optimizes and presents travel plans based on that analysis.

[0903] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0904] In this invention, the server includes means for receiving travel destination information entered by the user, means for selecting a mode of transportation based on the received information, means for presenting the mode of transportation information to the user, means for searching for accommodation based on the travel destination information entered by the user, means for presenting the accommodation information to the user, means for generating a list of tourist spots and activities based on the user's preferences, means for presenting the generated list of tourist spots and activities to the user, means for collecting and analyzing the user's emotional data, means for optimizing the travel plan based on the emotional data, means for receiving the user's request for changes and optimizing the plan again, means for presenting the optimized travel plan to the user, means for receiving final booking confirmation information and confirming the booking of transportation, accommodation, tourist spots / activities, and the booking confirmation information to the user. This enables the generation of an optimal travel plan based on the user's emotions and preferences, and real-time optimization of the plan in response to the user's request for changes.

[0905] A "user" is someone who uses this system to input travel planning information and generate and optimize travel plans.

[0906] A "terminal" is a device (such as a personal computer, smartphone, or tablet) that a user uses to input information.

[0907] A "server" is a central processing unit that receives, analyzes, optimizes, and transmits information sent by users.

[0908] "Travel destination information" refers to data entered by the user, such as the travel destination, travel itinerary, number of nights, budget, and purpose of travel.

[0909] "Transportation" refers to the means of getting around during a trip (air, train, bus, etc.).

[0910] "Accommodation facilities" refer to places of stay during travel (such as hotels and inns).

[0911] A "tourist spot" refers to famous places or attractions that users can visit at their travel destination.

[0912] "Activities" refer to activities that can be participated in during the trip (such as sports, cultural experiences, and relaxation).

[0913] "Preferences" refer to a user's personal likes and interests (such as history, food, nature, etc.).

[0914] "Emotional data" refers to data that indicates the user's emotional state (obtained through methods such as facial expression analysis and voice analysis).

[0915] A "travel plan" is a comprehensive travel plan that includes selected transportation, accommodation, sightseeing spots, activities, and other details.

[0916] "Optimization" refers to adjusting travel plans to best suit the user based on their input information and emotional data.

[0917] "Booking confirmation information" refers to the final booking confirmation data for transportation, accommodation, sightseeing spots, and activities.

[0918] This invention is a system in which a user inputs travel destination information, the system automatically generates an optimal travel plan, and further optimizes the plan based on the user's emotions using an emotion engine.

[0919] Entering user information and sending it to the server.

[0920] Users input travel planning information using their devices. Specifically, they enter information such as the destination, travel dates, number of nights, budget, and purpose of travel. The device collects this information as digital data and sends it to the server. Devices such as personal computers and smartphones are used.

[0921] Travel plan generation

[0922] When the server receives data sent by the user, it first selects a mode of transportation. The server searches a database of transportation options such as flights, trains, and buses to find the best choice and selects the most suitable mode of transport based on the user's budget and preferences. Next, it searches for accommodations near the travel destination and selects facilities that are within the budget and meet the user's preferences.

[0923] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, it suggests information on historical sites and landmarks for history buffs, and popular local restaurants for foodies. In this way, a comprehensive travel plan is initially generated from multiple perspectives.

[0924] User presentation of the generated plan

[0925] The generated initial travel plan is sent from the server to the terminal and displayed to the user. The user can review the details of the presented plan and enter any desired changes or additions. For example, they can add specific tourist attractions or change accommodations.

[0926] Using an Emotion Engine

[0927] This system incorporates an emotion engine. It collects and analyzes user emotional data (e.g., facial expression analysis and voice analysis). The emotion engine understands the user's emotional state (e.g., joy, surprise, fatigue) in real time and optimizes the travel plan based on the results.

[0928] For example, if the emotion engine determines that the user is "tired," the server will add rest spots and relaxing activities to the plan. Similarly, if the user is determined to be "excited," it will suggest more active spots and exciting activities.

[0929] Optimizing and regenerating travel plans

[0930] When a user enters a change request, the device sends it to the server, along with information from the sentiment engine. The server uses this information to optimize the plan again, searching for new transportation options, accommodations, and tourist attractions, and generating an optimized plan.

[0931] Reservation Confirmation

[0932] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button to finalize their travel plans. The device sends this information to the server, which then makes all the necessary bookings for transportation, accommodation, and activities. Once the bookings are complete, the server sends confirmation information (such as ticket booking numbers or accommodation confirmation numbers) to the device. The device then notifies the user of this confirmation information, informing them that their travel plans have been finalized.

[0933] Specific example

[0934] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[0935] Transportation: Shinkansen ticket from Tokyo to Kyoto

[0936] Accommodation: 3-star hotels near Gion

[0937] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[0938] Gourmet spot: A famous local Japanese restaurant

[0939] If the emotion engine detects that the user is "tired," the server can add a rest cafe or a relaxing hot spring facility to the plan. In this way, a plan adapted to the user's state can be provided.

[0940] Example of a prompt

[0941] Please automatically generate a travel plan for "Kyoto, 3 nights / 4 days from December 1st, 2023, 100,000 yen, focusing on history and gourmet food." Please also consider the emotional data "tired" and add relaxing activities.

[0942] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0943] Step 1:

[0944] Users enter travel plan information using their devices. This includes destination, travel dates, number of nights, budget, and preferences (history, food, etc.).

[0945] Input: User's travel plan information (destination, dates, number of nights, budget, preferences)

[0946] Output: Saved to the device as JSON data.

[0947] Specific actions:

[0948] When a user enters information into a form and presses the "Submit" button, the entered information is saved to the device as digital data in JSON format. At this point, a list of the entered information is displayed on the interface.

[0949] Step 2:

[0950] The terminal sends the travel plan information entered by the user to the server.

[0951] Input: JSON data of travel plan information saved on the device

[0952] Output: Sent to the server as an HTTP POST request

[0953] Specific actions:

[0954] The JSON data saved on the device is sent as an HTTP POST request, and when the send button is pressed, a "Sending data" message is displayed. Once the data has been sent, a notification of successful transmission is displayed.

[0955] Step 3:

[0956] The server receives travel plan information sent from the terminal and stores it in the database.

[0957] Input: JSON data sent as an HTTP request

[0958] Output: Travel plan information stored in the database

[0959] Specific actions:

[0960] The server receives the HTTP request and saves the data to its database. At this time, it returns a status code 200 (OK) as confirmation of receipt.

[0961] Step 4:

[0962] The server selects the mode of transportation based on the information it receives.

[0963] Input: Travel plan information stored in the database

[0964] Output: List of optimal modes of transport

[0965] Specific actions:

[0966] A program on the server executes an SQL query to search for transportation options (air, rail, bus, etc.) that match the specified criteria. The results are stored in memory as a list of candidates.

[0967] Step 5:

[0968] The server searches for accommodations that match the user's budget and preferences based on the selected mode of transportation.

[0969] Input: List of optimal transportation options, user's budget and preferences

[0970] Output: List of optimal accommodations

[0971] Specific actions:

[0972] A similar SQL query is executed from the accommodation database to search for accommodations that match the criteria. The results are added as a list of candidates.

[0973] Step 6:

[0974] The server generates a list of tourist spots and activities based on the user's preferences.

[0975] Input: User preferences (history, food, etc.)

[0976] Output: List of tourist attractions and activities

[0977] Specific actions:

[0978] The system filters and lists tourist spots and activities based on preference data. It extracts necessary data from the database and generates the list.

[0979] Step 7:

[0980] The server sends the generated travel plan to the terminal and displays it to the user.

[0981] Input: Information on the best transportation options, accommodations, attractions, and activities.

[0982] Output: Initial travel plan

[0983] Specific actions:

[0984] The generated travel plan information is constructed in JSON format and returned to the device as an HTTP response. The device then parses the received JSON data and displays it on the screen in a user-friendly format.

[0985] Step 8:

[0986] The device collects user emotional data.

[0987] Input: Real-time user emotion data (facial expression analysis, voice analysis, etc.)

[0988] Output: Analyzed emotional state data

[0989] Specific actions:

[0990] The device uses its built-in camera and microphone to collect emotional data in real time, which is then analyzed by an emotion analysis algorithm. The analysis results are stored in memory as digital data.

[0991] Step 9:

[0992] The device sends the analyzed emotion data to the server.

[0993] Input: Analyzed emotional state data

[0994] Output: Sent to the server as an HTTP POST request

[0995] Specific actions:

[0996] The analyzed emotional state data is converted into JSON format and sent to the server as an HTTP POST request.

[0997] Step 10:

[0998] The server optimizes and regenerates travel plans based on emotional data.

[0999] Input: User's emotional state data, initial travel plan

[1000] Output: Optimized travel plan

[1001] Specific actions:

[1002] The plan is adjusted by running an optimization algorithm based on the user's emotional state. For example, if the user is "tired," a rest spot is added; if they are "excited," an active spot is added. The optimized plan information is then reconstructed in JSON format.

[1003] Step 11:

[1004] The server then sends the optimized travel plan back to the device and presents it to the user.

[1005] Input: Optimized travel plan

[1006] Output: Display of the adjusted travel plan

[1007] Specific actions:

[1008] The optimized travel plan information is sent to the device in JSON format, and the new plan is displayed on the device.

[1009] Step 12:

[1010] If the user is satisfied with the optimized plan, they will confirm their reservation.

[1011] Input: User's reservation confirmation information

[1012] Output: Booking confirmation information (ticket booking number, accommodation confirmation number, etc.)

[1013] Specific actions:

[1014] When a user presses the "Confirm Reservation" button, their device sends that information to the server. The server then processes the reservations for transportation, accommodation, tourist attractions, and activities all at once and retrieves the reservation completion information.

[1015] Step 13:

[1016] The server sends reservation completion information to the terminal and notifies the user.

[1017] Input: Reservation confirmation information

[1018] Output: Display of confirmation information

[1019] Specific actions:

[1020] The acquired reservation confirmation information is sent to the device in JSON format, and the user's device displays the confirmation information. This finalizes the travel plan.

[1021] (Application Example 2)

[1022] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1023] In recent years, the use of autonomous vehicles has been rapidly increasing, but travelers are increasingly expecting real-time plan changes and optimizations based on their emotional state. However, conventional travel plan generation systems have been unable to take into account the user's emotional state, resulting in an insufficient user experience. Furthermore, the lack of real-time plan changes and optimizations has led to a decrease in convenience during travel.

[1024] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving travel destination information entered by the user, means for selecting a means of transportation based on the received information, means for presenting the selected means of transportation information to the user, means for searching for accommodation based on the travel destination information entered by the user, means for presenting the searched accommodation information to the user, means for generating a list of tourist spots and activities based on the user's preferences, means for presenting the generated list of tourist spots and activities to the user, means for receiving the user's request for changes and re-optimizing the plan, means for presenting the optimized travel plan to the user, means for receiving final reservation confirmation information and confirming reservations for transportation, accommodation, tourist spots, and activities, means for notifying the user of the reservation confirmation information, and means for collecting and analyzing the user's emotional data using the equipment of an autonomous vehicle and re-optimizing the travel plan in real time based on the emotional state. As a result, the user is continuously provided with a plan that is optimal for their state even while traveling, improving the quality of their experience.

[1025] "User-entered travel destination information" refers to data such as the destination, travel dates, budget, and purpose specified by the user planning the trip.

[1026] "Method of selecting transportation" refers to the process of selecting the most suitable mode of transport (air, train, bus, etc.) based on the entered travel destination information.

[1027] "Methods for searching for accommodations" refers to the process of finding potential places to stay based on the entered travel destination information.

[1028] "Methods for generating lists of tourist attractions and activities" refers to the process of listing places of interest and activities to visit based on the user's preferences.

[1029] "User emotion data" refers to data related to emotions collected using cameras and microphones, and analyzed from the user's facial expressions and voice.

[1030] "Real-time re-optimization" refers to a process that instantly re-optimizes the travel plan based on the user's current emotional state and new input data.

[1031] An "infotainment system" is an information and entertainment system installed inside an autonomous vehicle that allows the user to input and confirm information.

[1032] "Confirmed booking information" refers to the detailed booking information for transportation, accommodation, and sightseeing / activity that has been finalized in the travel plan.

[1033] An "emotion engine" is an algorithm or system that analyzes a user's facial expressions and voice to determine their emotional state.

[1034] "Means for receiving change requests" refers to the process by which the system receives information about changes and additions that users wish to make to their existing travel plans.

[1035] This invention is a system that receives travel destination information entered by a user within an autonomous vehicle, generates an optimal travel plan based on that information, and further optimizes the plan in real time based on the user's emotional state using an emotion engine. The system includes the following means:

[1036] Program generation

[1037] Hardware and software to use

[1038] Hardware: Infotainment system, cameras, and microphones for autonomous vehicles.

[1039] software:

[1040] Language: Python

[1041] Frameworks: Flask (web application frontend), OpenCV (for facial expression analysis), NLP library (for speech analysis), TensorFlow (emotion engine)

[1042] Explanation in natural language

[1043] The basic configuration of the system is as follows: First, the user enters travel information through the infotainment system of the autonomous vehicle. This includes the destination, travel itinerary, budget, and purpose of the trip. The entered information is collected as digital data and transmitted to the server.

[1044] Data collection and transmission

[1045] The server selects transportation methods based on the received information. It searches a database of transportation options such as flights, trains, and buses to find the best choice and selects the most suitable mode of transport based on the user's budget and preferences. It also searches for accommodations near the travel destination and selects facilities that are within the budget and meet the user's preferences. Furthermore, it generates a list of tourist attractions and activities based on the user's preferences.

[1046] Plan generation and display

[1047] The generated initial travel plan is sent from the server to the vehicle's infotainment system and displayed to the user. The user can review the details of the presented plan and enter any desired changes or additions through the infotainment system. The infotainment system then sends these requested changes to the server.

[1048] Emotional Engine

[1049] This system incorporates an emotion engine that collects and analyzes user emotion data (facial expression analysis and voice analysis). Specifically, facial expression data is acquired in real time using cameras mounted on autonomous vehicles and analyzed using OpenCV. Voice data is also collected using microphones and analyzed with an NLP library. TensorFlow is used to implement the emotion engine.

[1050] Real-time optimization

[1051] The emotion engine monitors the user's emotional state (joy, surprise, fatigue, etc.) in real time. Based on this, the server optimizes the travel plan. For example, if the system determines the user is "tired," it adds rest spots and relaxing activities to the plan. If the system determines the user is "excited," it suggests more active spots and exciting activities.

[1052] Reservation Confirmation

[1053] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button to finalize their travel plans. The device sends this information to the server, which then makes all the necessary bookings for transportation, accommodation, and activities. Once the booking is complete, the server sends confirmation information (ticket booking number and accommodation confirmation number) to the device and notifies the user.

[1054] Specific example

[1055] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[1056] Transportation: Shinkansen (bullet train) ticket from a major city to your destination.

[1057] Accommodation: Standard class hotels in the city center area

[1058] Tourist attractions: Historical sites, historical temples

[1059] Foodie spots: Famous local eateries

[1060] Additionally, if the emotion engine detects that the user is "tired," the server will add a rest cafe or a relaxing hot spring facility to the plan. An example of a prompt message is as follows:

[1061] "We're starting the Kyoto travel plan. Please consider the user's needs and add the most suitable relaxation points."

[1062] In this way, users can continuously receive plans that are best suited to their needs throughout their trip, thereby improving the quality of their experience.

[1063] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1064] Step 1:

[1065] The user inputs travel destination information through the infotainment system of the autonomous vehicle. The user enters information such as the destination, travel dates, budget, and purpose of the trip. The input data is transmitted digitally from the terminal (the vehicle's infotainment system) to the server. This allows the server to receive the necessary information.

[1066] Input: Travel information (destination, itinerary, budget, purpose of travel, etc.)

[1067] Output: Travel destination information sent to the server

[1068] Step 2:

[1069] The server selects a mode of transportation based on the received travel destination information. It searches a database of transportation options such as flights, trains, and buses, and selects the most suitable mode of transport based on the user's budget and preferences. The selected mode of transport information is then sent from the server to the terminal.

[1070] Input: Travel destination information

[1071] Data processing and computation: Database search and optimization algorithms for transportation methods.

[1072] Output: Selected transportation information

[1073] Step 3:

[1074] The server queries a database of accommodations based on travel destination information and selects facilities that meet the user's budget and preferences. The selected accommodation information is then transmitted from the server to the terminal and presented to the user.

[1075] Input: Travel destination information

[1076] Data processing and calculation: Database querying and selection algorithms for accommodation facilities.

[1077] Output: Selected accommodation information

[1078] Step 4:

[1079] The server generates a list of tourist attractions and activities based on the user's preferences. It retrieves detailed information about tourist attractions and activities from a database and lists those that match the user's preferences. The generated list is sent from the server to the terminal and presented to the user.

[1080] Input: Travel destination information, user preference information

[1081] Data processing and calculation: Database search and filtering of tourist spot and activity information.

[1082] Output: Listed information on tourist attractions and activities

[1083] Step 5:

[1084] Users review the details of the presented travel plan and enter any desired changes or additions through the infotainment system. These requested changes are sent to the server via the terminal, allowing the server to receive user feedback.

[1085] Input: User's request for change

[1086] Output: Requested changes sent to the server

[1087] Step 6:

[1088] The server receives the user's request for changes and optimizes the plan again. It searches for new transportation, accommodation, and sightseeing information as needed and generates an optimized plan. The regenerated plan is sent from the server to the terminal and presented to the user again.

[1089] Input: User's request for change

[1090] Data processing and calculation: Plan re-optimization algorithms, new data search.

[1091] Output: Optimized travel plan

[1092] Step 7:

[1093] The server uses an emotion engine to collect and analyze user emotion data. Cameras and microphones in the autonomous vehicle are used to acquire user facial and voice data in real time, which is then analyzed using OpenCV and NLP libraries. The analysis results are then used with an emotion engine based on TensorFlow to understand the emotional state.

[1094] Input: Facial expression data, audio data

[1095] Data processing and calculation: Facial expression analysis (OpenCV), speech analysis (NLP library), emotion analysis (TensorFlow)

[1096] Output: User's emotional state

[1097] Step 8:

[1098] The server re-optimizes the travel plan based on the emotional state analyzed by the emotion engine. For example, if the server determines that the user is "tired," it adds rest spots and relaxing activities to the plan. If the server determines that the user is "excited," it suggests more active spots and exciting activities. The optimized plan is sent from the server to the device and presented to the user.

[1099] Input: User's emotional state

[1100] Data processing and calculation: Plan re-optimization algorithms, sentiment data feedback

[1101] Output: Re-optimized travel plan

[1102] Step 9:

[1103] If the user is satisfied with the optimized plan, they press the "Confirm Reservation" button through the infotainment system. This information is then sent from the device to the server.

[1104] Input: User confirms booking

[1105] Output: Reservation confirmation information sent to the server

[1106] Step 10:

[1107] The server receives the final booking confirmation information and makes all reservations for transportation, accommodation, and sightseeing / activity in one place. Once the reservation is complete, the server sends confirmation information (such as ticket booking numbers and accommodation confirmation numbers) to the user's device and notifies them.

[1108] Input: Booking confirmation information

[1109] Data processing and calculation: Sending data to each reservation system, receiving reservation numbers.

[1110] Output: Confirmation information (ticket reservation number, accommodation confirmation number, etc.)

[1111] This allows users to continuously receive plans that are best suited to their needs throughout their trip, thereby improving the quality of their experience.

[1112] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1113] The data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1114] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1115] [Third Embodiment]

[1116] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1117] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1118] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1119] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1120] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1121] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1122] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1123] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1124] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1125] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1126] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1127] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1128] This invention is a system that allows users to input travel destination information and handle everything from selecting transportation, accommodations, sightseeing spots, and activities, to making reservations, all in one place. This system can generate and continuously optimize optimal travel plans based on the user's preferences and requirements.

[1129] Entering user information and sending it to the server.

[1130] The user uses a terminal to input travel planning information. Specifically, they input the destination, travel dates, number of nights, budget, and purpose of the trip (e.g., historical site visits or gourmet tours). The terminal collects this information as digital data and sends it to the server.

[1131] Travel plan generation

[1132] When the server receives the above data, it first selects a mode of transportation. The server searches a database of transportation options such as flights, trains, and buses to find the best choice and selects the most suitable mode of transport based on the user's budget and preferences. Next, it searches for accommodations near the travel destination and selects facilities that are within the budget and meet the user's preferences.

[1133] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, it suggests information on historical sites and landmarks for history buffs, and popular local restaurants for foodies. In this way, a comprehensive travel plan is initially generated from multiple perspectives.

[1134] User presentation of the generated plan

[1135] The generated initial travel plan is sent from the server to the terminal and displayed to the user. The user can review the details of the presented plan and enter any desired changes or additions. For example, they can add specific tourist attractions or change accommodations.

[1136] Optimizing travel plans

[1137] When the user enters their requested changes, the device sends that information back to the server. The server then searches and selects again based on the user's requests, optimizing the travel plan. The updated plan is then sent back to the device and presented to the user.

[1138] Reservation Confirmation

[1139] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button to finalize their travel plans. The device sends this information to the server, which then makes all the necessary bookings for transportation, accommodation, and activities. Once the bookings are complete, the server sends confirmation information (such as ticket booking numbers or accommodation confirmation numbers) to the device. The device then notifies the user of this confirmation information, informing them that their travel plans have been finalized.

[1140] Specific example

[1141] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[1142] Transportation: Shinkansen ticket from Tokyo to Kyoto

[1143] Accommodation: 3-star hotels near Gion

[1144] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[1145] Gourmet spot: A famous local Japanese restaurant

[1146] If a user presented with this plan adds "I'd also like to visit Arashiyama," the server recalculates and generates a new plan that includes "Arashiyama." Once the user is satisfied with the optimized plan and confirms their booking, transportation, accommodation, and activities are all booked together.

[1147] As described above, this system allows users to easily plan their trips and flexibly respond to their individual needs.

[1148] The following describes the processing flow.

[1149] Step 1:

[1150] The user enters travel destination information.

[1151] The user enters their travel destination, travel dates, number of nights, budget, and purpose of travel into their device.

[1152] The terminal collects the input information as data packets.

[1153] Step 2:

[1154] The terminal sends the input information to the server.

[1155] The terminal sends the collected data packets to the server.

[1156] The terminal waits for confirmation of receipt from the server.

[1157] Step 3:

[1158] Server receives data

[1159] The server receives data packets sent from the terminal and analyzes their contents.

[1160] The server establishes a database connection.

[1161] Step 4:

[1162] The server selects the mode of transport.

[1163] Based on the received travel destination information, the server consults a database of transportation options to select the most suitable mode of transport.

[1164] The server retrieves detailed information about the selected mode of transport (e.g., flight number, time, price).

[1165] Step 5:

[1166] Server searches for accommodations

[1167] The server references a database of accommodations in the travel destination to search for accommodations that fit the user's budget.

[1168] We obtain real-time information on available rooms and select the most suitable accommodation.

[1169] Step 6:

[1170] The server suggests tourist spots and activities.

[1171] The server searches for tourist attractions and activities in the user's travel destination based on their preferences.

[1172] The server prioritizes the list of selected tourist spots and activities.

[1173] Step 7:

[1174] The server generates an initial travel plan.

[1175] The server generates an initial travel plan that combines information on transportation, accommodation, tourist attractions, and activities.

[1176] Step 8:

[1177] The server sends the generated plan to the terminal.

[1178] The server sends the generated travel plan to the device.

[1179] Step 9:

[1180] The device displays the plan to the user.

[1181] The device displays the user details of the travel plan.

[1182] Users can review the plan details and enter any requested modifications on their device.

[1183] Step 10:

[1184] The user enters their request to change their plan.

[1185] Users enter their requests for plan changes (e.g., adding or changing specific tourist spots).

[1186] Step 11:

[1187] The device sends the change request to the server.

[1188] The device sends the user's change request to the server.

[1189] Step 12:

[1190] The server will regenerate to reflect the requested changes.

[1191] The server regenerates the travel plan based on the user's requested changes.

[1192] Obtain information on new transportation options, accommodations, and tourist attractions as needed.

[1193] Step 13:

[1194] The server sends the optimized plan to the device.

[1195] The server sends an optimized travel plan to the device.

[1196] Step 14:

[1197] The device displays the optimized plan to the user.

[1198] The device displays an optimized travel plan to the user.

[1199] Step 15:

[1200] The user confirms the final plan.

[1201] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button on their device.

[1202] Step 16:

[1203] The terminal sends confirmed information to the server.

[1204] The terminal sends the final reservation confirmation information to the server.

[1205] Step 17:

[1206] The server executes the reservation.

[1207] The server handles reservations for transportation, accommodation, and sightseeing / activity.

[1208] The server obtains confirmation information from each service provider.

[1209] Step 18:

[1210] The server sends reservation confirmation information to the terminal.

[1211] The server sends the confirmed reservation details to the terminal.

[1212] Step 19:

[1213] The device notifies the user that the reservation is complete.

[1214] The device notifies the user that the reservation is complete and displays the necessary information (e.g., reservation number, confirmation documents).

[1215] (Example 1)

[1216] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1217] Traditional travel planning systems required users to individually research and book transportation, accommodations, tourist attractions, and activities, which was a time-consuming and laborious process. Furthermore, the systems lacked sufficient automated suggestions to match users' preferences and budgets, making it difficult to optimize plans as users had to manually select and choose from options.

[1218] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1219] In this invention, the server includes means for receiving travel destination information entered by the user, means for collecting the received information as digital data and transmitting it to the server, means for selecting a mode of transportation based on the received information, means for presenting the selected mode of transportation information to the user, means for searching for accommodation based on the travel destination information entered by the user, means for presenting the searched accommodation information to the user, means for generating a list of tourist spots and activities based on the user's preferences, means for presenting the generated list of tourist spots and activities to the user, means for receiving the user's requests for changes and optimizing the plan again, means for presenting the optimized travel plan to the user, means for receiving final booking confirmation information and confirming the bookings for transportation, accommodation, tourist spots, and activities, and means for notifying the user of the booking confirmation information. This enables the user to plan their trip all at once, optimize that plan, and make reservations based on that plan with confidence.

[1220] A "user" is an individual or group that inputs travel planning information and uses the system.

[1221] A "device" is an electronic device used by a user to input travel planning information and receive notifications such as plan presentations and booking confirmations. Specific examples include smartphones and personal computers.

[1222] A "server" is a central processing unit that receives travel plan information sent by users and uses that information to select and optimize transportation, accommodation, tourist attractions, and activities.

[1223] "Travel destination information" refers to detailed travel information entered by the user, specifically including the travel destination, travel itinerary, number of nights of stay, budget, and purpose of travel.

[1224] "Transportation" refers to the means of getting around for travel, and specific examples include airplanes, bullet trains, railways, and buses.

[1225] "Accommodation facilities" refer to places where users stay during their travels, and specific examples include hotels, inns, and guesthouses.

[1226] A "tourist spot" is a place or landmark that should be visited at a travel destination, and specific examples include historical buildings and natural landscapes.

[1227] An "activity" refers to an activity or experience that a user can have at their travel destination, and specific examples include sightseeing tours and leisure activities.

[1228] "Digital data" refers to the electronic representation of travel plan information entered by a user using a device.

[1229] A "list" is a list containing a series of tourist attractions or activity items generated based on specific criteria.

[1230] A "plan" is the overall outline of a travel plan generated based on the user's input information and preferences.

[1231] "Booking confirmation information" refers to confirmation information regarding reservations for transportation, accommodation, sightseeing spots, and activities that is sent from the server when the user finalizes their travel plans.

[1232] "Validation" is the process of verifying that user input is accurate and complete.

[1233] An "algorithm" is a set of calculations or processing steps to optimize a travel plan again based on the user's requested changes.

[1234] This invention is a system that allows users to input travel destination information and handle everything from selecting transportation, accommodations, sightseeing spots, and activities, to making reservations, all in one go. This system can generate and continuously optimize optimal travel plans based on the user's preferences and circumstances.

[1235] First, the user enters travel plan information using a device, such as a smartphone or computer. The user enters detailed information such as the travel destination, itinerary, number of nights, budget, and purpose of travel. The device collects this information as digital data and sends it to the server. During this process, the device validates the user's input in real time and displays error messages as needed.

[1236] When the server receives user information, it first selects a mode of transportation. The server queries a database of transportation options such as air travel, trains, and buses, and selects the most suitable mode of transport based on the user's budget and preferences. Specifically, it uses a Python library (e.g., pandas) to analyze data and select the appropriate mode of transport. This process utilizes a specific algorithm to efficiently search for transportation options that meet the user's criteria.

[1237] Next, the server selects accommodations. It searches the database for the most suitable accommodations based on the user's budget and requirements. The server uses SQL queries to extract properties that match the criteria and ranks the accommodations considering reviews and ratings. Finally, the most suitable property is selected and presented to the user.

[1238] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, if the user is a history buff, the server will list spots related to historical landmarks and ruins. This process uses a specific search engine (e.g., Elasticsearch) to quickly find relevant data and generate it as a list.

[1239] The generated initial travel plan is sent to the device and presented to the user. The user reviews the proposed plan through the device's interface and enters any desired changes. For example, they can add specific tourist attractions or change accommodations.

[1240] When a user submits a request for changes, the server searches and selects the most suitable plan again, optimizing the travel plan. This process can utilize a generative AI model to provide the user with the most appropriate plan. The recalculated plan is then sent back to the device and presented to the user.

[1241] If the user is satisfied with the optimized plan, they can confirm their travel plans by pressing the "Confirm Booking" button. The device sends this information to the server, which confirms the bookings for transportation, accommodation, sightseeing, and activities. Once the booking is complete, the server generates confirmation information (e.g., ticket booking number or accommodation confirmation number) and sends it to the device. The device notifies the user of this confirmation information, informing them that their travel plans have been confirmed.

[1242] Specific example

[1243] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[1244] Transportation: Shinkansen ticket from Tokyo to Kyoto

[1245] Accommodation: 3-star hotels near Gion

[1246] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[1247] Gourmet spot: A famous local Japanese restaurant

[1248] If a user presented with this plan adds "I'd also like to visit Arashiyama," the server recalculates and generates a new plan that includes "Arashiyama." Once the user is satisfied with the optimized plan and confirms their booking, transportation, accommodation, and activities are all booked together.

[1249] This allows the system to easily plan trips and flexibly accommodate the individual needs of each user.

[1250] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1251] The processing flow of this system's program

[1252] Step 1:

[1253] The user enters travel planning information using a device. Specifically, the user enters the destination, travel dates, number of nights, budget, and purpose of travel on the interface screen. The device collects the information entered by the user as digital data. It also validates the input data, displays error messages, and prompts the user to make corrections as needed. The input data is saved in a structured format (e.g., JSON) and sent to the server.

[1254] Step 2:

[1255] The server receives travel plan information sent from the terminal. The server stores this information in a database. Specifically, it uses a relational database management system (RDBMS) such as MySQL to organize and store the information appropriately in each table. The received data includes travel destination, travel itinerary, number of nights of stay, budget, and purpose of travel.

[1256] Step 3:

[1257] The server selects transportation options based on the user's travel plan information. The server queries a database of transportation options to find the best option based on the user's budget and preferences. Specifically, it uses Python libraries (e.g., pandas) to analyze the data and suggest appropriate options. The output includes detailed information on the best transportation option (e.g., air travel, bullet train, bus, etc.).

[1258] Step 4:

[1259] The server presents the user with information on the selected mode of transport. The selected transport is then sent to the user's terminal and displayed. The user can review the information and proceed to the next step. The output includes detailed information about the transport option displayed on the screen.

[1260] Step 5:

[1261] The server searches for accommodations based on the user's travel plan information. The server queries a database of accommodations to find properties that meet the user's budget and preferences. Specifically, it uses SQL queries to extract properties that match the criteria and selects the best accommodation based on reviews and ratings. The output includes detailed information about the best-suited accommodation.

[1262] Step 6:

[1263] The server presents the user with searched accommodation information. The selected accommodations are sent to the terminal and displayed to the user. The user can review the presented accommodation information and proceed to the next step. The output displays detailed information about the accommodation on the screen.

[1264] Step 7:

[1265] The server generates a list of tourist attractions and activities based on the user's preferences. Specifically, it lists suggested tourist attractions and activities based on the travel purpose and preferences entered by the user. For example, it might suggest historical sites to a history enthusiast and famous local restaurants to a foodie. The output is a list of tourist attractions and activities.

[1266] Step 8:

[1267] The server generates a list of tourist attractions and activities and presents it to the user. The list is sent to the user's device and displayed. The user can review the list and enter any desired changes or additions. The output is a list of tourist attractions and activities displayed on the screen.

[1268] Step 9:

[1269] The system receives user requests for changes and resends them to the server. When a user enters corrections or additions, the terminal sends that information to the server and requests a recalculation. Specific input data includes tourist spots to be added and accommodation information to be changed.

[1270] Step 10:

[1271] The server re-optimizes the travel plan based on the user's requested changes. It queries the databases of transportation, accommodation, and attractions again using the new information, generating the optimal plan based on the latest conditions. The output includes the updated travel plan.

[1272] Step 11:

[1273] The optimized travel plan is sent to the user's device and presented to them. The recalculated plan is sent to the device for the user to review and display. The user can review the optimized plan and proceed to confirm their booking. The output displays detailed information about the optimized travel plan on the screen.

[1274] Step 12:

[1275] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button. The device sends this information to the server, which then confirms each booking. The specific input data includes the final booking confirmation information.

[1276] Step 13:

[1277] The server confirms reservations for transportation, accommodation, and tourist attractions / activities. It updates the relevant database based on each reservation to verify its success. The output includes confirmation information (e.g., ticket number and accommodation confirmation number).

[1278] Step 14:

[1279] The server sends confirmation information to the terminal and notifies the user. Final confirmation information is sent to the terminal and notified the user. The user receives the confirmation information and knows that their travel plan is confirmed. The output is the confirmation information displayed on the user's terminal.

[1280] The above outlines the specific processing steps of this system's program. This process allows users to easily plan their trips, obtain the optimal plan, and make secure reservations. A key feature of this system is its ability to automatically generate and provide optimized travel plans based on user input.

[1281] (Application Example 1)

[1282] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1283] The process of planning a trip is complex due to the many factors to consider, and it is not easy to create a plan that perfectly matches the user's desires in one go. Furthermore, users need to individually search for and book transportation, accommodations, attractions, and activities, which is also time-consuming. Additionally, when users plan and book trips in a virtual space, the lack of a suitable interface prevents them from providing the same experience as using a real travel agency.

[1284] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1285] In this invention, the server includes means for receiving travel destination information entered by the user, means for selecting a mode of transportation based on the received information, and means for providing an interface for the user to plan and book a trip in a virtual space. This allows the user to search and book transportation, accommodation, sightseeing spots, and activities all at once, and to intuitively plan, modify, and finalize a trip through an interactive experience in a virtual space.

[1286] "User" refers to the person who plans the trip.

[1287] "Travel destination information" refers to information about places that users want to visit.

[1288] "Transportation" refers to the means that users use to travel to their destination.

[1289] "Accommodation" refers to the place where a user stays during their trip.

[1290] "Tourist spots" refer to points of interest and tourist attractions at the destination the user visits.

[1291] "Activities" refer to the activities and events that users undertake while traveling.

[1292] "Preferences" refers to information about a user's interests and preferences.

[1293] "Virtual space" refers to a computer-generated three-dimensional or two-dimensional environment that users can experience through sight and sound.

[1294] An "interface" refers to the means or methods by which a user interacts with a system.

[1295] A "generative AI model" refers to an algorithm that uses artificial intelligence to create travel plans from data.

[1296] A "prompt statement" refers to an input statement used to give instructions to a generative AI model.

[1297] "Booking confirmation information" refers to information indicating that the travel plan has been finalized and all bookings have been completed.

[1298] This invention provides a travel planning system that allows users to intuitively plan, modify, and finalize travel plans in a virtual space. This system utilizes servers, terminals, and a generative AI model to streamline the entire process from travel planning to booking, providing users with an interactive experience.

[1299] Hardware and software

[1300] The hardware includes user terminals (smartphones, head-mounted displays) and servers. The software includes generative AI models (OpenAI APIs) and travel planning system programs.

[1301] Data processing and data calculation

[1302] 1. User Input Collection: Users input information such as travel destination, itinerary, budget, and preferences from their devices. This information is sent to the server, which processes the received data as digital data.

[1303] 2. Plan Generation: The server generates the optimal travel plan using a generation AI model based on travel destination information, dates, budget, and preferences. The following prompts are used during the generation process.

[1304] Travel destination: Kyoto

[1305] Duration: 4 days from December 1, 2023

[1306] Budget: 100,000 yen

[1307] Interests: History and gourmet food

[1308] Please generate the optimal travel plan.

[1309] The generated plan will include transportation, accommodation, sightseeing spots, and activities.

[1310] 3. Presentation and modification of the plan: The initial travel plan is sent from the server to the terminal and presented to the user. If the user enters a request for changes, that information is sent back to the server, which uses a generative AI model to further optimize the plan.

[1311] 4. Virtual Experience: The device provides users with a virtual interface, allowing them to intuitively plan, modify, and finalize their travel plans. This interface provides users with an interactive experience that feels as if they are visiting a real travel agency.

[1312] 5. Booking Confirmation and Notification: If the user is satisfied with the final travel plan, they press the "Confirm Booking" button to confirm the booking. The server makes all bookings for transportation, accommodation, and sightseeing / activity attractions at once and sends the booking confirmation information to the device. The device then notifies the user of this confirmation information.

[1313] Specific example

[1314] For example, if a user selects "Kyoto" as their travel destination in a virtual space, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[1315] Transportation: Shinkansen ticket from Tokyo to Kyoto

[1316] Accommodation: 3-star hotels near tourist attractions

[1317] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[1318] Gourmet spot: A famous local Japanese restaurant

[1319] If a user requests to visit Arashiyama, the server recalculates and generates a new plan that includes Arashiyama. Once the user is satisfied with the optimized plan and confirms their booking, transportation, accommodation, and activities are all booked together.

[1320] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1321] Step 1:

[1322] The user enters travel destination information. The user uses a terminal to enter travel plan information (destination, itinerary, number of nights, budget, purpose of travel, preferences). The terminal collects this information as digital data and sends it to the server. Input: Travel plan information. Output: Digital data transmission to the server.

[1323] Step 2:

[1324] The server selects a mode of transportation based on the information it receives. The server chooses the most suitable mode of transportation from a database of options such as air travel, trains, and buses, based on the user's budget and preferences. The selection is made using a generated AI model based on prompt messages. Input: Travel plan information. Output: Selected mode of transportation information.

[1325] Step 3:

[1326] The server transmits the selected transportation information to the terminal and presents it to the user. The user can then view the details of the presented transportation. Input: Selected transportation information. Output: Presentation of transportation information to the user.

[1327] Step 4:

[1328] The server searches for accommodations based on the travel destination information entered by the user. It selects and presents accommodations within the user's budget and preferences from a database of accommodations near the travel destination. The search is performed using a generated AI model based on prompt messages. Input: Travel plan information. Output: Searched accommodation information.

[1329] Step 5:

[1330] The server displays the searched accommodation information to the user. The user can view the accommodation details through their device. Input: Searched accommodation information. Output: Presentation of accommodation information to the user.

[1331] Step 6:

[1332] The server generates a list of tourist attractions and activities based on the user's preferences. It creates the most suitable list from a database of tourist attractions and activities related to those preferences. Input: Travel plan information and preference information. Output: Generated list of tourist attractions and activities.

[1333] Step 7:

[1334] The server presents the user with a list of generated tourist spots and activities. The user can view the list through their device. Input: List of generated tourist spots and activities. Output: Presentation of the list to the user.

[1335] Step 8:

[1336] The server receives the user's change requests and optimizes the plan again. The user enters their change requests via their device and sends them to the server. The server uses a generated AI model to optimize the plan again. Input: Change request information. Output: Optimized travel plan.

[1337] Step 9:

[1338] The server presents the user with an optimized travel plan. The user can view the details of the optimized plan through their device. Input: Optimized travel plan. Output: Presentation of the optimized plan to the user.

[1339] Step 10:

[1340] The server receives the final booking confirmation information and confirms reservations for transportation, accommodation, and sightseeing / activity. If the user is satisfied with the optimized plan, they press the "Confirm Booking" button and send the information to the server. The server receives this and makes the bookings in bulk. Input: Final booking confirmation information. Output: Generation of booking confirmation information.

[1341] Step 11:

[1342] The server notifies the user of the reservation confirmation information. The user can receive the confirmation information via their device. Input: Reservation confirmation information. Output: Notification to the user.

[1343] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1344] This invention is a system in which a user inputs travel destination information, the system automatically generates an optimal travel plan, and further optimizes the plan based on the user's emotions using an emotion engine.

[1345] Entering user information and sending it to the server.

[1346] Users input travel planning information using a device. Specifically, they enter information such as the destination, travel dates, number of nights, budget, and purpose of travel. The device collects this information as digital data and sends it to the server.

[1347] Travel plan generation

[1348] When the server receives the above data, it first selects a mode of transportation. The server searches its database of transportation options, such as flights, trains, and buses, and selects the most suitable mode of transport based on the user's budget and preferences. Next, it searches for accommodations near the travel destination and selects facilities that are within the budget and meet the user's preferences.

[1349] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, it suggests information on historical sites and landmarks for history buffs, and popular local restaurants for foodies. In this way, a comprehensive travel plan is initially generated from multiple perspectives.

[1350] User presentation of the generated plan

[1351] The generated initial travel plan is sent from the server to the terminal and displayed to the user. The user can review the details of the presented plan and enter any desired changes or additions. For example, they can add specific tourist attractions or change accommodations.

[1352] Using an Emotion Engine

[1353] This system incorporates an emotion engine that collects and analyzes user emotional data (e.g., facial expression analysis and voice analysis). The emotion engine understands the user's emotional state (e.g., joy, surprise, fatigue) in real time and optimizes the travel plan based on the results.

[1354] For example, if the emotion engine determines that the user is "tired," the server can add rest spots or relaxing activities to the plan. Similarly, if the engine determines that the user is "excited," it can suggest more active spots or exciting activities.

[1355] Optimizing and regenerating travel plans

[1356] When a user enters a change request, the device sends it to the server, along with information from the sentiment engine. The server uses this information to optimize the plan again. If necessary, it searches for new transportation, accommodation, and sightseeing information and generates an optimized plan.

[1357] Reservation Confirmation

[1358] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button to finalize their travel plans. The device sends this information to the server, which then makes all the necessary bookings for transportation, accommodation, and activities. Once the bookings are complete, the server sends confirmation information (such as ticket booking numbers or accommodation confirmation numbers) to the device. The device then notifies the user of this confirmation information, informing them that their travel plans have been finalized.

[1359] Specific example

[1360] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[1361] Transportation: Shinkansen ticket from Tokyo to Kyoto

[1362] Accommodation: 3-star hotels near Gion

[1363] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[1364] Gourmet spot: A famous local Japanese restaurant

[1365] If the emotion engine detects that the user is "tired," the server can add a rest cafe or a relaxing hot spring facility to the plan. In this way, a plan adapted to the user's state can be provided.

[1366] As described above, this system allows users to easily create travel plans that are perfectly tailored to their circumstances and preferences. The introduction of an emotion engine enables the provision of more personalized services and enhances user satisfaction.

[1367] The following describes the processing flow.

[1368] Step 1:

[1369] The user enters travel destination information.

[1370] Users use their device to enter their travel destination, itinerary, number of nights, budget, and purpose of travel.

[1371] The terminal collects the input information as data packets.

[1372] Step 2:

[1373] The terminal sends the input information to the server.

[1374] The terminal sends the collected data packets to the server.

[1375] The terminal monitors the transmission status and waits for confirmation of receipt from the server.

[1376] Step 3:

[1377] Server receives data

[1378] The server receives data packets sent from the terminal and analyzes their contents.

[1379] Establish a connection to the required database.

[1380] Step 4:

[1381] The server selects the mode of transport.

[1382] Based on the received travel destination information, the server selects the most suitable mode of transportation from its database of transportation options.

[1383] Obtain details of the selected mode of transport (e.g., flight number, time, price).

[1384] Step 5:

[1385] Server searches for accommodations

[1386] The server uses a database to search for accommodations near the travel destination.

[1387] We select facilities that match the user's budget and preferences, and obtain information on available rooms.

[1388] Step 6:

[1389] The server suggests tourist spots and activities.

[1390] The server analyzes the user's preferences and searches for tourist attractions and activities in their travel destination.

[1391] Generate a list of selected spots and activities.

[1392] Step 7:

[1393] The server generates an initial travel plan.

[1394] The server integrates selected transportation, accommodation, tourist attractions, and activities to generate an initial travel plan.

[1395] Step 8:

[1396] The server sends the generated plan to the terminal.

[1397] The server sends the generated travel plan to the device.

[1398] Step 9:

[1399] The device displays the plan to the user.

[1400] The device displays the travel plan details to the user.

[1401] Users review the plan details and enter any requested modifications.

[1402] Step 10:

[1403] The emotion engine recognizes the user's emotions.

[1404] The emotion engine uses sensor data from the device (e.g., camera and microphone) to analyze the user's emotions.

[1405] The detected emotion data is sent to the server.

[1406] Step 11:

[1407] The user enters their request to change their plan.

[1408] Users enter their desired changes to the plan (e.g., adding or changing specific tourist spots) into the device.

[1409] Step 12:

[1410] The device sends change requests and sentiment data to the server.

[1411] The device sends user change requests and sentiment data to the server.

[1412] Step 13:

[1413] The server will regenerate to reflect requested changes and sentiment data.

[1414] The server regenerates the travel plan based on the user's requested changes and sentiment data.

[1415] Obtain information on new transportation options, accommodations, and tourist attractions as needed.

[1416] Step 14:

[1417] The server sends the optimized plan to the device.

[1418] The server sends an optimized travel plan to the device.

[1419] Step 15:

[1420] The device displays the optimized plan to the user.

[1421] The device displays an optimized travel plan to the user.

[1422] Step 16:

[1423] The user confirms the final plan.

[1424] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button on their device.

[1425] Step 17:

[1426] The terminal sends confirmed information to the server.

[1427] The terminal sends the final reservation confirmation information to the server.

[1428] Step 18:

[1429] The server executes the reservation.

[1430] The server confirms reservations for transportation, accommodation, and sightseeing / activity.

[1431] The server obtains confirmation information from each service provider.

[1432] Step 19:

[1433] The server sends reservation confirmation information to the terminal.

[1434] The server sends the confirmed reservation details to the terminal.

[1435] Step 20:

[1436] The device notifies the user that the reservation is complete.

[1437] The device notifies the user that the reservation is complete and displays the necessary information (e.g., reservation number, confirmation documents).

[1438] (Example 2)

[1439] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1440] Conventional travel plan generation systems only generate basic travel plans based on user input, making it difficult to provide optimal plans that reflect the user's emotions and individual preferences. Furthermore, it is difficult to quickly reflect user requests for changes to the travel plan. In addition, the lack of a function to dynamically optimize the plan based on the user's emotions prevents increasing user satisfaction during the trip. To solve these problems, a system is needed that analyzes the user's emotional state in real time and optimizes and presents travel plans based on that analysis.

[1441] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1442] In this invention, the server includes means for receiving travel destination information entered by the user, means for selecting a mode of transportation based on the received information, means for presenting the mode of transportation information to the user, means for searching for accommodation based on the travel destination information entered by the user, means for presenting the accommodation information to the user, means for generating a list of tourist spots and activities based on the user's preferences, means for presenting the generated list of tourist spots and activities to the user, means for collecting and analyzing the user's emotional data, means for optimizing the travel plan based on the emotional data, means for receiving the user's request for changes and optimizing the plan again, means for presenting the optimized travel plan to the user, means for receiving final booking confirmation information and confirming the booking of transportation, accommodation, tourist spots / activities, and the booking confirmation information to the user. This enables the generation of an optimal travel plan based on the user's emotions and preferences, and real-time optimization of the plan in response to the user's request for changes.

[1443] A "user" is someone who uses this system to input travel planning information and generate and optimize travel plans.

[1444] A "terminal" is a device (such as a personal computer, smartphone, or tablet) that a user uses to input information.

[1445] A "server" is a central processing unit that receives, analyzes, optimizes, and transmits information sent by users.

[1446] "Travel destination information" refers to data entered by the user, such as the travel destination, travel itinerary, number of nights, budget, and purpose of travel.

[1447] "Transportation" refers to the means of getting around during a trip (air, train, bus, etc.).

[1448] "Accommodation facilities" refer to places of stay during travel (such as hotels and inns).

[1449] A "tourist spot" refers to famous places or attractions that users can visit at their travel destination.

[1450] "Activities" refer to activities that can be participated in during the trip (such as sports, cultural experiences, and relaxation).

[1451] "Preferences" refer to a user's personal likes and interests (such as history, food, nature, etc.).

[1452] "Emotional data" refers to data that indicates the user's emotional state (obtained through methods such as facial expression analysis and voice analysis).

[1453] A "travel plan" is a comprehensive travel plan that includes selected transportation, accommodation, sightseeing spots, activities, and other details.

[1454] "Optimization" refers to adjusting travel plans to best suit the user based on their input information and emotional data.

[1455] "Booking confirmation information" refers to the final booking confirmation data for transportation, accommodation, sightseeing spots, and activities.

[1456] This invention is a system in which a user inputs travel destination information, the system automatically generates an optimal travel plan, and further optimizes the plan based on the user's emotions using an emotion engine.

[1457] Entering user information and sending it to the server.

[1458] Users input travel planning information using their devices. Specifically, they enter information such as the destination, travel dates, number of nights, budget, and purpose of travel. The device collects this information as digital data and sends it to the server. Devices such as personal computers and smartphones are used.

[1459] Travel plan generation

[1460] When the server receives data sent by the user, it first selects a mode of transportation. The server searches a database of transportation options such as flights, trains, and buses to find the best choice and selects the most suitable mode of transport based on the user's budget and preferences. Next, it searches for accommodations near the travel destination and selects facilities that are within the budget and meet the user's preferences.

[1461] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, it suggests information on historical sites and landmarks for history buffs, and popular local restaurants for foodies. In this way, a comprehensive travel plan is initially generated from multiple perspectives.

[1462] User presentation of the generated plan

[1463] The generated initial travel plan is sent from the server to the terminal and displayed to the user. The user can review the details of the presented plan and enter any desired changes or additions. For example, they can add specific tourist attractions or change accommodations.

[1464] Using an Emotion Engine

[1465] This system incorporates an emotion engine. It collects and analyzes user emotional data (e.g., facial expression analysis and voice analysis). The emotion engine understands the user's emotional state (e.g., joy, surprise, fatigue) in real time and optimizes the travel plan based on the results.

[1466] For example, if the emotion engine determines that the user is "tired," the server will add rest spots and relaxing activities to the plan. Similarly, if the user is determined to be "excited," it will suggest more active spots and exciting activities.

[1467] Optimizing and regenerating travel plans

[1468] When a user enters a change request, the device sends it to the server, along with information from the sentiment engine. The server uses this information to optimize the plan again, searching for new transportation options, accommodations, and tourist attractions, and generating an optimized plan.

[1469] Reservation Confirmation

[1470] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button to finalize their travel plans. The device sends this information to the server, which then makes all the necessary bookings for transportation, accommodation, and activities. Once the bookings are complete, the server sends confirmation information (such as ticket booking numbers or accommodation confirmation numbers) to the device. The device then notifies the user of this confirmation information, informing them that their travel plans have been finalized.

[1471] Specific example

[1472] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[1473] Transportation: Shinkansen ticket from Tokyo to Kyoto

[1474] Accommodation: 3-star hotels near Gion

[1475] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[1476] Gourmet spot: A famous local Japanese restaurant

[1477] If the emotion engine detects that the user is "tired," the server can add a rest cafe or a relaxing hot spring facility to the plan. In this way, a plan adapted to the user's state can be provided.

[1478] Example of a prompt

[1479] Please automatically generate a travel plan for "Kyoto, 3 nights / 4 days from December 1st, 2023, 100,000 yen, focusing on history and gourmet food." Please also consider the emotional data "tired" and add relaxing activities.

[1480] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1481] Step 1:

[1482] Users enter travel plan information using their devices. This includes destination, travel dates, number of nights, budget, and preferences (history, food, etc.).

[1483] Input: User's travel plan information (destination, dates, number of nights, budget, preferences)

[1484] Output: Saved to the device as JSON data.

[1485] Specific actions:

[1486] When a user enters information into a form and presses the "Submit" button, the entered information is saved to the device as digital data in JSON format. At this point, a list of the entered information is displayed on the interface.

[1487] Step 2:

[1488] The terminal sends the travel plan information entered by the user to the server.

[1489] Input: JSON data of travel plan information saved on the device

[1490] Output: Sent to the server as an HTTP POST request

[1491] Specific actions:

[1492] The JSON data saved on the device is sent as an HTTP POST request, and when the send button is pressed, a "Sending data" message is displayed. Once the data has been sent, a notification of successful transmission is displayed.

[1493] Step 3:

[1494] The server receives travel plan information sent from the terminal and stores it in the database.

[1495] Input: JSON data sent as an HTTP request

[1496] Output: Travel plan information stored in the database

[1497] Specific actions:

[1498] The server receives the HTTP request and saves the data to its database. At this time, it returns a status code 200 (OK) as confirmation of receipt.

[1499] Step 4:

[1500] The server selects the mode of transportation based on the information it receives.

[1501] Input: Travel plan information stored in the database

[1502] Output: List of optimal modes of transport

[1503] Specific actions:

[1504] A program on the server executes an SQL query to search for transportation options (air, rail, bus, etc.) that match the specified criteria. The results are stored in memory as a list of candidates.

[1505] Step 5:

[1506] The server searches for accommodations that match the user's budget and preferences based on the selected mode of transportation.

[1507] Input: List of optimal transportation options, user's budget and preferences

[1508] Output: List of optimal accommodations

[1509] Specific actions:

[1510] A similar SQL query is executed from the accommodation database to search for accommodations that match the criteria. The results are added as a list of candidates.

[1511] Step 6:

[1512] The server generates a list of tourist spots and activities based on the user's preferences.

[1513] Input: User preferences (history, food, etc.)

[1514] Output: List of tourist attractions and activities

[1515] Specific actions:

[1516] The system filters and lists tourist spots and activities based on preference data. It extracts necessary data from the database and generates the list.

[1517] Step 7:

[1518] The server sends the generated travel plan to the terminal and displays it to the user.

[1519] Input: Information on the best transportation options, accommodations, attractions, and activities.

[1520] Output: Initial travel plan

[1521] Specific actions:

[1522] The generated travel plan information is constructed in JSON format and returned to the device as an HTTP response. The device then parses the received JSON data and displays it on the screen in a user-friendly format.

[1523] Step 8:

[1524] The device collects user emotional data.

[1525] Input: Real-time user emotion data (facial expression analysis, voice analysis, etc.)

[1526] Output: Analyzed emotional state data

[1527] Specific actions:

[1528] The device uses its built-in camera and microphone to collect emotional data in real time, which is then analyzed by an emotion analysis algorithm. The analysis results are stored in memory as digital data.

[1529] Step 9:

[1530] The device sends the analyzed emotion data to the server.

[1531] Input: Analyzed emotional state data

[1532] Output: Sent to the server as an HTTP POST request

[1533] Specific actions:

[1534] The analyzed emotional state data is converted into JSON format and sent to the server as an HTTP POST request.

[1535] Step 10:

[1536] The server optimizes and regenerates travel plans based on emotional data.

[1537] Input: User's emotional state data, initial travel plan

[1538] Output: Optimized travel plan

[1539] Specific actions:

[1540] The plan is adjusted by running an optimization algorithm based on the user's emotional state. For example, if the user is "tired," a rest spot is added; if they are "excited," an active spot is added. The optimized plan information is then reconstructed in JSON format.

[1541] Step 11:

[1542] The server then sends the optimized travel plan back to the device and presents it to the user.

[1543] Input: Optimized travel plan

[1544] Output: Display of the adjusted travel plan

[1545] Specific actions:

[1546] The optimized travel plan information is sent to the device in JSON format, and the new plan is displayed on the device.

[1547] Step 12:

[1548] If the user is satisfied with the optimized plan, they will confirm their reservation.

[1549] Input: User's reservation confirmation information

[1550] Output: Booking confirmation information (ticket booking number, accommodation confirmation number, etc.)

[1551] Specific actions:

[1552] When a user presses the "Confirm Reservation" button, their device sends that information to the server. The server then processes the reservations for transportation, accommodation, tourist attractions, and activities all at once and retrieves the reservation completion information.

[1553] Step 13:

[1554] The server sends reservation completion information to the terminal and notifies the user.

[1555] Input: Reservation confirmation information

[1556] Output: Display of confirmation information

[1557] Specific actions:

[1558] The acquired reservation confirmation information is sent to the device in JSON format, and the user's device displays the confirmation information. This finalizes the travel plan.

[1559] (Application Example 2)

[1560] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1561] In recent years, the use of autonomous vehicles has been rapidly increasing, but travelers are increasingly expecting real-time plan changes and optimizations based on their emotional state. However, conventional travel plan generation systems have been unable to take into account the user's emotional state, resulting in an insufficient user experience. Furthermore, the lack of real-time plan changes and optimizations has led to a decrease in convenience during travel.

[1562] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving travel destination information entered by the user, means for selecting a means of transportation based on the received information, means for presenting the selected means of transportation information to the user, means for searching for accommodation based on the travel destination information entered by the user, means for presenting the searched accommodation information to the user, means for generating a list of tourist spots and activities based on the user's preferences, means for presenting the generated list of tourist spots and activities to the user, means for receiving the user's request for changes and re-optimizing the plan, means for presenting the optimized travel plan to the user, means for receiving final reservation confirmation information and confirming reservations for transportation, accommodation, tourist spots, and activities, means for notifying the user of the reservation confirmation information, and means for collecting and analyzing the user's emotional data using the equipment of an autonomous vehicle and re-optimizing the travel plan in real time based on the emotional state. As a result, the user is continuously provided with a plan that is optimal for their state even while traveling, improving the quality of their experience.

[1563] "User-entered travel destination information" refers to data such as the destination, travel dates, budget, and purpose specified by the user planning the trip.

[1564] "Method of selecting transportation" refers to the process of selecting the most suitable mode of transport (air, train, bus, etc.) based on the entered travel destination information.

[1565] "Methods for searching for accommodations" refers to the process of finding potential places to stay based on the entered travel destination information.

[1566] "Methods for generating lists of tourist attractions and activities" refers to the process of listing places of interest and activities to visit based on the user's preferences.

[1567] "User emotion data" refers to data related to emotions collected using cameras and microphones, and analyzed from the user's facial expressions and voice.

[1568] "Real-time re-optimization" refers to a process that instantly re-optimizes the travel plan based on the user's current emotional state and new input data.

[1569] An "infotainment system" is an information and entertainment system installed inside an autonomous vehicle that allows the user to input and confirm information.

[1570] "Confirmed booking information" refers to the detailed booking information for transportation, accommodation, and sightseeing / activity that has been finalized in the travel plan.

[1571] An "emotion engine" is an algorithm or system that analyzes a user's facial expressions and voice to determine their emotional state.

[1572] "Means for receiving change requests" refers to the process by which the system receives information about changes and additions that users wish to make to their existing travel plans.

[1573] This invention is a system that receives travel destination information entered by a user within an autonomous vehicle, generates an optimal travel plan based on that information, and further optimizes the plan in real time based on the user's emotional state using an emotion engine. The system includes the following means:

[1574] Program generation

[1575] Hardware and software to use

[1576] Hardware: Infotainment system, cameras, and microphones for autonomous vehicles.

[1577] software:

[1578] Language: Python

[1579] Frameworks: Flask (web application frontend), OpenCV (for facial expression analysis), NLP library (for speech analysis), TensorFlow (emotion engine)

[1580] Explanation in natural language

[1581] The basic configuration of the system is as follows: First, the user enters travel information through the infotainment system of the autonomous vehicle. This includes the destination, travel itinerary, budget, and purpose of the trip. The entered information is collected as digital data and transmitted to the server.

[1582] Data collection and transmission

[1583] The server selects transportation methods based on the received information. It searches a database of transportation options such as flights, trains, and buses to find the best choice and selects the most suitable mode of transport based on the user's budget and preferences. It also searches for accommodations near the travel destination and selects facilities that are within the budget and meet the user's preferences. Furthermore, it generates a list of tourist attractions and activities based on the user's preferences.

[1584] Plan generation and display

[1585] The generated initial travel plan is sent from the server to the vehicle's infotainment system and displayed to the user. The user can review the details of the presented plan and enter any desired changes or additions through the infotainment system. The infotainment system then sends these requested changes to the server.

[1586] Emotional Engine

[1587] This system incorporates an emotion engine that collects and analyzes user emotion data (facial expression analysis and voice analysis). Specifically, facial expression data is acquired in real time using cameras mounted on autonomous vehicles and analyzed using OpenCV. Voice data is also collected using microphones and analyzed with an NLP library. TensorFlow is used to implement the emotion engine.

[1588] Real-time optimization

[1589] The emotion engine monitors the user's emotional state (joy, surprise, fatigue, etc.) in real time. Based on this, the server optimizes the travel plan. For example, if the system determines the user is "tired," it adds rest spots and relaxing activities to the plan. If the system determines the user is "excited," it suggests more active spots and exciting activities.

[1590] Reservation Confirmation

[1591] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button to finalize their travel plans. The device sends this information to the server, which then makes all the necessary bookings for transportation, accommodation, and activities. Once the booking is complete, the server sends confirmation information (ticket booking number and accommodation confirmation number) to the device and notifies the user.

[1592] Specific example

[1593] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[1594] Transportation: Shinkansen (bullet train) ticket from a major city to your destination.

[1595] Accommodation: Standard class hotels in the city center area

[1596] Tourist attractions: Historical sites, historical temples

[1597] Foodie spots: Famous local eateries

[1598] Additionally, if the emotion engine detects that the user is "tired," the server will add a rest cafe or a relaxing hot spring facility to the plan. An example of a prompt message is as follows:

[1599] "We're starting the Kyoto travel plan. Please consider the user's needs and add the most suitable relaxation points."

[1600] In this way, users can continuously receive plans that are best suited to their needs throughout their trip, thereby improving the quality of their experience.

[1601] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1602] Step 1:

[1603] The user inputs travel destination information through the infotainment system of the autonomous vehicle. The user enters information such as the destination, travel dates, budget, and purpose of the trip. The input data is transmitted digitally from the terminal (the vehicle's infotainment system) to the server. This allows the server to receive the necessary information.

[1604] Input: Travel information (destination, itinerary, budget, purpose of travel, etc.)

[1605] Output: Travel destination information sent to the server

[1606] Step 2:

[1607] The server selects a mode of transportation based on the received travel destination information. It searches a database of transportation options such as flights, trains, and buses, and selects the most suitable mode of transport based on the user's budget and preferences. The selected mode of transport information is then sent from the server to the terminal.

[1608] Input: Travel destination information

[1609] Data processing and computation: Database search and optimization algorithms for transportation methods.

[1610] Output: Selected transportation information

[1611] Step 3:

[1612] The server queries a database of accommodations based on travel destination information and selects facilities that meet the user's budget and preferences. The selected accommodation information is then transmitted from the server to the terminal and presented to the user.

[1613] Input: Travel destination information

[1614] Data processing and calculation: Database querying and selection algorithms for accommodation facilities.

[1615] Output: Selected accommodation information

[1616] Step 4:

[1617] The server generates a list of tourist attractions and activities based on the user's preferences. It retrieves detailed information about tourist attractions and activities from a database and lists those that match the user's preferences. The generated list is sent from the server to the terminal and presented to the user.

[1618] Input: Travel destination information, user preference information

[1619] Data processing and calculation: Database search and filtering of tourist spot and activity information.

[1620] Output: Listed information on tourist attractions and activities

[1621] Step 5:

[1622] Users review the details of the presented travel plan and enter any desired changes or additions through the infotainment system. These requested changes are sent to the server via the terminal, allowing the server to receive user feedback.

[1623] Input: User's request for change

[1624] Output: Requested changes sent to the server

[1625] Step 6:

[1626] The server receives the user's request for changes and optimizes the plan again. It searches for new transportation, accommodation, and sightseeing information as needed and generates an optimized plan. The regenerated plan is sent from the server to the terminal and presented to the user again.

[1627] Input: User's request for change

[1628] Data processing and calculation: Plan re-optimization algorithms, new data search.

[1629] Output: Optimized travel plan

[1630] Step 7:

[1631] The server uses an emotion engine to collect and analyze user emotion data. Cameras and microphones in the autonomous vehicle are used to acquire user facial and voice data in real time, which is then analyzed using OpenCV and NLP libraries. The analysis results are then used with an emotion engine based on TensorFlow to understand the emotional state.

[1632] Input: Facial expression data, audio data

[1633] Data processing and calculation: Facial expression analysis (OpenCV), speech analysis (NLP library), emotion analysis (TensorFlow)

[1634] Output: User's emotional state

[1635] Step 8:

[1636] The server re-optimizes the travel plan based on the emotional state analyzed by the emotion engine. For example, if the server determines that the user is "tired," it adds rest spots and relaxing activities to the plan. If the server determines that the user is "excited," it suggests more active spots and exciting activities. The optimized plan is sent from the server to the device and presented to the user.

[1637] Input: User's emotional state

[1638] Data processing and calculation: Plan re-optimization algorithms, sentiment data feedback

[1639] Output: Re-optimized travel plan

[1640] Step 9:

[1641] If the user is satisfied with the optimized plan, they press the "Confirm Reservation" button through the infotainment system. This information is then sent from the device to the server.

[1642] Input: User confirms booking

[1643] Output: Reservation confirmation information sent to the server

[1644] Step 10:

[1645] The server receives the final booking confirmation information and makes all reservations for transportation, accommodation, and sightseeing / activity in one place. Once the reservation is complete, the server sends confirmation information (such as ticket booking numbers and accommodation confirmation numbers) to the user's device and notifies them.

[1646] Input: Booking confirmation information

[1647] Data processing and calculation: Sending data to each reservation system, receiving reservation numbers.

[1648] Output: Confirmation information (ticket reservation number, accommodation confirmation number, etc.)

[1649] This allows users to continuously receive plans that are best suited to their needs throughout their trip, thereby improving the quality of their experience.

[1650] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1651] The data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1652] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1653] [Fourth Embodiment]

[1654] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1655] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1656] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1657] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1658] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1659] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1660] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1661] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1662] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1663] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1664] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1665] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1666] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1667] This invention is a system that allows users to input travel destination information and handle everything from selecting transportation, accommodations, sightseeing spots, and activities, to making reservations, all in one place. This system can generate and continuously optimize optimal travel plans based on the user's preferences and requirements.

[1668] Entering user information and sending it to the server.

[1669] The user uses a terminal to input travel planning information. Specifically, they input the destination, travel dates, number of nights, budget, and purpose of the trip (e.g., historical site visits or gourmet tours). The terminal collects this information as digital data and sends it to the server.

[1670] Travel plan generation

[1671] When the server receives the above data, it first selects a mode of transportation. The server searches a database of transportation options such as flights, trains, and buses to find the best choice and selects the most suitable mode of transport based on the user's budget and preferences. Next, it searches for accommodations near the travel destination and selects facilities that are within the budget and meet the user's preferences.

[1672] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, it suggests information on historical sites and landmarks for history buffs, and popular local restaurants for foodies. In this way, a comprehensive travel plan is initially generated from multiple perspectives.

[1673] User presentation of the generated plan

[1674] The generated initial travel plan is sent from the server to the terminal and displayed to the user. The user can review the details of the presented plan and enter any desired changes or additions. For example, they can add specific tourist attractions or change accommodations.

[1675] Optimizing travel plans

[1676] When the user enters their requested changes, the device sends that information back to the server. The server then searches and selects again based on the user's requests, optimizing the travel plan. The updated plan is then sent back to the device and presented to the user.

[1677] Reservation Confirmation

[1678] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button to finalize their travel plans. The device sends this information to the server, which then makes all the necessary bookings for transportation, accommodation, and activities. Once the bookings are complete, the server sends confirmation information (such as ticket booking numbers or accommodation confirmation numbers) to the device. The device then notifies the user of this confirmation information, informing them that their travel plans have been finalized.

[1679] Specific example

[1680] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[1681] Transportation: Shinkansen ticket from Tokyo to Kyoto

[1682] Accommodation: 3-star hotels near Gion

[1683] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[1684] Gourmet spot: A famous local Japanese restaurant

[1685] If a user presented with this plan adds "I'd also like to visit Arashiyama," the server recalculates and generates a new plan that includes "Arashiyama." Once the user is satisfied with the optimized plan and confirms their booking, transportation, accommodation, and activities are all booked together.

[1686] As described above, this system allows users to easily plan their trips and flexibly respond to their individual needs.

[1687] The following describes the processing flow.

[1688] Step 1:

[1689] The user enters travel destination information.

[1690] The user enters their travel destination, travel dates, number of nights, budget, and purpose of travel into their device.

[1691] The terminal collects the input information as data packets.

[1692] Step 2:

[1693] The terminal sends the input information to the server.

[1694] The terminal sends the collected data packets to the server.

[1695] The terminal waits for confirmation of receipt from the server.

[1696] Step 3:

[1697] Server receives data

[1698] The server receives data packets sent from the terminal and analyzes their contents.

[1699] The server establishes a database connection.

[1700] Step 4:

[1701] The server selects the mode of transport.

[1702] Based on the received travel destination information, the server consults a database of transportation options to select the most suitable mode of transport.

[1703] The server retrieves detailed information about the selected mode of transport (e.g., flight number, time, price).

[1704] Step 5:

[1705] Server searches for accommodations

[1706] The server references a database of accommodations in the travel destination to search for accommodations that fit the user's budget.

[1707] We obtain real-time information on available rooms and select the most suitable accommodation.

[1708] Step 6:

[1709] The server suggests tourist spots and activities.

[1710] The server searches for tourist attractions and activities in the user's travel destination based on their preferences.

[1711] The server prioritizes the list of selected tourist spots and activities.

[1712] Step 7:

[1713] The server generates an initial travel plan.

[1714] The server generates an initial travel plan that combines information on transportation, accommodation, tourist attractions, and activities.

[1715] Step 8:

[1716] The server sends the generated plan to the terminal.

[1717] The server sends the generated travel plan to the device.

[1718] Step 9:

[1719] The device displays the plan to the user.

[1720] The device displays the user details of the travel plan.

[1721] Users can review the plan details and enter any requested modifications on their device.

[1722] Step 10:

[1723] The user enters their request to change their plan.

[1724] Users enter their requests for plan changes (e.g., adding or changing specific tourist spots).

[1725] Step 11:

[1726] The device sends the change request to the server.

[1727] The device sends the user's change request to the server.

[1728] Step 12:

[1729] The server will regenerate to reflect the requested changes.

[1730] The server regenerates the travel plan based on the user's requested changes.

[1731] Obtain information on new transportation options, accommodations, and tourist attractions as needed.

[1732] Step 13:

[1733] The server sends the optimized plan to the device.

[1734] The server sends an optimized travel plan to the device.

[1735] Step 14:

[1736] The device displays the optimized plan to the user.

[1737] The device displays an optimized travel plan to the user.

[1738] Step 15:

[1739] The user confirms the final plan.

[1740] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button on their device.

[1741] Step 16:

[1742] The terminal sends confirmed information to the server.

[1743] The terminal sends the final reservation confirmation information to the server.

[1744] Step 17:

[1745] The server executes the reservation.

[1746] The server handles reservations for transportation, accommodation, and sightseeing / activity.

[1747] The server obtains confirmation information from each service provider.

[1748] Step 18:

[1749] The server sends reservation confirmation information to the terminal.

[1750] The server sends the confirmed reservation details to the terminal.

[1751] Step 19:

[1752] The device notifies the user that the reservation is complete.

[1753] The device notifies the user that the reservation is complete and displays the necessary information (e.g., reservation number, confirmation documents).

[1754] (Example 1)

[1755] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1756] Traditional travel planning systems required users to individually research and book transportation, accommodations, tourist attractions, and activities, which was a time-consuming and laborious process. Furthermore, the systems lacked sufficient automated suggestions to match users' preferences and budgets, making it difficult to optimize plans as users had to manually select and choose from options.

[1757] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1758] In this invention, the server includes means for receiving travel destination information entered by the user, means for collecting the received information as digital data and transmitting it to the server, means for selecting a mode of transportation based on the received information, means for presenting the selected mode of transportation information to the user, means for searching for accommodation based on the travel destination information entered by the user, means for presenting the searched accommodation information to the user, means for generating a list of tourist spots and activities based on the user's preferences, means for presenting the generated list of tourist spots and activities to the user, means for receiving the user's requests for changes and optimizing the plan again, means for presenting the optimized travel plan to the user, means for receiving final booking confirmation information and confirming the bookings for transportation, accommodation, tourist spots, and activities, and means for notifying the user of the booking confirmation information. This enables the user to plan their trip all at once, optimize that plan, and make reservations based on that plan with confidence.

[1759] A "user" is an individual or group that inputs travel planning information and uses the system.

[1760] A "device" is an electronic device used by a user to input travel planning information and receive notifications such as plan presentations and booking confirmations. Specific examples include smartphones and personal computers.

[1761] A "server" is a central processing unit that receives travel plan information sent by users and uses that information to select and optimize transportation, accommodation, tourist attractions, and activities.

[1762] "Travel destination information" refers to detailed travel information entered by the user, specifically including the travel destination, travel itinerary, number of nights of stay, budget, and purpose of travel.

[1763] "Transportation" refers to the means of getting around for travel, and specific examples include airplanes, bullet trains, railways, and buses.

[1764] "Accommodation facilities" refer to places where users stay during their travels, and specific examples include hotels, inns, and guesthouses.

[1765] A "tourist spot" is a place or landmark that should be visited at a travel destination, and specific examples include historical buildings and natural landscapes.

[1766] An "activity" refers to an activity or experience that a user can have at their travel destination, and specific examples include sightseeing tours and leisure activities.

[1767] "Digital data" refers to the electronic representation of travel plan information entered by a user using a device.

[1768] A "list" is a list containing a series of tourist attractions or activity items generated based on specific criteria.

[1769] A "plan" is the overall outline of a travel plan generated based on the user's input information and preferences.

[1770] "Booking confirmation information" refers to confirmation information regarding reservations for transportation, accommodation, sightseeing spots, and activities that is sent from the server when the user finalizes their travel plans.

[1771] "Validation" is the process of verifying that user input is accurate and complete.

[1772] An "algorithm" is a set of calculations or processing steps to optimize a travel plan again based on the user's requested changes.

[1773] This invention is a system that allows users to input travel destination information and handle everything from selecting transportation, accommodations, sightseeing spots, and activities, to making reservations, all in one go. This system can generate and continuously optimize optimal travel plans based on the user's preferences and circumstances.

[1774] First, the user enters travel plan information using a device, such as a smartphone or computer. The user enters detailed information such as the travel destination, itinerary, number of nights, budget, and purpose of travel. The device collects this information as digital data and sends it to the server. During this process, the device validates the user's input in real time and displays error messages as needed.

[1775] When the server receives user information, it first selects a mode of transportation. The server queries a database of transportation options such as air travel, trains, and buses, and selects the most suitable mode of transport based on the user's budget and preferences. Specifically, it uses a Python library (e.g., pandas) to analyze data and select the appropriate mode of transport. This process utilizes a specific algorithm to efficiently search for transportation options that meet the user's criteria.

[1776] Next, the server selects accommodations. It searches the database for the most suitable accommodations based on the user's budget and requirements. The server uses SQL queries to extract properties that match the criteria and ranks the accommodations considering reviews and ratings. Finally, the most suitable property is selected and presented to the user.

[1777] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, if the user is a history buff, the server will list spots related to historical landmarks and ruins. This process uses a specific search engine (e.g., Elasticsearch) to quickly find relevant data and generate it as a list.

[1778] The generated initial travel plan is sent to the device and presented to the user. The user reviews the proposed plan through the device's interface and enters any desired changes. For example, they can add specific tourist attractions or change accommodations.

[1779] When a user submits a request for changes, the server searches and selects the most suitable plan again, optimizing the travel plan. This process can utilize a generative AI model to provide the user with the most appropriate plan. The recalculated plan is then sent back to the device and presented to the user.

[1780] If the user is satisfied with the optimized plan, they can confirm their travel plans by pressing the "Confirm Booking" button. The device sends this information to the server, which confirms the bookings for transportation, accommodation, sightseeing, and activities. Once the booking is complete, the server generates confirmation information (e.g., ticket booking number or accommodation confirmation number) and sends it to the device. The device notifies the user of this confirmation information, informing them that their travel plans have been confirmed.

[1781] Specific example

[1782] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[1783] Transportation: Shinkansen ticket from Tokyo to Kyoto

[1784] Accommodation: 3-star hotels near Gion

[1785] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[1786] Gourmet spot: A famous local Japanese restaurant

[1787] If a user presented with this plan adds "I'd also like to visit Arashiyama," the server recalculates and generates a new plan that includes "Arashiyama." Once the user is satisfied with the optimized plan and confirms their booking, transportation, accommodation, and activities are all booked together.

[1788] This allows the system to easily plan trips and flexibly accommodate the individual needs of each user.

[1789] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1790] The processing flow of this system's program

[1791] Step 1:

[1792] The user enters travel planning information using a device. Specifically, the user enters the destination, travel dates, number of nights, budget, and purpose of travel on the interface screen. The device collects the information entered by the user as digital data. It also validates the input data, displays error messages, and prompts the user to make corrections as needed. The input data is saved in a structured format (e.g., JSON) and sent to the server.

[1793] Step 2:

[1794] The server receives travel plan information sent from the terminal. The server stores this information in a database. Specifically, it uses a relational database management system (RDBMS) such as MySQL to organize and store the information appropriately in each table. The received data includes travel destination, travel itinerary, number of nights of stay, budget, and purpose of travel.

[1795] Step 3:

[1796] The server selects transportation options based on the user's travel plan information. The server queries a database of transportation options to find the best option based on the user's budget and preferences. Specifically, it uses Python libraries (e.g., pandas) to analyze the data and suggest appropriate options. The output includes detailed information on the best transportation option (e.g., air travel, bullet train, bus, etc.).

[1797] Step 4:

[1798] The server presents the user with information on the selected mode of transport. The selected transport is then sent to the user's terminal and displayed. The user can review the information and proceed to the next step. The output includes detailed information about the transport option displayed on the screen.

[1799] Step 5:

[1800] The server searches for accommodations based on the user's travel plan information. The server queries a database of accommodations to find properties that meet the user's budget and preferences. Specifically, it uses SQL queries to extract properties that match the criteria and selects the best accommodation based on reviews and ratings. The output includes detailed information about the best-suited accommodation.

[1801] Step 6:

[1802] The server presents the user with searched accommodation information. The selected accommodations are sent to the terminal and displayed to the user. The user can review the presented accommodation information and proceed to the next step. The output displays detailed information about the accommodation on the screen.

[1803] Step 7:

[1804] The server generates a list of tourist attractions and activities based on the user's preferences. Specifically, it lists suggested tourist attractions and activities based on the travel purpose and preferences entered by the user. For example, it might suggest historical sites to a history enthusiast and famous local restaurants to a foodie. The output is a list of tourist attractions and activities.

[1805] Step 8:

[1806] The server generates a list of tourist attractions and activities and presents it to the user. The list is sent to the user's device and displayed. The user can review the list and enter any desired changes or additions. The output is a list of tourist attractions and activities displayed on the screen.

[1807] Step 9:

[1808] The system receives user requests for changes and resends them to the server. When a user enters corrections or additions, the terminal sends that information to the server and requests a recalculation. Specific input data includes tourist spots to be added and accommodation information to be changed.

[1809] Step 10:

[1810] The server re-optimizes the travel plan based on the user's requested changes. It queries the databases of transportation, accommodation, and attractions again using the new information, generating the optimal plan based on the latest conditions. The output includes the updated travel plan.

[1811] Step 11:

[1812] The optimized travel plan is sent to the user's device and presented to them. The recalculated plan is sent to the device for the user to review and display. The user can review the optimized plan and proceed to confirm their booking. The output displays detailed information about the optimized travel plan on the screen.

[1813] Step 12:

[1814] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button. The device sends this information to the server, which then confirms each booking. The specific input data includes the final booking confirmation information.

[1815] Step 13:

[1816] The server confirms reservations for transportation, accommodation, and tourist attractions / activities. It updates the relevant database based on each reservation to verify its success. The output includes confirmation information (e.g., ticket number and accommodation confirmation number).

[1817] Step 14:

[1818] The server sends confirmation information to the terminal and notifies the user. Final confirmation information is sent to the terminal and notified the user. The user receives the confirmation information and knows that their travel plan is confirmed. The output is the confirmation information displayed on the user's terminal.

[1819] The above outlines the specific processing steps of this system's program. This process allows users to easily plan their trips, obtain the optimal plan, and make secure reservations. A key feature of this system is its ability to automatically generate and provide optimized travel plans based on user input.

[1820] (Application Example 1)

[1821] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1822] The process of planning a trip is complex due to the many factors to consider, and it is not easy to create a plan that perfectly matches the user's desires in one go. Furthermore, users need to individually search for and book transportation, accommodations, attractions, and activities, which is also time-consuming. Additionally, when users plan and book trips in a virtual space, the lack of a suitable interface prevents them from providing the same experience as using a real travel agency.

[1823] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1824] In this invention, the server includes means for receiving travel destination information entered by the user, means for selecting a mode of transportation based on the received information, and means for providing an interface for the user to plan and book a trip in a virtual space. This allows the user to search and book transportation, accommodation, sightseeing spots, and activities all at once, and to intuitively plan, modify, and finalize a trip through an interactive experience in a virtual space.

[1825] "User" refers to the person who plans the trip.

[1826] "Travel destination information" refers to information about places that users want to visit.

[1827] "Transportation" refers to the means that users use to travel to their destination.

[1828] "Accommodation" refers to the place where a user stays during their trip.

[1829] "Tourist spots" refer to points of interest and tourist attractions at the destination the user visits.

[1830] "Activities" refer to the activities and events that users undertake while traveling.

[1831] "Preferences" refers to information about a user's interests and preferences.

[1832] "Virtual space" refers to a computer-generated three-dimensional or two-dimensional environment that users can experience through sight and sound.

[1833] An "interface" refers to the means or methods by which a user interacts with a system.

[1834] A "generative AI model" refers to an algorithm that uses artificial intelligence to create travel plans from data.

[1835] A "prompt statement" refers to an input statement used to give instructions to a generative AI model.

[1836] "Booking confirmation information" refers to information indicating that the travel plan has been finalized and all bookings have been completed.

[1837] This invention provides a travel planning system that allows users to intuitively plan, modify, and finalize travel plans in a virtual space. This system utilizes servers, terminals, and a generative AI model to streamline the entire process from travel planning to booking, providing users with an interactive experience.

[1838] Hardware and software

[1839] The hardware includes user terminals (smartphones, head-mounted displays) and servers. The software includes generative AI models (OpenAI APIs) and travel planning system programs.

[1840] Data processing and data calculation

[1841] 1. User Input Collection: Users input information such as travel destination, itinerary, budget, and preferences from their devices. This information is sent to the server, which processes the received data as digital data.

[1842] 2. Plan Generation: The server generates the optimal travel plan using a generation AI model based on travel destination information, dates, budget, and preferences. The following prompts are used during the generation process.

[1843] Travel destination: Kyoto

[1844] Duration: 4 days from December 1, 2023

[1845] Budget: 100,000 yen

[1846] Interests: History and gourmet food

[1847] Please generate the optimal travel plan.

[1848] The generated plan will include transportation, accommodation, sightseeing spots, and activities.

[1849] 3. Presentation and modification of the plan: The initial travel plan is sent from the server to the terminal and presented to the user. If the user enters a request for changes, that information is sent back to the server, which uses a generative AI model to further optimize the plan.

[1850] 4. Virtual Experience: The device provides users with a virtual interface, allowing them to intuitively plan, modify, and finalize their travel plans. This interface provides users with an interactive experience that feels as if they are visiting a real travel agency.

[1851] 5. Booking Confirmation and Notification: If the user is satisfied with the final travel plan, they press the "Confirm Booking" button to confirm the booking. The server makes all bookings for transportation, accommodation, and sightseeing / activity attractions at once and sends the booking confirmation information to the device. The device then notifies the user of this confirmation information.

[1852] Specific example

[1853] For example, if a user selects "Kyoto" as their travel destination in a virtual space, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[1854] Transportation: Shinkansen ticket from Tokyo to Kyoto

[1855] Accommodation: 3-star hotels near tourist attractions

[1856] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[1857] Gourmet spot: A famous local Japanese restaurant

[1858] If a user requests to visit Arashiyama, the server recalculates and generates a new plan that includes Arashiyama. Once the user is satisfied with the optimized plan and confirms their booking, transportation, accommodation, and activities are all booked together.

[1859] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1860] Step 1:

[1861] The user enters travel destination information. The user uses a terminal to enter travel plan information (destination, itinerary, number of nights, budget, purpose of travel, preferences). The terminal collects this information as digital data and sends it to the server. Input: Travel plan information. Output: Digital data transmission to the server.

[1862] Step 2:

[1863] The server selects a mode of transportation based on the information it receives. The server chooses the most suitable mode of transportation from a database of options such as air travel, trains, and buses, based on the user's budget and preferences. The selection is made using a generated AI model based on prompt messages. Input: Travel plan information. Output: Selected mode of transportation information.

[1864] Step 3:

[1865] The server transmits the selected transportation information to the terminal and presents it to the user. The user can then view the details of the presented transportation. Input: Selected transportation information. Output: Presentation of transportation information to the user.

[1866] Step 4:

[1867] The server searches for accommodations based on the travel destination information entered by the user. It selects and presents accommodations within the user's budget and preferences from a database of accommodations near the travel destination. The search is performed using a generated AI model based on prompt messages. Input: Travel plan information. Output: Searched accommodation information.

[1868] Step 5:

[1869] The server displays the searched accommodation information to the user. The user can view the accommodation details through their device. Input: Searched accommodation information. Output: Presentation of accommodation information to the user.

[1870] Step 6:

[1871] The server generates a list of tourist attractions and activities based on the user's preferences. It creates the most suitable list from a database of tourist attractions and activities related to those preferences. Input: Travel plan information and preference information. Output: Generated list of tourist attractions and activities.

[1872] Step 7:

[1873] The server presents the user with a list of generated tourist spots and activities. The user can view the list through their device. Input: List of generated tourist spots and activities. Output: Presentation of the list to the user.

[1874] Step 8:

[1875] The server receives the user's change requests and optimizes the plan again. The user enters their change requests via their device and sends them to the server. The server uses a generated AI model to optimize the plan again. Input: Change request information. Output: Optimized travel plan.

[1876] Step 9:

[1877] The server presents the user with an optimized travel plan. The user can view the details of the optimized plan through their device. Input: Optimized travel plan. Output: Presentation of the optimized plan to the user.

[1878] Step 10:

[1879] The server receives the final booking confirmation information and confirms reservations for transportation, accommodation, and sightseeing / activity. If the user is satisfied with the optimized plan, they press the "Confirm Booking" button and send the information to the server. The server receives this and makes the bookings in bulk. Input: Final booking confirmation information. Output: Generation of booking confirmation information.

[1880] Step 11:

[1881] The server notifies the user of the reservation confirmation information. The user can receive the confirmation information via their device. Input: Reservation confirmation information. Output: Notification to the user.

[1882] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1883] This invention is a system in which a user inputs travel destination information, the system automatically generates an optimal travel plan, and further optimizes the plan based on the user's emotions using an emotion engine.

[1884] Entering user information and sending it to the server.

[1885] Users input travel planning information using a device. Specifically, they enter information such as the destination, travel dates, number of nights, budget, and purpose of travel. The device collects this information as digital data and sends it to the server.

[1886] Travel plan generation

[1887] When the server receives the above data, it first selects a mode of transportation. The server searches its database of transportation options, such as flights, trains, and buses, and selects the most suitable mode of transport based on the user's budget and preferences. Next, it searches for accommodations near the travel destination and selects facilities that are within the budget and meet the user's preferences.

[1888] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, it suggests information on historical sites and landmarks for history buffs, and popular local restaurants for foodies. In this way, a comprehensive travel plan is initially generated from multiple perspectives.

[1889] User presentation of the generated plan

[1890] The generated initial travel plan is sent from the server to the terminal and displayed to the user. The user can review the details of the presented plan and enter any desired changes or additions. For example, they can add specific tourist attractions or change accommodations.

[1891] Using an Emotion Engine

[1892] This system incorporates an emotion engine that collects and analyzes user emotional data (e.g., facial expression analysis and voice analysis). The emotion engine understands the user's emotional state (e.g., joy, surprise, fatigue) in real time and optimizes the travel plan based on the results.

[1893] For example, if the emotion engine determines that the user is "tired," the server can add rest spots or relaxing activities to the plan. Similarly, if the engine determines that the user is "excited," it can suggest more active spots or exciting activities.

[1894] Optimizing and regenerating travel plans

[1895] When a user enters a change request, the device sends it to the server, along with information from the sentiment engine. The server uses this information to optimize the plan again. If necessary, it searches for new transportation, accommodation, and sightseeing information and generates an optimized plan.

[1896] Reservation Confirmation

[1897] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button to finalize their travel plans. The device sends this information to the server, which then makes all the necessary bookings for transportation, accommodation, and activities. Once the bookings are complete, the server sends confirmation information (such as ticket booking numbers or accommodation confirmation numbers) to the device. The device then notifies the user of this confirmation information, informing them that their travel plans have been finalized.

[1898] Specific example

[1899] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," and preferences of "history and gourmet food," the server might generate a plan like the following.

[1900] Transportation: Shinkansen ticket from Tokyo to Kyoto

[1901] Accommodation: 3-star hotels near Gion

[1902] Tourist attractions: Fushimi Inari Taisha Shrine, Kinkaku-ji Temple, Kiyomizu-dera Temple

[1903] Gourmet spot: A famous local Japanese restaurant

[1904] If the emotion engine detects that the user is "tired," the server can add a rest cafe or a relaxing hot spring facility to the plan. In this way, a plan adapted to the user's state can be provided.

[1905] As described above, this system allows users to easily create travel plans that are perfectly tailored to their circumstances and preferences. The introduction of an emotion engine enables the provision of more personalized services and enhances user satisfaction.

[1906] The following describes the processing flow.

[1907] Step 1:

[1908] The user enters travel destination information.

[1909] Users use their device to enter their travel destination, itinerary, number of nights, budget, and purpose of travel.

[1910] The terminal collects the input information as data packets.

[1911] Step 2:

[1912] The terminal sends the input information to the server.

[1913] The terminal sends the collected data packets to the server.

[1914] The terminal monitors the transmission status and waits for confirmation of receipt from the server.

[1915] Step 3:

[1916] Server receives data

[1917] The server receives data packets sent from the terminal and analyzes their contents.

[1918] Establish a connection to the required database.

[1919] Step 4:

[1920] The server selects the mode of transport.

[1921] Based on the received travel destination information, the server selects the most suitable mode of transportation from its database of transportation options.

[1922] Obtain details of the selected mode of transport (e.g., flight number, time, price).

[1923] Step 5:

[1924] Server searches for accommodations

[1925] The server uses a database to search for accommodations near the travel destination.

[1926] We select facilities that match the user's budget and preferences, and obtain information on available rooms.

[1927] Step 6:

[1928] The server suggests tourist spots and activities.

[1929] The server analyzes the user's preferences and searches for tourist attractions and activities in their travel destination.

[1930] Generate a list of selected spots and activities.

[1931] Step 7:

[1932] The server generates an initial travel plan.

[1933] The server integrates selected transportation, accommodation, tourist attractions, and activities to generate an initial travel plan.

[1934] Step 8:

[1935] The server sends the generated plan to the terminal.

[1936] The server sends the generated travel plan to the device.

[1937] Step 9:

[1938] The device displays the plan to the user.

[1939] The device displays the travel plan details to the user.

[1940] Users review the plan details and enter any requested modifications.

[1941] Step 10:

[1942] The emotion engine recognizes the user's emotions.

[1943] The emotion engine uses sensor data from the device (e.g., camera and microphone) to analyze the user's emotions.

[1944] The detected emotion data is sent to the server.

[1945] Step 11:

[1946] The user enters their request to change their plan.

[1947] Users enter their desired changes to the plan (e.g., adding or changing specific tourist spots) into the device.

[1948] Step 12:

[1949] The device sends change requests and sentiment data to the server.

[1950] The device sends user change requests and sentiment data to the server.

[1951] Step 13:

[1952] The server will regenerate to reflect requested changes and sentiment data.

[1953] The server regenerates the travel plan based on the user's requested changes and sentiment data.

[1954] Obtain information on new transportation options, accommodations, and tourist attractions as needed.

[1955] Step 14:

[1956] The server sends the optimized plan to the device.

[1957] The server sends an optimized travel plan to the device.

[1958] Step 15:

[1959] The device displays the optimized plan to the user.

[1960] The device displays an optimized travel plan to the user.

[1961] Step 16:

[1962] The user confirms the final plan.

[1963] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button on their device.

[1964] Step 17:

[1965] The terminal sends confirmed information to the server.

[1966] The terminal sends the final reservation confirmation information to the server.

[1967] Step 18:

[1968] The server executes the reservation.

[1969] The server confirms reservations for transportation, accommodation, and sightseeing / activity.

[1970] The server obtains confirmation information from each service provider.

[1971] Step 19:

[1972] The server sends reservation confirmation information to the terminal.

[1973] The server sends the confirmed reservation details to the terminal.

[1974] Step 20:

[1975] The device notifies the user that the reservation is complete.

[1976] The device notifies the user that the reservation is complete and displays the necessary information (e.g., reservation number, confirmation documents).

[1977] (Example 2)

[1978] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1979] Conventional travel plan generation systems only generate basic travel plans based on user input, making it difficult to provide optimal plans that reflect the user's emotions and individual preferences. Furthermore, it is difficult to quickly reflect user requests for changes to the travel plan. In addition, the lack of a function to dynamically optimize the plan based on the user's emotions prevents increasing user satisfaction during the trip. To solve these problems, a system is needed that analyzes the user's emotional state in real time and optimizes and presents travel plans based on that analysis.

[1980] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1981] In this invention, the server includes means for receiving travel destination information entered by the user, means for selecting a mode of transportation based on the received information, means for presenting the mode of transportation information to the user, means for searching for accommodation based on the travel destination information entered by the user, means for presenting the accommodation information to the user, means for generating a list of tourist spots and activities based on the user's preferences, means for presenting the generated list of tourist spots and activities to the user, means for collecting and analyzing the user's emotional data, means for optimizing the travel plan based on the emotional data, means for receiving the user's request for changes and optimizing the plan again, means for presenting the optimized travel plan to the user, means for receiving final booking confirmation information and confirming the booking of transportation, accommodation, tourist spots / activities, and the booking confirmation information to the user. This enables the generation of an optimal travel plan based on the user's emotions and preferences, and real-time optimization of the plan in response to the user's request for changes.

[1982] A "user" is someone who uses this system to input travel planning information and generate and optimize travel plans.

[1983] A "terminal" is a device (such as a personal computer, smartphone, or tablet) that a user uses to input information.

[1984] A "server" is a central processing unit that receives, analyzes, optimizes, and transmits information sent by users.

[1985] "Travel destination information" refers to data entered by the user, such as the travel destination, travel itinerary, number of nights, budget, and purpose of travel.

[1986] "Transportation" refers to the means of getting around during a trip (air, train, bus, etc.).

[1987] "Accommodation facilities" refer to places of stay during travel (such as hotels and inns).

[1988] A "tourist spot" refers to famous places or attractions that users can visit at their travel destination.

[1989] "Activities" refer to activities that can be participated in during the trip (such as sports, cultural experiences, and relaxation).

[1990] "Preferences" refer to a user's personal likes and interests (such as history, food, nature, etc.).

[1991] "Emotional data" refers to data that indicates the user's emotional state (obtained through methods such as facial expression analysis and voice analysis).

[1992] A "travel plan" is a comprehensive travel plan that includes selected transportation, accommodation, sightseeing spots, activities, and other details.

[1993] "Optimization" refers to adjusting travel plans to best suit the user based on their input information and emotional data.

[1994] "Booking confirmation information" refers to the final booking confirmation data for transportation, accommodation, sightseeing spots, and activities.

[1995] This invention is a system in which a user inputs travel destination information, the system automatically generates an optimal travel plan, and further optimizes the plan based on the user's emotions using an emotion engine.

[1996] Entering user information and sending it to the server.

[1997] Users input travel planning information using their devices. Specifically, they enter information such as the destination, travel dates, number of nights, budget, and purpose of travel. The device collects this information as digital data and sends it to the server. Devices such as personal computers and smartphones are used.

[1998] Travel plan generation

[1999] When the server receives data sent by the user, it first selects a mode of transportation. The server searches a database of transportation options such as flights, trains, and buses to find the best choice and selects the most suitable mode of transport based on the user's budget and preferences. Next, it searches for accommodations near the travel destination and selects facilities that are within the budget and meet the user's preferences.

[2000] Furthermore, the server generates lists of tourist attractions and activities based on the user's preferences. For example, it suggests information on historical sites and landmarks for history buffs, and popular local restaurants for foodies. In this way, a comprehensive travel plan is initially generated from multiple perspectives.

[2001] User presentation of the generated plan

[2002] The generated initial travel plan is sent from the server to the terminal and displayed to the user. The user can review the details of the presented plan and enter any desired changes or additions. For example, they can add specific tourist attractions or change accommodations.

[2003] Using an Emotion Engine

[2004] This system incorporates an emotion engine. It collects and analyzes user emotional data (e.g., facial expression analysis and voice analysis). The emotion engine understands the user's emotional state (e.g., joy, surprise, fatigue) in real time and optimizes the travel plan based on the results.

[2005] For example, if the emotion engine determines that the user is "tired," the server will add rest spots and relaxing activities to the plan. Similarly, if the user is determined to be "excited," it will suggest more active spots and exciting activities.

[2006] Optimizing and regenerating travel plans

[2007] When a user enters a change request, the device sends it to the server, along with information from the sentiment engine. The server uses this information to optimize the plan again, searching for new transportation options, accommodations, and tourist attractions, and generating an optimized plan.

[2008] Reservation Confirmation

[2009] If the user is satisfied with the optimized plan, they press the "Confirm Booking" button to finalize their travel plans. The device sends this information to the server, which then makes all the necessary bookings for transportation, accommodation, and activities. Once the bookings are complete, the server sends confirmation information (such as ticket booking numbers or accommodation confirmation numbers) to the device. The device then notifies the user of this confirmation information, informing them that their travel plans have been finalized.

[2010] Specific example

[2011] For example, if a user selects "Kyoto" as their travel destination, enters "December 1st, 2023 for 3 nights and 4 days" as their travel dates, a budget of "100,000 yen," an...

Claims

1. A means of receiving travel destination information entered by the user, A means of selecting a mode of transportation based on the received information, A means of presenting selected transportation information to the user, A method for searching for accommodations based on travel destination information entered by the user, A means of presenting searched accommodation information to the user, A means of generating a list of tourist spots and activities based on user preferences, A means of presenting a generated list of tourist spots and activities to the user, A means of receiving user change requests and re-optimizing the plan, A means of presenting users with optimized travel plans, You will receive final booking confirmation information and a means to confirm your reservations for transportation, accommodation, and sightseeing / activity. A means of notifying users of reservation confirmation information, A system that includes this.

2. The system according to claim 1, which searches for and generates transportation, accommodation, tourist attractions, and activities in a single batch based on travel destination information entered by the user.

3. The system according to claim 1, comprising an algorithm that receives a user's change request and re-optimizes the plan.

Images