System

Abstract

A system is provided.SOLUTION: This system is provided with a means for inputting the desire of travel by a user in an interactive form, a means for proposing an optimum travel destination, moving means and accommodation facilities based on the inputted desire of travel, a means for displaying proposed travel plans and reserving the plan selected by the user and a means for performing a payment procedure in the process of reservation.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The challenge is to reduce the enormous amount of research and preparation required when planning a trip, and to enable users to easily obtain the optimal travel plan.In addition, there is a need to improve the current situation, where the efficiency of travel planning is significantly reduced due to the lack of a system that can handle everything from travel planning to reservations and payment procedures all at once. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system that includes a means for a user to interactively input their travel preferences, a means for suggesting optimal travel destinations, means of transportation, and accommodations based on the input preferences, a means for displaying suggested travel plans and reserving the plan selected by the user, and a means for carrying out payment procedures during the reservation process. The user's input information is sent to a server, which searches a database for travel plans and presents suggested travel plans to the user for selection. Furthermore, by simplifying on-site payments using electronic money, the system improves the convenience of the entire travel planning process.

[0006] A "user" is an entity that uses the system to create travel plans and make reservations.

[0007] "Dialogue" refers to a method of exchanging information with the user through questions and answers and choices.

[0008] "Travel wishes" is a general term for a user's travel-related requests, including the place they want to travel, the number of days, their budget, the activities they want to experience, and so on.

[0009] A "server" is a computer system that receives and processes input information from a user.

[0010] A "database" is a collection of information that stores and can be searched for, such as travel destinations, accommodations, and transportation methods.

[0011] "Proposal" refers to the act of selecting the most suitable travel plan based on the user's wishes and presenting it to the user.

[0012] A "travel destination" is a geographic location that a user wishes to visit.

[0013] "Transportation" refers to the means of transportation, such as an airplane, train, or automobile, that a user uses to reach their travel destination.

[0014] "Accommodation" refers to residential facilities such as hotels, inns, and private lodgings where people stay when traveling.

[0015] "Display" refers to the act of visually showing the proposed travel plan on the user's terminal screen.

[0016] "Selection" refers to the act of the user deciding on one or more travel plans from among the multiple travel plans presented.

[0017] "Reservation" means the act of confirming the selected travel plan and completing the necessary procedures.

[0018] A "payment procedure" is a series of operations required to settle the cost of a travel plan.

[0019] "Electronic money" is a currency managed in digital form that enables online payments without using cash.

[0020] "Simplification" means simplifying the procedure so that users can operate it easily. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] The present invention is a system that allows a user to interactively input their travel preferences, proposes an optimal travel plan based on the preferences, and performs reservation and payment procedures all at once. Specific embodiments are described below.

[0043] First, the user enters their travel preferences. Using a device such as a smartphone or computer, the user enters their travel destination, number of days, budget, activities they would like to experience, and other preferences. For example, they might enter something like, "I would like to go to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[0044] Next, the device sends this input information to the server. The data is structured in a format such as JSON and transmitted to the server. For example, the format might be {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[0045] The server analyzes the input data and searches the database for information based on the user's preferences. Specifically, it extracts information about hotels, transportation options (flights and trains), and activities suitable for the destination. For example, it uses the following SQL query:

[0046] sql

[0047] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[0048] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0049] The server generates multiple travel plans based on the search results and builds details for each plan, such as a three-night stay at a beach resort hotel in Okinawa for a total of 50,000 yen, or a round-trip flight for 30,000 yen.

[0050] The generated travel plan is sent to the terminal and displayed to the user. The user reviews each plan and selects the one that best suits their needs. Once the selection is complete, the selection is sent to the server.

[0051] The server proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. To do this, it accesses the API of each provider using an HTTP request. For example:

[0052] POST / reserve

[0053] Host: hotelprovider.com

[0054] Content-Type: application / json

[0055] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0056] POST / book

[0057] Host: airlineprovider.com

[0058] Content-Type: application / json

[0059] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0060] At the payment stage, a payment page appears on the terminal, offering options such as credit card and electronic money. The user selects a payment method and enters the required information. The server processes the payment information and sends a confirmation message to the user once the transaction is complete. A confirmation email is also automatically sent, containing payment confirmation and trip details.

[0061] For example, if you say you want to travel to Okinawa, the server will search for beach resort hotels and flights in Okinawa and generate the following three plans:

[0062] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[0063] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[0064] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[0065] If the user selects plan 1, the server will confirm the reservation and process the payment accordingly.

[0066] The above is a description of a specific embodiment for carrying out the present invention. In this way, the user can easily obtain the optimal travel plan and complete the reservation and payment procedures in one go.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] The user inputs their travel preferences.

[0070] The user uses a device (smartphone or computer) to enter information such as the desired travel location, number of days, budget, desired activities, etc. into an input form. For example, the user might enter information such as, "I want to travel to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen."

[0071] Step 2:

[0072] The terminal sends the input information to the server.

[0073] When a user submits the form, the information is sent to the server in a structured data format (e.g., JSON). An example of the data sent is {"Location": "Okinawa", "Number of Days": "3", "Budget": "100,000 yen", "Activity": "Beach Resort"}.

[0074] Step 3:

[0075] The server parses the data it receives.

[0076] The server analyzes the received travel preference data, breaks down and recognizes each item (location, number of days, budget, activity), and generates the necessary search query based on this analyzed data.

[0077] Step 4:

[0078] The server retrieves the information from the database.

[0079] Based on the analysis results, the server searches the database for the best travel destinations, accommodations, and transportation options, for example by executing the following SQL query:

[0080] sql

[0081] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[0082] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0083] Step 5:

[0084] The server generates the itinerary.

[0085] The server generates multiple optimal travel plans from the search results. Each plan includes information on accommodation, transportation, prices, and activities. It builds data to propose these plans to users.

[0086] Step 6:

[0087] The device displays the suggested itinerary.

[0088] The generated travel plans are sent to the user's device and displayed on the screen, allowing the user to compare and consider these plans.

[0089] Step 7:

[0090] The user selects a plan.

[0091] The user selects the most suitable plan from the presented plans and clicks or taps the selection button, and the selection information is sent to the server.

[0092] Step 8:

[0093] The server starts the reservation procedure for the selected plan.

[0094] The server then processes the booking of accommodation and transportation based on the plan selected by the user, sending the booking information to each provider via API. For example:

[0095] POST / reserve

[0096] Host: hotelprovider.com

[0097] Content-Type: application / json

[0098] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0099] POST / book

[0100] Host: airlineprovider.com

[0101] Content-Type: application / json

[0102] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0103] Step 9:

[0104] The server sends the reservation confirmation information to the user.

[0105] If the accommodation and transportation reservations are successful, the server notifies the user of the result. The reservation confirmation information is sent back to the user's device and a confirmation is displayed.

[0106] Step 10:

[0107] The terminal displays the payment page.

[0108] Once the reservation is confirmed, the terminal displays a payment page to the user, offering payment options such as credit card and electronic money.

[0109] Step 11:

[0110] The user enters payment information.

[0111] The user enters their payment information and makes the payment, which is then sent to the server.

[0112] Step 12:

[0113] The server processes the payment transaction.

[0114] The server processes the payment information received and completes the transaction. If successful, a payment confirmation message is sent to the user and the payment is recorded as successful in the system.

[0115] Step 13:

[0116] The server sends a confirmation email.

[0117] Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to double-check their trip details.

[0118] By following the above steps, the system automatically proposes the most suitable travel plan and processes reservations and payment procedures all in one go, simply by the user inputting their desired travel preferences.

[0119] Example 1

[0120] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0121] In today's world, responding to diverse travel needs requires searching through a wide range of information to find the best plan. However, finding the best plan from a vast amount of information takes time and effort, placing a significant burden on users. Furthermore, reservation and payment procedures must be completed separately, which is time-consuming. There is a need for a system that can solve these issues and enable users to easily obtain travel plans and complete reservations and payment procedures all in one go.

[0122] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0123] In this invention, the server includes means for a user to interactively input travel preferences, means for transmitting the input preference information in JSON format and for the server to analyze it, means for searching a database using an SQL query for optimal travel destinations, means of transportation, and accommodations based on the analysis results, means for generating multiple travel plans based on the search results and presenting them to the user, and means for reserving the plan selected by the user and performing payment procedures. This allows the user to easily obtain the optimal travel plan and complete the reservation and payment procedures all at once.

[0124] "User" refers to an individual or entity that inputs travel preferences.

[0125] "Server" refers to a device that analyzes input information sent by users, proposes optimal travel plans, and manages reservation and payment procedures.

[0126] "Terminal" refers to a smartphone, computer, or other electronic device used by a User to input their travel preferences.

[0127] "Travel preferences" refers to travel-related requests of the user, including the destination, number of days, budget, activities they would like to experience, etc.

[0128] "JSON format" is an abbreviation for JavaScript Object Notation, and refers to a lightweight data description format primarily used for data exchange.

[0129] "Parsing" refers to the process by which the server understands the user's input and extracts the necessary information.

[0130] "SQL query" is an abbreviation for Structured Query Language and refers to a question used to retrieve necessary information from a database.

[0131] "Travel plan" refers to a set of suggestions including travel destinations, transportation, accommodations, etc., generated by the server based on the user's preferences.

[0132] "Reservation" refers to the process of actually securing accommodation and transportation based on the travel plan selected by the user.

[0133] "Payment procedure" refers to the process a user goes through to pay for a travel plan.

[0134] "Generating multiple itineraries" refers to the server suggesting multiple available options based on the user's input information.

[0135] The present invention is a system that allows a user to interactively input their travel preferences, proposes an optimal travel plan based on the preferences, and performs reservation and payment procedures all at once. Specific embodiments are described below.

[0136] First, the user uses a device such as a smartphone or computer to input their travel preferences. Specifically, they enter information such as the destination, number of days, budget, and activities they want to experience through a dedicated application or web form. In this example, let's say, "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[0137] The device then structures the input information into JSON format and sends it to the server. The input data is sent to a specific endpoint on the server using an HTTP POST request. For example, the following JSON data is sent:

[0138] json

[0139] {

[0140] "Location": "Okinawa",

[0141] "days": "3",

[0142] "Budget": "100,000 yen",

[0143] "Activity": "Beach Resort"

[0144] }

[0145] The server parses the received data and analyzes the user's desired conditions. During this analysis, the server extracts conditions such as destination, number of days, budget, and activities, and generates SQL queries. The server uses these queries to search its database for relevant hotel, flight, and activity information. For example, the following SQL queries are executed:

[0146] sql

[0147] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[0148] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0149] The server generates multiple itineraries based on the information retrieved from the database, each containing accommodation, transportation, and activities that match the user's specified criteria. For example, the following itinerary may be generated:

[0150] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[0151] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[0152] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[0153] Next, the server structures the generated travel plan in JSON format and sends it to the terminal as an HTTP response. The terminal displays the received travel plan information to the user. The user selects the plan that seems best from multiple plans. The selected plan is sent back to the server, and the server proceeds with the reservation process based on that information.

[0154] The server uses an HTTP request to send the reservation information to the accommodation or transportation provider, for example, a request like this:

[0155] POST / reserve

[0156] Host: hotelprovider.com

[0157] Content-Type: application / json

[0158] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0159] POST / book

[0160] Host: airlineprovider.com

[0161] Content-Type: application / json

[0162] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0163] If the booking is successful, the terminal displays a payment page. The user selects a payment method and enters the required payment information. The server processes the payment information and sends a confirmation message to the user once the transaction is complete. A confirmation email is also automatically sent containing payment confirmation and trip details.

[0164] An example prompt sentence to be input to a generative AI model is as follows:

[0165] What travel plans would be suggested if a user said, "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[0166] The above is a specific embodiment for carrying out the present invention, which allows users to easily obtain the most suitable travel plan and complete the reservation and payment procedures all at once.

[0167] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0168] Step 1:

[0169] The user inputs their travel preferences. Specifically, they use a smartphone or computer to enter their travel destination, number of days, budget, and desired activities through a dedicated application or web form. For example, a user might enter their preference as "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen." At this stage, text information is generated as input data.

[0170] Step 2:

[0171] The terminal converts the input desired information into JSON format. For example, the information input by the user is converted into the following JSON format:

[0172] json

[0173] {

[0174] "Location": "Okinawa",

[0175] "days": "3",

[0176] "Budget": "100,000 yen",

[0177] "Activity": "Beach Resort"

[0178] }

[0179] The converted JSON data is then sent to the server via an HTTP POST request, with the terminal outputting the input data as structured data.

[0180] Step 3:

[0181] The server parses the received JSON data and analyzes the user's desired conditions. Specifically, it reads the JSON data and extracts key information such as "location," "number of days," "budget," and "activity." Based on the results of this analysis, it prepares to search for related information in the database. At this time, the server outputs the analyzed data of the user's desired conditions.

[0182] Step 4:

[0183] The server generates an SQL query based on the parsed data and searches the database for the relevant information. For example, it executes a query like this:

[0184] sql

[0185] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[0186] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0187] This query causes the server to retrieve hotel, flight, and activity data that matches the criteria from the database, and then outputs the information retrieved from the database.

[0188] Step 5:

[0189] The server generates multiple travel plans based on the acquired information. For example, it creates multiple plans combining hotels and flights in Okinawa:

[0190] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[0191] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[0192] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[0193] The server outputs detailed information about these plans in structured JSON format.

[0194] Step 6:

[0195] The server sends the generated travel plan to the terminal as an HTTP response. The terminal displays the received travel plan information to the user. At this time, the travel plan information is output to the terminal.

[0196] Step 7:

[0197] The user selects the desired plan from the multiple plans presented. When the selected plan is input into the device, the device again sends the selection information in JSON format to the server. At this time, the selection information is output as structured data.

[0198] Step 8:

[0199] The server receives the user's selection information and proceeds with the reservation process based on the selected plan. Specifically, it analyzes the selection information and sends the reservation information to the accommodation and transportation providers via HTTP request. For example, it sends the following request:

[0200] POST / reserve

[0201] Host: hotelprovider.com

[0202] Content-Type: application / json

[0203] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0204] POST / book

[0205] Host: airlineprovider.com

[0206] Content-Type: application / json

[0207] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0208] At this time, reservation confirmation information after the request is sent is output.

[0209] Step 9:

[0210] After the reservation is completed, the payment page will be displayed on the terminal. The user can select the desired payment method from the presented payment methods and enter the necessary payment information. At this time, the payment information will be entered into the terminal.

[0211] Step 10:

[0212] The server receives the payment information and processes the payment. After the payment is completed, the server sends a payment confirmation message to the user and automatically sends a confirmation email containing the payment confirmation and trip details. At this time, the payment confirmation information and trip details are output.

[0213] (Application example 1)

[0214] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0215] Currently, many travel plan suggestion and reservation systems on the market struggle to efficiently reflect user preferences, making it cumbersome to compare and select travel plans. Payment procedures are also complicated, resulting in an overall unsatisfactory user experience. Furthermore, because they use physical devices, it is difficult to provide intuitive visual feedback, making it difficult for users to easily select the optimal travel plan.

[0216] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0217] In this invention, the server includes a means for a user to interactively input their travel preferences, a means for suggesting optimal travel destinations, means of transportation, and accommodations based on the input travel preferences, a visual feedback means for displaying the proposed travel plans, a means for booking the plan selected by the user, a means for performing payment procedures during the booking process, and a means for the user to select and operate travel plans in a virtual space using a smart device. This allows users to easily input their preferences interactively and intuitively compare and select travel plans through visual feedback. Furthermore, the reservation and payment procedures can be completed seamlessly in one go by operating in the virtual space, improving the overall user experience.

[0218] The "means for the user to interactively input travel preferences" is a function that allows the user to interactively input travel preferences into the terminal through voice input or keyboard input.

[0219] "Means for suggesting optimal travel destinations, means of transportation, and accommodations based on input travel preferences" is a function that allows the server to analyze the information input by the user and suggest appropriate travel destinations, means of transportation, and accommodations.

[0220] The "visual feedback means for displaying the proposed itinerary" is a function that displays the itinerary to the user in a visual format to help the user intuitively understand it.

[0221] The "means for reserving a plan selected by the user" is a function that allows the user to select a presented travel plan and confirm a reservation based on that plan.

[0222] "Means for carrying out payment procedures during the reservation process" is a function for carrying out all the payment procedures required when reserving a travel plan in one go.

[0223] "Means for users to select and operate travel plans in a virtual space using a smart device" refers to a function that allows users to visually check and operate travel plans in a virtual reality space using a smart device such as smart glasses or a head-mounted display.

[0224] The present invention is a system that allows users to interactively input their travel preferences, proposes optimal travel plans based on those preferences, and handles reservations and payment procedures all at once. The system for implementing the present invention mainly consists of a smart device, a back-end server, and related software.

[0225] First, the user inputs their travel preferences. Using smart glasses or a head-mounted display (HMD), the user inputs their preferences using voice or gestures. For example, they might input something like, "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen." This input is analyzed in real time using voice recognition and gesture recognition technology and converted into JSON format.

[0226] Next, the device sends the input information to the server. The data is structured in JSON format and transmitted to the server. For example, it is sent in the form {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[0227] The server analyzes the input data and searches the database for information based on the user's preferences. Specifically, it extracts information about hotels, transportation options (flights and trains), and activities suitable for the destination. For example, it uses the following SQL query:

[0228] sql

[0229] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[0230] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0231] The server generates multiple travel plans based on the search results and builds details for each plan. The generated travel plans are sent to the device and displayed in the virtual space as visual feedback to the user. The user can use the HMD to review each plan and select the one that best suits their needs using gestures and voice controls.

[0232] Once the selection is complete, the selection is sent to the server. The server then proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. This involves accessing the API of each provider using an HTTP request. For example:

[0233] POST / reserve

[0234] Host: hotelprovider.com

[0235] Content-Type: application / json

[0236] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0237] POST / book

[0238] Host: airlineprovider.com

[0239] Content-Type: application / json

[0240] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0241] At the payment stage, a payment page appears on the terminal, offering options such as credit card and electronic money. The user selects a payment method and enters the required information. The server processes the payment information and sends a confirmation message to the user once the transaction is complete. A confirmation email is also automatically sent, containing payment confirmation and trip details.

[0242] For example, if you say you want to travel to Okinawa, the server will search for beach resort hotels and flights in Okinawa and generate the following three plans:

[0243] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[0244] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[0245] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[0246] If the user selects plan 1, the server will confirm the reservation and process the payment accordingly.

[0247] Examples of prompts:

[0248] "Planning my next trip: Destination is Okinawa, looking for a beach resort, budget under 100,000 yen. Any recommendations?"

[0249] In this way, users can easily obtain the optimal travel plan, make decisions through visual feedback, and complete the booking and payment procedures all in one place within the virtual space.

[0250] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0251] Step 1: The user uses a smart device to interactively input their travel preferences

[0252] Users use smart glasses or a head-mounted display to input their travel preferences using voice input or gestures. What the user says is captured by the device's microphone and sensors. The voice input is converted into text using speech recognition technology, and the data is structured into JSON format (e.g., {"Location": "Okinawa", "Number of Days": "3", "Budget": "100,000 yen", "Activity": "Beach Resort"}).

[0253] Step 2: The device sends the input information to the server

[0254] The device sends the JSON-formatted data generated in step 1 to the server, which uses an HTTP request to send the data to the backend server. The server then analyzes the data and prepares a search for a travel plan based on the user's preferences.

[0255] Step 3: The server retrieves the itinerary from the database

[0256] The server parses the received JSON data and searches the database for the appropriate itinerary (accommodation, transport, activities) using Structured Query Language (SQL). For example, the following query is executed:

[0257] sql

[0258] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[0259] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0260] The query results are temporarily stored in the server.

[0261] Step 4: The server sends the generated itinerary to the device.

[0262] The server generates multiple itineraries based on the search results, converts them into JSON format data, and sends it to the device. This data includes details of each itinerary (e.g., hotel name, price, flight information, etc.).

[0263] Step 5: The device displays the travel plan in the virtual space.

[0264] Based on the received data, the terminal displays the travel plan in a virtual space on the user's smart device (HMD or smart glasses). The displayed plan is presented in a visual format that the user can intuitively understand. The user can operate and select each plan using gestures and voice commands.

[0265] Step 6: User confirms selected plan and sends it to the server

[0266] The plan selected by the user in the virtual space is converted back into JSON format data via the device and sent to the server, which then analyzes the data and initiates the reservation process based on the selection.

[0267] Step 7: The server processes the reservation

[0268] The server accesses the API of each provider (hotel, airline) based on the selected plan and sends a reservation request. For example, the following API request is used:

[0269] POST / reserve

[0270] Host: hotelprovider.com

[0271] Content-Type: application / json

[0272] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0273] POST / book

[0274] Host: airlineprovider.com

[0275] Content-Type: application / json

[0276] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0277] If the reservation is successful, the result is notified to the terminal.

[0278] Step 8: Make a payment

[0279] The terminal displays a payment page, offering the user payment options such as credit card or electronic money. The user selects the desired payment method and enters the required information. The server processes the payment information and, if the payment is successful, sends the user a confirmation message with further details. A confirmation email is also automatically sent to the user.

[0280] Examples of prompts:

[0281] "Planning my next trip: Destination is Okinawa, looking for a beach resort, budget under 100,000 yen. Any recommendations?"

[0282] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0283] The present invention achieves a higher level of personalization by combining a system in which a user interactively inputs their travel preferences, proposes an optimal travel plan based on those preferences, and handles all booking and payment procedures in one go with an emotion engine that recognizes the user's emotions. Specific embodiments are described below.

[0284] First, the user enters their travel preferences. Using a device such as a smartphone or computer, the user enters information such as the travel destination, number of days, budget, and desired activities into an input form. For example, the user might enter information such as, "I want to travel to Okinawa, relax at a beach resort, and my budget is 100,000 yen."

[0285] Next, the device sends the input information to the server. The data is structured in a format such as JSON and transmitted to the server. An example of the data sent is {"Location": "Okinawa", "Number of Days": "3", "Budget": "100,000 yen", "Activity": "Beach Resort"}.

[0286] The server analyzes the received data and searches for information from a database based on the user's preferences. Specifically, it extracts information on hotels, transportation (flights or trains), and activities suitable for the travel destination. At this time, the emotion engine analyzes the user's facial expressions and voice when inputting information to evaluate the user's emotional state. For example, if the user is excited, it will prioritize suggestions with plenty of activities, or if the user wants to relax, it will prioritize suggestions for quiet resorts.

[0287] The emotion engine analyzes emotions based on the following information:

[0288] Facial expression analysis (using a camera to analyze facial movements and expressions)

[0289] Voice analysis (analyzing voice tone and speed using a microphone)

[0290] Text analysis (guessing emotions from input text)

[0291] Based on the analysis results of the emotion engine, the server generates multiple travel plans from the search results and builds information on the optimal accommodation, transportation, price, and activities for each plan. For example, a plan might include a three-night stay at a beach resort hotel in Okinawa for a total of 50,000 yen, or a round-trip flight for 30,000 yen.

[0292] The generated travel plan is sent to the terminal and displayed to the user. The user reviews each plan and selects the one that best suits their needs. Once the selection is complete, the selection is sent back to the server.

[0293] The server proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. At this time, it accesses each provider via API and sends the reservation information. For example:

[0294] POST / reserve

[0295] Host: hotelprovider.com

[0296] Content-Type: application / json

[0297] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0298] POST / book

[0299] Host: airlineprovider.com

[0300] Content-Type: application / json

[0301] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0302] Once the reservation is confirmed, the terminal displays a payment procedure page, where options such as credit card and electronic money are offered. The user enters their payment information and completes the payment. The payment information is then sent to the server.

[0303] The server processes the payment information it receives, sends a confirmation message to the user if the payment is successful, and records the payment as successful within the system. Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to review their trip details.

[0304] The analysis results of the emotion engine are saved as reference data for the next travel plan generation. For example, if a user previously expressed the emotion "I want to relax," that information will be used to suggest future plans.

[0305] Through the above steps, the present invention allows users to simply input their travel preferences, automatically suggest optimal travel plans, and process reservations and payment procedures all in one go. Furthermore, by utilizing an emotion engine, it is possible to provide a personalized experience that matches the user's emotions.

[0306] The processing flow will be explained below.

[0307] Step 1:

[0308] The user inputs their travel preferences.

[0309] The user uses a device (smartphone or computer) to input the desired travel destination, number of days, budget, desired activities, etc. For example, the user inputs information such as "I want to travel to Okinawa, relax at a beach resort, and my budget is 100,000 yen."

[0310] Step 2:

[0311] The terminal transmits the input information and the user's emotion information to the server.

[0312] Once the input is complete, the device sends the input information in JSON format, along with the user's emotional data (facial expression and voice analysis data), to the server. For example, the emotional data is sent as {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[0313] Step 3:

[0314] The server analyzes the travel preference data.

[0315] The server analyzes the received input data and recognizes each item (location, number of days, budget, activity), and generates a query for database search based on the analysis results.

[0316] Step 4:

[0317] The emotion engine analyzes the user's emotion data.

[0318] The emotion engine in the server analyzes the user's facial expression data and voice data to identify the user's emotional state (excited, relaxed, anxious, etc.). For example, it is determined that the user is in a relaxed mode.

[0319] Step 5:

[0320] The server retrieves information from the database based on the results of the emotion engine.

[0321] The server takes into account the results of the emotion engine and searches the database for the most suitable travel plans. For example, if the emotion of wanting to relax is detected, it will prioritize plans that include quiet beach resorts and spas. An example SQL query is as follows:

[0322] sql

[0323] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000 AND features LIKE '%Relax%';

[0324] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0325] Step 6:

[0326] The server generates the itinerary.

[0327] The server generates multiple travel plans based on the search results, incorporating information on the best accommodations, transportation, prices, and activities for each plan, such as a three-night stay at a beach resort hotel in Okinawa for a total of 50,000 yen, or a round-trip flight for 30,000 yen.

[0328] Step 7:

[0329] The device displays the suggested itinerary.

[0330] The generated travel plans are sent to the terminal and displayed on the user's screen, allowing the user to compare and consider these plans.

[0331] Step 8:

[0332] The user selects a plan.

[0333] The user selects the desired plan from the presented plans and clicks or taps the selection button. The selection information is sent to the server.

[0334] Step 9:

[0335] The server starts the reservation procedure for the selected plan.

[0336] The server processes the reservations for accommodation and transportation based on the plan selected by the user. This is done via the API of each provider. For example, it sends the following request:

[0337] POST / reserve

[0338] Host: hotelprovider.com

[0339] Content-Type: application / json

[0340] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0341] POST / book

[0342] Host: airlineprovider.com

[0343] Content-Type: application / json

[0344] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0345] Step 10:

[0346] The server sends the reservation confirmation information to the user.

[0347] If the accommodation and transportation reservations are successful, the result is notified to the user's terminal. The server generates and transmits data including reservation confirmation information.

[0348] Step 11:

[0349] The terminal displays the payment page.

[0350] Once the reservation is confirmed, the terminal displays a payment procedure page to the user, offering payment methods such as credit card and electronic money.

[0351] Step 12:

[0352] The user enters payment information.

[0353] The user enters their payment information and completes the transaction, which is then sent to the server.

[0354] Step 13:

[0355] The server processes the payment transaction.

[0356] The server processes the received payment information, sends a confirmation message to the user when the payment is complete, and records the payment completion information in the system.

[0357] Step 14:

[0358] The server sends a confirmation email.

[0359] Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to double-check their trip details.

[0360] Step 15:

[0361] The emotion engine stores the emotion data.

[0362] The emotion engine saves the user's emotional data and uses it as reference data when generating the next travel plan. For example, if a user previously expressed the emotion "I want to relax," that information will be used to suggest future plans.

[0363] Through the above steps, the present invention allows users to simply input their travel preferences, automatically suggest optimal travel plans, and process reservations and payment procedures all in one go. Furthermore, by utilizing an emotion engine, it is possible to provide a personalized experience that matches the user's emotions.

[0364] Example 2

[0365] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0366] Many conventional travel reservation systems propose travel plans based on the user's preferences and handle reservations and payment procedures, but because they do not take the user's emotional state into consideration, they have the problem of making it difficult to provide a personalized experience that matches the user's emotions.In addition, because plans are proposed based only on information entered by the user, they may not always propose the optimal travel plan.

[0367] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0368] In this invention, the server includes means for allowing a user to interactively input travel preferences, means for proposing optimal travel destinations, means of transportation, and accommodations based on the input travel preferences, and means for customizing a travel plan based on the user's emotional state using an analysis device for analyzing the user's emotions. This makes it possible to propose an optimal travel plan based on the user's emotions and to complete reservations and payment procedures all at once.

[0369] "User" refers to a person who uses this system, who inputs travel preferences, selects a proposed plan, and completes payment procedures.

[0370] "Travel preferences" refers to information that a user interactively inputs, such as travel destination, number of days, budget, desired activities, etc.

[0371] "Interactive" refers to a format in which a user interacts with a system to input information.

[0372] "Means of suggestion" refers to a method for selecting the most suitable travel destination, means of transportation, and accommodation based on the input information and presenting them to the user.

[0373] The term "means for making a reservation" refers to a method for carrying out the procedure for actually making a reservation for the travel plan selected by the user.

[0374] "Means for making payment" refers to the method by which a user makes payment after completing a travel reservation.

[0375] "Analysis device" refers to a device that analyzes facial expressions, voice, and text data input by a user and evaluates the user's emotional state.

[0376] "Means for customization" refers to ways to adjust or change travel plans based on the user's emotional state.

[0377] "Server" refers to the computer system that processes and analyzes input data, generates suggested travel plans, and provides them to users.

[0378] A "database" refers to a collection of information that stores information on travel destinations, accommodations, means of transportation, etc., and that can be searched and extracted.

[0379] "Emotional state" refers to the psychological and emotional state analyzed from the user's facial expressions, voice, and text input.

[0380] This invention achieves a higher level of personalization by combining a system in which a user interactively inputs their travel preferences, proposes an optimal travel plan based on those preferences, and performs all the booking and payment procedures in one go with an emotion engine that recognizes the user's emotions. Specific embodiments of the present invention are described in detail below.

[0381] First, the user uses a device such as a smartphone or computer to input their travel preferences. They enter information such as the travel destination, number of days, budget, and desired activities into an input form. For example, they might enter information like, "I want to travel to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen."

[0382] The device then sends the input information to the server, which converts the data into a structured data format such as JSON and transmits it to the server.

[0383] The server analyzes the received data and searches for information from a database based on the user's preferences. Specifically, it extracts information about hotels, transportation options (flights or trains), and activities suitable for the travel destination. At this time, the emotion engine analyzes the user's facial expressions and voice when they input information and evaluates their emotional state. The emotion engine analyzes the user's emotions using technologies such as facial expression analysis (using a camera to analyze facial movements and expressions), voice analysis (using a microphone to analyze the tone and speed of voice), and text analysis (inferring emotions from input text).

[0384] For example, if a user inputs "I want to travel to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen," and their facial expression is relaxed, the server will prioritize suggesting quiet resorts. As a specific example, if the emotion engine reads the emotion "I want to relax" from the user's facial expression, the server will prioritize generating plans that include quiet beach resort hotels in Okinawa. Also, if a user inputs "I want to travel to Okinawa. I want to enjoy the beach and diving as activities," and their facial expression shows signs of excitement, the server will suggest plans with plenty of activities.

[0385] The generated travel plans are sent from the server to the terminal as multiple options and displayed to the user. The user reviews each plan and selects the one that best suits their needs. Once the selection is complete, the selection is sent back to the server.

[0386] The server proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. At this time, it accesses each provider via API and sends the reservation information. For example, the following request is made:

[0387] POST / reserve

[0388] Host: hotelprovider.com

[0389] Content-Type: application / json

[0390] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0391] Once the reservation is confirmed, the terminal displays a payment procedure page, where options such as credit card and electronic money are offered. The user enters their payment information and completes the payment. The payment information is then sent to the server.

[0392] The server processes the payment information it receives, sends a confirmation message to the user if the payment is successful, and records the payment as successful within the system. Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to review their trip details.

[0393] The analysis results of the emotion engine are saved as reference data for the next travel plan generation. For example, if a user previously expressed the emotion "I want to relax," that information will be used to suggest future plans.

[0394] Through the above steps, the present invention allows users to simply input their travel preferences, automatically suggest optimal travel plans, and process reservations and payment procedures all in one go. Furthermore, by utilizing an emotion engine, it is possible to provide a personalized experience that matches the user's emotions.

[0395] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0396] Step 1:

[0397] The user inputs his / her travel preferences.

[0398] Input: The user uses a smartphone or computer to enter information such as the travel destination, number of days, budget, desired activities, etc. into an input form. For example, the user might enter information such as, "I want to travel to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen."

[0399] Output: The input information is temporarily stored in the device's memory.

[0400] Step 2:

[0401] The terminal sends the input information to the server.

[0402] Input: Travel preference information entered by the user.

[0403] Data processing: Converting input information into structured data such as JSON format.

[0404] Output: The converted data is sent to the server. For example, the following data is sent: {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[0405] Step 3:

[0406] The server receives and analyzes the data.

[0407] Input: JSON data of travel preference information sent from the terminal.

[0408] Data calculation: Data is analyzed to extract information such as travel destination, budget, number of days, and desired activities. Furthermore, an emotion engine is activated at this time to evaluate the user's emotional state based on their facial expressions and voice input. The emotion engine uses facial expression analysis technology (using the camera), voice analysis technology (using the microphone), and text analysis technology.

[0409] Output: The analysis results in information that reflects the user's wishes and emotional state.

[0410] Step 4:

[0411] The server searches the database and generates the optimal travel plan.

[0412] Input: Travel preference information and emotional evaluation data.

[0413] Data calculation: Searches the database for hotels, transportation (flights and trains), and activity information suitable for the travel destination, and combines this data to generate multiple travel plans. For example, if a user is rated as wanting to relax, a plan including a quiet beach resort hotel will be generated.

[0414] Output: Multiple itineraries are generated and their details are stored as data.

[0415] Step 5:

[0416] The server transmits the generated travel plan to the terminal.

[0417] Input: The generated itinerary data.

[0418] Data processing: Convert travel plan data into JSON format or other format for sending to the device.

[0419] Output: The travel plan sent to the device is displayed on the user's UI. For example, a three-night stay at a beach resort hotel in Okinawa (total price: 50,000 yen) and a round-trip flight (30,000 yen) are displayed as a list.

[0420] Step 6:

[0421] The user reviews and selects a travel plan.

[0422] Input: Multiple travel plans displayed on the device.

[0423] How it works: The user reviews each plan in detail and selects the one that best suits their needs.

[0424] Output: The data of the selected plan is sent from the device to the server.

[0425] Step 7:

[0426] The server will proceed with the reservation procedure for the selected plan.

[0427] Input: Travel plan data selected by the user.

[0428] Data Computing: Generate API requests to send booking information to accommodation and transportation providers. For example, send requests to hotel and airline booking APIs.

[0429] Output: The reservation information is sent to the provider, and if successful, the result is returned to the server and the user is notified.

[0430] Step 8:

[0431] The user completes the payment process.

[0432] Input: Information about your confirmed travel plan and the payment options displayed on the payment page (credit card, e-money, etc.).

[0433] How it works: The user enters payment information and makes a payment.

[0434] Output: Payment information is sent from the terminal to the server.

[0435] Step 9:

[0436] The server processes the payment and sends a confirmation email.

[0437] Input: Payment information submitted by the user.

[0438] Data calculation: Executes the payment process and checks whether the payment is successful. If successful, records the payment as completed in the system.

[0439] Output: A confirmation message confirming payment is completed and a confirmation email containing the trip details is sent to the user.

[0440] (Application example 2)

[0441] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0442] Conventional travel plan suggestion systems have struggled to provide optimal travel experiences because they lack the ability to personalize based on users' emotions and real-time situations. Furthermore, they lack the ability to obtain and suggest detailed information about travel destinations and activities in real time, making it difficult to respond to users' changing needs. Furthermore, it has been difficult to provide real-time guidance and reservation functions during travel.

[0443] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recognizing emotions by analyzing the user's facial expressions and voice, means for personalizing plan suggestions based on the user's emotions, means for providing real-time information on tourist spots and activities being visited, and means for checking real-time availability information on proposed activities and accommodations and making reservations. This makes it possible to provide a personalized travel experience optimized for the user's emotions and real-time situation.

[0444] A "means for interactively inputting user travel preferences" is a device or software that allows a user to input travel preferences via voice or text through an interface.

[0445] "Means for suggesting optimal travel destinations, means of transportation, and accommodations based on input travel preferences" refers to an algorithm or software function that analyzes the desired information input by the user and selects and suggests optimal travel destinations, means of transportation, and accommodations accordingly.

[0446] "Means for displaying proposed travel plans and reserving the plan selected by the user" is a function that displays the travel plan selected by the user on the screen and allows the user to proceed with the reservation procedure for that plan after selection.

[0447] The "means for carrying out payment procedures during the reservation process" refers to a processing means for making payment for the travel plan selected by the user using a credit card or electronic money.

[0448] "Means for recognizing emotions by analyzing a user's facial expressions and voice" refers to technology that uses a user's camera or microphone to analyze their facial expressions and voice, and infers the user's emotions from the analysis results.

[0449] "Means for personalizing plan suggestions based on the user's emotions" is a function that generates travel plans that are optimal for the user's psychological state based on the results of emotion analysis.

[0450] "Means for providing real-time information on tourist spots and activities being visited" is a system for providing real-time information on activities and spots related to the tourist spots that the user is visiting.

[0451] "A means to check real-time availability of suggested activities and accommodations and make reservations" is a function that allows you to check the current availability of suggested activities and accommodations and confirm reservations on the spot.

[0452] The present invention achieves a higher level of personalization by combining a system in which a user interactively inputs their travel preferences, proposes an optimal travel plan based on those preferences, and performs all the booking and payment procedures in one go with an emotion engine that recognizes the user's emotions. An embodiment of the present invention will be described below.

[0453] The system consists of devices such as smart glasses, smartphones, and head-mounted displays, servers, databases, an emotion analysis engine, and a payment processing system. Specific hardware and software used include Google Glass, Microsoft HoloLens, iPhones, Android devices, Affectiva SDK, Google Cloud Speech-to-Text, Amazon Transcribe, Google Maps API, Mapbox, Stripe API, and PayPal.

[0454] The system operates in the following steps:

[0455] First, the user uses smart glasses, a smartphone, or a head-mounted display to input their travel preferences by voice or text. This input information is sent to the server via the device. For example, the user might input, "I want to go to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen."

[0456] The server then receives the user's input and uses the Affectiva SDK or a similar emotion analysis engine to analyze the user's facial expressions and voice to recognize their emotions. Based on the recognized emotion, the server searches a database for the best travel destinations, accommodations, and transportation options, and generates multiple itineraries. The itinerary is personalized based on the results of the emotion analysis; for example, if the server recognizes that the user wants to relax, quiet resorts will be prioritized.

[0457] The generated travel plan is sent to the device and displayed to the user. The user selects the most suitable plan from the displayed plans. The selected plan is sent back to the server, and the reservation process begins. The server then sends the reservation information to accommodation and transportation providers via API.

[0458] In addition, the user can check real-time availability information for suggested activities and accommodations, and make instant reservations. Payments are processed using Stripe API or PayPal, with credit card or electronic money payments. Once payment is complete, the server sends a confirmation message to the user, providing them with their trip details.

[0459] In addition, the results of the emotion analysis are stored in the system and used as reference data when generating future travel plans. The next time you plan a trip, more advanced personalization based on past emotion data will be possible.

[0460] Examples of specific prompts include:

[0461] "I want to travel to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[0462] In this way, the present invention can provide a highly personalized travel experience that is optimized to the user's emotions and real-time circumstances.

[0463] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0464] Step 1:

[0465] The user interactively inputs their travel preferences using smart glasses, a smartphone, or a head-mounted display. The user inputs "I want to go to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen" by voice, and the device converts this into text data. This input text is generated.

[0466] Step 2:

[0467] The device sends the generated text data to the server. The device then structures the entered travel desire data into JSON format and sends it to the server. For example, the format might be {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[0468] Step 3:

[0469] The server analyzes the received data. To determine the user's desired information, it processes the text data and uses the Affectiva SDK and Google Cloud Speech-to-Text to analyze voice and facial expression data to recognize the user's emotional state. This analysis confirms that the user is seeking relaxation.

[0470] Step 4:

[0471] The server then searches the database for multiple travel plans based on the analysis results. The search results include information on destinations, accommodations, transportation, and activities. For example, a plan for a three-night stay at a beach resort hotel in Okinawa and a round-trip flight for 30,000 yen might be included.

[0472] Step 5:

[0473] The server sends the generated itinerary to the terminal, which displays the proposed itineraries on the screen. The user can select and confirm a specific itinerary from the displayed itinerary.

[0474] Step 6:

[0475] The user selects the desired travel plan. The selected plan is retransmitted to the server via the terminal. For example, the selection might be "Resort A, 3 nights, round-trip flight included, total price 80,000 yen."

[0476] Step 7:

[0477] The server sends the reservation information to the accommodation and transportation providers according to the selected plan. For example, the reservation information is sent via API in the format {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}.

[0478] Step 8:

[0479] If the reservation is confirmed, the terminal displays a payment procedure page, where the user selects a payment method such as credit card or electronic money and enters payment information.

[0480] Step 9:

[0481] The server processes the entered payment information and executes the payment using the Stripe API or PayPal. Once payment is complete, the server sends a confirmation message to the user and provides the trip details.

[0482] Step 10:

[0483] The system stores the results of the sentiment analysis as reference data for generating future travel plans, allowing for even more personalized planning the next time you plan a trip.

[0484] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0485] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0486] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0487] [Second embodiment]

[0488] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0489] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0490] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0491] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0492] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0493] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0494] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0495] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0496] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0497] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0498] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0499] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0500] The present invention is a system that allows a user to interactively input their travel preferences, proposes an optimal travel plan based on the preferences, and performs reservation and payment procedures all at once. Specific embodiments are described below.

[0501] First, the user enters their travel preferences. Using a device such as a smartphone or computer, the user enters their travel destination, number of days, budget, activities they would like to experience, and other preferences. For example, they might enter something like, "I would like to go to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[0502] Next, the device sends this input information to the server. The data is structured in a format such as JSON and transmitted to the server. For example, the format might be {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[0503] The server analyzes the input data and searches the database for information based on the user's preferences. Specifically, it extracts information about hotels, transportation options (flights and trains), and activities suitable for the destination. For example, it uses the following SQL query:

[0504] sql

[0505] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[0506] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0507] The server generates multiple travel plans based on the search results and builds details for each plan, such as a three-night stay at a beach resort hotel in Okinawa for a total of 50,000 yen, or a round-trip flight for 30,000 yen.

[0508] The generated travel plan is sent to the terminal and displayed to the user. The user reviews each plan and selects the one that best suits their needs. Once the selection is complete, the selection is sent to the server.

[0509] The server proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. To do this, it accesses the API of each provider using an HTTP request. For example:

[0510] POST / reserve

[0511] Host: hotelprovider.com

[0512] Content-Type: application / json

[0513] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0514] POST / book

[0515] Host: airlineprovider.com

[0516] Content-Type: application / json

[0517] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0518] At the payment stage, a payment page appears on the terminal, offering options such as credit card and electronic money. The user selects a payment method and enters the required information. The server processes the payment information and sends a confirmation message to the user once the transaction is complete. A confirmation email is also automatically sent, containing payment confirmation and trip details.

[0519] For example, if you say you want to travel to Okinawa, the server will search for beach resort hotels and flights in Okinawa and generate the following three plans:

[0520] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[0521] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[0522] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[0523] If the user selects plan 1, the server will confirm the reservation and process the payment accordingly.

[0524] The above is a description of a specific embodiment for carrying out the present invention. In this way, the user can easily obtain the optimal travel plan and complete the reservation and payment procedures in one go.

[0525] The processing flow will be explained below.

[0526] Step 1:

[0527] The user inputs their travel preferences.

[0528] The user uses a device (smartphone or computer) to enter information such as the desired travel location, number of days, budget, desired activities, etc. into an input form. For example, the user enters information such as "I want to travel to Okinawa, relax at a beach resort, and my budget is 100,000 yen."

[0529] Step 2:

[0530] The terminal sends the input information to the server.

[0531] When a user submits the form, the information is sent to the server in a structured data format (e.g., JSON). An example of the data sent is {"Location": "Okinawa", "Number of Days": "3", "Budget": "100,000 yen", "Activity": "Beach Resort"}.

[0532] Step 3:

[0533] The server parses the data it receives.

[0534] The server analyzes the received travel preference data, breaks down and recognizes each item (location, number of days, budget, activity), and generates the necessary search query based on this analyzed data.

[0535] Step 4:

[0536] The server retrieves the information from the database.

[0537] Based on the analysis results, the server searches the database for the best travel destinations, accommodations, and transportation options, for example by executing the following SQL query:

[0538] sql

[0539] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[0540] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0541] Step 5:

[0542] The server generates the itinerary.

[0543] The server generates multiple optimal travel plans from the search results. Each plan includes information on accommodation, transportation, prices, and activities. It builds data to propose these plans to users.

[0544] Step 6:

[0545] The device displays the suggested itinerary.

[0546] The generated travel plans are sent to the user's device and displayed on the screen, allowing the user to compare and consider these plans.

[0547] Step 7:

[0548] The user selects a plan.

[0549] The user selects the most suitable plan from the presented plans and clicks or taps the selection button, and the selection information is sent to the server.

[0550] Step 8:

[0551] The server starts the reservation procedure for the selected plan.

[0552] The server then processes the booking of accommodation and transportation based on the plan selected by the user, sending the booking information to each provider via API. For example:

[0553] POST / reserve

[0554] Host: hotelprovider.com

[0555] Content-Type: application / json

[0556] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0557] POST / book

[0558] Host: airlineprovider.com

[0559] Content-Type: application / json

[0560] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0561] Step 9:

[0562] The server sends the reservation confirmation information to the user.

[0563] If the accommodation and transportation reservations are successful, the server notifies the user of the result. The reservation confirmation information is sent back to the user's device and a confirmation is displayed.

[0564] Step 10:

[0565] The terminal displays the payment page.

[0566] Once the reservation is confirmed, the terminal displays a payment page to the user, offering payment options such as credit card and electronic money.

[0567] Step 11:

[0568] The user enters payment information.

[0569] The user enters their payment information and makes the payment, which is then sent to the server.

[0570] Step 12:

[0571] The server processes the payment transaction.

[0572] The server processes the payment information received and completes the transaction. If successful, a payment confirmation message is sent to the user and the payment is recorded as successful in the system.

[0573] Step 13:

[0574] The server sends a confirmation email.

[0575] Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to double-check their trip details.

[0576] By following the above steps, the system automatically proposes the most suitable travel plan and processes reservations and payment procedures all in one go, simply by the user inputting their desired travel preferences.

[0577] Example 1

[0578] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0579] In today's world, responding to diverse travel needs requires searching through a wide range of information to find the best plan. However, finding the best plan from a vast amount of information takes time and effort, placing a significant burden on users. Furthermore, reservation and payment procedures must be completed separately, which is time-consuming. There is a need for a system that can solve these issues and enable users to easily obtain travel plans and complete reservations and payment procedures all in one go.

[0580] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0581] In this invention, the server includes means for a user to interactively input travel preferences, means for transmitting the input preference information in JSON format and for the server to analyze it, means for searching a database using an SQL query for optimal travel destinations, means of transportation, and accommodations based on the analysis results, means for generating multiple travel plans based on the search results and presenting them to the user, and means for reserving the plan selected by the user and performing payment procedures. This allows the user to easily obtain the optimal travel plan and complete the reservation and payment procedures all at once.

[0582] "User" refers to an individual or entity that inputs travel preferences.

[0583] "Server" refers to a device that analyzes input information sent by users, proposes optimal travel plans, and manages reservation and payment procedures.

[0584] "Terminal" refers to a smartphone, computer, or other electronic device used by a User to input their travel preferences.

[0585] "Travel preferences" refers to travel-related requests of the user, including the destination, number of days, budget, activities they would like to experience, etc.

[0586] "JSON format" is an abbreviation for JavaScript Object Notation, and refers to a lightweight data description format primarily used for data exchange.

[0587] "Parsing" refers to the process by which the server understands the user's input and extracts the necessary information.

[0588] "SQL query" is an abbreviation for Structured Query Language and refers to a question used to retrieve necessary information from a database.

[0589] "Travel plan" refers to a set of suggestions including travel destinations, transportation, accommodations, etc., generated by the server based on the user's preferences.

[0590] "Reservation" refers to the process of actually securing accommodation and transportation based on the travel plan selected by the user.

[0591] "Payment procedure" refers to the process a user goes through to pay for a travel plan.

[0592] "Generating multiple itineraries" refers to the server suggesting multiple available options based on the user's input information.

[0593] The present invention is a system that allows a user to interactively input their travel preferences, proposes an optimal travel plan based on the preferences, and performs reservation and payment procedures all at once. Specific embodiments are described below.

[0594] First, the user uses a device such as a smartphone or computer to input their travel preferences. Specifically, they enter information such as the destination, number of days, budget, and activities they want to experience through a dedicated application or web form. In this example, let's say, "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[0595] The device then structures the input information into JSON format and sends it to the server. The input data is sent to a specific endpoint on the server using an HTTP POST request. For example, the following JSON data is sent:

[0596] json

[0597] {

[0598] "Location": "Okinawa",

[0599] "days": "3",

[0600] "Budget": "100,000 yen",

[0601] "Activity": "Beach Resort"

[0602] }

[0603] The server parses the received data and analyzes the user's desired conditions. During this analysis, the server extracts conditions such as destination, number of days, budget, and activities, and generates SQL queries. The server uses these queries to search its database for relevant hotel, flight, and activity information. For example, the following SQL queries are executed:

[0604] sql

[0605] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[0606] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0607] The server generates multiple itineraries based on the information retrieved from the database, each containing accommodation, transportation, and activities that match the user's specified criteria. For example, the following itinerary may be generated:

[0608] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[0609] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[0610] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[0611] Next, the server structures the generated travel plan in JSON format and sends it to the terminal as an HTTP response. The terminal displays the received travel plan information to the user. The user selects the plan that seems best from multiple plans. The selected plan is sent back to the server, and the server proceeds with the reservation process based on that information.

[0612] The server uses an HTTP request to send the reservation information to the accommodation or transportation provider, for example, a request like this:

[0613] POST / reserve

[0614] Host: hotelprovider.com

[0615] Content-Type: application / json

[0616] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0617] POST / book

[0618] Host: airlineprovider.com

[0619] Content-Type: application / json

[0620] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0621] If the booking is successful, the terminal displays a payment page. The user selects a payment method and enters the required payment information. The server processes the payment information and sends a confirmation message to the user once the transaction is complete. A confirmation email is also automatically sent containing payment confirmation and trip details.

[0622] An example prompt sentence to be input to a generative AI model is as follows:

[0623] What travel plans would be suggested if a user said, "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[0624] The above is a specific embodiment for carrying out the present invention, which allows users to easily obtain the most suitable travel plan and complete the reservation and payment procedures all at once.

[0625] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0626] Step 1:

[0627] The user inputs their travel preferences. Specifically, they use a smartphone or computer to enter their travel destination, number of days, budget, and desired activities through a dedicated application or web form. For example, a user might enter their preference as "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen." At this stage, text information is generated as input data.

[0628] Step 2:

[0629] The terminal converts the input desired information into JSON format. For example, the information input by the user is converted into the following JSON format:

[0630] json

[0631] {

[0632] "Location": "Okinawa",

[0633] "days": "3",

[0634] "Budget": "100,000 yen",

[0635] "Activity": "Beach Resort"

[0636] }

[0637] The converted JSON data is then sent to the server via an HTTP POST request, with the terminal outputting the input data as structured data.

[0638] Step 3:

[0639] The server parses the received JSON data and analyzes the user's desired conditions. Specifically, it reads the JSON data and extracts key information such as "location," "number of days," "budget," and "activity." Based on the results of this analysis, it prepares to search for related information in the database. At this time, the server outputs the analyzed data of the user's desired conditions.

[0640] Step 4:

[0641] The server generates an SQL query based on the parsed data and searches the database for the relevant information. For example, it executes a query like this:

[0642] sql

[0643] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[0644] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0645] This query causes the server to retrieve hotel, flight, and activity data that matches the criteria from the database, and then outputs the information retrieved from the database.

[0646] Step 5:

[0647] The server generates multiple travel plans based on the acquired information. For example, it creates multiple plans combining hotels and flights in Okinawa:

[0648] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[0649] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[0650] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[0651] The server outputs detailed information about these plans in structured JSON format.

[0652] Step 6:

[0653] The server sends the generated travel plan to the terminal as an HTTP response. The terminal displays the received travel plan information to the user. At this time, the travel plan information is output to the terminal.

[0654] Step 7:

[0655] The user selects the desired plan from the multiple plans presented. When the selected plan is input into the device, the device again sends the selection information in JSON format to the server. At this time, the selection information is output as structured data.

[0656] Step 8:

[0657] The server receives the user's selection information and proceeds with the reservation process based on the selected plan. Specifically, it analyzes the selection information and sends the reservation information to the accommodation and transportation providers via HTTP request. For example, it sends the following request:

[0658] POST / reserve

[0659] Host: hotelprovider.com

[0660] Content-Type: application / json

[0661] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0662] POST / book

[0663] Host: airlineprovider.com

[0664] Content-Type: application / json

[0665] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0666] At this time, reservation confirmation information after the request is sent is output.

[0667] Step 9:

[0668] After the reservation is completed, the payment page will be displayed on the terminal. The user can select the desired payment method from the presented payment methods and enter the necessary payment information. At this time, the payment information will be entered into the terminal.

[0669] Step 10:

[0670] The server receives the payment information and processes the payment. After the payment is completed, the server sends a payment confirmation message to the user and automatically sends a confirmation email containing the payment confirmation and trip details. At this time, the payment confirmation information and trip details are output.

[0671] (Application example 1)

[0672] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0673] Currently, many travel plan suggestion and reservation systems on the market struggle to efficiently reflect user preferences, making it cumbersome to compare and select travel plans. Payment procedures are also complicated, resulting in an overall unsatisfactory user experience. Furthermore, because they use physical devices, it is difficult to provide intuitive visual feedback, making it difficult for users to easily select the optimal travel plan.

[0674] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0675] In this invention, the server includes a means for a user to interactively input their travel preferences, a means for suggesting optimal travel destinations, means of transportation, and accommodations based on the input travel preferences, a visual feedback means for displaying the proposed travel plans, a means for booking the plan selected by the user, a means for performing payment procedures during the booking process, and a means for the user to select and operate travel plans in a virtual space using a smart device. This allows users to easily input their preferences interactively and intuitively compare and select travel plans through visual feedback. Furthermore, the reservation and payment procedures can be completed seamlessly in one go by operating in the virtual space, improving the overall user experience.

[0676] The "means for the user to interactively input travel preferences" is a function that allows the user to interactively input travel preferences into the terminal through voice input or keyboard input.

[0677] "Means for suggesting optimal travel destinations, means of transportation, and accommodations based on input travel preferences" is a function that allows the server to analyze the information input by the user and suggest appropriate travel destinations, means of transportation, and accommodations.

[0678] The "visual feedback means for displaying the proposed itinerary" is a function that displays the itinerary to the user in a visual format to help the user intuitively understand it.

[0679] The "means for reserving a plan selected by the user" is a function that allows the user to select a presented travel plan and confirm a reservation based on that plan.

[0680] "Means for carrying out payment procedures during the reservation process" is a function for carrying out all the payment procedures required when reserving a travel plan in one go.

[0681] "Means for users to select and operate travel plans in a virtual space using a smart device" refers to a function that allows users to visually check and operate travel plans in a virtual reality space using a smart device such as smart glasses or a head-mounted display.

[0682] The present invention is a system that allows users to interactively input their travel preferences, proposes optimal travel plans based on those preferences, and handles reservations and payment procedures all at once. The system for implementing the present invention mainly consists of a smart device, a back-end server, and related software.

[0683] First, the user inputs their travel preferences. Using smart glasses or a head-mounted display (HMD), the user inputs their preferences using voice or gestures. For example, they might input something like, "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen." This input is analyzed in real time using voice recognition and gesture recognition technology and converted into JSON format.

[0684] Next, the device sends the input information to the server. The data is structured in JSON format and transmitted to the server. For example, it is sent in the form {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[0685] The server analyzes the input data and searches the database for information based on the user's preferences. Specifically, it extracts information about hotels, transportation options (flights and trains), and activities suitable for the destination. For example, it uses the following SQL query:

[0686] sql

[0687] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[0688] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0689] The server generates multiple travel plans based on the search results and builds details for each plan. The generated travel plans are sent to the device and displayed in the virtual space as visual feedback to the user. The user can use the HMD to review each plan and select the one that best suits their needs using gestures and voice controls.

[0690] Once the selection is complete, the selection is sent to the server. The server then proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. This involves accessing the API of each provider using an HTTP request. For example:

[0691] POST / reserve

[0692] Host: hotelprovider.com

[0693] Content-Type: application / json

[0694] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0695] POST / book

[0696] Host: airlineprovider.com

[0697] Content-Type: application / json

[0698] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0699] At the payment stage, a payment page appears on the terminal, offering options such as credit card and electronic money. The user selects a payment method and enters the required information. The server processes the payment information and sends a confirmation message to the user once the transaction is complete. A confirmation email is also automatically sent, containing payment confirmation and trip details.

[0700] For example, if you say you want to travel to Okinawa, the server will search for beach resort hotels and flights in Okinawa and generate the following three plans:

[0701] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[0702] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[0703] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[0704] If the user selects plan 1, the server will confirm the reservation and process the payment accordingly.

[0705] Examples of prompts:

[0706] "Planning my next trip: Destination is Okinawa, looking for a beach resort, budget under 100,000 yen. Any recommendations?"

[0707] In this way, users can easily obtain the optimal travel plan, make decisions through visual feedback, and complete the booking and payment procedures all in one place within the virtual space.

[0708] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0709] Step 1: The user uses a smart device to interactively input their travel preferences

[0710] Users use smart glasses or a head-mounted display to input their travel preferences using voice input or gestures. What the user says is captured by the device's microphone and sensors. The voice input is converted into text using speech recognition technology, and the data is structured into JSON format (e.g., {"Location": "Okinawa", "Number of Days": "3", "Budget": "100,000 yen", "Activity": "Beach Resort"}).

[0711] Step 2: The device sends the input information to the server

[0712] The device sends the JSON-formatted data generated in step 1 to the server, which uses an HTTP request to send the data to the backend server. The server then analyzes the data and prepares a search for a travel plan based on the user's preferences.

[0713] Step 3: The server retrieves the itinerary from the database

[0714] The server parses the received JSON data and searches the database for the appropriate itinerary (accommodation, transport, activities) using Structured Query Language (SQL). For example, the following query is executed:

[0715] sql

[0716] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[0717] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0718] The query results are temporarily stored in the server.

[0719] Step 4: The server sends the generated itinerary to the device.

[0720] The server generates multiple itineraries based on the search results, converts them into JSON format data, and sends it to the device. This data includes details of each itinerary (e.g., hotel name, price, flight information, etc.).

[0721] Step 5: The device displays the travel plan in the virtual space.

[0722] Based on the received data, the terminal displays the travel plan in a virtual space on the user's smart device (HMD or smart glasses). The displayed plan is presented in a visual format that the user can intuitively understand. The user can operate and select each plan using gestures and voice commands.

[0723] Step 6: User confirms selected plan and sends it to the server

[0724] The plan selected by the user in the virtual space is converted back into JSON format data via the device and sent to the server, which then analyzes the data and initiates the reservation process based on the selection.

[0725] Step 7: The server processes the reservation

[0726] Based on the selected plan, the server accesses the API of each provider (hotel, airline) and sends a reservation request. For example, the following API request is used:

[0727] POST / reserve

[0728] Host: hotelprovider.com

[0729] Content-Type: application / json

[0730] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0731] POST / book

[0732] Host: airlineprovider.com

[0733] Content-Type: application / json

[0734] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0735] If the reservation is successful, the result is notified to the terminal.

[0736] Step 8: Make a payment

[0737] The terminal displays a payment page, offering the user payment options such as credit card or electronic money. The user selects the desired payment method and enters the required information. The server processes the payment information and, if the payment is successful, sends the user a confirmation message with further details. A confirmation email is also automatically sent to the user.

[0738] Examples of prompts:

[0739] "Planning my next trip: Destination is Okinawa, looking for a beach resort, budget under 100,000 yen. Any recommendations?"

[0740] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0741] The present invention achieves a higher level of personalization by combining a system in which a user interactively inputs their travel preferences, proposes an optimal travel plan based on those preferences, and handles all booking and payment procedures in one go with an emotion engine that recognizes the user's emotions. Specific embodiments are described below.

[0742] First, the user enters their travel preferences. Using a device such as a smartphone or computer, the user enters information such as the travel destination, number of days, budget, and desired activities into an input form. For example, the user might enter information such as, "I want to travel to Okinawa, relax at a beach resort, and my budget is 100,000 yen."

[0743] Next, the device sends the input information to the server. The data is structured in a format such as JSON and transmitted to the server. An example of the data sent is {"Location": "Okinawa", "Number of Days": "3", "Budget": "100,000 yen", "Activity": "Beach Resort"}.

[0744] The server analyzes the received data and searches for information from a database based on the user's preferences. Specifically, it extracts information on hotels, transportation (flights or trains), and activities suitable for the travel destination. At this time, the emotion engine analyzes the user's facial expressions and voice when inputting information to evaluate the user's emotional state. For example, if the user is excited, it will prioritize suggestions with plenty of activities, or if the user wants to relax, it will prioritize suggestions for quiet resorts.

[0745] The emotion engine analyzes emotions based on the following information:

[0746] Facial expression analysis (using a camera to analyze facial movements and expressions)

[0747] Voice analysis (analyzing voice tone and speed using a microphone)

[0748] Text analysis (guessing emotions from input text)

[0749] Based on the analysis results of the emotion engine, the server generates multiple travel plans from the search results and builds information on the optimal accommodation, transportation, price, and activities for each plan. For example, a plan might include a three-night stay at a beach resort hotel in Okinawa for a total of 50,000 yen, or a round-trip flight for 30,000 yen.

[0750] The generated travel plan is sent to the terminal and displayed to the user. The user reviews each plan and selects the one that best suits their needs. Once the selection is complete, the selection is sent back to the server.

[0751] The server proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. At this time, it accesses each provider via API and sends the reservation information. For example:

[0752] POST / reserve

[0753] Host: hotelprovider.com

[0754] Content-Type: application / json

[0755] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0756] POST / book

[0757] Host: airlineprovider.com

[0758] Content-Type: application / json

[0759] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0760] Once the reservation is confirmed, the terminal displays a payment procedure page, where options such as credit card and electronic money are offered. The user enters their payment information and completes the payment. The payment information is then sent to the server.

[0761] The server processes the payment information it receives, sends a confirmation message to the user if the payment is successful, and records the payment as successful within the system. Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to review their trip details.

[0762] The analysis results of the emotion engine are saved as reference data for the next travel plan generation. For example, if a user previously expressed the emotion "I want to relax," that information will be used to suggest future plans.

[0763] Through the above steps, the present invention allows users to simply input their travel preferences, automatically suggest optimal travel plans, and process reservations and payment procedures all in one go. Furthermore, by utilizing an emotion engine, it is possible to provide a personalized experience that matches the user's emotions.

[0764] The processing flow will be explained below.

[0765] Step 1:

[0766] The user inputs their travel preferences.

[0767] The user uses a device (smartphone or computer) to input the desired travel destination, number of days, budget, desired activities, etc. For example, the user inputs information such as "I want to travel to Okinawa, relax at a beach resort, and my budget is 100,000 yen."

[0768] Step 2:

[0769] The terminal transmits the input information and the user's emotion information to the server.

[0770] Once the input is complete, the device sends the input information in JSON format, along with the user's emotional data (facial expression and voice analysis data), to the server. For example, the emotional data is sent as {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[0771] Step 3:

[0772] The server analyzes the travel preference data.

[0773] The server analyzes the received input data and recognizes each item (location, number of days, budget, activity), and generates a query for database search based on the analysis results.

[0774] Step 4:

[0775] The emotion engine analyzes the user's emotion data.

[0776] The emotion engine in the server analyzes the user's facial expression data and voice data to identify the user's emotional state (excited, relaxed, anxious, etc.). For example, it is determined that the user is in a relaxed mode.

[0777] Step 5:

[0778] The server retrieves information from the database based on the results of the emotion engine.

[0779] The server takes into account the results of the emotion engine and searches the database for the most suitable travel plans. For example, if the emotion of wanting to relax is detected, it will prioritize plans that include quiet beach resorts and spas. An example SQL query is as follows:

[0780] sql

[0781] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000 AND features LIKE '%Relax%';

[0782] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0783] Step 6:

[0784] The server generates the itinerary.

[0785] The server generates multiple travel plans based on the search results, incorporating information on the best accommodations, transportation, prices, and activities for each plan, such as a three-night stay at a beach resort hotel in Okinawa for a total of 50,000 yen, or a round-trip flight for 30,000 yen.

[0786] Step 7:

[0787] The device displays the suggested itinerary.

[0788] The generated travel plans are sent to the terminal and displayed on the user's screen, allowing the user to compare and consider these plans.

[0789] Step 8:

[0790] The user selects a plan.

[0791] The user selects the desired plan from the presented plans and clicks or taps the selection button. The selection information is sent to the server.

[0792] Step 9:

[0793] The server starts the reservation procedure for the selected plan.

[0794] The server processes the reservations for accommodation and transportation based on the plan selected by the user. This is done via the API of each provider. For example, it sends the following request:

[0795] POST / reserve

[0796] Host: hotelprovider.com

[0797] Content-Type: application / json

[0798] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0799] POST / book

[0800] Host: airlineprovider.com

[0801] Content-Type: application / json

[0802] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0803] Step 10:

[0804] The server sends the reservation confirmation information to the user.

[0805] If the accommodation and transportation reservations are successful, the result is notified to the user's terminal. The server generates and transmits data including reservation confirmation information.

[0806] Step 11:

[0807] The terminal displays the payment page.

[0808] Once the reservation is confirmed, the terminal displays a payment procedure page to the user, offering payment methods such as credit card and electronic money.

[0809] Step 12:

[0810] The user enters payment information.

[0811] The user enters their payment information and completes the transaction, which is then sent to the server.

[0812] Step 13:

[0813] The server processes the payment transaction.

[0814] The server processes the received payment information, sends a confirmation message to the user when the payment is complete, and records the payment completion information in the system.

[0815] Step 14:

[0816] The server sends a confirmation email.

[0817] Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to double-check their trip details.

[0818] Step 15:

[0819] The emotion engine stores the emotion data.

[0820] The emotion engine saves the user's emotional data and uses it as reference data when generating the next travel plan. For example, if a user previously expressed the emotion "I want to relax," that information will be used to suggest future plans.

[0821] Through the above steps, the present invention allows users to simply input their travel preferences, automatically suggest optimal travel plans, and process reservations and payment procedures all in one go. Furthermore, by utilizing an emotion engine, it is possible to provide a personalized experience that matches the user's emotions.

[0822] Example 2

[0823] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0824] Many conventional travel reservation systems propose travel plans based on the user's preferences and handle reservations and payment procedures, but because they do not take the user's emotional state into consideration, they have the problem of making it difficult to provide a personalized experience that matches the user's emotions.In addition, because plans are proposed based only on information entered by the user, they may not always propose the optimal travel plan.

[0825] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0826] In this invention, the server includes means for allowing a user to interactively input travel preferences, means for proposing optimal travel destinations, means of transportation, and accommodations based on the input travel preferences, and means for customizing a travel plan based on the user's emotional state using an analysis device for analyzing the user's emotions. This makes it possible to propose an optimal travel plan based on the user's emotions and to complete reservations and payment procedures all at once.

[0827] "User" refers to a person who uses this system, who inputs travel preferences, selects a proposed plan, and completes payment procedures.

[0828] "Travel preferences" refers to information that a user interactively inputs, such as travel destination, number of days, budget, desired activities, etc.

[0829] "Interactive" refers to a format in which a user interacts with a system to input information.

[0830] "Means of suggestion" refers to a method for selecting the most suitable travel destination, means of transportation, and accommodation based on the input information and presenting them to the user.

[0831] The term "means for making a reservation" refers to a method for carrying out the procedure for actually making a reservation for the travel plan selected by the user.

[0832] "Means for making payment" refers to the method by which a user makes payment after completing a travel reservation.

[0833] "Analysis device" refers to a device that analyzes facial expressions, voice, and text data input by a user and evaluates the user's emotional state.

[0834] "Means for customization" refers to ways to adjust or change travel plans based on the user's emotional state.

[0835] "Server" refers to the computer system that processes and analyzes input data, generates suggested travel plans, and provides them to users.

[0836] A "database" refers to a collection of information that stores information on travel destinations, accommodations, means of transportation, etc., and that can be searched and extracted.

[0837] "Emotional state" refers to the psychological and emotional state analyzed from the user's facial expressions, voice, and text input.

[0838] This invention achieves a higher level of personalization by combining a system in which a user interactively inputs their travel preferences, proposes an optimal travel plan based on those preferences, and performs all the booking and payment procedures in one go with an emotion engine that recognizes the user's emotions. Specific embodiments of the present invention are described in detail below.

[0839] First, the user uses a device such as a smartphone or computer to input their travel preferences. They enter information such as the travel destination, number of days, budget, and desired activities into an input form. For example, they might enter information like, "I want to travel to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen."

[0840] The device then sends the input information to the server, which converts the data into a structured data format such as JSON and transmits it to the server.

[0841] The server analyzes the received data and searches for information from a database based on the user's preferences. Specifically, it extracts information about hotels, transportation options (flights or trains), and activities suitable for the travel destination. At this time, the emotion engine analyzes the user's facial expressions and voice when they input information and evaluates their emotional state. The emotion engine analyzes the user's emotions using technologies such as facial expression analysis (using a camera to analyze facial movements and expressions), voice analysis (using a microphone to analyze the tone and speed of voice), and text analysis (inferring emotions from input text).

[0842] For example, if a user inputs "I want to travel to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen," and their facial expression is relaxed, the server will prioritize suggesting quiet resorts. As a specific example, if the emotion engine reads the emotion "I want to relax" from the user's facial expression, the server will prioritize generating plans that include quiet beach resort hotels in Okinawa. Also, if a user inputs "I want to travel to Okinawa. I want to enjoy the beach and diving as activities," and their facial expression shows signs of excitement, the server will suggest plans with plenty of activities.

[0843] The generated travel plans are sent from the server to the terminal as multiple options and displayed to the user. The user reviews each plan and selects the one that best suits their needs. Once the selection is complete, the selection is sent back to the server.

[0844] The server then proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. At this time, it accesses each provider via API and sends the reservation information. For example, the following request is made:

[0845] POST / reserve

[0846] Host: hotelprovider.com

[0847] Content-Type: application / json

[0848] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0849] Once the reservation is confirmed, the terminal displays a payment procedure page, where options such as credit card and electronic money are offered. The user enters their payment information and completes the payment. The payment information is then sent to the server.

[0850] The server processes the payment information it receives, sends a confirmation message to the user if the payment is successful, and records the payment as successful within the system. Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to review their trip details.

[0851] The analysis results of the emotion engine are saved as reference data for the next travel plan generation. For example, if a user previously expressed the emotion "I want to relax," that information will be used to suggest future plans.

[0852] Through the above steps, the present invention allows users to simply input their travel preferences, automatically suggest optimal travel plans, and process reservations and payment procedures all in one go. Furthermore, by utilizing an emotion engine, it is possible to provide a personalized experience that matches the user's emotions.

[0853] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0854] Step 1:

[0855] The user inputs his / her travel preferences.

[0856] Input: The user uses a smartphone or computer to enter information such as the travel destination, number of days, budget, desired activities, etc. into an input form. For example, the user might enter information such as, "I want to travel to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen."

[0857] Output: The input information is temporarily stored in the device's memory.

[0858] Step 2:

[0859] The terminal sends the input information to the server.

[0860] Input: Travel preference information entered by the user.

[0861] Data processing: Converting input information into structured data such as JSON format.

[0862] Output: The converted data is sent to the server. For example, the following data is sent: {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[0863] Step 3:

[0864] The server receives and analyzes the data.

[0865] Input: JSON data of travel preference information sent from the terminal.

[0866] Data calculation: Data is analyzed to extract information such as travel destination, budget, number of days, and desired activities. Furthermore, an emotion engine is activated at this time to evaluate the user's emotional state based on their facial expressions and voice input. The emotion engine uses facial expression analysis technology (using the camera), voice analysis technology (using the microphone), and text analysis technology.

[0867] Output: The analysis results in information that reflects the user's wishes and emotional state.

[0868] Step 4:

[0869] The server searches the database and generates the optimal travel plan.

[0870] Input: Travel preference information and emotional evaluation data.

[0871] Data calculation: Searches the database for hotels, transportation (flights and trains), and activity information suitable for the travel destination, and combines this data to generate multiple travel plans. For example, if a user is rated as wanting to relax, a plan including a quiet beach resort hotel will be generated.

[0872] Output: Multiple itineraries are generated and their details are stored as data.

[0873] Step 5:

[0874] The server transmits the generated travel plan to the terminal.

[0875] Input: The generated itinerary data.

[0876] Data processing: Convert travel plan data into JSON format or other format for sending to the device.

[0877] Output: The travel plan sent to the device is displayed on the user's UI. For example, a three-night stay at a beach resort hotel in Okinawa (total price: 50,000 yen) and a round-trip flight (30,000 yen) are displayed as a list.

[0878] Step 6:

[0879] The user reviews and selects a travel plan.

[0880] Input: Multiple travel plans displayed on the device.

[0881] How it works: The user reviews each plan in detail and selects the one that best suits their needs.

[0882] Output: The data of the selected plan is sent from the device to the server.

[0883] Step 7:

[0884] The server will proceed with the reservation procedure for the selected plan.

[0885] Input: Travel plan data selected by the user.

[0886] Data Computing: Generate API requests to send booking information to accommodation and transportation providers. For example, send requests to hotel and airline booking APIs.

[0887] Output: The reservation information is sent to the provider, and if successful, the result is returned to the server and the user is notified.

[0888] Step 8:

[0889] The user completes the payment process.

[0890] Input: Information about your confirmed travel plan and the payment options displayed on the payment page (credit card, e-money, etc.).

[0891] How it works: The user enters payment information and makes a payment.

[0892] Output: Payment information is sent from the terminal to the server.

[0893] Step 9:

[0894] The server processes the payment and sends a confirmation email.

[0895] Input: Payment information submitted by the user.

[0896] Data calculation: Executes the payment process and checks whether the payment is successful. If successful, records the payment as completed in the system.

[0897] Output: A confirmation message confirming payment is completed and a confirmation email containing the trip details is sent to the user.

[0898] (Application example 2)

[0899] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0900] Conventional travel plan suggestion systems have struggled to provide optimal travel experiences because they lack the ability to personalize based on users' emotions and real-time situations. Furthermore, they lack the ability to obtain and suggest detailed information about travel destinations and activities in real time, making it difficult to respond to users' changing needs. Furthermore, it has been difficult to provide real-time guidance and reservation functions during travel.

[0901] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recognizing emotions by analyzing the user's facial expressions and voice, means for personalizing plan suggestions based on the user's emotions, means for providing real-time information on tourist spots and activities being visited, and means for checking real-time availability information on proposed activities and accommodations and making reservations. This makes it possible to provide a personalized travel experience optimized for the user's emotions and real-time situation.

[0902] A "means for interactively inputting user travel preferences" is a device or software that allows a user to input travel preferences via voice or text through an interface.

[0903] "Means for suggesting optimal travel destinations, means of transportation, and accommodations based on input travel preferences" refers to an algorithm or software function that analyzes the desired information input by the user and selects and suggests optimal travel destinations, means of transportation, and accommodations accordingly.

[0904] "Means for displaying proposed travel plans and reserving the plan selected by the user" is a function that displays the travel plan selected by the user on the screen and allows the user to proceed with the reservation procedure for that plan after selection.

[0905] The "means for carrying out payment procedures during the reservation process" refers to a processing means for making payment for the travel plan selected by the user using a credit card or electronic money.

[0906] "Means for recognizing emotions by analyzing a user's facial expressions and voice" refers to technology that uses a user's camera or microphone to analyze their facial expressions and voice, and infers the user's emotions from the analysis results.

[0907] "Means for personalizing plan suggestions based on the user's emotions" is a function that generates travel plans that are optimal for the user's psychological state based on the results of emotion analysis.

[0908] "Means for providing real-time information on tourist spots and activities being visited" is a system for providing real-time information on activities and spots related to the tourist spots that the user is visiting.

[0909] "A means to check real-time availability of suggested activities and accommodations and make reservations" is a function that allows you to check the current availability of suggested activities and accommodations and confirm reservations on the spot.

[0910] The present invention achieves a higher level of personalization by combining a system in which a user interactively inputs their travel preferences, proposes an optimal travel plan based on those preferences, and performs all the booking and payment procedures in one go with an emotion engine that recognizes the user's emotions. An embodiment of the present invention will be described below.

[0911] The system consists of devices such as smart glasses, smartphones, and head-mounted displays, servers, databases, an emotion analysis engine, and a payment processing system. Specific hardware and software used include Google Glass, Microsoft HoloLens, iPhones, Android devices, Affectiva SDK, Google Cloud Speech-to-Text, Amazon Transcribe, Google Maps API, Mapbox, Stripe API, and PayPal.

[0912] The system operates in the following steps:

[0913] First, the user uses smart glasses, a smartphone, or a head-mounted display to input their travel preferences by voice or text. This input information is sent to the server via the device. For example, the user might input, "I want to go to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen."

[0914] The server then receives the user's input and uses the Affectiva SDK or a similar emotion analysis engine to analyze the user's facial expressions and voice to recognize their emotions. Based on the recognized emotion, the server searches a database for the best travel destinations, accommodations, and transportation options, and generates multiple itineraries. The itinerary is personalized based on the results of the emotion analysis; for example, if the server recognizes that the user wants to relax, quiet resorts will be prioritized.

[0915] The generated travel plan is sent to the device and displayed to the user. The user selects the most suitable plan from the displayed plans. The selected plan is sent back to the server, and the reservation process begins. The server then sends the reservation information to accommodation and transportation providers via API.

[0916] In addition, the user can check real-time availability information for suggested activities and accommodations, and make instant reservations. Payments are processed using Stripe API or PayPal, with credit card or electronic money payments. Once payment is complete, the server sends a confirmation message to the user, providing them with their trip details.

[0917] In addition, the results of the emotion analysis are stored in the system and used as reference data when generating future travel plans. The next time you plan a trip, more advanced personalization based on past emotion data will be possible.

[0918] Examples of specific prompts include:

[0919] "I want to travel to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[0920] In this way, the present invention can provide a highly personalized travel experience that is optimized to the user's emotions and real-time circumstances.

[0921] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0922] Step 1:

[0923] The user interactively inputs their travel preferences using smart glasses, a smartphone, or a head-mounted display. The user inputs "I want to go to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen" by voice, and the device converts this into text data. This input text is generated.

[0924] Step 2:

[0925] The device sends the generated text data to the server. The device then structures the entered travel desire data into JSON format and sends it to the server. For example, the format might be {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[0926] Step 3:

[0927] The server analyzes the received data. To determine the user's desired information, it processes the text data and uses the Affectiva SDK and Google Cloud Speech-to-Text to analyze voice and facial expression data to recognize the user's emotional state. This analysis confirms that the user is seeking relaxation.

[0928] Step 4:

[0929] The server then searches the database for multiple travel plans based on the analysis results. The search results include information on destinations, accommodations, transportation, and activities. For example, a plan for a three-night stay at a beach resort hotel in Okinawa and a round-trip flight for 30,000 yen might be included.

[0930] Step 5:

[0931] The server sends the generated itinerary to the terminal, which displays the proposed itineraries on the screen. The user can select and confirm a specific itinerary from the displayed itinerary.

[0932] Step 6:

[0933] The user selects the desired travel plan. The selected plan is retransmitted to the server via the terminal. For example, the selection might be "Resort A, 3 nights, round-trip flight included, total price 80,000 yen."

[0934] Step 7:

[0935] The server sends the reservation information to the accommodation and transportation providers according to the selected plan. For example, the reservation information is sent via API in the format {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}.

[0936] Step 8:

[0937] If the reservation is confirmed, the terminal displays a payment procedure page, where the user selects a payment method such as credit card or electronic money and enters payment information.

[0938] Step 9:

[0939] The server processes the entered payment information and executes the payment using the Stripe API or PayPal. Once payment is complete, the server sends a confirmation message to the user and provides the trip details.

[0940] Step 10:

[0941] The system stores the results of the sentiment analysis as reference data for generating future travel plans, allowing for even more personalized planning the next time you plan a trip.

[0942] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0943] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0944] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0945] [Third embodiment]

[0946] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0947] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0948] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0949] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0950] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0951] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0952] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0953] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0954] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0955] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0956] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0957] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0958] The present invention is a system that allows a user to interactively input their travel preferences, proposes an optimal travel plan based on the preferences, and performs reservation and payment procedures all at once. Specific embodiments are described below.

[0959] First, the user enters their travel preferences. Using a device such as a smartphone or computer, the user enters their travel destination, number of days, budget, activities they would like to experience, and other preferences. For example, they might enter something like, "I would like to go to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[0960] Next, the device sends this input information to the server. The data is structured in a format such as JSON and transmitted to the server. For example, the format might be {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[0961] The server analyzes the input data and searches the database for information based on the user's preferences. Specifically, it extracts information about hotels, transportation options (flights and trains), and activities suitable for the destination. For example, it uses the following SQL query:

[0962] sql

[0963] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[0964] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0965] The server generates multiple travel plans based on the search results and builds details for each plan, such as a three-night stay at a beach resort hotel in Okinawa for a total of 50,000 yen, or a round-trip flight for 30,000 yen.

[0966] The generated travel plan is sent to the terminal and displayed to the user. The user reviews each plan and selects the one that best suits their needs. Once the selection is complete, the selection is sent to the server.

[0967] The server proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. To do this, it accesses the API of each provider using an HTTP request. For example:

[0968] POST / reserve

[0969] Host: hotelprovider.com

[0970] Content-Type: application / json

[0971] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[0972] POST / book

[0973] Host: airlineprovider.com

[0974] Content-Type: application / json

[0975] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[0976] At the payment stage, a payment page appears on the terminal, offering options such as credit card and electronic money. The user selects a payment method and enters the required information. The server processes the payment information and sends a confirmation message to the user once the transaction is complete. A confirmation email is also automatically sent, containing payment confirmation and trip details.

[0977] For example, if you say you want to travel to Okinawa, the server will search for beach resort hotels and flights in Okinawa and generate the following three plans:

[0978] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[0979] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[0980] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[0981] If the user selects plan 1, the server will confirm the reservation and process the payment accordingly.

[0982] The above is a description of a specific embodiment for carrying out the present invention. In this way, the user can easily obtain the optimal travel plan and complete the reservation and payment procedures in one go.

[0983] The processing flow will be explained below.

[0984] Step 1:

[0985] The user inputs their travel preferences.

[0986] The user uses a device (smartphone or computer) to enter information such as the desired travel location, number of days, budget, desired activities, etc. into an input form. For example, the user enters information such as "I want to travel to Okinawa, relax at a beach resort, and my budget is 100,000 yen."

[0987] Step 2:

[0988] The terminal sends the input information to the server.

[0989] When a user submits the form, the information is sent to the server in a structured data format (e.g., JSON). An example of the data sent is {"Location": "Okinawa", "Number of Days": "3", "Budget": "100,000 yen", "Activity": "Beach Resort"}.

[0990] Step 3:

[0991] The server parses the data it receives.

[0992] The server analyzes the received travel preference data, breaks down and recognizes each item (location, number of days, budget, activity), and generates the necessary search query based on this analyzed data.

[0993] Step 4:

[0994] The server retrieves the information from the database.

[0995] Based on the analysis results, the server searches the database for the best travel destinations, accommodations, and transportation options, for example by executing the following SQL query:

[0996] sql

[0997] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[0998] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[0999] Step 5:

[1000] The server generates the itinerary.

[1001] The server generates multiple optimal travel plans from the search results. Each plan includes information on accommodation, transportation, prices, and activities. It builds data to propose these plans to users.

[1002] Step 6:

[1003] The device displays the suggested itinerary.

[1004] The generated travel plans are sent to the user's device and displayed on the screen, allowing the user to compare and consider these plans.

[1005] Step 7:

[1006] The user selects a plan.

[1007] The user selects the most suitable plan from the presented plans and clicks or taps the selection button, and the selection information is sent to the server.

[1008] Step 8:

[1009] The server starts the reservation procedure for the selected plan.

[1010] The server then processes the booking of accommodation and transportation based on the plan selected by the user, sending the booking information to each provider via API. For example:

[1011] POST / reserve

[1012] Host: hotelprovider.com

[1013] Content-Type: application / json

[1014] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1015] POST / book

[1016] Host: airlineprovider.com

[1017] Content-Type: application / json

[1018] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[1019] Step 9:

[1020] The server sends the reservation confirmation information to the user.

[1021] If the accommodation and transportation reservations are successful, the server notifies the user of the result. The reservation confirmation information is sent back to the user's device and a confirmation is displayed.

[1022] Step 10:

[1023] The terminal displays the payment page.

[1024] Once the reservation is confirmed, the terminal displays a payment page to the user, offering payment options such as credit card and electronic money.

[1025] Step 11:

[1026] The user enters payment information.

[1027] The user enters their payment information and makes the payment, which is then sent to the server.

[1028] Step 12:

[1029] The server processes the payment transaction.

[1030] The server processes the payment information received and completes the transaction. If successful, a payment confirmation message is sent to the user and the payment is recorded as successful in the system.

[1031] Step 13:

[1032] The server sends a confirmation email.

[1033] Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to double-check their trip details.

[1034] By following the above steps, the system automatically proposes the most suitable travel plan and processes reservations and payment procedures all in one go, simply by the user inputting their desired travel preferences.

[1035] Example 1

[1036] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1037] In today's world, responding to diverse travel needs requires searching through a wide range of information to find the best plan. However, finding the best plan from a vast amount of information takes time and effort, placing a significant burden on users. Furthermore, reservation and payment procedures must be completed separately, which is time-consuming. There is a need for a system that can solve these issues and enable users to easily obtain travel plans and complete reservations and payment procedures all in one go.

[1038] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1039] In this invention, the server includes means for a user to interactively input travel preferences, means for transmitting the input preference information in JSON format and for the server to analyze it, means for searching a database using an SQL query for optimal travel destinations, means of transportation, and accommodations based on the analysis results, means for generating multiple travel plans based on the search results and presenting them to the user, and means for reserving the plan selected by the user and performing payment procedures. This allows the user to easily obtain the optimal travel plan and complete the reservation and payment procedures all at once.

[1040] "User" refers to an individual or entity that inputs travel preferences.

[1041] "Server" refers to a device that analyzes input information sent by users, proposes optimal travel plans, and manages reservation and payment procedures.

[1042] "Terminal" refers to a smartphone, computer, or other electronic device used by a User to input their travel preferences.

[1043] "Travel preferences" refers to travel-related requests of the user, including the destination, number of days, budget, activities they would like to experience, etc.

[1044] "JSON format" is an abbreviation for JavaScript Object Notation, and refers to a lightweight data description format primarily used for data exchange.

[1045] "Parsing" refers to the process by which the server understands the user's input and extracts the necessary information.

[1046] "SQL query" is an abbreviation for Structured Query Language and refers to a question used to retrieve necessary information from a database.

[1047] "Travel plan" refers to a set of suggestions including travel destinations, transportation, accommodations, etc., generated by the server based on the user's preferences.

[1048] "Reservation" refers to the process of actually securing accommodation and transportation based on the travel plan selected by the user.

[1049] "Payment procedure" refers to the process a user goes through to pay for a travel plan.

[1050] "Generating multiple itineraries" refers to the server suggesting multiple available options based on the user's input information.

[1051] The present invention is a system that allows a user to interactively input their travel preferences, proposes an optimal travel plan based on the preferences, and performs reservation and payment procedures all at once. Specific embodiments are described below.

[1052] First, the user uses a device such as a smartphone or computer to input their travel preferences. Specifically, they enter information such as the destination, number of days, budget, and activities they want to experience through a dedicated application or web form. In this example, let's say, "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[1053] The device then structures the input information into JSON format and sends it to the server. The input data is sent to a specific endpoint on the server using an HTTP POST request. For example, the following JSON data is sent:

[1054] json

[1055] {

[1056] "Location": "Okinawa",

[1057] "days": "3",

[1058] "Budget": "100,000 yen",

[1059] "Activity": "Beach Resort"

[1060] }

[1061] The server parses the received data and analyzes the user's desired conditions. During this analysis, the server extracts conditions such as destination, number of days, budget, and activities, and generates SQL queries. The server uses these queries to search its database for relevant hotel, flight, and activity information. For example, the following SQL queries are executed:

[1062] sql

[1063] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[1064] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[1065] The server generates multiple itineraries based on the information retrieved from the database, each containing accommodation, transportation, and activities that match the user's specified criteria. For example, the following itinerary may be generated:

[1066] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[1067] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[1068] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[1069] Next, the server structures the generated travel plan in JSON format and sends it to the terminal as an HTTP response. The terminal displays the received travel plan information to the user. The user selects the plan that seems best from multiple plans. The selected plan is sent back to the server, and the server proceeds with the reservation process based on that information.

[1070] The server uses an HTTP request to send the reservation information to the accommodation or transportation provider, for example, a request like this:

[1071] POST / reserve

[1072] Host: hotelprovider.com

[1073] Content-Type: application / json

[1074] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1075] POST / book

[1076] Host: airlineprovider.com

[1077] Content-Type: application / json

[1078] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[1079] If the booking is successful, the terminal displays a payment page. The user selects a payment method and enters the required payment information. The server processes the payment information and sends a confirmation message to the user once the transaction is complete. A confirmation email is also automatically sent containing payment confirmation and trip details.

[1080] An example prompt sentence to be input to a generative AI model is as follows:

[1081] What travel plans would be suggested if a user said, "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[1082] The above is a specific embodiment for carrying out the present invention, which allows users to easily obtain the most suitable travel plan and complete the reservation and payment procedures all at once.

[1083] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1084] Step 1:

[1085] The user inputs their travel preferences. Specifically, they use a smartphone or computer to enter their travel destination, number of days, budget, and desired activities through a dedicated application or web form. For example, a user might enter their preference as "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen." At this stage, text information is generated as input data.

[1086] Step 2:

[1087] The terminal converts the input desired information into JSON format. For example, the information input by the user is converted into the following JSON format:

[1088] json

[1089] {

[1090] "Location": "Okinawa",

[1091] "days": "3",

[1092] "Budget": "100,000 yen",

[1093] "Activity": "Beach Resort"

[1094] }

[1095] The converted JSON data is then sent to the server via an HTTP POST request, with the terminal outputting the input data as structured data.

[1096] Step 3:

[1097] The server parses the received JSON data and analyzes the user's desired conditions. Specifically, it reads the JSON data and extracts key information such as "location," "number of days," "budget," and "activity." Based on the results of this analysis, it prepares to search for related information in the database. At this time, the server outputs the analyzed data of the user's desired conditions.

[1098] Step 4:

[1099] The server generates an SQL query based on the parsed data and searches the database for the relevant information. For example, it executes a query like this:

[1100] sql

[1101] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[1102] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[1103] This query causes the server to retrieve hotel, flight, and activity data that matches the criteria from the database, and then outputs the information retrieved from the database.

[1104] Step 5:

[1105] The server generates multiple travel plans based on the acquired information. For example, it creates multiple plans combining hotels and flights in Okinawa:

[1106] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[1107] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[1108] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[1109] The server outputs detailed information about these plans in structured JSON format.

[1110] Step 6:

[1111] The server sends the generated travel plan to the terminal as an HTTP response. The terminal displays the received travel plan information to the user. At this time, the travel plan information is output to the terminal.

[1112] Step 7:

[1113] The user selects the desired plan from the multiple plans presented. When the selected plan is input into the device, the device again sends the selection information in JSON format to the server. At this time, the selection information is output as structured data.

[1114] Step 8:

[1115] The server receives the user's selection information and proceeds with the reservation process based on the selected plan. Specifically, it analyzes the selection information and sends the reservation information to the accommodation and transportation providers via HTTP request. For example, it sends the following request:

[1116] POST / reserve

[1117] Host: hotelprovider.com

[1118] Content-Type: application / json

[1119] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1120] POST / book

[1121] Host: airlineprovider.com

[1122] Content-Type: application / json

[1123] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[1124] At this time, reservation confirmation information after the request is sent is output.

[1125] Step 9:

[1126] After the reservation is completed, the payment page will be displayed on the terminal. The user can select the desired payment method from the presented payment methods and enter the necessary payment information. At this time, the payment information will be entered into the terminal.

[1127] Step 10:

[1128] The server receives the payment information and processes the payment. After the payment is completed, the server sends a payment confirmation message to the user and automatically sends a confirmation email containing the payment confirmation and trip details. At this time, the payment confirmation information and trip details are output.

[1129] (Application example 1)

[1130] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1131] Currently, many travel plan suggestion and reservation systems on the market struggle to efficiently reflect user preferences, making it cumbersome to compare and select travel plans. Payment procedures are also complicated, resulting in an overall unsatisfactory user experience. Furthermore, because they use physical devices, it is difficult to provide intuitive visual feedback, making it difficult for users to easily select the optimal travel plan.

[1132] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1133] In this invention, the server includes a means for a user to interactively input their travel preferences, a means for suggesting optimal travel destinations, means of transportation, and accommodations based on the input travel preferences, a visual feedback means for displaying the proposed travel plans, a means for booking the plan selected by the user, a means for performing payment procedures during the booking process, and a means for the user to select and operate travel plans in a virtual space using a smart device. This allows users to easily input their preferences interactively and intuitively compare and select travel plans through visual feedback. Furthermore, the reservation and payment procedures can be completed seamlessly in one go by operating in the virtual space, improving the overall user experience.

[1134] The "means for the user to interactively input travel preferences" is a function that allows the user to interactively input travel preferences into the terminal through voice input or keyboard input.

[1135] "Means for suggesting optimal travel destinations, means of transportation, and accommodations based on input travel preferences" is a function that allows the server to analyze the information input by the user and suggest appropriate travel destinations, means of transportation, and accommodations.

[1136] The "visual feedback means for displaying the proposed itinerary" is a function that displays the itinerary to the user in a visual format to help the user intuitively understand it.

[1137] The "means for reserving a plan selected by the user" is a function that allows the user to select a presented travel plan and confirm a reservation based on that plan.

[1138] "Means for carrying out payment procedures during the reservation process" is a function for carrying out all the payment procedures required when reserving a travel plan in one go.

[1139] "Means for users to select and operate travel plans in a virtual space using a smart device" refers to a function that allows users to visually check and operate travel plans in a virtual reality space using a smart device such as smart glasses or a head-mounted display.

[1140] The present invention is a system that allows users to interactively input their travel preferences, proposes optimal travel plans based on those preferences, and handles reservations and payment procedures all at once. The system for implementing the present invention mainly consists of a smart device, a back-end server, and related software.

[1141] First, the user inputs their travel preferences. Using smart glasses or a head-mounted display (HMD), the user inputs their preferences using voice or gestures. For example, they might input something like, "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen." This input is analyzed in real time using voice recognition and gesture recognition technology and converted into JSON format.

[1142] Next, the device sends the input information to the server. The data is structured in JSON format and transmitted to the server. For example, it is sent in the form {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[1143] The server analyzes the input data and searches the database for information based on the user's preferences. Specifically, it extracts information about hotels, transportation options (flights and trains), and activities suitable for the destination. For example, it uses the following SQL query:

[1144] sql

[1145] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[1146] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[1147] The server generates multiple travel plans based on the search results and builds details for each plan. The generated travel plans are sent to the device and displayed in the virtual space as visual feedback to the user. The user can use the HMD to review each plan and select the one that best suits their needs using gestures and voice controls.

[1148] Once the selection is complete, the selection is sent to the server. The server then proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. This involves accessing the API of each provider using an HTTP request. For example:

[1149] POST / reserve

[1150] Host: hotelprovider.com

[1151] Content-Type: application / json

[1152] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1153] POST / book

[1154] Host: airlineprovider.com

[1155] Content-Type: application / json

[1156] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[1157] At the payment stage, a payment page appears on the terminal, offering options such as credit card and electronic money. The user selects a payment method and enters the required information. The server processes the payment information and sends a confirmation message to the user once the transaction is complete. A confirmation email is also automatically sent, containing payment confirmation and trip details.

[1158] For example, if you say you want to travel to Okinawa, the server will search for beach resort hotels and flights in Okinawa and generate the following three plans:

[1159] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[1160] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[1161] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[1162] If the user selects plan 1, the server will confirm the reservation and process the payment accordingly.

[1163] Examples of prompts:

[1164] "Planning my next trip: Destination is Okinawa, looking for a beach resort, budget under 100,000 yen. Any recommendations?"

[1165] In this way, users can easily obtain the optimal travel plan, make decisions through visual feedback, and complete the booking and payment procedures all in one place within the virtual space.

[1166] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1167] Step 1: The user uses a smart device to interactively input their travel preferences

[1168] Users use smart glasses or a head-mounted display to input their travel preferences using voice input or gestures. What the user says is captured by the device's microphone and sensors. The voice input is converted into text using speech recognition technology, and the data is structured into JSON format (e.g., {"Location": "Okinawa", "Number of Days": "3", "Budget": "100,000 yen", "Activity": "Beach Resort"}).

[1169] Step 2: The device sends the input information to the server

[1170] The device sends the JSON-formatted data generated in step 1 to the server, which uses an HTTP request to send the data to the backend server. The server then analyzes the data and prepares a search for a travel plan based on the user's preferences.

[1171] Step 3: The server retrieves the itinerary from the database

[1172] The server parses the received JSON data and searches the database for the appropriate itinerary (accommodation, transport, activities) using Structured Query Language (SQL). For example, the following query is executed:

[1173] sql

[1174] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[1175] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[1176] The query results are temporarily stored in the server.

[1177] Step 4: The server sends the generated itinerary to the device.

[1178] The server generates multiple itineraries based on the search results, converts them into JSON format data, and sends it to the device. This data includes details of each itinerary (e.g., hotel name, price, flight information, etc.).

[1179] Step 5: The device displays the travel plan in the virtual space.

[1180] Based on the received data, the terminal displays the travel plan in a virtual space on the user's smart device (HMD or smart glasses). The displayed plan is presented in a visual format that the user can intuitively understand. The user can operate and select each plan using gestures and voice commands.

[1181] Step 6: User confirms selected plan and sends it to the server

[1182] The plan selected by the user in the virtual space is converted back into JSON format data via the device and sent to the server, which then analyzes the data and initiates the reservation process based on the selection.

[1183] Step 7: The server processes the reservation

[1184] The server accesses the API of each provider (hotel, airline) based on the selected plan and sends a reservation request. For example, the following API request is used:

[1185] POST / reserve

[1186] Host: hotelprovider.com

[1187] Content-Type: application / json

[1188] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1189] POST / book

[1190] Host: airlineprovider.com

[1191] Content-Type: application / json

[1192] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[1193] If the reservation is successful, the result is notified to the terminal.

[1194] Step 8: Make a payment

[1195] The terminal displays a payment page, offering the user payment options such as credit card or electronic money. The user selects the desired payment method and enters the required information. The server processes the payment information and, if the payment is successful, sends the user a confirmation message with further details. A confirmation email is also automatically sent to the user.

[1196] Examples of prompts:

[1197] "Planning my next trip: Destination is Okinawa, looking for a beach resort, budget under 100,000 yen. Any recommendations?"

[1198] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1199] The present invention achieves a higher level of personalization by combining a system in which a user interactively inputs their travel preferences, proposes an optimal travel plan based on those preferences, and handles all booking and payment procedures in one go with an emotion engine that recognizes the user's emotions. Specific embodiments are described below.

[1200] First, the user enters their travel preferences. Using a device such as a smartphone or computer, the user enters information such as the travel destination, number of days, budget, and desired activities into an input form. For example, the user might enter information such as, "I want to travel to Okinawa, relax at a beach resort, and my budget is 100,000 yen."

[1201] Next, the device sends the input information to the server. The data is structured in a format such as JSON and transmitted to the server. An example of the data sent is {"Location": "Okinawa", "Number of Days": "3", "Budget": "100,000 yen", "Activity": "Beach Resort"}.

[1202] The server analyzes the received data and searches for information from a database based on the user's preferences. Specifically, it extracts information on hotels, transportation (flights or trains), and activities suitable for the travel destination. At this time, the emotion engine analyzes the user's facial expressions and voice when inputting information to evaluate the user's emotional state. For example, if the user is excited, it will prioritize suggestions with plenty of activities, or if the user wants to relax, it will prioritize suggestions for quiet resorts.

[1203] The emotion engine analyzes emotions based on the following information:

[1204] Facial expression analysis (using a camera to analyze facial movements and expressions)

[1205] Voice analysis (analyzing voice tone and speed using a microphone)

[1206] Text analysis (guessing emotions from input text)

[1207] Based on the analysis results of the emotion engine, the server generates multiple travel plans from the search results and builds information on the optimal accommodation, transportation, price, and activities for each plan. For example, a plan might include a three-night stay at a beach resort hotel in Okinawa for a total of 50,000 yen, or a round-trip flight for 30,000 yen.

[1208] The generated travel plan is sent to the terminal and displayed to the user. The user reviews each plan and selects the one that best suits their needs. Once the selection is complete, the selection is sent back to the server.

[1209] The server proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. At this time, it accesses each provider via API and sends the reservation information. For example:

[1210] POST / reserve

[1211] Host: hotelprovider.com

[1212] Content-Type: application / json

[1213] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1214] POST / book

[1215] Host: airlineprovider.com

[1216] Content-Type: application / json

[1217] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[1218] Once the reservation is confirmed, the terminal displays a payment procedure page, where options such as credit card and electronic money are offered. The user enters their payment information and completes the payment. The payment information is then sent to the server.

[1219] The server processes the payment information it receives, sends a confirmation message to the user if the payment is successful, and records the payment as successful within the system. Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to review their trip details.

[1220] The analysis results of the emotion engine are saved as reference data for the next travel plan generation. For example, if a user previously expressed the emotion "I want to relax," that information will be used to suggest future plans.

[1221] Through the above steps, the present invention allows users to simply input their travel preferences, automatically suggest optimal travel plans, and process reservations and payment procedures all in one go. Furthermore, by utilizing an emotion engine, it is possible to provide a personalized experience that matches the user's emotions.

[1222] The processing flow will be explained below.

[1223] Step 1:

[1224] The user inputs their travel preferences.

[1225] The user uses a device (smartphone or computer) to input the desired travel destination, number of days, budget, desired activities, etc. For example, the user inputs information such as "I want to travel to Okinawa, relax at a beach resort, and my budget is 100,000 yen."

[1226] Step 2:

[1227] The terminal transmits the input information and the user's emotion information to the server.

[1228] Once the input is complete, the device sends the input information in JSON format, along with the user's emotional data (facial expression and voice analysis data), to the server. For example, the emotional data is sent as {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[1229] Step 3:

[1230] The server analyzes the travel preference data.

[1231] The server analyzes the received input data and recognizes each item (location, number of days, budget, activity), and generates a query for database search based on the analysis results.

[1232] Step 4:

[1233] The emotion engine analyzes the user's emotion data.

[1234] The emotion engine in the server analyzes the user's facial expression data and voice data to identify the user's emotional state (excited, relaxed, anxious, etc.). For example, it is determined that the user is in a relaxed mode.

[1235] Step 5:

[1236] The server retrieves information from the database based on the results of the emotion engine.

[1237] The server takes into account the results of the emotion engine and searches the database for the most suitable travel plans. For example, if the emotion of wanting to relax is detected, it will prioritize plans that include quiet beach resorts and spas. An example SQL query is as follows:

[1238] sql

[1239] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000 AND features LIKE '%Relax%';

[1240] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[1241] Step 6:

[1242] The server generates the itinerary.

[1243] The server generates multiple travel plans based on the search results, incorporating information on the best accommodations, transportation, prices, and activities for each plan, such as a three-night stay at a beach resort hotel in Okinawa for a total of 50,000 yen, or a round-trip flight for 30,000 yen.

[1244] Step 7:

[1245] The device displays the suggested itinerary.

[1246] The generated travel plans are sent to the terminal and displayed on the user's screen, allowing the user to compare and consider these plans.

[1247] Step 8:

[1248] The user selects a plan.

[1249] The user selects the desired plan from the presented plans and clicks or taps the selection button. The selection information is sent to the server.

[1250] Step 9:

[1251] The server starts the reservation procedure for the selected plan.

[1252] The server processes the reservations for accommodation and transportation based on the plan selected by the user. This is done via the API of each provider. For example, it sends the following request:

[1253] POST / reserve

[1254] Host: hotelprovider.com

[1255] Content-Type: application / json

[1256] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1257] POST / book

[1258] Host: airlineprovider.com

[1259] Content-Type: application / json

[1260] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[1261] Step 10:

[1262] The server sends the reservation confirmation information to the user.

[1263] If the accommodation and transportation reservations are successful, the result is notified to the user's terminal. The server generates and transmits data including reservation confirmation information.

[1264] Step 11:

[1265] The terminal displays the payment page.

[1266] Once the reservation is confirmed, the terminal displays a payment procedure page to the user, offering payment methods such as credit card and electronic money.

[1267] Step 12:

[1268] The user enters payment information.

[1269] The user enters their payment information and completes the transaction, which is then sent to the server.

[1270] Step 13:

[1271] The server processes the payment transaction.

[1272] The server processes the received payment information, sends a confirmation message to the user when the payment is complete, and records the payment completion information in the system.

[1273] Step 14:

[1274] The server sends a confirmation email.

[1275] Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to double-check their trip details.

[1276] Step 15:

[1277] The emotion engine stores the emotion data.

[1278] The emotion engine saves the user's emotional data and uses it as reference data when generating the next travel plan. For example, if a user previously expressed the emotion "I want to relax," that information will be used to suggest future plans.

[1279] Through the above steps, the present invention allows users to simply input their travel preferences, automatically suggest optimal travel plans, and process reservations and payment procedures all in one go. Furthermore, by utilizing an emotion engine, it is possible to provide a personalized experience that matches the user's emotions.

[1280] Example 2

[1281] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1282] Many conventional travel reservation systems propose travel plans based on the user's preferences and handle reservations and payment procedures, but because they do not take the user's emotional state into consideration, they have the problem of making it difficult to provide a personalized experience that matches the user's emotions.In addition, because plans are proposed based only on information entered by the user, they may not always propose the optimal travel plan.

[1283] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1284] In this invention, the server includes means for allowing a user to interactively input travel preferences, means for proposing optimal travel destinations, means of transportation, and accommodations based on the input travel preferences, and means for customizing a travel plan based on the user's emotional state using an analysis device for analyzing the user's emotions. This makes it possible to propose an optimal travel plan based on the user's emotions and to complete reservations and payment procedures all at once.

[1285] "User" refers to a person who uses this system, who inputs travel preferences, selects a proposed plan, and completes payment procedures.

[1286] "Travel preferences" refers to information that a user interactively inputs, such as travel destination, number of days, budget, desired activities, etc.

[1287] "Interactive" refers to a format in which a user interacts with a system to input information.

[1288] "Means of suggestion" refers to a method for selecting the most suitable travel destination, means of transportation, and accommodation based on the input information and presenting them to the user.

[1289] The term "means for making a reservation" refers to a method for carrying out the procedure for actually making a reservation for the travel plan selected by the user.

[1290] "Means for making payment" refers to the method by which a user makes payment after completing a travel reservation.

[1291] "Analysis device" refers to a device that analyzes facial expressions, voice, and text data input by a user and evaluates the user's emotional state.

[1292] "Means for customization" refers to ways to adjust or change travel plans based on the user's emotional state.

[1293] "Server" refers to the computer system that processes and analyzes input data, generates suggested travel plans, and provides them to users.

[1294] A "database" refers to a collection of information that stores information on travel destinations, accommodations, means of transportation, etc., and that can be searched and extracted.

[1295] "Emotional state" refers to the psychological and emotional state analyzed from the user's facial expressions, voice, and text input.

[1296] This invention achieves a higher level of personalization by combining a system in which a user interactively inputs their travel preferences, proposes an optimal travel plan based on those preferences, and performs all the booking and payment procedures in one go with an emotion engine that recognizes the user's emotions. Specific embodiments of the present invention are described in detail below.

[1297] First, the user uses a device such as a smartphone or computer to input their travel preferences. They enter information such as the travel destination, number of days, budget, and desired activities into an input form. For example, they might enter information like, "I want to travel to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen."

[1298] The device then sends the input information to the server, which converts the data into a structured data format such as JSON and transmits it to the server.

[1299] The server analyzes the received data and searches for information from a database based on the user's preferences. Specifically, it extracts information about hotels, transportation options (flights or trains), and activities suitable for the travel destination. At this time, the emotion engine analyzes the user's facial expressions and voice when they input information and evaluates their emotional state. The emotion engine analyzes the user's emotions using technologies such as facial expression analysis (using a camera to analyze facial movements and expressions), voice analysis (using a microphone to analyze the tone and speed of voice), and text analysis (inferring emotions from input text).

[1300] For example, if a user inputs "I want to travel to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen," and their facial expression is relaxed, the server will prioritize suggesting quiet resorts. As a specific example, if the emotion engine reads the emotion "I want to relax" from the user's facial expression, the server will prioritize generating plans that include quiet beach resort hotels in Okinawa. Also, if a user inputs "I want to travel to Okinawa. I want to enjoy the beach and diving as activities," and their facial expression shows signs of excitement, the server will suggest plans with plenty of activities.

[1301] The generated travel plans are sent from the server to the terminal as multiple options and displayed to the user. The user reviews each plan and selects the one that best suits their needs. Once the selection is complete, the selection is sent back to the server.

[1302] The server proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. At this time, it accesses each provider via API and sends the reservation information. For example, the following request is made:

[1303] POST / reserve

[1304] Host: hotelprovider.com

[1305] Content-Type: application / json

[1306] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1307] Once the reservation is confirmed, the terminal displays a payment procedure page, where options such as credit card and electronic money are offered. The user enters their payment information and completes the payment. The payment information is then sent to the server.

[1308] The server processes the payment information it receives, sends a confirmation message to the user if the payment is successful, and records the payment as successful within the system. Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to review their trip details.

[1309] The analysis results of the emotion engine are saved as reference data for the next travel plan generation. For example, if a user previously expressed the emotion "I want to relax," that information will be used to suggest future plans.

[1310] Through the above steps, the present invention allows users to simply input their travel preferences, automatically suggest optimal travel plans, and process reservations and payment procedures all in one go. Furthermore, by utilizing an emotion engine, it is possible to provide a personalized experience that matches the user's emotions.

[1311] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1312] Step 1:

[1313] The user inputs his / her travel preferences.

[1314] Input: The user uses a smartphone or computer to enter information such as the travel destination, number of days, budget, desired activities, etc. into an input form. For example, the user might enter information such as, "I want to travel to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen."

[1315] Output: The input information is temporarily stored in the device's memory.

[1316] Step 2:

[1317] The terminal sends the input information to the server.

[1318] Input: Travel preference information entered by the user.

[1319] Data processing: Converting input information into structured data such as JSON format.

[1320] Output: The converted data is sent to the server. For example, the following data is sent: {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[1321] Step 3:

[1322] The server receives and analyzes the data.

[1323] Input: JSON data of travel preference information sent from the terminal.

[1324] Data calculation: Data is analyzed to extract information such as travel destination, budget, number of days, and desired activities. Furthermore, an emotion engine is activated at this time to evaluate the user's emotional state based on their facial expressions and voice input. The emotion engine uses facial expression analysis technology (using the camera), voice analysis technology (using the microphone), and text analysis technology.

[1325] Output: The analysis results in information that reflects the user's wishes and emotional state.

[1326] Step 4:

[1327] The server searches the database and generates the optimal travel plan.

[1328] Input: Travel preference information and emotional evaluation data.

[1329] Data calculation: Searches the database for hotels, transportation (flights and trains), and activity information suitable for the travel destination, and combines this data to generate multiple travel plans. For example, if a user is rated as wanting to relax, a plan including a quiet beach resort hotel will be generated.

[1330] Output: Multiple itineraries are generated and their details are stored as data.

[1331] Step 5:

[1332] The server transmits the generated travel plan to the terminal.

[1333] Input: The generated itinerary data.

[1334] Data processing: Convert travel plan data into JSON format or other format for sending to the device.

[1335] Output: The travel plan sent to the device is displayed on the user's UI. For example, a three-night stay at a beach resort hotel in Okinawa (total price: 50,000 yen) and a round-trip flight (30,000 yen) are displayed as a list.

[1336] Step 6:

[1337] The user reviews and selects a travel plan.

[1338] Input: Multiple travel plans displayed on the device.

[1339] How it works: The user reviews each plan in detail and selects the one that best suits their needs.

[1340] Output: The data of the selected plan is sent from the device to the server.

[1341] Step 7:

[1342] The server will proceed with the reservation procedure for the selected plan.

[1343] Input: Travel plan data selected by the user.

[1344] Data Computing: Generate API requests to send booking information to accommodation and transportation providers. For example, send requests to hotel and airline booking APIs.

[1345] Output: The reservation information is sent to the provider, and if successful, the result is returned to the server and the user is notified.

[1346] Step 8:

[1347] The user completes the payment process.

[1348] Input: Information about your confirmed travel plan and the payment options displayed on the payment page (credit card, e-money, etc.).

[1349] How it works: The user enters payment information and makes a payment.

[1350] Output: Payment information is sent from the terminal to the server.

[1351] Step 9:

[1352] The server processes the payment and sends a confirmation email.

[1353] Input: Payment information submitted by the user.

[1354] Data calculation: Executes the payment process and checks whether the payment is successful. If successful, records the payment as completed in the system.

[1355] Output: A confirmation message confirming payment is completed and a confirmation email containing the trip details is sent to the user.

[1356] (Application example 2)

[1357] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1358] Conventional travel plan suggestion systems have struggled to provide optimal travel experiences because they lack the ability to personalize based on users' emotions and real-time situations. Furthermore, they lack the ability to obtain and suggest detailed information about travel destinations and activities in real time, making it difficult to respond to users' changing needs. Furthermore, it has been difficult to provide real-time guidance and reservation functions during travel.

[1359] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recognizing emotions by analyzing the user's facial expressions and voice, means for personalizing plan suggestions based on the user's emotions, means for providing real-time information on tourist spots and activities being visited, and means for checking real-time availability information on proposed activities and accommodations and making reservations. This makes it possible to provide a personalized travel experience optimized for the user's emotions and real-time situation.

[1360] A "means for interactively inputting user travel preferences" is a device or software that allows a user to input travel preferences via voice or text through an interface.

[1361] "Means for suggesting optimal travel destinations, means of transportation, and accommodations based on input travel preferences" refers to an algorithm or software function that analyzes the desired information input by the user and selects and suggests optimal travel destinations, means of transportation, and accommodations accordingly.

[1362] "Means for displaying proposed travel plans and reserving the plan selected by the user" is a function that displays the travel plan selected by the user on the screen and allows the user to proceed with the reservation procedure for that plan after selection.

[1363] The "means for carrying out payment procedures during the reservation process" refers to a processing means for making payment for the travel plan selected by the user using a credit card or electronic money.

[1364] "Means for recognizing emotions by analyzing a user's facial expressions and voice" refers to technology that uses a user's camera or microphone to analyze their facial expressions and voice, and infers the user's emotions from the analysis results.

[1365] "Means for personalizing plan suggestions based on the user's emotions" is a function that generates travel plans that are optimal for the user's psychological state based on the results of emotion analysis.

[1366] "Means for providing real-time information on tourist spots and activities currently being visited" is a system for providing real-time information on activities and spots related to the tourist spots that the user is visiting.

[1367] "A means to check real-time availability of suggested activities and accommodations and make reservations" is a function that allows you to check the current availability of suggested activities and accommodations and confirm reservations on the spot.

[1368] The present invention achieves a higher level of personalization by combining a system in which a user interactively inputs their travel preferences, proposes an optimal travel plan based on those preferences, and performs all the booking and payment procedures in one go with an emotion engine that recognizes the user's emotions. An embodiment of the present invention will be described below.

[1369] The system consists of devices such as smart glasses, smartphones, and head-mounted displays, servers, databases, an emotion analysis engine, and a payment processing system. Specific hardware and software used include Google Glass, Microsoft HoloLens, iPhones, Android devices, Affectiva SDK, Google Cloud Speech-to-Text, Amazon Transcribe, Google Maps API, Mapbox, Stripe API, and PayPal.

[1370] The system operates in the following steps:

[1371] First, the user uses smart glasses, a smartphone, or a head-mounted display to input their travel preferences by voice or text. This input information is sent to the server via the device. For example, the user might input, "I want to go to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen."

[1372] The server then receives the user's input and uses the Affectiva SDK or a similar emotion analysis engine to analyze the user's facial expressions and voice to recognize their emotions. Based on the recognized emotion, the server searches a database for the best travel destinations, accommodations, and transportation options, and generates multiple itineraries. The itinerary is personalized based on the results of the emotion analysis; for example, if the server recognizes that the user wants to relax, quiet resorts will be prioritized.

[1373] The generated travel plan is sent to the device and displayed to the user. The user selects the most suitable plan from the displayed plans. The selected plan is sent back to the server, and the reservation process begins. The server then sends the reservation information to accommodation and transportation providers via API.

[1374] In addition, the user can check real-time availability information for suggested activities and accommodations, and make instant reservations. Payments are processed using Stripe API or PayPal, with credit card or electronic money payments. Once payment is complete, the server sends a confirmation message to the user, providing them with their trip details.

[1375] In addition, the results of the emotion analysis are stored in the system and used as reference data when generating future travel plans. The next time you plan a trip, more advanced personalization based on past emotion data will be possible.

[1376] Examples of specific prompts include:

[1377] "I want to travel to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[1378] In this way, the present invention can provide a highly personalized travel experience that is optimized to the user's emotions and real-time circumstances.

[1379] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1380] Step 1:

[1381] The user interactively inputs their travel preferences using smart glasses, a smartphone, or a head-mounted display. The user inputs "I want to go to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen" by voice, and the device converts this into text data. This input text is generated.

[1382] Step 2:

[1383] The device sends the generated text data to the server. The device then structures the entered travel desire data into JSON format and sends it to the server. For example, the format might be {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[1384] Step 3:

[1385] The server analyzes the received data. To determine the user's desired information, it processes the text data and uses the Affectiva SDK and Google Cloud Speech-to-Text to analyze voice and facial expression data to recognize the user's emotional state. This analysis confirms that the user is seeking relaxation.

[1386] Step 4:

[1387] The server then searches the database for multiple travel plans based on the analysis results. The search results include information on destinations, accommodations, transportation, and activities. For example, a plan for a three-night stay at a beach resort hotel in Okinawa and a round-trip flight for 30,000 yen might be included.

[1388] Step 5:

[1389] The server sends the generated itinerary to the terminal, which displays the proposed itineraries on the screen. The user can select and confirm a specific itinerary from the displayed itinerary.

[1390] Step 6:

[1391] The user selects the desired travel plan. The selected plan is retransmitted to the server via the terminal. For example, the selection might be "Resort A, 3 nights, round-trip flight included, total price 80,000 yen."

[1392] Step 7:

[1393] The server sends the reservation information to the accommodation and transportation providers according to the selected plan. For example, the reservation information is sent via API in the format {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}.

[1394] Step 8:

[1395] If the reservation is confirmed, the terminal displays a payment procedure page, where the user selects a payment method such as credit card or electronic money and enters payment information.

[1396] Step 9:

[1397] The server processes the entered payment information and executes the payment using the Stripe API or PayPal. Once payment is complete, the server sends a confirmation message to the user and provides the trip details.

[1398] Step 10:

[1399] The system stores the results of the sentiment analysis as reference data for generating future travel plans, allowing for even more personalized planning the next time you plan a trip.

[1400] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1401] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1402] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1403] [Fourth embodiment]

[1404] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1405] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1406] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1407] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1408] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1409] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1410] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1411] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1412] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1413] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1414] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1415] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1416] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1417] The present invention is a system that allows a user to interactively input their travel preferences, proposes an optimal travel plan based on the preferences, and performs reservation and payment procedures all at once. Specific embodiments are described below.

[1418] First, the user enters their travel preferences. Using a device such as a smartphone or computer, the user enters their travel destination, number of days, budget, activities they would like to experience, and other preferences. For example, they might enter something like, "I would like to go to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[1419] Next, the device sends this input information to the server. The data is structured in a format such as JSON and transmitted to the server. For example, the format might be {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[1420] The server analyzes the input data and searches the database for information based on the user's preferences. Specifically, it extracts information about hotels, transportation options (flights and trains), and activities suitable for the destination. For example, it uses the following SQL query:

[1421] sql

[1422] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[1423] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[1424] The server generates multiple travel plans based on the search results and builds details for each plan, such as a three-night stay at a beach resort hotel in Okinawa for a total of 50,000 yen, or a round-trip flight for 30,000 yen.

[1425] The generated travel plan is sent to the terminal and displayed to the user. The user reviews each plan and selects the one that best suits their needs. Once the selection is complete, the selection is sent to the server.

[1426] The server proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. To do this, it accesses the API of each provider using an HTTP request. For example:

[1427] POST / reserve

[1428] Host: hotelprovider.com

[1429] Content-Type: application / json

[1430] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1431] POST / book

[1432] Host: airlineprovider.com

[1433] Content-Type: application / json

[1434] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[1435] At the payment stage, a payment page appears on the terminal, offering options such as credit card and electronic money. The user selects a payment method and enters the required information. The server processes the payment information and sends a confirmation message to the user once the transaction is complete. A confirmation email is also automatically sent, containing payment confirmation and trip details.

[1436] For example, if you say you want to travel to Okinawa, the server will search for beach resort hotels and flights in Okinawa and generate the following three plans:

[1437] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[1438] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[1439] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[1440] If the user selects plan 1, the server will confirm the reservation and process the payment accordingly.

[1441] The above is a description of a specific embodiment for carrying out the present invention. In this way, the user can easily obtain the optimal travel plan and complete the reservation and payment procedures in one go.

[1442] The processing flow will be explained below.

[1443] Step 1:

[1444] The user inputs their travel preferences.

[1445] The user uses a device (smartphone or computer) to enter information such as the desired travel location, number of days, budget, desired activities, etc. into an input form. For example, the user enters information such as "I want to travel to Okinawa, relax at a beach resort, and my budget is 100,000 yen."

[1446] Step 2:

[1447] The terminal sends the input information to the server.

[1448] When a user submits the form, the information is sent to the server in a structured data format (e.g., JSON). An example of the data sent is {"Location": "Okinawa", "Number of Days": "3", "Budget": "100,000 yen", "Activity": "Beach Resort"}.

[1449] Step 3:

[1450] The server parses the data it receives.

[1451] The server analyzes the received travel preference data, breaks down and recognizes each item (location, number of days, budget, activity), and generates the necessary search query based on this analyzed data.

[1452] Step 4:

[1453] The server retrieves the information from the database.

[1454] Based on the analysis results, the server searches the database for the best travel destinations, accommodations, and transportation options, for example by executing the following SQL query:

[1455] sql

[1456] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[1457] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[1458] Step 5:

[1459] The server generates the itinerary.

[1460] The server generates multiple optimal travel plans from the search results. Each plan includes information on accommodation, transportation, prices, and activities. It builds data to propose these plans to users.

[1461] Step 6:

[1462] The device displays the suggested itinerary.

[1463] The generated travel plans are sent to the user's device and displayed on the screen, allowing the user to compare and consider these plans.

[1464] Step 7:

[1465] The user selects a plan.

[1466] The user selects the most suitable plan from the presented plans and clicks or taps the selection button, and the selection information is sent to the server.

[1467] Step 8:

[1468] The server starts the reservation procedure for the selected plan.

[1469] The server then processes the booking of accommodation and transportation based on the plan selected by the user, sending the booking information to each provider via API. For example:

[1470] POST / reserve

[1471] Host: hotelprovider.com

[1472] Content-Type: application / json

[1473] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1474] POST / book

[1475] Host: airlineprovider.com

[1476] Content-Type: application / json

[1477] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[1478] Step 9:

[1479] The server sends the reservation confirmation information to the user.

[1480] If the accommodation and transportation reservations are successful, the server notifies the user of the result. The reservation confirmation information is sent back to the user's device and a confirmation is displayed.

[1481] Step 10:

[1482] The terminal displays the payment page.

[1483] Once the reservation is confirmed, the terminal displays a payment page to the user, offering payment options such as credit card and electronic money.

[1484] Step 11:

[1485] The user enters payment information.

[1486] The user enters their payment information and makes the payment, which is then sent to the server.

[1487] Step 12:

[1488] The server processes the payment transaction.

[1489] The server processes the payment information received and completes the transaction. If successful, a payment confirmation message is sent to the user and the payment is recorded as successful in the system.

[1490] Step 13:

[1491] The server sends a confirmation email.

[1492] Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to double-check their trip details.

[1493] By following the above steps, the system automatically proposes the most suitable travel plan and processes reservations and payment procedures all in one go, simply by the user inputting their desired travel preferences.

[1494] Example 1

[1495] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1496] In today's world, responding to diverse travel needs requires searching through a wide range of information to find the best plan. However, finding the best plan from a vast amount of information takes time and effort, placing a significant burden on users. Furthermore, reservation and payment procedures must be completed separately, which is time-consuming. There is a need for a system that can solve these issues and enable users to easily obtain travel plans and complete reservations and payment procedures all in one go.

[1497] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1498] In this invention, the server includes means for a user to interactively input travel preferences, means for transmitting the input preference information in JSON format and for the server to analyze it, means for searching a database using an SQL query for optimal travel destinations, means of transportation, and accommodations based on the analysis results, means for generating multiple travel plans based on the search results and presenting them to the user, and means for reserving the plan selected by the user and performing payment procedures. This allows the user to easily obtain the optimal travel plan and complete the reservation and payment procedures all at once.

[1499] "User" refers to an individual or entity that inputs travel preferences.

[1500] "Server" refers to a device that analyzes input information sent by users, proposes optimal travel plans, and manages reservation and payment procedures.

[1501] "Terminal" refers to a smartphone, computer, or other electronic device used by a User to input their travel preferences.

[1502] "Travel preferences" refers to travel-related requests of the user, including the destination, number of days, budget, activities they would like to experience, etc.

[1503] "JSON format" is an abbreviation for JavaScript Object Notation, and refers to a lightweight data description format primarily used for data exchange.

[1504] "Parsing" refers to the process by which the server understands the user's input and extracts the necessary information.

[1505] "SQL query" is an abbreviation for Structured Query Language and refers to a question used to retrieve necessary information from a database.

[1506] "Travel plan" refers to a set of suggestions including travel destinations, transportation, accommodations, etc., generated by the server based on the user's preferences.

[1507] "Reservation" refers to the process of actually securing accommodation and transportation based on the travel plan selected by the user.

[1508] "Payment procedure" refers to the process a user goes through to pay for a travel plan.

[1509] "Generating multiple itineraries" refers to the server suggesting multiple available options based on the user's input information.

[1510] The present invention is a system that allows a user to interactively input their travel preferences, proposes an optimal travel plan based on the preferences, and performs reservation and payment procedures all at once. Specific embodiments are described below.

[1511] First, the user uses a device such as a smartphone or computer to input their travel preferences. Specifically, they enter information such as the destination, number of days, budget, and activities they want to experience through a dedicated application or web form. In this example, let's say, "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[1512] The device then structures the input information into JSON format and sends it to the server. The input data is sent to a specific endpoint on the server using an HTTP POST request. For example, the following JSON data is sent:

[1513] json

[1514] {

[1515] "Location": "Okinawa",

[1516] "days": "3",

[1517] "Budget": "100,000 yen",

[1518] "Activity": "Beach Resort"

[1519] }

[1520] The server parses the received data and analyzes the user's desired conditions. During this analysis, the server extracts conditions such as destination, number of days, budget, and activities, and generates SQL queries. The server uses these queries to search its database for relevant hotel, flight, and activity information. For example, the following SQL queries are executed:

[1521] sql

[1522] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[1523] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[1524] The server generates multiple itineraries based on the information retrieved from the database, each containing accommodation, transportation, and activities that match the user's specified criteria. For example, the following itinerary may be generated:

[1525] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[1526] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[1527] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[1528] Next, the server structures the generated travel plan in JSON format and sends it to the terminal as an HTTP response. The terminal displays the received travel plan information to the user. The user selects the plan that seems best from multiple plans. The selected plan is sent back to the server, and the server proceeds with the reservation process based on that information.

[1529] The server uses an HTTP request to send the reservation information to the accommodation or transportation provider, for example, a request like this:

[1530] POST / reserve

[1531] Host: hotelprovider.com

[1532] Content-Type: application / json

[1533] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1534] POST / book

[1535] Host: airlineprovider.com

[1536] Content-Type: application / json

[1537] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[1538] If the booking is successful, the terminal displays a payment page. The user selects a payment method and enters the required payment information. The server processes the payment information and sends a confirmation message to the user once the transaction is complete. A confirmation email is also automatically sent containing payment confirmation and trip details.

[1539] An example prompt sentence to be input to a generative AI model is as follows:

[1540] What travel plans would be suggested if a user said, "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[1541] The above is a specific embodiment for carrying out the present invention, which allows users to easily obtain the most suitable travel plan and complete the reservation and payment procedures all at once.

[1542] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1543] Step 1:

[1544] The user inputs their travel preferences. Specifically, they use a smartphone or computer to enter their travel destination, number of days, budget, and desired activities through a dedicated application or web form. For example, a user might enter their preference as "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen." At this stage, text information is generated as input data.

[1545] Step 2:

[1546] The terminal converts the input desired information into JSON format. For example, the information input by the user is converted into the following JSON format:

[1547] json

[1548] {

[1549] "Location": "Okinawa",

[1550] "days": "3",

[1551] "Budget": "100,000 yen",

[1552] "Activity": "Beach Resort"

[1553] }

[1554] The converted JSON data is then sent to the server via an HTTP POST request, with the terminal outputting the input data as structured data.

[1555] Step 3:

[1556] The server parses the received JSON data and analyzes the user's desired conditions. Specifically, it reads the JSON data and extracts key information such as "location," "number of days," "budget," and "activity." Based on the results of this analysis, it prepares to search for related information in the database. At this time, the server outputs the analyzed data of the user's desired conditions.

[1557] Step 4:

[1558] The server generates an SQL query based on the parsed data and searches the database for the relevant information. For example, it executes a query like this:

[1559] sql

[1560] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[1561] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[1562] This query causes the server to retrieve hotel, flight, and activity data that matches the criteria from the database, and then outputs the information retrieved from the database.

[1563] Step 5:

[1564] The server generates multiple travel plans based on the acquired information. For example, it creates multiple plans combining hotels and flights in Okinawa:

[1565] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[1566] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[1567] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[1568] The server outputs detailed information about these plans in structured JSON format.

[1569] Step 6:

[1570] The server sends the generated travel plan to the terminal as an HTTP response. The terminal displays the received travel plan information to the user. At this time, the travel plan information is output to the terminal.

[1571] Step 7:

[1572] The user selects the desired plan from the multiple plans presented. When the selected plan is input into the device, the device again sends the selection information in JSON format to the server. At this time, the selection information is output as structured data.

[1573] Step 8:

[1574] The server receives the user's selection information and proceeds with the reservation process based on the selected plan. Specifically, it analyzes the selection information and sends the reservation information to the accommodation and transportation providers via HTTP request. For example, it sends the following request:

[1575] POST / reserve

[1576] Host: hotelprovider.com

[1577] Content-Type: application / json

[1578] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1579] POST / book

[1580] Host: airlineprovider.com

[1581] Content-Type: application / json

[1582] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[1583] At this time, reservation confirmation information after the request is sent is output.

[1584] Step 9:

[1585] After the reservation is completed, the payment page will be displayed on the terminal. The user can select the desired payment method from the presented payment methods and enter the necessary payment information. At this time, the payment information will be entered into the terminal.

[1586] Step 10:

[1587] The server receives the payment information and processes the payment. After the payment is completed, the server sends a payment confirmation message to the user and automatically sends a confirmation email containing the payment confirmation and trip details. At this time, the payment confirmation information and trip details are output.

[1588] (Application example 1)

[1589] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1590] Currently, many travel plan suggestion and reservation systems on the market struggle to efficiently reflect user preferences, making it cumbersome to compare and select travel plans. Payment procedures are also complicated, resulting in an overall unsatisfactory user experience. Furthermore, because they use physical devices, it is difficult to provide intuitive visual feedback, making it difficult for users to easily select the optimal travel plan.

[1591] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1592] In this invention, the server includes a means for a user to interactively input their travel preferences, a means for suggesting optimal travel destinations, means of transportation, and accommodations based on the input travel preferences, a visual feedback means for displaying the proposed travel plans, a means for booking the plan selected by the user, a means for performing payment procedures during the booking process, and a means for the user to select and operate travel plans in a virtual space using a smart device. This allows users to easily input their preferences interactively and intuitively compare and select travel plans through visual feedback. Furthermore, the reservation and payment procedures can be completed seamlessly in one go by operating in the virtual space, improving the overall user experience.

[1593] The "means for the user to interactively input travel preferences" is a function that allows the user to interactively input travel preferences into the terminal through voice input or keyboard input.

[1594] "Means for suggesting optimal travel destinations, means of transportation, and accommodations based on input travel preferences" is a function that allows the server to analyze the information input by the user and suggest appropriate travel destinations, means of transportation, and accommodations.

[1595] The "visual feedback means for displaying the proposed itinerary" is a function that displays the itinerary to the user in a visual format to help the user intuitively understand it.

[1596] The "means for reserving a plan selected by the user" is a function that allows the user to select a presented travel plan and confirm a reservation based on that plan.

[1597] "Means for carrying out payment procedures during the reservation process" is a function for carrying out all the payment procedures required when reserving a travel plan in one go.

[1598] "Means for users to select and operate travel plans in a virtual space using a smart device" refers to a function that allows users to visually check and operate travel plans in a virtual reality space using a smart device such as smart glasses or a head-mounted display.

[1599] The present invention is a system that allows users to interactively input their travel preferences, proposes optimal travel plans based on those preferences, and handles reservations and payment procedures all at once. The system for implementing the present invention mainly consists of a smart device, a back-end server, and related software.

[1600] First, the user inputs their travel preferences. Using smart glasses or a head-mounted display (HMD), the user inputs their preferences using voice or gestures. For example, they might input something like, "I want to go to Okinawa and relax at a beach resort. My budget is 100,000 yen." This input is analyzed in real time using voice recognition and gesture recognition technology and converted into JSON format.

[1601] Next, the device sends the input information to the server. The data is structured in JSON format and transmitted to the server. For example, it is sent in the form {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[1602] The server analyzes the input data and searches the database for information based on the user's preferences. Specifically, it extracts information about hotels, transportation options (flights and trains), and activities suitable for the destination. For example, it uses the following SQL query:

[1603] sql

[1604] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[1605] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[1606] The server generates multiple travel plans based on the search results and builds details for each plan. The generated travel plans are sent to the device and displayed in the virtual space as visual feedback to the user. The user can use the HMD to review each plan and select the one that best suits their needs using gestures and voice controls.

[1607] Once the selection is complete, the selection is sent to the server. The server then proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. This involves accessing the API of each provider using an HTTP request. For example:

[1608] POST / reserve

[1609] Host: hotelprovider.com

[1610] Content-Type: application / json

[1611] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1612] POST / book

[1613] Host: airlineprovider.com

[1614] Content-Type: application / json

[1615] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[1616] At the payment stage, a payment page appears on the terminal, offering options such as credit card and electronic money. The user selects a payment method and enters the required information. The server processes the payment information and sends a confirmation message to the user once the transaction is complete. A confirmation email is also automatically sent, containing payment confirmation and trip details.

[1617] For example, if you say you want to travel to Okinawa, the server will search for beach resort hotels and flights in Okinawa and generate the following three plans:

[1618] Plan 1: Hotel A (5,000 yen / night, 3 nights), Flight X (30,000 yen round trip)

[1619] Plan 2: Hotel B (6,000 yen / night, 3 nights), Flight Y (25,000 yen round trip)

[1620] Plan 3: Hotel C (7,000 yen / night, 3 nights), Flight Z (20,000 yen round trip)

[1621] If the user selects plan 1, the server will confirm the reservation and process the payment accordingly.

[1622] Examples of prompts:

[1623] "Planning my next trip: Destination is Okinawa, looking for a beach resort, budget under 100,000 yen. Any recommendations?"

[1624] In this way, users can easily obtain the optimal travel plan, make decisions through visual feedback, and complete the booking and payment procedures all in one place within the virtual space.

[1625] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1626] Step 1: The user uses a smart device to interactively input their travel preferences

[1627] Users use smart glasses or a head-mounted display to input their travel preferences using voice input or gestures. What the user says is captured by the device's microphone and sensors. The voice input is converted into text using speech recognition technology, and the data is structured into JSON format (e.g., {"Location": "Okinawa", "Number of Days": "3", "Budget": "100,000 yen", "Activity": "Beach Resort"}).

[1628] Step 2: The device sends the input information to the server

[1629] The device sends the JSON-formatted data generated in step 1 to the server, which uses an HTTP request to send the data to the backend server. The server then analyzes the data and prepares a search for a travel plan based on the user's preferences.

[1630] Step 3: The server retrieves the itinerary from the database

[1631] The server parses the received JSON data and searches the database for the appropriate itinerary (accommodation, transport, activities) using Structured Query Language (SQL). For example, the following query is executed:

[1632] sql

[1633] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000;

[1634] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[1635] The query results are temporarily stored in the server.

[1636] Step 4: The server sends the generated itinerary to the device.

[1637] The server generates multiple itineraries based on the search results, converts them into JSON format data, and sends it to the device. This data includes details of each itinerary (e.g., hotel name, price, flight information, etc.).

[1638] Step 5: The device displays the travel plan in the virtual space.

[1639] Based on the received data, the terminal displays the travel plan in a virtual space on the user's smart device (HMD or smart glasses). The displayed plan is presented in a visual format that the user can intuitively understand. The user can operate and select each plan using gestures and voice commands.

[1640] Step 6: User confirms selected plan and sends it to the server

[1641] The plan selected by the user in the virtual space is converted back into JSON format data via the device and sent to the server, which then analyzes the data and initiates the reservation process based on the selection.

[1642] Step 7: The server processes the reservation

[1643] The server accesses the API of each provider (hotel, airline) based on the selected plan and sends a reservation request. For example, the following API request is used:

[1644] POST / reserve

[1645] Host: hotelprovider.com

[1646] Content-Type: application / json

[1647] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1648] POST / book

[1649] Host: airlineprovider.com

[1650] Content-Type: application / json

[1651] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[1652] If the reservation is successful, the result is notified to the terminal.

[1653] Step 8: Make a payment

[1654] The terminal displays a payment page, offering the user payment options such as credit card or electronic money. The user selects the desired payment method and enters the required information. The server processes the payment information and, if the payment is successful, sends the user a confirmation message with further details. A confirmation email is also automatically sent to the user.

[1655] Examples of prompts:

[1656] "Planning my next trip: Destination is Okinawa, looking for a beach resort, budget under 100,000 yen. Any recommendations?"

[1657] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1658] The present invention achieves a higher level of personalization by combining a system in which a user interactively inputs their travel preferences, proposes an optimal travel plan based on those preferences, and handles all booking and payment procedures in one go with an emotion engine that recognizes the user's emotions. Specific embodiments are described below.

[1659] First, the user enters their travel preferences. Using a device such as a smartphone or computer, the user enters information such as the travel destination, number of days, budget, and desired activities into an input form. For example, the user might enter information such as, "I want to travel to Okinawa, relax at a beach resort, and my budget is 100,000 yen."

[1660] Next, the device sends the input information to the server. The data is structured in a format such as JSON and transmitted to the server. An example of the data sent is {"Location": "Okinawa", "Number of Days": "3", "Budget": "100,000 yen", "Activity": "Beach Resort"}.

[1661] The server analyzes the received data and searches for information from a database based on the user's preferences. Specifically, it extracts information on hotels, transportation (flights or trains), and activities suitable for the travel destination. At this time, the emotion engine analyzes the user's facial expressions and voice when inputting information to evaluate the user's emotional state. For example, if the user is excited, it will prioritize suggestions with plenty of activities, or if the user wants to relax, it will prioritize suggestions for quiet resorts.

[1662] The emotion engine analyzes emotions based on the following information:

[1663] Facial expression analysis (using a camera to analyze facial movements and expressions)

[1664] Voice analysis (analyzing voice tone and speed using a microphone)

[1665] Text analysis (guessing emotions from input text)

[1666] Based on the analysis results of the emotion engine, the server generates multiple travel plans from the search results and builds information on the optimal accommodation, transportation, price, and activities for each plan. For example, a plan might include a three-night stay at a beach resort hotel in Okinawa for a total of 50,000 yen, or a round-trip flight for 30,000 yen.

[1667] The generated travel plan is sent to the terminal and displayed to the user. The user reviews each plan and selects the one that best suits their needs. Once the selection is complete, the selection is sent back to the server.

[1668] The server proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. At this time, it accesses each provider via API and sends the reservation information. For example:

[1669] POST / reserve

[1670] Host: hotelprovider.com

[1671] Content-Type: application / json

[1672] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1673] POST / book

[1674] Host: airlineprovider.com

[1675] Content-Type: application / json

[1676] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[1677] Once the reservation is confirmed, the terminal displays a payment procedure page, where options such as credit card and electronic money are offered. The user enters their payment information and completes the payment. The payment information is then sent to the server.

[1678] The server processes the payment information it receives, sends a confirmation message to the user if the payment is successful, and records the payment as successful within the system. Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to review their trip details.

[1679] The analysis results of the emotion engine are saved as reference data for the next travel plan generation. For example, if a user previously expressed the emotion "I want to relax," that information will be used to suggest future plans.

[1680] Through the above steps, the present invention allows users to simply input their travel preferences, automatically suggest optimal travel plans, and process reservations and payment procedures all in one go. Furthermore, by utilizing an emotion engine, it is possible to provide a personalized experience that matches the user's emotions.

[1681] The processing flow will be explained below.

[1682] Step 1:

[1683] The user inputs their travel preferences.

[1684] The user uses a device (smartphone or computer) to input the desired travel destination, number of days, budget, desired activities, etc. For example, the user inputs information such as "I want to travel to Okinawa, relax at a beach resort, and my budget is 100,000 yen."

[1685] Step 2:

[1686] The terminal transmits the input information and the user's emotion information to the server.

[1687] Once the input is complete, the device sends the input information in JSON format, along with the user's emotional data (facial expression and voice analysis data), to the server. For example, the emotional data is sent as {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[1688] Step 3:

[1689] The server analyzes the travel preference data.

[1690] The server analyzes the received input data and recognizes each item (location, number of days, budget, activity), and generates a query for database search based on the analysis results.

[1691] Step 4:

[1692] The emotion engine analyzes the user's emotion data.

[1693] The emotion engine in the server analyzes the user's facial expression data and voice data to identify the user's emotional state (excited, relaxed, anxious, etc.). For example, it is determined that the user is in a relaxed mode.

[1694] Step 5:

[1695] The server retrieves information from the database based on the results of the emotion engine.

[1696] The server takes into account the results of the emotion engine and searches the database for the most suitable travel plans. For example, if the emotion of wanting to relax is detected, it will prioritize plans that include quiet beach resorts and spas. An example SQL query is as follows:

[1697] sql

[1698] SELECT FROM hotels WHERE location='Okinawa' AND price <= 50000 AND features LIKE '%Relax%';

[1699] SELECT FROM flights WHERE destination='Okinawa' AND price <= 50000;

[1700] Step 6:

[1701] The server generates the itinerary.

[1702] The server generates multiple travel plans based on the search results, incorporating information on the best accommodations, transportation, prices, and activities for each plan, such as a three-night stay at a beach resort hotel in Okinawa for a total of 50,000 yen, or a round-trip flight for 30,000 yen.

[1703] Step 7:

[1704] The device displays the suggested itinerary.

[1705] The generated travel plans are sent to the terminal and displayed on the user's screen, allowing the user to compare and consider these plans.

[1706] Step 8:

[1707] The user selects a plan.

[1708] The user selects the desired plan from the presented plans and clicks or taps the selection button. The selection information is sent to the server.

[1709] Step 9:

[1710] The server starts the reservation procedure for the selected plan.

[1711] The server processes the reservations for accommodation and transportation based on the plan selected by the user. This is done via the API of each provider. For example, it sends the following request:

[1712] POST / reserve

[1713] Host: hotelprovider.com

[1714] Content-Type: application / json

[1715] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1716] POST / book

[1717] Host: airlineprovider.com

[1718] Content-Type: application / json

[1719] {"flight_id": "456", "departure_date": "2023-12-01", "return_date": "2023-12-04"}

[1720] Step 10:

[1721] The server sends the reservation confirmation information to the user.

[1722] If the accommodation and transportation reservations are successful, the result is notified to the user's terminal. The server generates and transmits data including reservation confirmation information.

[1723] Step 11:

[1724] The terminal displays the payment page.

[1725] Once the reservation is confirmed, the terminal displays a payment procedure page to the user, offering payment methods such as credit card and electronic money.

[1726] Step 12:

[1727] The user enters payment information.

[1728] The user enters their payment information and completes the transaction, which is then sent to the server.

[1729] Step 13:

[1730] The server processes the payment transaction.

[1731] The server processes the received payment information, sends a confirmation message to the user when the payment is complete, and records the payment completion information in the system.

[1732] Step 14:

[1733] The server sends a confirmation email.

[1734] Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to double-check their trip details.

[1735] Step 15:

[1736] The emotion engine stores the emotion data.

[1737] The emotion engine saves the user's emotional data and uses it as reference data when generating the next travel plan. For example, if a user previously expressed the emotion "I want to relax," that information will be used to suggest future plans.

[1738] Through the above steps, the present invention allows users to simply input their travel preferences, automatically suggest optimal travel plans, and process reservations and payment procedures all in one go. Furthermore, by utilizing an emotion engine, it is possible to provide a personalized experience that matches the user's emotions.

[1739] Example 2

[1740] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1741] Many conventional travel reservation systems propose travel plans based on the user's preferences and handle reservations and payment procedures, but because they do not take the user's emotional state into consideration, they have the problem of making it difficult to provide a personalized experience that matches the user's emotions.In addition, because plans are proposed based only on information entered by the user, they may not always propose the optimal travel plan.

[1742] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1743] In this invention, the server includes means for allowing a user to interactively input travel preferences, means for proposing optimal travel destinations, means of transportation, and accommodations based on the input travel preferences, and means for customizing a travel plan based on the user's emotional state using an analysis device for analyzing the user's emotions. This makes it possible to propose an optimal travel plan based on the user's emotions and to complete reservations and payment procedures all at once.

[1744] "User" refers to a person who uses this system, who inputs travel preferences, selects a proposed plan, and completes payment procedures.

[1745] "Travel preferences" refers to information that a user interactively inputs, such as travel destination, number of days, budget, desired activities, etc.

[1746] "Interactive" refers to a format in which a user interacts with a system to input information.

[1747] "Means of suggestion" refers to a method for selecting the most suitable travel destination, means of transportation, and accommodation based on the input information and presenting them to the user.

[1748] The term "means for making a reservation" refers to a method for carrying out the procedure for actually making a reservation for the travel plan selected by the user.

[1749] "Means for making payment" refers to the method by which a user makes payment after completing a travel reservation.

[1750] "Analysis device" refers to a device that analyzes facial expressions, voice, and text data input by a user and evaluates the user's emotional state.

[1751] "Means for customization" refers to ways to adjust or change travel plans based on the user's emotional state.

[1752] "Server" refers to the computer system that processes and analyzes input data, generates suggested travel plans, and provides them to users.

[1753] A "database" refers to a collection of information that stores information on travel destinations, accommodations, means of transportation, etc., and that can be searched and extracted.

[1754] "Emotional state" refers to the psychological and emotional state analyzed from the user's facial expressions, voice, and text input.

[1755] This invention achieves a higher level of personalization by combining a system in which a user interactively inputs their travel preferences, proposes an optimal travel plan based on those preferences, and performs all the booking and payment procedures in one go with an emotion engine that recognizes the user's emotions. Specific embodiments of the present invention are described in detail below.

[1756] First, the user uses a device such as a smartphone or computer to input their travel preferences. They enter information such as the travel destination, number of days, budget, and desired activities into an input form. For example, they might enter information like, "I want to travel to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen."

[1757] The device then sends the input information to the server, which converts the data into a structured data format such as JSON and transmits it to the server.

[1758] The server analyzes the received data and searches for information from a database based on the user's preferences. Specifically, it extracts information about hotels, transportation options (flights or trains), and activities suitable for the travel destination. At this time, the emotion engine analyzes the user's facial expressions and voice when they input information and evaluates their emotional state. The emotion engine analyzes the user's emotions using technologies such as facial expression analysis (using a camera to analyze facial movements and expressions), voice analysis (using a microphone to analyze the tone and speed of voice), and text analysis (inferring emotions from input text).

[1759] For example, if a user inputs "I want to travel to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen," and their facial expression is relaxed, the server will prioritize suggesting quiet resorts. As a specific example, if the emotion engine reads the emotion "I want to relax" from the user's facial expression, the server will prioritize generating plans that include quiet beach resort hotels in Okinawa. Also, if a user inputs "I want to travel to Okinawa. I want to enjoy the beach and diving as activities," and their facial expression shows signs of excitement, the server will suggest plans with plenty of activities.

[1760] The generated travel plans are sent from the server to the terminal as multiple options and displayed to the user. The user reviews each plan and selects the one that best suits their needs. Once the selection is complete, the selection is sent back to the server.

[1761] The server proceeds with the reservation process based on the selected plan. It sends the reservation information to the accommodation and transportation providers, and if successful, notifies the user of the result. At this time, it accesses each provider via API and sends the reservation information. For example, the following request is made:

[1762] POST / reserve

[1763] Host: hotelprovider.com

[1764] Content-Type: application / json

[1765] {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}

[1766] Once the reservation is confirmed, the terminal displays a payment procedure page, where options such as credit card and electronic money are offered. The user enters their payment information and completes the payment. The payment information is then sent to the server.

[1767] The server processes the payment information it receives, sends a confirmation message to the user if the payment is successful, and records the payment as successful within the system. Finally, the server automatically sends the user a confirmation email containing payment confirmation and trip details, allowing the user to review their trip details.

[1768] The analysis results of the emotion engine are saved as reference data for the next travel plan generation. For example, if a user previously expressed the emotion "I want to relax," that information will be used to suggest future plans.

[1769] Through the above steps, the present invention allows users to simply input their travel preferences, automatically suggest optimal travel plans, and process reservations and payment procedures all in one go. Furthermore, by utilizing an emotion engine, it is possible to provide a personalized experience that matches the user's emotions.

[1770] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1771] Step 1:

[1772] The user inputs his / her travel preferences.

[1773] Input: The user uses a smartphone or computer to enter information such as the travel destination, number of days, budget, desired activities, etc. into an input form. For example, the user might enter information such as, "I want to travel to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen."

[1774] Output: The input information is temporarily stored in the device's memory.

[1775] Step 2:

[1776] The terminal sends the input information to the server.

[1777] Input: Travel preference information entered by the user.

[1778] Data processing: Converting input information into structured data such as JSON format.

[1779] Output: The converted data is sent to the server. For example, the following data is sent: {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[1780] Step 3:

[1781] The server receives and analyzes the data.

[1782] Input: JSON data of travel preference information sent from the terminal.

[1783] Data calculation: Data is analyzed to extract information such as travel destination, budget, number of days, and desired activities. Furthermore, an emotion engine is activated at this time to evaluate the user's emotional state based on their facial expressions and voice input. The emotion engine uses facial expression analysis technology (using the camera), voice analysis technology (using the microphone), and text analysis technology.

[1784] Output: The analysis results in information that reflects the user's wishes and emotional state.

[1785] Step 4:

[1786] The server searches the database and generates the optimal travel plan.

[1787] Input: Travel preference information and emotional evaluation data.

[1788] Data calculation: Searches the database for hotels, transportation (flights and trains), and activity information suitable for the travel destination, and combines this data to generate multiple travel plans. For example, if a user is rated as wanting to relax, a plan including a quiet beach resort hotel will be generated.

[1789] Output: Multiple itineraries are generated and their details are stored as data.

[1790] Step 5:

[1791] The server transmits the generated travel plan to the terminal.

[1792] Input: The generated itinerary data.

[1793] Data processing: Convert travel plan data into JSON format or other format for sending to the device.

[1794] Output: The travel plan sent to the device is displayed on the user's UI. For example, a three-night stay at a beach resort hotel in Okinawa (total price: 50,000 yen) and a round-trip flight (30,000 yen) are displayed as a list.

[1795] Step 6:

[1796] The user reviews and selects a travel plan.

[1797] Input: Multiple travel plans displayed on the device.

[1798] How it works: The user reviews each plan in detail and selects the one that best suits their needs.

[1799] Output: The data of the selected plan is sent from the device to the server.

[1800] Step 7:

[1801] The server will proceed with the reservation procedure for the selected plan.

[1802] Input: Travel plan data selected by the user.

[1803] Data Computing: Generate API requests to send booking information to accommodation and transportation providers. For example, send requests to hotel and airline booking APIs.

[1804] Output: The reservation information is sent to the provider, and if successful, the result is returned to the server and the user is notified.

[1805] Step 8:

[1806] The user completes the payment process.

[1807] Input: Information about your confirmed travel plan and the payment options displayed on the payment page (credit card, e-money, etc.).

[1808] How it works: The user enters payment information and makes a payment.

[1809] Output: Payment information is sent from the terminal to the server.

[1810] Step 9:

[1811] The server processes the payment and sends a confirmation email.

[1812] Input: Payment information submitted by the user.

[1813] Data calculation: Executes the payment process and checks whether the payment is successful. If successful, records the payment as completed in the system.

[1814] Output: A confirmation message confirming payment is completed and a confirmation email containing the trip details is sent to the user.

[1815] (Application example 2)

[1816] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1817] Conventional travel plan suggestion systems have struggled to provide optimal travel experiences because they lack the ability to personalize based on users' emotions and real-time situations. Furthermore, they lack the ability to obtain and suggest detailed information about travel destinations and activities in real time, making it difficult to respond to users' changing needs. Furthermore, it has been difficult to provide real-time guidance and reservation functions during travel.

[1818] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recognizing emotions by analyzing the user's facial expressions and voice, means for personalizing plan suggestions based on the user's emotions, means for providing real-time information on tourist spots and activities being visited, and means for checking real-time availability information on proposed activities and accommodations and making reservations. This makes it possible to provide a personalized travel experience optimized for the user's emotions and real-time situation.

[1819] A "means for interactively inputting user travel preferences" is a device or software that allows a user to input travel preferences via voice or text through an interface.

[1820] "Means for suggesting optimal travel destinations, means of transportation, and accommodations based on input travel preferences" refers to an algorithm or software function that analyzes the desired information input by the user and selects and suggests optimal travel destinations, means of transportation, and accommodations accordingly.

[1821] "Means for displaying proposed travel plans and reserving the plan selected by the user" is a function that displays the travel plan selected by the user on the screen and allows the user to proceed with the reservation procedure for that plan after selection.

[1822] The "means for carrying out payment procedures during the reservation process" refers to a processing means for making payment for the travel plan selected by the user using a credit card or electronic money.

[1823] "Means for recognizing emotions by analyzing a user's facial expressions and voice" refers to technology that uses a user's camera or microphone to analyze their facial expressions and voice, and infers the user's emotions from the analysis results.

[1824] "Means for personalizing plan suggestions based on the user's emotions" is a function that generates travel plans that are optimal for the user's psychological state based on the results of emotion analysis.

[1825] "Means for providing real-time information on tourist spots and activities being visited" is a system for providing real-time information on activities and spots related to the tourist spots that the user is visiting.

[1826] "A means to check real-time availability of suggested activities and accommodations and make reservations" is a function that allows you to check the current availability of suggested activities and accommodations and confirm reservations on the spot.

[1827] The present invention achieves a higher level of personalization by combining a system in which a user interactively inputs their travel preferences, proposes an optimal travel plan based on those preferences, and performs all the booking and payment procedures in one go with an emotion engine that recognizes the user's emotions. An embodiment of the present invention will be described below.

[1828] The system consists of devices such as smart glasses, smartphones, and head-mounted displays, servers, databases, an emotion analysis engine, and a payment processing system. Specific hardware and software used include Google Glass, Microsoft HoloLens, iPhones, Android devices, Affectiva SDK, Google Cloud Speech-to-Text, Amazon Transcribe, Google Maps API, Mapbox, Stripe API, and PayPal.

[1829] The system operates in the following steps:

[1830] First, the user uses smart glasses, a smartphone, or a head-mounted display to input their travel preferences by voice or text. This input information is sent to the server via the device. For example, the user might input, "I want to go to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen."

[1831] The server then receives the user's input and uses the Affectiva SDK or a similar emotion analysis engine to analyze the user's facial expressions and voice to recognize their emotions. Based on the recognized emotion, the server searches a database for the best travel destinations, accommodations, and transportation options, and generates multiple itineraries. The itinerary is personalized based on the results of the emotion analysis; for example, if the server recognizes that the user wants to relax, quiet resorts will be prioritized.

[1832] The generated travel plan is sent to the device and displayed to the user. The user selects the most suitable plan from the displayed plans. The selected plan is sent back to the server, and the reservation process begins. The server then sends the reservation information to accommodation and transportation providers via API.

[1833] In addition, the user can check real-time availability information for suggested activities and accommodations, and make instant reservations. Payments are processed using Stripe API or PayPal, with credit card or electronic money payments. Once payment is complete, the server sends a confirmation message to the user, providing them with their trip details.

[1834] In addition, the results of the emotion analysis are stored in the system and used as reference data when generating future travel plans. The next time you plan a trip, more advanced personalization based on past emotion data will be possible.

[1835] Examples of specific prompts include:

[1836] "I want to travel to Okinawa and relax at a beach resort. My budget is 100,000 yen."

[1837] In this way, the present invention can provide a highly personalized travel experience that is optimized to the user's emotions and real-time circumstances.

[1838] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1839] Step 1:

[1840] The user interactively inputs their travel preferences using smart glasses, a smartphone, or a head-mounted display. The user inputs "I want to go to Okinawa. I want to relax at a beach resort. My budget is 100,000 yen" by voice, and the device converts this into text data. This input text is generated.

[1841] Step 2:

[1842] The device sends the generated text data to the server. The device then structures the entered travel desire data into JSON format and sends it to the server. For example, the format might be {"Location": "Okinawa", "Number of days": "3", "Budget": "100,000 yen", "Activity": "Beach resort"}.

[1843] Step 3:

[1844] The server analyzes the received data. To determine the user's desired information, it processes the text data and uses the Affectiva SDK and Google Cloud Speech-to-Text to analyze voice and facial expression data to recognize the user's emotional state. This analysis confirms that the user is seeking relaxation.

[1845] Step 4:

[1846] The server then searches the database for multiple travel plans based on the analysis results. The search results include information on destinations, accommodations, transportation, and activities. For example, a plan for a three-night stay at a beach resort hotel in Okinawa and a round-trip flight for 30,000 yen might be included.

[1847] Step 5:

[1848] The server sends the generated itinerary to the terminal, which displays the proposed itineraries on the screen. The user can select and confirm a specific itinerary from the displayed itinerary.

[1849] Step 6:

[1850] The user selects the desired travel plan. The selected plan is retransmitted to the server via the terminal. For example, the selection might be "Resort A, 3 nights, round-trip flight included, total price 80,000 yen."

[1851] Step 7:

[1852] The server sends the reservation information to the accommodation and transportation providers according to the selected plan. For example, it sends the reservation information via API in the format {"hotel_id": "123", "checkin_date": "2023-12-01", "checkout_date": "2023-12-04"}.

[1853] Step 8:

[1854] If the reservation is confirmed, the terminal displays a payment procedure page, where the user selects a payment method such as credit card or electronic money and enters payment information.

[1855] Step 9:

[1856] The server processes the entered payment information and executes the payment using the Stripe API or PayPal. Once payment is complete, the server sends a confirmation message to the user and provides the trip details.

[1857] Step 10:

[1858] The system stores the results of the sentiment analysis as reference data for generating future travel plans, allowing for even more personalized planning the next time you plan a trip.

[1859] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1860] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1861] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1862] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1863] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1864] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1865] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1866] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1867] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1868] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1869] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1870] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1871] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1872] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1873] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1874] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1875] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1876] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-men...

Claims

1. means for a user to interactively input travel preferences; A means of suggesting the best travel destinations, transportation and accommodations based on input travel preferences; means for displaying suggested travel plans and allowing the user to book a plan selected by the user; A means of making payment during the booking process; A system including:

2. 10. The system of claim 1, wherein the user's input is transmitted to a server, which retrieves the travel plan from a database.

3. 10. The system of claim 1, further comprising means for presenting suggested travel plans to the user and for allowing the user to confirm a selected plan.

4. 2. The system according to claim 1, further comprising means for facilitating on-site payments by electronic money.

Images