system

Abstract

Provide a system. A means for a user to input travel itineraries, destinations, and activities of interest; means for obtaining weather information based on input information; a means for collecting event information; A means for obtaining accommodation information; means for integrating weather information, event information, and accommodation information to generate travel suggestions; means for calculating travel costs based on the travel proposal; means for displaying the generated travel plans and their costs to the user; a means for the user to review the travel suggestions and input their booking intent; A way to make reservations for accommodation, events, and transportation; means for notifying the user of the reservation result; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional travel planning systems, after users input their travel itinerary and destination, they had to separately collect weather information, event information, and accommodation information based on that information, making it difficult to integrate this information to create optimal travel plans. Furthermore, various reservation procedures had to be completed separately, which was time-consuming and laborious for users. Furthermore, travel plans that did not take weather and events into account could result in lower travel satisfaction due to bad weather or unexpected events. [Means for solving the problem]

[0005] The present invention provides a system in which a user inputs a travel itinerary, destination, and activities of interest, and then acquires weather information, event information, and accommodation information based on that information. It also provides a system that integrates the acquired weather, event, and accommodation information to generate a travel plan, and calculates travel costs based on the generated travel plan. It also provides a system that displays the generated travel plan and its costs to the user, and a system that allows the user to confirm the travel plan and input their reservation intention. Furthermore, by incorporating a system for making reservations for accommodation, events, and transportation, and a system for notifying the user of the reservation results, it provides a system that efficiently performs travel planning and reservation procedures all at once.

[0006] "Travel itinerary" refers to a specific period of time during which a user wishes to travel.

[0007] "Destination" refers to a specific place or city to which a user plans to travel.

[0008] "Activities of Interest" refers to specific activities or events that a user wishes to undertake while traveling.

[0009] "Weather Information" refers to weather data for a particular destination during the duration of the trip.

[0010] "Event Information" refers to data about events and activities taking place at a particular destination during your trip.

[0011] "Accommodation Information" means data relating to the accommodation available and prices at a particular destination during the travel period.

[0012] A "trip suggestion" is a travel plan generated based on the user's travel itinerary, destinations, activities of interest, weather information, event information, and accommodation information.

[0013] "Travel Cost" means the total cost of the trip calculated based on the Trip Plan.

[0014] "Booking intent" refers to a user's expressed desire to actually make a reservation based on the presented travel proposal.

[0015] "Accommodation reservation" refers to the process of confirming the use of accommodation during a trip.

[0016] "Event Booking" refers to the process of securing tickets to events or activities you wish to participate in during your trip.

[0017] "Transportation reservation" refers to the process of booking the means of transportation to be used during the travel period.

[0018] "Reservation result" refers to information indicating the success or failure of the reservation procedure made by the user and details thereof.

[0019] "User" refers to a person who uses the system to make travel plans. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] The present invention is a system that allows a user to plan and book a trip all at once. A specific embodiment of this system will be described below.

[0042] System configuration

[0043] This system consists of a user's device, a server, and external data sources (weather API, event API, accommodation API, etc.). Users operate their own devices to make travel plans, and the information is sent to the server, which then collects and analyzes the necessary data to create travel plans.

[0044] Program processing overview

[0045] 1. User enters travel information

[0046] Users open the app and enter their travel dates, destinations, and activities of interest.

[0047] 2. The device sends the input information to the server

[0048] The device sends the entered travel information to the server using a method such as an HTTP request.

[0049] 3. The server retrieves weather information

[0050] The server calls the weather API to obtain weather information for the destination during the trip, using an external weather information provider.

[0051] 4. The server retrieves the event information

[0052] The server calls the event API to obtain information about events held at the travel destination during the travel period, by collecting data from an event information service.

[0053] 5. The server obtains the accommodation information

[0054] The server calls the accommodation information API to obtain the availability and price information of accommodations at the destination during the travel period. For this purpose, the accommodation information provider service is used.

[0055] 6. The server generates travel suggestions

[0056] The server integrates the acquired weather, event, and accommodation information to generate an optimal travel plan based on the user's interests. For example, it suggests outdoor activities on sunny days and indoor activities on rainy days.

[0057] 7. The server calculates the cost of the trip

[0058] The server calculates the total cost of the trip by adding up accommodation costs, event participation fees, transportation costs, etc. according to the generated travel plan.

[0059] 8. The server sends the travel plan and costs to the device.

[0060] The server sends the generated travel plan and its cost to the terminal so that the user can check it.

[0061] 9. The device displays trip suggestions and costs to the user.

[0062] The terminal displays the travel proposal and cost information received from the server to the user, who can then check the details of the proposed travel plan (daily activity plan, cost breakdown).

[0063] 10. User reviews trip suggestions and makes booking

[0064] The user checks the proposed travel plans and costs, and if satisfied, makes a reservation by clicking the "Book" button on the screen.

[0065] 11. The device sends the reservation information to the server

[0066] The terminal sends the user's reservation intention to the server using an HTTP request.

[0067] 12. The server sends a reservation request to the partner service.

[0068] The server calls the accommodation reservation API, event reservation API, and transportation reservation API to make each reservation.

[0069] 13. The server checks the reservation result and notifies the device

[0070] The server checks the responses from each reservation service, determines whether the reservation was successful, and notifies the terminal of the result.

[0071] 14. The device displays a notification to the user that the reservation is complete.

[0072] The terminal displays the reservation result received from the server to the user. If successful, it displays the reservation details (such as accommodation information and event ticket information), and if unsuccessful, it displays an appropriate error message.

[0073] Specific example explanation

[0074] For example, if a user inputs "I would like to visit art museums in Tokyo from May 20th to May 23rd," the device sends this information to the server. The server obtains weather information for Tokyo from May 20th to 23rd, then collects information on art events and special exhibitions at art museums being held in Tokyo during the travel period. It also obtains information on accommodations in Tokyo during that period. The server then generates a travel itinerary that includes outdoor art events on sunny days and indoor museum visits on rainy days, and calculates the total cost based on that itinerary.

[0075] This information is sent to the device, where the user can confirm the proposed trip and its cost. Once the user confirms and clicks the "Book" button, the device sends the reservation intent to the server. The server then sends a reservation request to each linked service, and if the reservation is successful, the result is notified to the device and displayed to the user.

[0076] By implementing the present invention, users can efficiently plan their trips without hassle and easily complete the reservation procedure.

[0077] The processing flow will be explained below.

[0078] Step 1:

[0079] A user launches the app and enters their travel dates, destinations, and activities they are interested in. For example, a user might enter, "I want to visit museums in Tokyo from May 20th to May 23rd."

[0080] Step 2:

[0081] The terminal obtains the user's input information and sends the data to the server, usually in the form of an HTTP request.

[0082] Step 3:

[0083] The server calls an external weather API to obtain weather information. The server obtains and analyzes weather data for the relevant period according to the travel period and destination entered by the user. For example, it obtains information that May 20th and 21st will be sunny, and May 22nd and 23rd will be rainy.

[0084] Step 4:

[0085] The server calls an external event API to get event information. The server gets a list of events taking place at the destination during the travel period and filters the event information based on the user's interests. For example, it gets information about special museum exhibitions and art festivals taking place during the period.

[0086] Step 5:

[0087] The server calls an external accommodation API to get accommodation information. The server gets the availability and price information of accommodation during the travel period. For example, it uses OYO service to get the list of accommodations available during the period and their prices.

[0088] Step 6:

[0089] The server generates travel plans by integrating weather, event, and accommodation information. The server creates plans that include outdoor art events on sunny days and indoor museum visits on rainy days. It also selects the most suitable accommodations.

[0090] Step 7:

[0091] The server calculates the travel costs based on the travel plan it generates. The server adds up the cost of accommodation, event participation, and transportation (car rental, taxi, etc.) to calculate the total cost. A detailed breakdown of the costs is also provided.

[0092] Step 8:

[0093] The server sends the generated travel plans and their costs to the device, usually in JSON format, so that the device can easily parse them.

[0094] Step 9:

[0095] The device analyzes the travel proposals and cost information received from the server and displays them to the user, who can then check the details of the proposed travel plan (daily schedule, cost breakdown).

[0096] Step 10:

[0097] The user checks the proposed travel plan and costs, and if satisfied, clicks the "Book" button, thereby indicating their intention to begin the booking process.

[0098] Step 11:

[0099] The terminal sends the user's reservation intention to the server. The reservation data is sent to the server as an HTTP request.

[0100] Step 12:

[0101] The server calls the APIs of each service it connects to and starts the specific reservation procedure. The server makes each reservation using the accommodation reservation API, event reservation API, and transportation reservation API.

[0102] Step 13:

[0103] The server checks the response from each reservation service and determines whether the reservation was successful. The server then sends data to notify the terminal of the reservation result (success / failure).

[0104] Step 14:

[0105] The terminal analyzes the reservation results received from the server and displays them to the user. If the reservation is successful, detailed reservation information (such as the accommodation address and event ticket information) is displayed, and if the reservation is unsuccessful, an error message is displayed to notify the user.

[0106] The above is a specific flow of processing from user input to reservation completion.

[0107] Example 1

[0108] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] Traditional travel planning systems require users to manually research and book trip itineraries, destinations, and activities of interest across multiple websites and applications. This manual process is time-consuming, inconvenient, and prone to errors. It also places a burden on users, making it difficult to integrate all aspects of a trip (weather, events, accommodations, etc.) into a single plan. To solve these problems, a system that can capture and integrate all travel information in one place is needed.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0111] In this invention, the server includes means for acquiring weather information from an external data providing service, means for acquiring event information from the external data providing service, and means for acquiring accommodation information from the external data providing service. As a result, when a user inputs a travel itinerary, destination, and activities of interest, the necessary weather information, event information, and accommodation information are automatically acquired and integrated, allowing the user to efficiently plan a trip without hassle.

[0112] A "user" is a person who uses the system to make travel plans.

[0113] "Travel itinerary" is information indicating the start and end dates of the period during which the user plans to travel.

[0114] "Destination" is information indicating a specific place or area that the user wants to visit during their trip.

[0115] "Interesting Activities" is information indicating specific activities or events that the user would like to do during the trip.

[0116] A "server" is a computer system that receives travel information sent by a user, obtains necessary data from an external data provider based on that information, and generates travel plans.

[0117] A "terminal" is a device through which a user inputs travel information and displays travel suggestions and costs sent from the server.

[0118] "External data providing services" are various online services that provide weather information, event information, accommodation information, etc.

[0119] "Weather information" is information that indicates weather data for the destination during the user's travel period.

[0120] "Event information" is information about various events that will be held at the destination during the user's travel period.

[0121] "Accommodation information" is data that indicates room availability and price information for accommodation facilities where the user can stay during their travel period.

[0122] A "travel plan" is a specific travel itinerary plan generated by the server based on information input by the user.

[0123] "Travel expenses" is information indicating the overall expenses, such as accommodation expenses, event participation fees, and transportation expenses, based on the generated travel plan.

[0124] "Booking intent" is an indication that the user agrees with the proposed travel plan and wishes to actually make a reservation.

[0125] The "reservation result" is the result of the server sending a reservation request to each linked service, and includes information on whether the reservation was successful or not, as well as detailed information.

[0126] System Overview

[0127] This system is a comprehensive support system that allows users to make travel plans and make reservations based on those plans. The main components are the user's device, a server, and external data provision services (weather API, event API, accommodation API, etc.). When the user enters travel information using their device, the server collects and analyzes the necessary data to generate optimal travel suggestions.

[0128] Hardware and software used

[0129] In this system, users use devices such as smartphones and tablets, and the server is a high-performance computer system installed in a remote data center. The main software tools used are the following services and APIs:

[0130] To obtain weather information, use a weather API (e.g., OpenWeatherMap).

[0131] To get event information, use an event API (e.g., Eventbrite API).

[0132] To obtain accommodation information, use an accommodation API, e.g., Booking.com API.

[0133] Program processing overview

[0134] First, a user opens the application on their device and inputs their travel itinerary, destination, and activities they are interested in. The device then sends this information to the server via an HTTP request. Based on the received data, the server retrieves weather, event, and accommodation information from various external data providers.

[0135] The server integrates the collected data and generates an optimal travel plan based on the user's requests. The generated travel plan includes the schedule for each day and a breakdown of costs. The server sends this information to the terminal and displays it for the user to review.

[0136] When the user is satisfied with the proposed travel plan and clicks the "Book" button, the device sends the reservation intent to the server. The server then calls the accommodation, event, and transportation reservation APIs to automatically complete the reservation process. Once the reservation result is received, the server notifies the device and displays detailed reservation information to the user.

[0137] Specific example explanation

[0138] For example, let's consider what happens when a user enters, "I want to visit art museums in Tokyo from May 20th to May 23rd." The user enters this information on their device and presses the send button. The device then sends this information to the server. The server calls the OpenWeatherMap API to obtain weather information for Tokyo from May 20th to 23rd. At the same time, it uses the Eventbrite API to collect information on art events and special museum exhibitions taking place in Tokyo during the travel period. It then calls the Booking.com API to obtain information on accommodations in Tokyo during that period.

[0139] Based on the information it has acquired, the server generates a travel plan that includes outdoor art events on days when fine weather is predicted, and indoor museum visits on days when rain is predicted. The total cost is calculated based on this plan, and the result is sent to the device. The user checks the travel plan and cost on the device and clicks the "Book" button. The device sends the reservation intent to the server, and the server calls each reservation API to complete the reservation procedure. If the reservation is successful, the result is notified to the device, and detailed reservation information is displayed to the user.

[0140] Prompt Sentence Examples

[0141] Below is an example of a prompt sentence to input to the generative AI model.

[0142] Please explain in detail each step of the system where the program creates travel plans and completes the booking process. In particular, please explain the flow from input to display of results and the APIs used.

[0143] The above is a detailed description of the embodiment of the present invention. This system allows users to efficiently plan their trips and complete reservation procedures without hassle.

[0144] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0145] Step 1:

[0146] User enters travel information

[0147] A user opens a travel planning app and enters their travel dates (e.g., May 20 to May 23, 2023), destination (e.g., Tokyo), and activities of interest (e.g., visiting museums). The information entered is stored in data entry fields on the device.

[0148] Step 2:

[0149] The device sends the input information to the server

[0150] The terminal sends the travel information entered by the user to the server as a single JSON-formatted data. Specifically, it uses an HTTP POST request to send the "travel information entered by the user." This data is prepared to be parsed by the server. The input is the user's travel information, and the output is the server's receipt of the request.

[0151] Step 3:

[0152] The server retrieves weather information

[0153] The server calls a weather API based on the received user travel information. For example, it sends a request to the OpenWeatherMap API saying, "Please get weather information for Tokyo from May 20th to May 23rd, 2023." The weather information (weather, temperature, etc.) returned by the API is received, analyzed, and stored in a database. The input is the travel dates and destination, and the output is the weather information for that period.

[0154] Step 4:

[0155] The server retrieves the event information

[0156] The server calls an event API based on the received weather information. For example, it sends a request to the Eventbrite API saying, "Get information about art events taking place in Tokyo from May 20th to May 23rd, 2023." The event information (event name, location, date and time, etc.) returned by the API is received and stored in a database. The input is the travel dates, destination, and activities of interest, and the output is the event information for that period.

[0157] Step 5:

[0158] The server obtains accommodation information

[0159] The server calls the accommodation information API based on the received event information. For example, it sends a request to the Booking.com API saying, "Please get information about accommodations available in Tokyo from May 20th to May 23rd, 2023." The accommodation information (facility name, availability, price, etc.) returned by the API is received and stored in a database. The input is the travel dates and destination, and the output is accommodation information for that period.

[0160] Step 6:

[0161] The server generates travel suggestions

[0162] The server integrates the acquired weather, event, and accommodation information. It automatically generates an optimal travel plan based on the user's interests. For example, a plan might be created to attend an outdoor art event on a sunny day, and visit a museum on a rainy day. Based on this information, a detailed travel plan is created, including a daily itinerary and transportation methods. The input is weather information, event information, and accommodation information, and the output is a travel plan.

[0163] Step 7:

[0164] The server calculates the trip cost

[0165] The server calculates the total cost of the trip based on the generated travel suggestions. It adds up accommodation costs, event participation fees, transportation costs, etc., and provides the optimal plan within the budget. For example, it calculates "accommodation costs 5,000 yen / night x 3 nights + event participation fee 2,000 yen + transportation costs 1,000 yen = total cost 17,000 yen." The input is the travel suggestions, and the output is the total cost.

[0166] Step 8:

[0167] The server sends travel plans and costs to the terminal

[0168] The server sends the generated travel plan and its cost to the terminal. Specifically, it returns the travel plan and detailed cost information to the terminal as a new HTTP response. It formats the data so that the user can check it and sends it. The input is the travel plan and detailed cost information, and the output is the response to the terminal.

[0169] Step 9:

[0170] The device displays travel suggestions and costs to the user

[0171] The terminal displays the travel plan and cost information received from the server. Details such as "Itinerary for May 20th: Tokyo art events" and "Itinerary for May 21st: Museum tours" are displayed on the screen, along with the total cost to the user. The input is the travel plan and cost details, and the output is an enjoyable travel plan presented to the user.

[0172] Step 10:

[0173] User reviews trip suggestions and makes booking

[0174] The user reviews the proposed itinerary and costs, and if satisfied, clicks the "Book" button. The input is the itinerary details, and the output is a click indicating the intention to book.

[0175] Step 11:

[0176] The device sends the reservation information to the server

[0177] The device then sends the information indicating the user's booking intention back to the server. Specifically, it uses an HTTP POST request to send JSON data containing the content "The user has booked a travel plan." The input is the user's booking intention, and the output is a request sent to the server.

[0178] Step 12:

[0179] The server sends a reservation request to the associated service.

[0180] The server sends each reservation request to the accommodation reservation API, event reservation API, and transportation reservation API. For example, it sends specific request details such as "confirm accommodation reservation," "purchase event tickets," or "reserve transportation tickets." The input is the user's reservation information, and the output is a request sent to each reservation service.

[0181] Step 13:

[0182] The server checks the reservation result and notifies the device

[0183] The server receives responses from each API and checks whether the reservation was successful. If successful, it notifies the terminal of the result "Reservation successful." If unsuccessful, it generates and sends an appropriate error message. The input is the result of the reservation (success or failure), and the output is feedback to the terminal.

[0184] Step 14:

[0185] The device displays a notification to the user that the reservation is complete.

[0186] The terminal displays the reservation results received from the server to the user. If the reservation is successful, it displays detailed information such as "Accommodation: XX Hotel, Event: XX Exhibition, Transportation: Shinkansen", and if the reservation is unsuccessful, it displays an error message such as "Reservation failed. Please try again." The input is the reservation result, and the output is the feedback displayed to the user.

[0187] (Application example 1)

[0188] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0189] Traditional travel planning and booking systems require users to manually gather information from multiple sources and create travel plans, which is time-consuming. They also lack easy access to real-time information during the trip, making it difficult to change or adjust plans. Furthermore, the lack of systems that support specialized visual devices, voice input, or gesture control limits the user experience.

[0190] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0191] In this invention, the server includes: means for a user to input a travel itinerary, destinations, and activities of interest; means for acquiring weather information based on the input information; means for collecting event information; means for acquiring accommodation information; means for generating a travel plan by integrating weather information, event information, and accommodation information; means for calculating travel costs based on the travel plan; means for a user to confirm the travel plan and input a reservation intention; means for making reservations for accommodation, events, and transportation; means for notifying the user of the reservation results; means for a user to confirm and operate the travel information via a visual device; means for inputting travel information by voice input and gesture operation; and means for acquiring and displaying weather information, event information, and accommodation information in real time. This allows a user to efficiently plan a travel plan all at once and obtain and operate information in real time, enabling a flexible and comfortable travel experience.

[0192] "Means for user input of travel itinerary, destinations, and activities of interest" refers to an interface that allows a user to input detailed information about their trip using smart glasses or other visual device.

[0193] "Means for obtaining weather information based on input information" refers to a process for obtaining weather data from an external weather API based on the travel itinerary and destination input by the user.

[0194] "Means of collecting event information" refers to the function for obtaining information about events taking place at the travel destination from an external event calendar API.

[0195] "Means for obtaining accommodation information" refers to the mechanism for obtaining accommodation availability and price information from an external accommodation information API during the user's travel period.

[0196] "Means for generating travel plans by integrating weather information, event information, and accommodation information" refers to algorithms or systems that integrate various collected information and generate optimal travel plans based on the user's interests.

[0197] The "means for calculating travel costs based on the travel plan" refers to a function for calculating the total travel costs based on the content of the generated travel plan.

[0198] The "means for displaying the generated travel proposals and their costs to the user" refers to an interface for displaying the travel plan and cost information so that the user can check it via a visual device.

[0199] "Means for users to confirm travel plans and input their intention to make a reservation" refers to a function that allows users to input their intention to make a reservation for the displayed travel plan using voice commands or gesture operations.

[0200] "Means for booking accommodation, events, and transportation" refers to the process of booking accommodation, event participation, and transportation through an external service based on the user's booking intention.

[0201] "Means for notifying the user of the reservation result" refers to a function for notifying the user whether the reservation was successful or not.

[0202] "Means for users to view and manipulate travel information via a visual device" refers to an interface that allows users to view and manipulate details of their travel plans using a visual device such as smart glasses.

[0203] "Means for inputting travel information by voice input and gesture operation" refers to a function that allows a user to input travel information using voice recognition and gesture recognition.

[0204] "Means for acquiring and displaying weather information, event information, and accommodation information in real time" refers to a system for instantly acquiring and displaying the latest weather information, event information, and accommodation information in response to a user request.

[0205] The travel planning and booking system of the present invention utilizes a user's visual device (e.g., smart glasses) to plan and book a trip all at once. The system is composed of a user terminal, a server, and external data sources (weather API, event API, accommodation API, etc.), and specific embodiments are described in the detailed description below.

[0206] System Configuration

[0207] User terminal

[0208] Users wear the smart glasses and input their travel itinerary, destinations, and activities of interest through voice and gesture control, reducing user effort and enabling more intuitive operation.

[0209] server

[0210] The server includes the following means:

[0211] 1. How to obtain weather information: Weather information is obtained from an external weather API based on the travel itinerary and destination entered by the user.

[0212] 2. Event information collection method: Event information of interest to users is collected from an external event calendar API.

[0213] 3. How to obtain accommodation information: Accommodation availability and pricing information is obtained from an external accommodation information API.

[0214] 4. Travel plan generation method: The acquired weather information, event information, and accommodation information are integrated to generate the optimal travel plan based on the user's interests.

[0215] 5. Travel cost calculation method: Calculate the total travel cost based on the generated travel plans.

[0216] 6. Result display means: The generated travel plans and their cost information are displayed on the user's smart glasses.

[0217] 7. Means for inputting reservation intention: The user inputs the reservation intention using voice commands or gesture operations.

[0218] 8. Reservation Methods: Make reservations for accommodation, events, and transportation through external services.

[0219] 9. Notification means: Notify the user of the reservation results.

[0220] Devices and Software

[0221] The specific hardware and software used are as follows:

[0222] Hardware: Smart glasses (e.g., Google® Glass®)

[0223] Software: Speech recognition library (Google Cloud Speech API), gesture recognition library, external API (weather API, event API, accommodation API), data analysis library

[0224] Data processing flow

[0225] 1. Data input: The user inputs by voice, "I would like to travel to Kyoto next weekend and am interested in visiting museums and cafes." Information can also be input using gestures, and this information is sent from the user's device to the server.

[0226] 2. Information collection: The server calls the weather API, event API, and accommodation API to collect the necessary weather information, event information, and accommodation information.

[0227] 3. Plan Generation: The system integrates the collected information and generates an optimal travel plan based on the user's interests. For example, it suggests outdoor art events on sunny days and includes a visit to a museum on rainy days.

[0228] 4. Cost calculation and display: The total travel cost is calculated based on the generated plan and the information is overlaid within the field of view of the smart glasses.

[0229] 5. Reservation procedure: If the user is satisfied with the displayed travel plan, he / she can make a reservation by using the voice command "Reserve." The server sends a reservation request to each linked service, collects the reservation results, and notifies the user.

[0230] Examples of concrete examples and prompts

[0231] For example, if a user says, "Plan a trip to Kyoto for next weekend to visit museums and cafes," the system will gather the appropriate information and display something like this:

[0232] "Dates: October 20th to October 23rd"

[0233] "Destination: Kyoto"

[0234] "Activities: Visiting museums and cafes"

[0235] "Travel plan: October 20th: Kyoto International Museum, XX Art Museum (sunny weather forecast)"

[0236] "October 21st: XX Cafe Tour (Rain Forecast)"

[0237] "Total cost:\XXXXX"

[0238] "Would you like to make a reservation?"

[0239] By implementing the present invention in this way, users can plan their trips efficiently and intuitively, and can obtain and manipulate information in real time, allowing them to enjoy a flexible and comfortable travel experience.

[0240] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0241] Step 1:

[0242] The user puts on the smart glasses and uses voice input or gesture control to input their travel itinerary, destinations, and activities they are interested in. For example, the user might say, "I'd like to visit museums and cafes in Kyoto next weekend." This input information is then converted into text data by the device.

[0243] Input: User voice or gesture control

[0244] Output: Text data (travel itinerary, destinations, activities of interest)

[0245] Step 2:

[0246] The device sends the entered text data to the server using a method such as an HTTP request.

[0247] Input: Text data

[0248] Output: HTTP request

[0249] Step 3:

[0250] The server analyzes the received text data and calls an external weather API based on the travel itinerary and destination to obtain weather information for the destination during the travel period. The weather information is obtained in JSON format.

[0251] Input: Text data (travel itinerary, destination)

[0252] Output: Weather information (JSON data)

[0253] Step 4:

[0254] The server calls the event API to retrieve information about events taking place at the travel destination during the trip. The event information is filtered based on the user's interests and retrieved in JSON format.

[0255] Input: Text data (travel itinerary, destination, activities of interest)

[0256] Output: Event information (JSON data)

[0257] Step 5:

[0258] The server calls the accommodation information API to obtain information about accommodations at the destinations during the trip. The accommodation information includes room availability and price information, and is obtained in JSON format.

[0259] Input: Text data (travel itinerary, destination)

[0260] Output: Accommodation information (JSON data)

[0261] Step 6:

[0262] The server integrates the acquired weather, event, and accommodation information to generate an optimal travel plan based on the user's interests. Based on the weather information, it suggests outdoor events on sunny days and indoor activities on rainy days.

[0263] Input: Weather information, event information, accommodation information

[0264] Output: Travel plan

[0265] Step 7:

[0266] The server calculates the travel expenses based on the generated travel plan, including accommodation, event participation fees, and transportation costs.

[0267] Input: Travel Plan

[0268] Output: Trip cost

[0269] Step 8:

[0270] The server sends the generated travel plan and its cost information to the user's smart glasses using an HTTP response.

[0271] Input: Travel plan, travel expenses

[0272] Output: HTTP response

[0273] Step 9:

[0274] The user reviews the travel plan and cost information displayed on the smart glasses, and if satisfied, inputs an intent such as "book" using voice commands or gestures.

[0275] Input: Travel plan, travel expenses

[0276] Output: Reservation intent (voice or gesture operation)

[0277] Step 10:

[0278] The device sends the user's reservation intention to the server, again using a method such as an HTTP request.

[0279] Input: Booking Intent

[0280] Output: HTTP request

[0281] Step 11:

[0282] The server calls an external booking API to make reservations for accommodation, events, and transportation. The booking results are returned in JSON format.

[0283] Input: Booking Intent

[0284] Output: Reservation results (JSON data)

[0285] Step 12:

[0286] The server analyzes the reservation result and generates a success message and reservation details if successful, or an error message if unsuccessful, and sends them to the user's smart glasses.

[0287] Input: Reservation result

[0288] Output: Success or error message

[0289] Step 13:

[0290] The user checks the reservation results displayed on the smart glasses. If successful, the trip details (including accommodation information and event ticket information) are displayed.

[0291] Input: Success message or error message

[0292] Output: Trip details or error message

[0293] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0294] The present invention is a travel planning and booking system incorporating an emotion engine that recognizes the user's emotions. A specific embodiment of this system will be described below.

[0295] System configuration

[0296] The system consists of a user's device, a server, external data sources (weather API, event API, accommodation API, etc.), and an emotion engine. Users operate their devices to make travel plans, and the information is sent to the server. The server collects and analyzes the necessary data to create travel plans. Furthermore, the emotion engine recognizes the user's emotions and adjusts the travel plans accordingly.

[0297] Program processing overview

[0298] 1. User enters travel information

[0299] Users open the app and enter their travel dates, destinations, and activities they are interested in. For example, a user might enter, "I want to visit museums in Tokyo from May 20th to May 23rd."

[0300] 2. The device sends the input information to the server

[0301] The device receives the travel information entered by the user and sends the data to the server, usually in the form of an HTTP request.

[0302] 3. Emotion engine recognizes user emotions

[0303] The device uses an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions, tone of voice, and / or text input to determine the user's emotional state (e.g., joy, sadness, excitement, stress, etc.).

[0304] 4. The server retrieves weather information

[0305] The server calls the weather API to obtain weather information for the destination during the trip. The server analyzes the weather data according to the trip period and destination entered by the user.

[0306] 5. The server retrieves the event information

[0307] The server calls the event API to retrieve event information held at the destination during the trip, and filters the event information based on the user's interests.

[0308] 6. The server obtains the accommodation information

[0309] The server calls the accommodation information API to obtain information on room availability and prices for accommodations during the travel period, and analyzes the obtained data to select the optimal accommodation plan.

[0310] 7. The server generates travel suggestions

[0311] The server integrates the acquired weather, event, and accommodation information and generates travel plans taking into account the user's emotional state. For example, if the user is feeling stressed, it will create a plan that includes many relaxing activities, while if the user is excited, it will create a plan that includes many active activities.

[0312] 8. The server calculates the cost of the trip

[0313] The server calculates the total cost of the trip by adding up accommodation costs, event participation fees, and transportation costs (such as rental cars and taxis) based on the generated travel plan. A detailed breakdown of the costs is also provided.

[0314] 9. The server sends the travel plan and costs to the device.

[0315] The server sends the generated travel plans and their costs to the device, usually in JSON format, which can be easily parsed by the device.

[0316] 10. The device displays trip suggestions and costs to the user.

[0317] The device analyzes the travel proposals and cost information received from the server and displays them to the user, who can then check the details of the proposed travel plan (daily schedule, cost breakdown).

[0318] 11. User reviews trip suggestions and makes booking

[0319] The user checks the proposed travel plans and costs, and if satisfied, clicks the "Book" button, thereby indicating their intention to begin the booking process.

[0320] 12. The device sends the reservation information to the server

[0321] The terminal sends the user's reservation intention to the server. The reservation data is sent to the server as an HTTP request.

[0322] 13. The server calls the API of each service it is connected to and starts the specific reservation procedure.

[0323] The server uses the accommodation reservation API, event reservation API, and transportation reservation API to make each reservation.

[0324] 14. The server checks the response from each reservation service and notifies the device.

[0325] The server checks the responses from each reservation service, determines whether the reservation was successful, and notifies the terminal of the result.

[0326] 15. The terminal displays the reservation results to the user

[0327] The terminal analyzes the reservation results received from the server and displays them to the user. If the reservation is successful, detailed reservation information (such as the accommodation address and event ticket information) is displayed, but if the reservation is unsuccessful, an error message is displayed to notify the user.

[0328] Specific example explanation

[0329] For example, if a user inputs "I want to visit art museums in Tokyo from May 20th to May 23rd," and the emotion engine determines that the user is "feeling stressed" based on their facial expression and tone of voice when they input, the device will send this information to the server. The server will then obtain weather, event, and accommodation information, and create a travel plan that prioritizes relaxing facilities and events based on the user's emotional data.

[0330] In this way, by implementing the present invention, it is possible to provide a travel plan that is customized according to the user's emotional state, thereby improving user satisfaction.

[0331] The processing flow will be explained below.

[0332] Step 1:

[0333] A user launches the app and enters their travel dates, destinations, and activities they are interested in. For example, a user might enter, "I want to visit museums in Tokyo from May 20th to May 23rd."

[0334] Step 2:

[0335] The terminal obtains the user's input information and sends the data to the server, usually using an HTTP request.

[0336] Step 3:

[0337] To recognize the user's emotions, the emotion engine captures and analyzes the user's facial expressions. It also analyzes the user's tone of voice and the content of the text entered by the user to determine their emotional state. For example, if the user enters text with a tired expression, it will determine that they are "feeling stressed."

[0338] Step 4:

[0339] The device sends the emotion data determined by the emotion engine to the server as an HTTP request.

[0340] Step 5:

[0341] The server calls an external weather API to obtain weather information. Based on the travel period and destination entered by the user, the server obtains and analyzes weather data for the relevant period.

[0342] Step 6:

[0343] The server calls an external event API to get event information. The server gets a list of events taking place at the destination during the travel period and filters the event information based on the user's interests. For example, it gets information about special museum exhibitions and art festivals taking place during the period.

[0344] Step 7:

[0345] The server calls an external accommodation API to get accommodation information. The server gets the availability and price information of accommodations during the travel period. For example, it gets the list of accommodations available during the period and their prices.

[0346] Step 8:

[0347] The server generates travel plans by integrating weather information, event information, accommodation information, and emotion data. For example, if the user is feeling stressed, the server creates a plan that includes many relaxing activities and quiet accommodations. If the user is excited, the server creates a plan that includes active activities.

[0348] Step 9:

[0349] The server calculates the travel costs based on the travel plan it generates. The server adds up the cost of accommodation, event participation, and transportation (car rental, taxi, etc.) to calculate the total cost. A detailed breakdown of the costs is also provided.

[0350] Step 10:

[0351] The server sends the generated travel plans and their costs to the device, usually in JSON format, so that the device can easily parse them.

[0352] Step 11:

[0353] The device analyzes the travel proposals and cost information received from the server and displays them to the user, who can then check the details of the proposed travel plan (daily schedule, cost breakdown).

[0354] Step 12:

[0355] The user checks the proposed travel plan and costs, and if satisfied, clicks the "Book" button, thereby indicating their intention to begin the booking process.

[0356] Step 13:

[0357] The terminal sends the user's reservation intention to the server. The reservation data is sent to the server as an HTTP request.

[0358] Step 14:

[0359] The server calls the APIs of each service it connects to and starts the specific reservation procedure. The server makes each reservation using the accommodation reservation API, event reservation API, and transportation reservation API.

[0360] Step 15:

[0361] The server checks the response from each reservation service and determines whether the reservation was successful. The server then sends data to notify the terminal of the reservation result (success / failure).

[0362] Step 16:

[0363] The terminal analyzes the reservation results received from the server and displays them to the user. If the reservation is successful, detailed reservation information (such as the accommodation address and event ticket information) is displayed, and if the reservation is unsuccessful, an error message is displayed to notify the user.

[0364] Step 17:

[0365] The user can confirm the reservation details and complete the trip planning. The user can then prepare for the trip based on the displayed information.

[0366] In this way, the present invention proposes travel plans customized according to the user's emotional state, and can handle everything from travel planning to booking all at once. By being provided with plans that take emotions into consideration, users can have a more satisfying travel experience.

[0367] Example 2

[0368] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0369] Conventional travel planning and booking systems generate itineraries without considering the user's emotions, which means that the user's satisfaction and relaxation level are not fully reflected. Furthermore, even when integrating weather, event, and accommodation information, providing information that ignores the user's emotional state is insufficient to optimize the user experience.

[0370] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for the user to input a travel itinerary, destination, and activities of interest; means for acquiring weather information based on the input information; means for collecting event information; means for acquiring accommodation information; means for recognizing the user's emotional state; means for generating a travel plan by integrating weather information, event information, accommodation information, and the user's emotional state; means for calculating travel costs based on the travel plan; means for displaying the generated travel plan and its costs to the user; means for the user to confirm the travel plan and input their reservation intention; means for making reservations for accommodation, events, and transportation; and means for notifying the user of the reservation results. This makes it possible to provide a travel plan customized according to the user's emotional state, thereby improving user satisfaction.

[0371] A "user" is an individual or entity that uses the travel planning and booking system to enter travel information, obtain travel suggestions, and make reservations.

[0372] A "travel itinerary" is a period consisting of the start and end dates of a trip specified by a user.

[0373] A "destination" is a place or area that a user visits during a trip.

[0374] An "interesting activity" is an event, activity, or interest that a user wishes to participate in while traveling.

[0375] "Weather information" refers to information about the weather conditions at the destination during the travel period, including temperature, probability of precipitation, wind speed, etc.

[0376] "Event information" refers to information about events and activities that will be held at the destination during the trip.

[0377] "Accommodation information" refers to information regarding the availability and prices of accommodations available for stay during the travel period.

[0378] "Emotional state" refers to a user's psychological and emotional state, determined by analyzing the user's facial expressions, tone of voice, text input, etc.

[0379] A "travel suggestion" is a specific travel plan proposed to a user, which is generated based on weather information, event information, accommodation information, and the user's emotional state.

[0380] "Travel Cost" means the total cost of the trip calculated based on the Trip Plan, including accommodation, event participation fees, and transportation costs.

[0381] "Booking intent" refers to the user's willingness to accept the proposed travel plan and to actually begin the booking process.

[0382] "Reservation Result" means the result of the reservation process for accommodation, events, and transportation, including information on the success or failure of the reservation.

[0383] This invention is a travel planning and reservation system incorporating an emotion engine that recognizes user emotions. The system consists of a user's terminal, a server, external data sources (weather data service, event data service, accommodation information service, etc.), and the emotion engine. The user operates their terminal to make travel plans, and the information is sent to the server. The server collects and analyzes the necessary data and creates travel plans. Furthermore, the emotion engine recognizes the user's emotions and adjusts the travel plans based on them.

[0384] System configuration

[0385] This system consists of a user's device, a server, external data sources (weather data service, event data service, accommodation information service, etc.), and an emotion engine. The user enters travel information through the device, and the input information is sent to the server. The server retrieves and analyzes the necessary data from the external data sources, while the emotion engine analyzes the user's emotions.

[0386] Hardware and Software

[0387] The devices used are smartphones or PCs, which must be equipped with a camera and microphone. The server is a system server with the computing power to process data, and has an emotion engine and data analysis engine installed. External data sources include weather data services (e.g., OpenWeatherMap API), event data services (e.g., Eventbrite API), and accommodation information services (e.g., Booking.com API).

[0388] How the system works

[0389] The user enters their travel itinerary, destinations, and activities of interest into the app. For example, they might enter, "I want to visit museums in Tokyo from May 20th to May 23rd." This information is sent to the server by the device. The device sends the information to the server through a REST API in the form of an HTTP request.

[0390] The emotion engine recognizes the user's facial expressions and tone of voice through the camera and microphone, and determines their emotional state (e.g., OpenFace or IBM Watson (registered trademark) emotion analysis API). For example, if the user is smiling, it will be determined to be "happy."

[0391] The server calls a weather data provider (e.g., OpenWeatherMap API) to obtain weather information for the destination during the travel period, then retrieves event information for the destination from an event data provider (e.g., Eventbrite API) and filters it based on the user's interests, and finally retrieves accommodation availability and price information from an accommodation information provider (e.g., Booking.com API).

[0392] The server integrates all the acquired information (weather, event, accommodation, and the user's emotional state) to generate travel suggestions. For example, if the user is feeling stressed, the server creates a plan that includes many relaxing activities.

[0393] Based on the generated travel plan, the server calculates the travel cost, including accommodation, event participation, and transportation costs, and provides a detailed breakdown. The travel plan and its cost are sent to the device in JSON format, which the device parses and displays to the user.

[0394] The user checks the travel plan and, if satisfied, clicks the "Book" button. The device sends the user's reservation intention to the server, which then calls the accommodation reservation API, event reservation API, and transportation reservation API to carry out the specific reservation procedure. The reservation result is notified to the device from the server, and the final reservation information is displayed to the user.

[0395] Specific example explanation

[0396] For example, if a user inputs "I want to visit art museums in Tokyo from May 20th to May 23rd," and the emotion engine determines that the user is "feeling stressed" based on their facial expression and tone of voice, that information is sent to the server. The server retrieves weather, event, and accommodation information from external data sources and creates a travel plan that prioritizes relaxing facilities and events based on the user's emotional state.

[0397] Examples of specific prompts are:

[0398] A system creates customized travel plans based on the user's emotional state. The user inputs that they want to visit museums in Tokyo from May 20th to May 23rd. The emotion engine determines that the user is stressed based on their facial expressions and tone of voice. Based on this information, create a travel plan that includes many relaxing activities.

[0399] In this way, the present invention provides a customized travel plan according to the user's emotional state, making it possible to improve user satisfaction.

[0400] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0401] Step 1:

[0402] Users input their travel itinerary, destinations, and activities they are interested in. Using a smartphone or PC application, users input travel information such as "I would like to visit art museums in Tokyo from May 20th to May 23rd." This information is stored as data entry on the device.

[0403] Step 2:

[0404] The terminal sends the input information to the server. The travel information entered by the user is sent to the server in the form of an HTTP request. Specifically, the information is transmitted to the server using the REST API and the POST method. The input data is received by the server and processing begins.

[0405] Step 3:

[0406] The emotion engine recognizes the user's emotions. The server analyzes the user's facial expressions and tone of voice data sent from the device to determine their emotional state. Specifically, emotion engines such as OpenFace and IBM Watson's emotion analysis API are used. The result of the data analysis is the user's emotional state (for example, "joy," "sadness," "excitement," "stress," etc.).

[0407] Step 4:

[0408] The server retrieves weather information. The server calls an external weather data provider (for example, OpenWeatherMap API) to retrieve weather information for the destination during the travel period. The input is the user's travel period and destination data, and the output is the corresponding weather data in JSON format.

[0409] Step 5:

[0410] The server obtains event information. The server calls an external event data provider (for example, Eventbrite API) to obtain information about events held at the destination during the travel period. It then filters the information based on the user's interests and extracts useful event information. The input is the user's interests and travel period / destination data, and the output is related event information.

[0411] Step 6:

[0412] The server retrieves accommodation information. The server calls an accommodation information service (e.g., Booking.com API) to retrieve the availability and price information of accommodations during the travel period. The input is the user's travel period and destination data, and the output is accommodation information and price data.

[0413] Step 7:

[0414] The server generates travel suggestions by integrating the acquired weather information, event information, accommodation information, and the user's emotional state. Data processing involves integrating data from different sources to create an optimal travel plan based on the user's emotional state. For example, for a user who is feeling stressed, a plan that includes many relaxing activities is created. The output is design data for specific travel suggestions.

[0415] Step 8:

[0416] The server calculates the travel costs. Based on the generated travel plan, the server adds up accommodation costs, event participation fees, transportation costs, etc. to calculate the total cost of the trip. A detailed breakdown of the costs is also provided. The input is the travel plan data, and the output is detailed travel costs.

[0417] Step 9:

[0418] The server sends the travel plan and cost to the terminal. The server sends the generated travel plan and its cost in JSON format to the terminal. The terminal receives and analyzes it. The input is the travel plan and cost data, and the analyzed data is transferred as the output.

[0419] Step 10:

[0420] The terminal displays the travel plan and costs to the user. The terminal analyzes the travel plan and cost information received from the server and presents it visually to the user. Specifically, the screen displays the daily plan and cost breakdown. The input is the analyzed travel plan and cost data, and the output is a visual display.

[0421] Step 11:

[0422] The user confirms the proposed trip and makes a reservation. The user checks the proposed trip and the cost, and if satisfied, clicks the "Book" button. This indicates the terminal's intention to start the reservation procedure. The input is the user's confirmation operation, and the output is a signal to start the reservation.

[0423] Step 12:

[0424] The terminal sends reservation information to the server. The terminal then sends the user's reservation intention to the server as an HTTP request. Specifically, the POST method is used. The input is the reservation start intention data, and the output is a request sent to the server.

[0425] Step 13:

[0426] The server calls the API of each linked service and starts the specific reservation procedure. The server calls the accommodation reservation API, event reservation API, and transportation reservation API to proceed with each reservation process. For example, it checks accommodation availability and confirms the reservation, secures event tickets, and arranges transportation. The input is reservation information data, and the output is a request to each reservation service.

[0427] Step 14:

[0428] The server checks the response from each reservation service and notifies the terminal. The server checks the response from each reservation service and determines whether the reservation was successful. The input is the response data from each service, and the reservation result is obtained as output and transferred to the terminal.

[0429] Step 15:

[0430] The terminal displays the reservation results to the user. The terminal analyzes the reservation results received from the server and displays them visually to the user. If successful, detailed reservation information (e.g., accommodation address, event ticket information, etc.) is displayed, and if unsuccessful, an error message is displayed. The input is the reservation result data, and the output is a visual display of the reservation result.

[0431] (Application example 2)

[0432] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0433] Conventional travel planning and reservation systems can automatically create itineraries based on user input, but they have the problem of being unable to customize them to take into account the user's emotional state. As a result, it is difficult to provide travel plans that reduce user stress and fatigue, limiting the improvement of user satisfaction. Furthermore, optimizing the work environment in factories while taking into account the emotions of operators is also a challenge.

[0434] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0435] In this invention, the server includes means for generating a travel plan by integrating weather information, event information, and accommodation information using an emotion engine that recognizes the user's emotional state, means for calculating travel costs based on the travel plan, and means for displaying the generated travel plan and its costs to the user, thereby enabling customized travel plans and optimization of the work environment at a factory site that take the user's emotional state into consideration.

[0436] A "user" is a person who operates the system to input information such as travel itinerary and destination.

[0437] "Travel itinerary" is information indicating the start and end dates of a trip planned by a user.

[0438] "Destination" is information indicating a place that the user wants to visit.

[0439] "Interesting Activities" is information indicating activities and events that the user wants to do during the trip.

[0440] "Weather information" is data related to the weather at the destination.

[0441] "Event information" is data about events and activities held at a destination.

[0442] "Accommodation information" is data related to accommodation facilities at the destination.

[0443] "Emotional state" refers to the user's psychological state (e.g., joy, sadness, stress, etc.).

[0444] An "emotion engine" is software or hardware for recognizing and analyzing a user's emotions.

[0445] A "travel suggestion" is a travel plan that is generated based on the user's input information and emotional state.

[0446] "Trip Cost" means the total cost calculated based on the proposed trip.

[0447] "Server" is a central computer system that processes information sent from user terminals, retrieves data, and generates travel proposals.

[0448] A "user terminal" is a device (e.g., a smartphone, tablet, etc.) that a user uses to input information and exchange information with a server.

[0449] The present invention is a travel planning and booking system incorporating an emotion engine that recognizes the user's emotions. The details of this system are described below.

[0450] System configuration

[0451] This system consists of a user device, a server, external data sources (weather API, event API, accommodation API, etc.), and an emotion engine. Users operate their devices to make travel plans, and the information is sent to the server. The server collects and analyzes the necessary data and creates travel plans. Furthermore, the emotion engine recognizes the user's emotions and adjusts the travel plans accordingly.

[0452] Hardware and software used

[0453] Emotion recognition hardware: camera, microphone (captures the user's facial expressions and tone of voice)

[0454] Server: A central computer system for data analysis and travel proposal generation

[0455] Emotion engine software: Microsoft® Azure® Cognitive Services and Google Cloud Vision API

[0456] User devices: smartphones, tablets, etc.

[0457] Program processing

[0458] The server processes the data using the following methods:

[0459] 1. Data collection: Collect travel information from user devices and data from weather API, event API, and accommodation API.

[0460] 2. Emotion Recognition: Analyze the user's emotional state in real time using an emotion engine.

[0461] 3. Data integration and analysis: Integrate the collected information and sentiment data and perform data analysis.

[0462] 4. Travel plan generation: Generate optimal travel plans based on the user's emotional state.

[0463] 5. Displaying the results: The generated travel plans and costs are sent to the user's terminal and displayed to the user.

[0464] Specific example explanation

[0465] For example, if a user inputs "I want to visit art museums in Tokyo from May 20th to May 23rd," and the emotion engine determines that the user is "feeling stressed" based on their facial expression and tone of voice when they input, the device will send this information to the server. The server will then obtain weather information, event information, and accommodation information, and create a travel plan that prioritizes relaxing facilities and events based on the user's emotional data. In this way, a travel plan customized to the user's emotional state can be provided, improving user satisfaction.

[0466] Generative AI model prompt example

[0467] "Draw up a stress reduction work plan based on the following data:

[0468] Facial expressions

[0469] Tone of voice

[0470] Past work history

[0471] Work environment data (temperature, humidity, light, sound level)

[0472] Work Contents

[0473] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0474] Step 1:

[0475] Users input their travel itinerary, destinations, and activities they are interested in. Specifically, they use a device such as a smartphone or tablet to enter travel information into the application, such as "I would like to visit art museums in Tokyo from May 20th to May 23rd." This information is captured as input data.

[0476] Step 2:

[0477] The terminal sends the entered travel information to the server. Specifically, it uses an HTTP request to transfer the input data (travel itinerary, destination, and activities of interest) to the server. The input of this step is the travel information from the user, and the output is the data sent to the server.

[0478] Step 3:

[0479] The device uses an emotion engine to recognize the user's emotions. Specifically, it uses a camera and microphone to capture the user's facial expressions and tone of voice, which are then analyzed by emotion engine software (e.g., Microsoft Azure Cognitive Services or Google Cloud Vision API). The input for this step is the user's real-time facial and voice data, and the output is a recognized emotional state (e.g., joy, sadness, stress, etc.).

[0480] Step 4:

[0481] The server retrieves weather information. Specifically, it calls an external weather API to retrieve weather data based on the user's travel itinerary and destination. The input for this step is the travel itinerary and destination, and the output is the weather information for the corresponding period.

[0482] Step 5:

[0483] The server retrieves event information. Specifically, it calls the event API to retrieve event information held at the destination during the trip period. It also filters based on the user's interests. The inputs for this step are the trip period, destination, and user interests, and the output is related event information.

[0484] Step 6:

[0485] The server obtains accommodation information. Specifically, it uses the accommodation API to collect and analyze information on room availability and prices for accommodations during the travel period. The input for this step is the travel dates and destination, and the output is the optimal accommodation plan.

[0486] Step 7:

[0487] The server integrates weather information, event information, accommodation information, and the user's recognized emotional state to generate optimal travel recommendations. Specifically, it uses a data analysis algorithm to create a plan that prioritizes relaxation facilities and events. The inputs for this step are weather information, event information, accommodation information, and emotional data, and the output is customized travel recommendations.

[0488] Step 8:

[0489] The server calculates the cost of the trip based on the trip suggestions. Specifically, it calculates the total cost by adding up accommodation costs, event participation fees, transportation costs, etc. A detailed breakdown of the costs is also provided. The input of this step is the generated trip suggestions, and the output is the total cost of the trip.

[0490] Step 9:

[0491] The server sends the generated travel plan and its cost to the user's device. Specifically, it converts the travel plan and cost into JSON format and sends it to the device. The input of this step is the travel plan and cost, and the output is the data sent to the device.

[0492] Step 10:

[0493] The device analyzes the trip suggestions and cost information received from the server and displays them to the user. Specifically, it provides the user with a daily plan and a breakdown of costs in an easy-to-understand format. The input for this step is the trip suggestions and cost data from the server, and the output is the information displayed to the user.

[0494] Step 11:

[0495] The user checks the proposed travel plan and costs, and if satisfied, clicks the "Book" button. This indicates the user's intention to begin the reservation process. The input to this step is confirmation of the travel plan and costs, and the output is the user's intention to make a reservation.

[0496] Step 12:

[0497] The terminal sends the user's reservation intention to the server. Specifically, it sends the reservation data to the server as an HTTP request. The input of this step is the reservation intention from the user, and the output is the reservation data sent to the server.

[0498] Step 13:

[0499] The server uses the accommodation reservation API, event reservation API, and transportation reservation API to make each reservation. Specifically, it calls each API to execute the reservation procedure. The input of this step is the reservation data, and the output is reservation confirmation data.

[0500] Step 14:

[0501] The server checks the response from each reservation service and determines whether the reservation was successful. It notifies the terminal of the result. The input of this step is the reservation confirmation data, and the output is the reservation result sent to the terminal.

[0502] Step 15:

[0503] The terminal analyzes the reservation results received from the server and displays them to the user. Specifically, if the reservation is successful, detailed reservation information (e.g., accommodation address, event ticket information, etc.) is displayed, and if the reservation is unsuccessful, an error message is displayed. The input to this step is the reservation result from the server, and the output is the reservation information displayed to the user.

[0504] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0505] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0506] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0507] [Second embodiment]

[0508] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0509] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0510] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0511] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0512] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0513] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0514] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0515] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0516] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0517] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0518] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0519] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0520] The present invention is a system that allows a user to plan and book a trip all at once. A specific embodiment of this system will be described below.

[0521] System configuration

[0522] This system consists of a user's device, a server, and external data sources (weather API, event API, accommodation API, etc.). Users operate their own devices to make travel plans, and the information is sent to the server, which then collects and analyzes the necessary data to create travel plans.

[0523] Program processing overview

[0524] 1. User enters travel information

[0525] Users open the app and enter their travel dates, destinations, and activities of interest.

[0526] 2. The device sends the input information to the server

[0527] The device sends the entered travel information to the server using a method such as an HTTP request.

[0528] 3. The server retrieves weather information

[0529] The server calls the weather API to obtain weather information for the destination during the trip, using an external weather information provider.

[0530] 4. The server retrieves the event information

[0531] The server calls the event API to obtain information about events held at the travel destination during the travel period, by collecting data from an event information service.

[0532] 5. The server obtains the accommodation information

[0533] The server calls the accommodation information API to obtain the availability and price information of accommodations at the destination during the travel period. For this purpose, the accommodation information provider service is used.

[0534] 6. The server generates travel suggestions

[0535] The server integrates the acquired weather, event, and accommodation information to generate an optimal travel plan based on the user's interests. For example, it suggests outdoor activities on sunny days and indoor activities on rainy days.

[0536] 7. The server calculates the cost of the trip

[0537] The server calculates the total cost of the trip by adding up accommodation costs, event participation fees, transportation costs, etc. according to the generated travel plan.

[0538] 8. The server sends the travel plan and costs to the device.

[0539] The server sends the generated travel plan and its cost to the terminal so that the user can check it.

[0540] 9. The device displays trip suggestions and costs to the user.

[0541] The terminal displays the travel proposal and cost information received from the server to the user, who can then check the details of the proposed travel plan (daily activity plan, cost breakdown).

[0542] 10. User reviews trip suggestions and makes booking

[0543] The user checks the proposed travel plans and costs, and if satisfied, makes a reservation by clicking the "Book" button on the screen.

[0544] 11. The device sends the reservation information to the server

[0545] The terminal sends the user's reservation intention to the server using an HTTP request.

[0546] 12. The server sends a reservation request to the partner service.

[0547] The server calls the accommodation reservation API, event reservation API, and transportation reservation API to make each reservation.

[0548] 13. The server checks the reservation result and notifies the device

[0549] The server checks the responses from each reservation service, determines whether the reservation was successful, and notifies the terminal of the result.

[0550] 14. The device displays a notification to the user that the reservation is complete.

[0551] The terminal displays the reservation result received from the server to the user. If successful, it displays the reservation details (such as accommodation information and event ticket information), and if unsuccessful, it displays an appropriate error message.

[0552] Specific example explanation

[0553] For example, if a user inputs "I would like to visit art museums in Tokyo from May 20th to May 23rd," the device sends this information to the server. The server obtains weather information for Tokyo from May 20th to 23rd, then collects information on art events and special exhibitions at art museums being held in Tokyo during the travel period. It also obtains information on accommodations in Tokyo during that period. The server then generates a travel itinerary that includes outdoor art events on sunny days and indoor museum visits on rainy days, and calculates the total cost based on that itinerary.

[0554] This information is sent to the device, where the user can confirm the proposed trip and its cost. Once the user confirms and clicks the "Book" button, the device sends the reservation intent to the server. The server then sends a reservation request to each linked service, and if the reservation is successful, the result is notified to the device and displayed to the user.

[0555] By implementing the present invention, users can efficiently plan their trips without hassle and easily complete the reservation procedure.

[0556] The processing flow will be explained below.

[0557] Step 1:

[0558] A user launches the app and enters their travel dates, destinations, and activities they are interested in. For example, a user might enter, "I want to visit museums in Tokyo from May 20th to May 23rd."

[0559] Step 2:

[0560] The terminal obtains the user's input information and sends the data to the server, usually in the form of an HTTP request.

[0561] Step 3:

[0562] The server calls an external weather API to obtain weather information. The server obtains and analyzes weather data for the relevant period according to the travel period and destination entered by the user. For example, it obtains information that May 20th and 21st will be sunny, and May 22nd and 23rd will be rainy.

[0563] Step 4:

[0564] The server calls an external event API to get event information. The server gets a list of events taking place at the destination during the travel period and filters the event information based on the user's interests. For example, it gets information about special museum exhibitions and art festivals taking place during the period.

[0565] Step 5:

[0566] The server calls an external accommodation API to get accommodation information. The server gets the availability and price information of accommodation during the travel period. For example, it uses OYO service to get the list of accommodations available during the period and their prices.

[0567] Step 6:

[0568] The server generates travel plans by integrating weather, event, and accommodation information. The server creates plans that include outdoor art events on sunny days and indoor museum visits on rainy days. It also selects the most suitable accommodations.

[0569] Step 7:

[0570] The server calculates the travel costs based on the travel plan it generates. The server adds up the cost of accommodation, event participation, and transportation (car rental, taxi, etc.) to calculate the total cost. A detailed breakdown of the costs is also provided.

[0571] Step 8:

[0572] The server sends the generated travel plans and their costs to the device, usually in JSON format, so that the device can easily parse them.

[0573] Step 9:

[0574] The device analyzes the travel proposals and cost information received from the server and displays them to the user, who can then check the details of the proposed travel plan (daily schedule, cost breakdown).

[0575] Step 10:

[0576] The user checks the proposed travel plan and costs, and if satisfied, clicks the "Book" button, thereby indicating their intention to begin the booking process.

[0577] Step 11:

[0578] The terminal sends the user's reservation intention to the server. The reservation data is sent to the server as an HTTP request.

[0579] Step 12:

[0580] The server calls the APIs of each service it connects to and starts the specific reservation procedure. The server makes each reservation using the accommodation reservation API, event reservation API, and transportation reservation API.

[0581] Step 13:

[0582] The server checks the response from each reservation service and determines whether the reservation was successful. The server then sends data to notify the terminal of the reservation result (success / failure).

[0583] Step 14:

[0584] The terminal analyzes the reservation results received from the server and displays them to the user. If the reservation is successful, detailed reservation information (such as the accommodation address and event ticket information) is displayed, and if the reservation is unsuccessful, an error message is displayed to notify the user.

[0585] The above is a specific flow of processing from user input to reservation completion.

[0586] Example 1

[0587] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0588] Traditional travel planning systems require users to manually research and book trip itineraries, destinations, and activities of interest across multiple websites and applications. This manual process is time-consuming, inconvenient, and prone to errors. It also places a burden on users, making it difficult to integrate all aspects of a trip (weather, events, accommodations, etc.) into a single plan. To solve these problems, a system that can capture and integrate all travel information in one place is needed.

[0589] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0590] In this invention, the server includes means for acquiring weather information from an external data providing service, means for acquiring event information from the external data providing service, and means for acquiring accommodation information from the external data providing service. As a result, when a user inputs a travel itinerary, destination, and activities of interest, the necessary weather information, event information, and accommodation information are automatically acquired and integrated, allowing the user to efficiently plan a trip without hassle.

[0591] A "user" is a person who uses the system to make travel plans.

[0592] "Travel itinerary" is information indicating the start and end dates of the period during which the user plans to travel.

[0593] "Destination" is information indicating a specific place or area that the user wants to visit during their trip.

[0594] "Interesting Activities" is information indicating specific activities or events that the user would like to do during the trip.

[0595] A "server" is a computer system that receives travel information sent by a user, obtains necessary data from an external data provider based on that information, and generates travel plans.

[0596] A "terminal" is a device through which a user inputs travel information and displays travel suggestions and costs sent from the server.

[0597] "External data providing services" are various online services that provide weather information, event information, accommodation information, etc.

[0598] "Weather information" is information that indicates weather data for the destination during the user's travel period.

[0599] "Event information" is information about various events that will be held at the destination during the user's travel period.

[0600] "Accommodation information" is data that indicates room availability and price information for accommodation facilities where the user can stay during their travel period.

[0601] A "travel plan" is a specific travel itinerary plan generated by the server based on information input by the user.

[0602] "Travel expenses" is information indicating the overall expenses, such as accommodation expenses, event participation fees, and transportation expenses, based on the generated travel plan.

[0603] "Booking intent" is an indication that the user agrees with the proposed travel plan and wishes to actually make a reservation.

[0604] The "reservation result" is the result of the server sending a reservation request to each linked service, and includes information on whether the reservation was successful or not, as well as detailed information.

[0605] System Overview

[0606] This system is a comprehensive support system that allows users to make travel plans and make reservations based on those plans. The main components are the user's device, a server, and external data provision services (weather API, event API, accommodation API, etc.). When the user enters travel information using their device, the server collects and analyzes the necessary data to generate optimal travel suggestions.

[0607] Hardware and software used

[0608] In this system, users use devices such as smartphones and tablets, and the server is a high-performance computer system installed in a remote data center. The main software tools used are the following services and APIs:

[0609] To obtain weather information, use a weather API (e.g., OpenWeatherMap).

[0610] To get event information, use an event API (e.g., Eventbrite API).

[0611] To obtain accommodation information, use an accommodation API, e.g., Booking.com API.

[0612] Program processing overview

[0613] First, a user opens the application on their device and inputs their travel itinerary, destination, and activities they are interested in. The device then sends this information to the server via an HTTP request. Based on the received data, the server retrieves weather, event, and accommodation information from various external data providers.

[0614] The server integrates the collected data and generates an optimal travel plan based on the user's requests. The generated travel plan includes the schedule for each day and a breakdown of costs. The server sends this information to the terminal and displays it for the user to review.

[0615] When the user is satisfied with the proposed travel plan and clicks the "Book" button, the device sends the reservation intent to the server. The server then calls the accommodation, event, and transportation reservation APIs to automatically complete the reservation process. Once the reservation result is received, the server notifies the device and displays detailed reservation information to the user.

[0616] Specific example explanation

[0617] For example, let's consider what happens when a user enters, "I want to visit art museums in Tokyo from May 20th to May 23rd." The user enters this information on their device and presses the send button. The device then sends this information to the server. The server calls the OpenWeatherMap API to obtain weather information for Tokyo from May 20th to 23rd. At the same time, it uses the Eventbrite API to collect information on art events and special museum exhibitions taking place in Tokyo during the travel period. It then calls the Booking.com API to obtain information on accommodations in Tokyo during that period.

[0618] Based on the information it has acquired, the server generates a travel plan that includes outdoor art events on days when fine weather is predicted, and indoor museum visits on days when rain is predicted. The total cost is calculated based on this plan, and the result is sent to the device. The user checks the travel plan and cost on the device and clicks the "Book" button. The device sends the reservation intent to the server, and the server calls each reservation API to complete the reservation procedure. If the reservation is successful, the result is notified to the device, and detailed reservation information is displayed to the user.

[0619] Prompt Sentence Examples

[0620] Below is an example of a prompt sentence to input to the generative AI model.

[0621] Please explain in detail each step of the system where the program creates travel plans and completes the booking process. In particular, please explain the flow from input to display of results and the APIs used.

[0622] The above is a detailed description of the embodiment of the present invention. This system allows users to efficiently plan their trips and complete reservation procedures without hassle.

[0623] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0624] Step 1:

[0625] User enters travel information

[0626] A user opens a travel planning app and enters their travel dates (e.g., May 20 to May 23, 2023), destination (e.g., Tokyo), and activities of interest (e.g., visiting museums). The information entered is stored in data entry fields on the device.

[0627] Step 2:

[0628] The device sends the input information to the server

[0629] The terminal sends the travel information entered by the user to the server as a single JSON-formatted data. Specifically, it uses an HTTP POST request to send the "travel information entered by the user." This data is prepared to be parsed by the server. The input is the user's travel information, and the output is the server's receipt of the request.

[0630] Step 3:

[0631] The server retrieves weather information

[0632] The server calls a weather API based on the received user travel information. For example, it sends a request to the OpenWeatherMap API saying, "Please get weather information for Tokyo from May 20th to May 23rd, 2023." The weather information (weather, temperature, etc.) returned by the API is received, analyzed, and stored in a database. The input is the travel dates and destination, and the output is the weather information for that period.

[0633] Step 4:

[0634] The server retrieves the event information

[0635] The server calls an event API based on the received weather information. For example, it sends a request to the Eventbrite API saying, "Get information about art events taking place in Tokyo from May 20th to May 23rd, 2023." The event information (event name, location, date and time, etc.) returned by the API is received and stored in a database. The input is the travel dates, destination, and activities of interest, and the output is the event information for that period.

[0636] Step 5:

[0637] The server obtains accommodation information

[0638] The server calls the accommodation information API based on the received event information. For example, it sends a request to the Booking.com API saying, "Please get information about accommodations available in Tokyo from May 20th to May 23rd, 2023." The accommodation information (facility name, availability, price, etc.) returned by the API is received and stored in a database. The input is the travel dates and destination, and the output is accommodation information for that period.

[0639] Step 6:

[0640] The server generates travel suggestions

[0641] The server integrates the acquired weather, event, and accommodation information. It automatically generates an optimal travel plan based on the user's interests. For example, a plan might be created to attend an outdoor art event on a sunny day, and visit a museum on a rainy day. Based on this information, a detailed travel plan is created, including a daily itinerary and transportation methods. The input is weather information, event information, and accommodation information, and the output is a travel plan.

[0642] Step 7:

[0643] The server calculates the trip cost

[0644] The server calculates the total cost of the trip based on the generated travel suggestions. It adds up accommodation costs, event participation fees, transportation costs, etc., and provides the optimal plan within the budget. For example, it calculates "accommodation costs 5,000 yen / night x 3 nights + event participation fee 2,000 yen + transportation costs 1,000 yen = total cost 17,000 yen." The input is the travel suggestions, and the output is the total cost.

[0645] Step 8:

[0646] The server sends travel plans and costs to the terminal

[0647] The server sends the generated travel plan and its cost to the terminal. Specifically, it returns the travel plan and detailed cost information to the terminal as a new HTTP response. It formats the data so that the user can check it and sends it. The input is the travel plan and detailed cost information, and the output is the response to the terminal.

[0648] Step 9:

[0649] The device displays travel suggestions and costs to the user

[0650] The terminal displays the travel plan and cost information received from the server. Details such as "Itinerary for May 20th: Tokyo art events" and "Itinerary for May 21st: Museum tours" are displayed on the screen, along with the total cost to the user. The input is the travel plan and cost details, and the output is an enjoyable travel plan presented to the user.

[0651] Step 10:

[0652] User reviews trip suggestions and makes booking

[0653] The user reviews the proposed itinerary and costs, and if satisfied, clicks the "Book" button. The input is the itinerary details, and the output is a click indicating the intention to book.

[0654] Step 11:

[0655] The device sends the reservation information to the server

[0656] The device then sends the information indicating the user's booking intention back to the server. Specifically, it uses an HTTP POST request to send JSON data containing the content "The user has booked a travel plan." The input is the user's booking intention, and the output is a request sent to the server.

[0657] Step 12:

[0658] The server sends a reservation request to the associated service.

[0659] The server sends each reservation request to the accommodation reservation API, event reservation API, and transportation reservation API. For example, it sends specific request details such as "confirm accommodation reservation," "purchase event tickets," or "reserve transportation tickets." The input is the user's reservation information, and the output is a request sent to each reservation service.

[0660] Step 13:

[0661] The server checks the reservation result and notifies the device

[0662] The server receives responses from each API and checks whether the reservation was successful. If successful, it notifies the terminal of the result "Reservation successful." If unsuccessful, it generates and sends an appropriate error message. The input is the result of the reservation (success or failure), and the output is feedback to the terminal.

[0663] Step 14:

[0664] The device displays a notification to the user that the reservation is complete.

[0665] The terminal displays the reservation results received from the server to the user. If the reservation is successful, it displays detailed information such as "Accommodation: XX Hotel, Event: XX Exhibition, Transportation: Shinkansen", and if the reservation is unsuccessful, it displays an error message such as "Reservation failed. Please try again." The input is the reservation result, and the output is the feedback displayed to the user.

[0666] (Application example 1)

[0667] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0668] Traditional travel planning and booking systems require users to manually gather information from multiple sources and create travel plans, which is time-consuming. They also lack easy access to real-time information during the trip, making it difficult to change or adjust plans. Furthermore, the lack of systems that support specialized visual devices, voice input, or gesture control limits the user experience.

[0669] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0670] In this invention, the server includes: means for a user to input a travel itinerary, destinations, and activities of interest; means for acquiring weather information based on the input information; means for collecting event information; means for acquiring accommodation information; means for generating a travel plan by integrating weather information, event information, and accommodation information; means for calculating travel costs based on the travel plan; means for a user to confirm the travel plan and input a reservation intention; means for making reservations for accommodation, events, and transportation; means for notifying the user of the reservation results; means for a user to confirm and operate the travel information via a visual device; means for inputting travel information by voice input and gesture operation; and means for acquiring and displaying weather information, event information, and accommodation information in real time. This allows a user to efficiently plan a travel plan all at once and obtain and operate information in real time, enabling a flexible and comfortable travel experience.

[0671] "Means for user input of travel itinerary, destinations, and activities of interest" refers to an interface that allows a user to input detailed information about their trip using smart glasses or other visual device.

[0672] "Means for obtaining weather information based on input information" refers to a process for obtaining weather data from an external weather API based on the travel itinerary and destination input by the user.

[0673] "Means of collecting event information" refers to the function for obtaining information about events taking place at the travel destination from an external event calendar API.

[0674] "Means for obtaining accommodation information" refers to the mechanism for obtaining accommodation availability and price information from an external accommodation information API during the user's travel period.

[0675] "Means for generating travel plans by integrating weather information, event information, and accommodation information" refers to algorithms or systems that integrate various collected information and generate optimal travel plans based on the user's interests.

[0676] The "means for calculating travel costs based on the travel plan" refers to a function for calculating the total travel costs based on the content of the generated travel plan.

[0677] The "means for displaying the generated travel proposals and their costs to the user" refers to an interface for displaying the travel plan and cost information so that the user can check it via a visual device.

[0678] "Means for users to confirm travel plans and input their intention to make a reservation" refers to a function that allows users to input their intention to make a reservation for the displayed travel plan using voice commands or gesture operations.

[0679] "Means for booking accommodation, events, and transportation" refers to the process of booking accommodation, event participation, and transportation through an external service based on the user's booking intention.

[0680] "Means for notifying the user of the reservation result" refers to a function for notifying the user whether the reservation was successful or not.

[0681] "Means for users to view and manipulate travel information via a visual device" refers to an interface that allows users to view and manipulate details of their travel plans using a visual device such as smart glasses.

[0682] "Means for inputting travel information by voice input and gesture operation" refers to a function that allows a user to input travel information using voice recognition and gesture recognition.

[0683] "Means for acquiring and displaying weather information, event information, and accommodation information in real time" refers to a system for instantly acquiring and displaying the latest weather information, event information, and accommodation information in response to a user request.

[0684] The travel planning and booking system of the present invention utilizes a user's visual device (e.g., smart glasses) to plan and book a trip all at once. The system is composed of a user terminal, a server, and external data sources (weather API, event API, accommodation API, etc.), and specific embodiments are described in the detailed description below.

[0685] System Configuration

[0686] User terminal

[0687] Users wear the smart glasses and input their travel itinerary, destinations, and activities of interest through voice and gesture control, reducing user effort and enabling more intuitive operation.

[0688] server

[0689] The server includes the following means:

[0690] 1. How to obtain weather information: Weather information is obtained from an external weather API based on the travel itinerary and destination entered by the user.

[0691] 2. Event information collection method: Event information of interest to users is collected from an external event calendar API.

[0692] 3. How to obtain accommodation information: Accommodation availability and pricing information is obtained from an external accommodation information API.

[0693] 4. Travel plan generation method: The acquired weather information, event information, and accommodation information are integrated to generate the optimal travel plan based on the user's interests.

[0694] 5. Travel cost calculation method: Calculate the total travel cost based on the generated travel plans.

[0695] 6. Result display means: The generated travel plans and their cost information are displayed on the user's smart glasses.

[0696] 7. Means for inputting reservation intention: The user inputs the reservation intention using voice commands or gesture operations.

[0697] 8. Reservation Methods: Make reservations for accommodation, events, and transportation through external services.

[0698] 9. Notification means: Notify the user of the reservation results.

[0699] Devices and Software

[0700] The specific hardware and software used are as follows:

[0701] Hardware: Smart glasses (e.g., Google Glass)

[0702] Software: Speech recognition library (Google Cloud Speech API), gesture recognition library, external API (weather API, event API, accommodation API), data analysis library

[0703] Data processing flow

[0704] 1. Data input: The user inputs by voice, "I would like to travel to Kyoto next weekend and am interested in visiting museums and cafes." Information can also be input using gestures, and this information is sent from the user's device to the server.

[0705] 2. Information collection: The server calls the weather API, event API, and accommodation API to collect the necessary weather information, event information, and accommodation information.

[0706] 3. Plan Generation: The system integrates the collected information and generates an optimal travel plan based on the user's interests. For example, it suggests outdoor art events on sunny days and includes a visit to a museum on rainy days.

[0707] 4. Cost calculation and display: The total travel cost is calculated based on the generated plan and the information is overlaid within the field of view of the smart glasses.

[0708] 5. Reservation procedure: If the user is satisfied with the displayed travel plan, he / she can make a reservation by using the voice command "Reserve." The server sends a reservation request to each linked service, collects the reservation results, and notifies the user.

[0709] Examples of concrete examples and prompts

[0710] For example, if a user says, "Plan a trip to Kyoto for next weekend to visit museums and cafes," the system will gather the appropriate information and display something like this:

[0711] "Dates: October 20th to October 23rd"

[0712] "Destination: Kyoto"

[0713] "Activities: Visiting museums and cafes"

[0714] "Travel plan: October 20th: Kyoto International Museum, XX Art Museum (sunny weather forecast)"

[0715] "October 21st: XX Cafe Tour (Rain Forecast)"

[0716] "Total cost:\XXXXX"

[0717] "Would you like to make a reservation?"

[0718] By implementing the present invention in this way, users can plan their trips efficiently and intuitively, and can obtain and manipulate information in real time, allowing them to enjoy a flexible and comfortable travel experience.

[0719] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0720] Step 1:

[0721] The user puts on the smart glasses and uses voice input or gesture control to input their travel itinerary, destinations, and activities they are interested in. For example, the user might say, "I'd like to visit museums and cafes in Kyoto next weekend." This input information is then converted into text data by the device.

[0722] Input: User voice or gesture control

[0723] Output: Text data (travel itinerary, destinations, activities of interest)

[0724] Step 2:

[0725] The device sends the entered text data to the server using a method such as an HTTP request.

[0726] Input: Text data

[0727] Output: HTTP request

[0728] Step 3:

[0729] The server analyzes the received text data and calls an external weather API based on the travel itinerary and destination to obtain weather information for the destination during the travel period. The weather information is obtained in JSON format.

[0730] Input: Text data (travel itinerary, destination)

[0731] Output: Weather information (JSON data)

[0732] Step 4:

[0733] The server calls the event API to retrieve information about events taking place at the travel destination during the trip. The event information is filtered based on the user's interests and retrieved in JSON format.

[0734] Input: Text data (travel itinerary, destination, activities of interest)

[0735] Output: Event information (JSON data)

[0736] Step 5:

[0737] The server calls the accommodation information API to obtain information about accommodations at the destinations during the trip. The accommodation information includes room availability and price information, and is obtained in JSON format.

[0738] Input: Text data (travel itinerary, destination)

[0739] Output: Accommodation information (JSON data)

[0740] Step 6:

[0741] The server integrates the acquired weather, event, and accommodation information to generate an optimal travel plan based on the user's interests. Based on the weather information, it suggests outdoor events on sunny days and indoor activities on rainy days.

[0742] Input: Weather information, event information, accommodation information

[0743] Output: Travel plan

[0744] Step 7:

[0745] The server calculates the travel expenses based on the generated travel plan, including accommodation, event participation fees, and transportation costs.

[0746] Input: Travel Plan

[0747] Output: Trip cost

[0748] Step 8:

[0749] The server sends the generated travel plan and its cost information to the user's smart glasses using an HTTP response.

[0750] Input: Travel plan, travel expenses

[0751] Output: HTTP response

[0752] Step 9:

[0753] The user reviews the travel plan and cost information displayed on the smart glasses, and if satisfied, inputs an intent such as "book" using voice commands or gestures.

[0754] Input: Travel plan, travel expenses

[0755] Output: Reservation intent (voice or gesture operation)

[0756] Step 10:

[0757] The device sends the user's reservation intention to the server, again using a method such as an HTTP request.

[0758] Input: Booking Intent

[0759] Output: HTTP request

[0760] Step 11:

[0761] The server calls an external booking API to make reservations for accommodation, events, and transportation. The booking results are returned in JSON format.

[0762] Input: Booking Intent

[0763] Output: Reservation results (JSON data)

[0764] Step 12:

[0765] The server analyzes the reservation result and generates a success message and reservation details if successful, or an error message if unsuccessful, and sends them to the user's smart glasses.

[0766] Input: Reservation result

[0767] Output: Success or error message

[0768] Step 13:

[0769] The user checks the reservation results displayed on the smart glasses. If successful, the trip details (including accommodation information and event ticket information) are displayed.

[0770] Input: Success message or error message

[0771] Output: Trip details or error message

[0772] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0773] The present invention is a travel planning and booking system incorporating an emotion engine that recognizes the user's emotions. A specific embodiment of this system will be described below.

[0774] System configuration

[0775] The system consists of a user's device, a server, external data sources (weather API, event API, accommodation API, etc.), and an emotion engine. Users operate their devices to make travel plans, and the information is sent to the server. The server collects and analyzes the necessary data to create travel plans. Furthermore, the emotion engine recognizes the user's emotions and adjusts the travel plans accordingly.

[0776] Program processing overview

[0777] 1. User enters travel information

[0778] Users open the app and enter their travel dates, destinations, and activities they are interested in. For example, a user might enter, "I want to visit museums in Tokyo from May 20th to May 23rd."

[0779] 2. The device sends the input information to the server

[0780] The device receives the travel information entered by the user and sends the data to the server, usually in the form of an HTTP request.

[0781] 3. Emotion engine recognizes user emotions

[0782] The device uses an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions, tone of voice, and / or text input to determine the user's emotional state (e.g., joy, sadness, excitement, stress, etc.).

[0783] 4. The server retrieves weather information

[0784] The server calls the weather API to obtain weather information for the destination during the trip. The server analyzes the weather data according to the trip period and destination entered by the user.

[0785] 5. The server retrieves the event information

[0786] The server calls the event API to retrieve event information held at the destination during the trip, and filters the event information based on the user's interests.

[0787] 6. The server obtains the accommodation information

[0788] The server calls the accommodation information API to obtain information on room availability and prices for accommodations during the travel period, and analyzes the obtained data to select the optimal accommodation plan.

[0789] 7. The server generates travel suggestions

[0790] The server integrates the acquired weather, event, and accommodation information and generates travel plans taking into account the user's emotional state. For example, if the user is feeling stressed, it will create a plan that includes many relaxing activities, while if the user is excited, it will create a plan that includes many active activities.

[0791] 8. The server calculates the cost of the trip

[0792] The server calculates the total cost of the trip by adding up accommodation costs, event participation fees, and transportation costs (such as rental cars and taxis) based on the generated travel plan. A detailed breakdown of the costs is also provided.

[0793] 9. The server sends the travel plan and costs to the device.

[0794] The server sends the generated travel plans and their costs to the device, usually in JSON format, which can be easily parsed by the device.

[0795] 10. The device displays trip suggestions and costs to the user.

[0796] The device analyzes the travel proposals and cost information received from the server and displays them to the user, who can then check the details of the proposed travel plan (daily schedule, cost breakdown).

[0797] 11. User reviews trip suggestions and makes booking

[0798] The user checks the proposed travel plans and costs, and if satisfied, clicks the "Book" button, thereby indicating their intention to begin the booking process.

[0799] 12. The device sends the reservation information to the server

[0800] The terminal sends the user's reservation intention to the server. The reservation data is sent to the server as an HTTP request.

[0801] 13. The server calls the API of each service it is connected to and starts the specific reservation procedure.

[0802] The server uses the accommodation reservation API, event reservation API, and transportation reservation API to make each reservation.

[0803] 14. The server checks the response from each reservation service and notifies the device.

[0804] The server checks the responses from each reservation service, determines whether the reservation was successful, and notifies the terminal of the result.

[0805] 15. The terminal displays the reservation results to the user

[0806] The terminal analyzes the reservation results received from the server and displays them to the user. If the reservation is successful, detailed reservation information (such as the accommodation address and event ticket information) is displayed, but if the reservation is unsuccessful, an error message is displayed to notify the user.

[0807] Specific example explanation

[0808] For example, if a user inputs "I want to visit art museums in Tokyo from May 20th to May 23rd," and the emotion engine determines that the user is "feeling stressed" based on their facial expression and tone of voice when they input, the device will send this information to the server. The server will then obtain weather, event, and accommodation information, and create a travel plan that prioritizes relaxing facilities and events based on the user's emotional data.

[0809] In this way, by implementing the present invention, it is possible to provide a travel plan that is customized according to the user's emotional state, thereby improving user satisfaction.

[0810] The processing flow will be explained below.

[0811] Step 1:

[0812] A user launches the app and enters their travel dates, destinations, and activities they are interested in. For example, a user might enter, "I want to visit museums in Tokyo from May 20th to May 23rd."

[0813] Step 2:

[0814] The terminal obtains the user's input information and sends the data to the server, usually using an HTTP request.

[0815] Step 3:

[0816] To recognize the user's emotions, the emotion engine captures and analyzes the user's facial expressions. It also analyzes the user's tone of voice and the content of the text entered by the user to determine their emotional state. For example, if the user enters text with a tired expression, it will determine that they are "feeling stressed."

[0817] Step 4:

[0818] The device sends the emotion data determined by the emotion engine to the server as an HTTP request.

[0819] Step 5:

[0820] The server calls an external weather API to obtain weather information. Based on the travel period and destination entered by the user, the server obtains and analyzes weather data for the relevant period.

[0821] Step 6:

[0822] The server calls an external event API to get event information. The server gets a list of events taking place at the destination during the travel period and filters the event information based on the user's interests. For example, it gets information about special museum exhibitions and art festivals taking place during the period.

[0823] Step 7:

[0824] The server calls an external accommodation API to get accommodation information. The server gets the availability and price information of accommodations during the travel period. For example, it gets the list of accommodations available during the period and their prices.

[0825] Step 8:

[0826] The server generates travel plans by integrating weather information, event information, accommodation information, and emotion data. For example, if the user is feeling stressed, the server creates a plan that includes many relaxing activities and quiet accommodations. If the user is excited, the server creates a plan that includes active activities.

[0827] Step 9:

[0828] The server calculates the travel costs based on the travel plan it generates. The server adds up the cost of accommodation, event participation, and transportation (car rental, taxi, etc.) to calculate the total cost. A detailed breakdown of the costs is also provided.

[0829] Step 10:

[0830] The server sends the generated travel plans and their costs to the device, usually in JSON format, so that the device can easily parse them.

[0831] Step 11:

[0832] The device analyzes the travel proposals and cost information received from the server and displays them to the user, who can then check the details of the proposed travel plan (daily schedule, cost breakdown).

[0833] Step 12:

[0834] The user checks the proposed travel plan and costs, and if satisfied, clicks the "Book" button, thereby indicating their intention to begin the booking process.

[0835] Step 13:

[0836] The terminal sends the user's reservation intention to the server. The reservation data is sent to the server as an HTTP request.

[0837] Step 14:

[0838] The server calls the APIs of each service it connects to and starts the specific reservation procedure. The server makes each reservation using the accommodation reservation API, event reservation API, and transportation reservation API.

[0839] Step 15:

[0840] The server checks the response from each reservation service and determines whether the reservation was successful. The server then sends data to notify the terminal of the reservation result (success / failure).

[0841] Step 16:

[0842] The terminal analyzes the reservation results received from the server and displays them to the user. If the reservation is successful, detailed reservation information (such as the accommodation address and event ticket information) is displayed, and if the reservation is unsuccessful, an error message is displayed to notify the user.

[0843] Step 17:

[0844] The user can confirm the reservation details and complete the trip planning. The user can then prepare for the trip based on the displayed information.

[0845] In this way, the present invention proposes travel plans customized according to the user's emotional state, and can handle everything from travel planning to booking all at once. By being provided with plans that take emotions into consideration, users can have a more satisfying travel experience.

[0846] Example 2

[0847] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0848] Conventional travel planning and booking systems generate itineraries without considering the user's emotions, which means that the user's satisfaction and relaxation level are not fully reflected. Furthermore, even when integrating weather, event, and accommodation information, providing information that ignores the user's emotional state is insufficient to optimize the user experience.

[0849] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for the user to input a travel itinerary, destination, and activities of interest; means for acquiring weather information based on the input information; means for collecting event information; means for acquiring accommodation information; means for recognizing the user's emotional state; means for generating a travel plan by integrating weather information, event information, accommodation information, and the user's emotional state; means for calculating travel costs based on the travel plan; means for displaying the generated travel plan and its costs to the user; means for the user to confirm the travel plan and input their reservation intention; means for making reservations for accommodation, events, and transportation; and means for notifying the user of the reservation results. This makes it possible to provide a travel plan customized according to the user's emotional state, thereby improving user satisfaction.

[0850] A "user" is an individual or entity that uses the travel planning and booking system to enter travel information, obtain travel suggestions, and make reservations.

[0851] A "travel itinerary" is a period consisting of the start and end dates of a trip specified by a user.

[0852] A "destination" is a place or area that a user visits during a trip.

[0853] An "interesting activity" is an event, activity, or interest that a user wishes to participate in while traveling.

[0854] "Weather information" refers to information about the weather conditions at the destination during the travel period, including temperature, probability of precipitation, wind speed, etc.

[0855] "Event information" refers to information about events and activities that will be held at the destination during the trip.

[0856] "Accommodation information" refers to information regarding the availability and prices of accommodations available for stay during the travel period.

[0857] "Emotional state" refers to a user's psychological and emotional state, determined by analyzing the user's facial expressions, tone of voice, text input, etc.

[0858] A "travel suggestion" is a specific travel plan proposed to a user, which is generated based on weather information, event information, accommodation information, and the user's emotional state.

[0859] "Travel Cost" means the total cost of the trip calculated based on the Trip Plan, including accommodation, event participation fees, and transportation costs.

[0860] "Booking intent" refers to the user's willingness to accept the proposed travel plan and to actually begin the booking process.

[0861] "Reservation Result" means the result of the reservation process for accommodation, events, and transportation, including information on the success or failure of the reservation.

[0862] This invention is a travel planning and reservation system incorporating an emotion engine that recognizes user emotions. The system consists of a user's terminal, a server, external data sources (weather data service, event data service, accommodation information service, etc.), and the emotion engine. The user operates their terminal to make travel plans, and the information is sent to the server. The server collects and analyzes the necessary data and creates travel plans. Furthermore, the emotion engine recognizes the user's emotions and adjusts the travel plans based on them.

[0863] System configuration

[0864] This system consists of a user's device, a server, external data sources (weather data service, event data service, accommodation information service, etc.), and an emotion engine. The user enters travel information through the device, and the input information is sent to the server. The server retrieves and analyzes the necessary data from the external data sources, while the emotion engine analyzes the user's emotions.

[0865] Hardware and Software

[0866] The devices used are smartphones or PCs, which must be equipped with a camera and microphone. The server is a system server with the computing power to process data, and has an emotion engine and data analysis engine installed. External data sources include weather data services (e.g., OpenWeatherMap API), event data services (e.g., Eventbrite API), and accommodation information services (e.g., Booking.com API).

[0867] How the system works

[0868] The user enters their travel itinerary, destinations, and activities of interest into the app. For example, they might enter, "I want to visit museums in Tokyo from May 20th to May 23rd." This information is sent to the server by the device. The device sends the information to the server through a REST API in the form of an HTTP request.

[0869] The emotion engine recognizes the user's facial expressions and tone of voice through the camera or microphone and determines their emotional state (e.g., OpenFace or IBM Watson emotion analysis API). For example, if the user is smiling, it will be determined to be "happy."

[0870] The server calls a weather data provider (e.g., OpenWeatherMap API) to obtain weather information for the destination during the travel period, then retrieves event information for the destination from an event data provider (e.g., Eventbrite API) and filters it based on the user's interests, and finally retrieves accommodation availability and price information from an accommodation information provider (e.g., Booking.com API).

[0871] The server integrates all the acquired information (weather, event, accommodation, and the user's emotional state) to generate travel suggestions. For example, if the user is feeling stressed, the server creates a plan that includes many relaxing activities.

[0872] Based on the generated travel plan, the server calculates the travel cost, including accommodation, event participation, and transportation costs, and provides a detailed breakdown. The travel plan and its cost are sent to the device in JSON format, which the device parses and displays to the user.

[0873] The user checks the travel plan and, if satisfied, clicks the "Book" button. The device sends the user's reservation intention to the server, which then calls the accommodation reservation API, event reservation API, and transportation reservation API to carry out the specific reservation procedure. The reservation result is notified to the device from the server, and the final reservation information is displayed to the user.

[0874] Specific example explanation

[0875] For example, if a user inputs "I want to visit art museums in Tokyo from May 20th to May 23rd," and the emotion engine determines that the user is "feeling stressed" based on their facial expression and tone of voice, that information is sent to the server. The server retrieves weather, event, and accommodation information from external data sources and creates a travel plan that prioritizes relaxing facilities and events based on the user's emotional state.

[0876] Examples of specific prompts are:

[0877] A system creates customized travel plans based on the user's emotional state. The user inputs that they want to visit museums in Tokyo from May 20th to May 23rd. The emotion engine determines that the user is stressed based on their facial expressions and tone of voice. Based on this information, create a travel plan that includes many relaxing activities.

[0878] In this way, the present invention provides a customized travel plan according to the user's emotional state, making it possible to improve user satisfaction.

[0879] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0880] Step 1:

[0881] Users input their travel itinerary, destinations, and activities they are interested in. Using a smartphone or PC application, users input travel information such as "I would like to visit art museums in Tokyo from May 20th to May 23rd." This information is stored as data entry on the device.

[0882] Step 2:

[0883] The terminal sends the input information to the server. The travel information entered by the user is sent to the server in the form of an HTTP request. Specifically, the information is transmitted to the server using the REST API and the POST method. The input data is received by the server and processing begins.

[0884] Step 3:

[0885] The emotion engine recognizes the user's emotions. The server analyzes the user's facial expressions and tone of voice data sent from the device to determine their emotional state. Specifically, emotion engines such as OpenFace and IBM Watson's emotion analysis API are used. The result of the data analysis is the user's emotional state (for example, "joy," "sadness," "excitement," "stress," etc.).

[0886] Step 4:

[0887] The server retrieves weather information. The server calls an external weather data provider (for example, OpenWeatherMap API) to retrieve weather information for the destination during the travel period. The input is the user's travel period and destination data, and the output is the corresponding weather data in JSON format.

[0888] Step 5:

[0889] The server obtains event information. The server calls an external event data provider (for example, Eventbrite API) to obtain information about events held at the destination during the travel period. It then filters the information based on the user's interests and extracts useful event information. The input is the user's interests and travel period / destination data, and the output is related event information.

[0890] Step 6:

[0891] The server retrieves accommodation information. The server calls an accommodation information service (e.g., Booking.com API) to retrieve the availability and price information of accommodations during the travel period. The input is the user's travel period and destination data, and the output is accommodation information and price data.

[0892] Step 7:

[0893] The server generates travel suggestions by integrating the acquired weather information, event information, accommodation information, and the user's emotional state. Data processing involves integrating data from different sources to create an optimal travel plan based on the user's emotional state. For example, for a user who is feeling stressed, a plan that includes many relaxing activities is created. The output is design data for specific travel suggestions.

[0894] Step 8:

[0895] The server calculates the travel costs. Based on the generated travel plan, the server adds up accommodation costs, event participation fees, transportation costs, etc. to calculate the total cost of the trip. A detailed breakdown of the costs is also provided. The input is the travel plan data, and the output is detailed travel costs.

[0896] Step 9:

[0897] The server sends the travel plan and cost to the terminal. The server sends the generated travel plan and its cost in JSON format to the terminal. The terminal receives and analyzes it. The input is the travel plan and cost data, and the analyzed data is transferred as the output.

[0898] Step 10:

[0899] The terminal displays the travel plan and costs to the user. The terminal analyzes the travel plan and cost information received from the server and presents it visually to the user. Specifically, the screen displays the daily plan and cost breakdown. The input is the analyzed travel plan and cost data, and the output is a visual display.

[0900] Step 11:

[0901] The user confirms the proposed trip and makes a reservation. The user checks the proposed trip and the cost, and if satisfied, clicks the "Book" button. This indicates the terminal's intention to start the reservation procedure. The input is the user's confirmation operation, and the output is a signal to start the reservation.

[0902] Step 12:

[0903] The terminal sends reservation information to the server. The terminal then sends the user's reservation intention to the server as an HTTP request. Specifically, the POST method is used. The input is the reservation start intention data, and the output is a request sent to the server.

[0904] Step 13:

[0905] The server calls the API of each linked service and starts the specific reservation procedure. The server calls the accommodation reservation API, event reservation API, and transportation reservation API to proceed with each reservation process. For example, it checks accommodation availability and confirms the reservation, secures event tickets, and arranges transportation. The input is reservation information data, and the output is a request to each reservation service.

[0906] Step 14:

[0907] The server checks the response from each reservation service and notifies the terminal. The server checks the response from each reservation service and determines whether the reservation was successful. The input is the response data from each service, and the reservation result is obtained as output and transferred to the terminal.

[0908] Step 15:

[0909] The terminal displays the reservation results to the user. The terminal analyzes the reservation results received from the server and displays them visually to the user. If successful, detailed reservation information (e.g., accommodation address, event ticket information, etc.) is displayed, and if unsuccessful, an error message is displayed. The input is the reservation result data, and the output is a visual display of the reservation result.

[0910] (Application example 2)

[0911] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0912] Conventional travel planning and reservation systems can automatically create itineraries based on user input, but they have the problem of being unable to customize them to take into account the user's emotional state. As a result, it is difficult to provide travel plans that reduce user stress and fatigue, limiting the improvement of user satisfaction. Furthermore, optimizing the work environment in factories while taking into account the emotions of operators is also a challenge.

[0913] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0914] In this invention, the server includes means for generating a travel plan by integrating weather information, event information, and accommodation information using an emotion engine that recognizes the user's emotional state, means for calculating travel costs based on the travel plan, and means for displaying the generated travel plan and its costs to the user, thereby enabling customized travel plans and optimization of the work environment at a factory site that take the user's emotional state into consideration.

[0915] A "user" is a person who operates the system to input information such as travel itinerary and destination.

[0916] "Travel itinerary" is information indicating the start and end dates of a trip planned by a user.

[0917] "Destination" is information indicating a place that the user wants to visit.

[0918] "Interesting Activities" is information indicating activities and events that the user wants to do during the trip.

[0919] "Weather information" is data related to the weather at the destination.

[0920] "Event information" is data about events and activities held at a destination.

[0921] "Accommodation information" is data related to accommodation facilities at the destination.

[0922] "Emotional state" refers to the user's psychological state (e.g., joy, sadness, stress, etc.).

[0923] An "emotion engine" is software or hardware for recognizing and analyzing a user's emotions.

[0924] A "travel suggestion" is a travel plan that is generated based on the user's input information and emotional state.

[0925] "Trip Cost" means the total cost calculated based on the proposed trip.

[0926] "Server" is a central computer system that processes information sent from user terminals, retrieves data, and generates travel proposals.

[0927] A "user terminal" is a device (e.g., a smartphone, tablet, etc.) that a user uses to input information and exchange information with a server.

[0928] The present invention is a travel planning and booking system incorporating an emotion engine that recognizes the user's emotions. The details of this system are described below.

[0929] System configuration

[0930] This system consists of a user device, a server, external data sources (weather API, event API, accommodation API, etc.), and an emotion engine. Users operate their devices to make travel plans, and the information is sent to the server. The server collects and analyzes the necessary data and creates travel plans. Furthermore, the emotion engine recognizes the user's emotions and adjusts the travel plans accordingly.

[0931] Hardware and software used

[0932] Emotion recognition hardware: camera, microphone (captures the user's facial expressions and tone of voice)

[0933] Server: A central computer system for data analysis and travel proposal generation

[0934] Emotion engine software: Microsoft Azure Cognitive Services and Google Cloud Vision API

[0935] User devices: smartphones, tablets, etc.

[0936] Program processing

[0937] The server processes the data using the following methods:

[0938] 1. Data collection: Collect travel information from user devices and data from weather API, event API, and accommodation API.

[0939] 2. Emotion Recognition: Analyze the user's emotional state in real time using an emotion engine.

[0940] 3. Data integration and analysis: Integrate the collected information and sentiment data and perform data analysis.

[0941] 4. Travel plan generation: Generate optimal travel plans based on the user's emotional state.

[0942] 5. Displaying the results: The generated travel plans and costs are sent to the user's terminal and displayed to the user.

[0943] Specific example explanation

[0944] For example, if a user inputs "I want to visit art museums in Tokyo from May 20th to May 23rd," and the emotion engine determines that the user is "feeling stressed" based on their facial expression and tone of voice when they input, the device will send this information to the server. The server will then obtain weather information, event information, and accommodation information, and create a travel plan that prioritizes relaxing facilities and events based on the user's emotional data. In this way, a travel plan customized to the user's emotional state can be provided, improving user satisfaction.

[0945] Generative AI model prompt example

[0946] "Draw up a stress reduction work plan based on the following data:

[0947] Facial expressions

[0948] Tone of voice

[0949] Past work history

[0950] Work environment data (temperature, humidity, light, sound level)

[0951] Work Contents

[0952] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0953] Step 1:

[0954] Users input their travel itinerary, destinations, and activities they are interested in. Specifically, they use a device such as a smartphone or tablet to enter travel information into the application, such as "I would like to visit art museums in Tokyo from May 20th to May 23rd." This information is captured as input data.

[0955] Step 2:

[0956] The terminal sends the entered travel information to the server. Specifically, it uses an HTTP request to transfer the input data (travel itinerary, destination, and activities of interest) to the server. The input of this step is the travel information from the user, and the output is the data sent to the server.

[0957] Step 3:

[0958] The device uses an emotion engine to recognize the user's emotions. Specifically, it uses a camera and microphone to capture the user's facial expressions and tone of voice, which are then analyzed by emotion engine software (e.g., Microsoft Azure Cognitive Services or Google Cloud Vision API). The input for this step is the user's real-time facial and voice data, and the output is a recognized emotional state (e.g., joy, sadness, stress, etc.).

[0959] Step 4:

[0960] The server retrieves weather information. Specifically, it calls an external weather API to retrieve weather data based on the user's travel itinerary and destination. The input for this step is the travel itinerary and destination, and the output is the weather information for the corresponding period.

[0961] Step 5:

[0962] The server retrieves event information. Specifically, it calls the event API to retrieve event information held at the destination during the trip period. It also filters based on the user's interests. The inputs for this step are the trip period, destination, and user interests, and the output is related event information.

[0963] Step 6:

[0964] The server obtains accommodation information. Specifically, it uses the accommodation API to collect and analyze information on room availability and prices for accommodations during the travel period. The input for this step is the travel dates and destination, and the output is the optimal accommodation plan.

[0965] Step 7:

[0966] The server integrates weather information, event information, accommodation information, and the user's recognized emotional state to generate optimal travel recommendations. Specifically, it uses a data analysis algorithm to create a plan that prioritizes relaxation facilities and events. The inputs for this step are weather information, event information, accommodation information, and emotional data, and the output is customized travel recommendations.

[0967] Step 8:

[0968] The server calculates the cost of the trip based on the trip suggestions. Specifically, it calculates the total cost by adding up accommodation costs, event participation fees, transportation costs, etc. A detailed breakdown of the costs is also provided. The input of this step is the generated trip suggestions, and the output is the total cost of the trip.

[0969] Step 9:

[0970] The server sends the generated travel plan and its cost to the user's device. Specifically, it converts the travel plan and cost into JSON format and sends it to the device. The input of this step is the travel plan and cost, and the output is the data sent to the device.

[0971] Step 10:

[0972] The device analyzes the trip suggestions and cost information received from the server and displays them to the user. Specifically, it provides the user with a daily plan and a breakdown of costs in an easy-to-understand format. The input for this step is the trip suggestions and cost data from the server, and the output is the information displayed to the user.

[0973] Step 11:

[0974] The user checks the proposed travel plan and costs, and if satisfied, clicks the "Book" button. This indicates the user's intention to begin the reservation process. The input to this step is confirmation of the travel plan and costs, and the output is the user's intention to make a reservation.

[0975] Step 12:

[0976] The terminal sends the user's reservation intention to the server. Specifically, it sends the reservation data to the server as an HTTP request. The input of this step is the reservation intention from the user, and the output is the reservation data sent to the server.

[0977] Step 13:

[0978] The server uses the accommodation reservation API, event reservation API, and transportation reservation API to make each reservation. Specifically, it calls each API to execute the reservation procedure. The input of this step is the reservation data, and the output is reservation confirmation data.

[0979] Step 14:

[0980] The server checks the response from each reservation service and determines whether the reservation was successful. It notifies the terminal of the result. The input of this step is the reservation confirmation data, and the output is the reservation result sent to the terminal.

[0981] Step 15:

[0982] The terminal analyzes the reservation results received from the server and displays them to the user. Specifically, if the reservation is successful, detailed reservation information (e.g., accommodation address, event ticket information, etc.) is displayed, and if the reservation is unsuccessful, an error message is displayed. The input to this step is the reservation result from the server, and the output is the reservation information displayed to the user.

[0983] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0984] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0985] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0986] [Third embodiment]

[0987] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0988] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0989] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0990] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0991] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0992] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0993] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0994] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0995] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0996] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0997] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0998] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0999] The present invention is a system that allows a user to plan and book a trip all at once. A specific embodiment of this system will be described below.

[1000] System configuration

[1001] This system consists of a user's device, a server, and external data sources (weather API, event API, accommodation API, etc.). Users operate their own devices to make travel plans, and the information is sent to the server, which then collects and analyzes the necessary data to create travel plans.

[1002] Program processing overview

[1003] 1. User enters travel information

[1004] Users open the app and enter their travel dates, destinations, and activities of interest.

[1005] 2. The device sends the input information to the server

[1006] The device sends the entered travel information to the server using a method such as an HTTP request.

[1007] 3. The server retrieves weather information

[1008] The server calls the weather API to obtain weather information for the destination during the trip, using an external weather information provider.

[1009] 4. The server retrieves the event information

[1010] The server calls the event API to obtain information about events held at the travel destination during the travel period, by collecting data from an event information service.

[1011] 5. The server obtains the accommodation information

[1012] The server calls the accommodation information API to obtain the availability and price information of accommodations at the destination during the travel period. For this purpose, the accommodation information provider service is used.

[1013] 6. The server generates travel suggestions

[1014] The server integrates the acquired weather, event, and accommodation information to generate an optimal travel plan based on the user's interests. For example, it suggests outdoor activities on sunny days and indoor activities on rainy days.

[1015] 7. The server calculates the cost of the trip

[1016] The server calculates the total cost of the trip by adding up accommodation costs, event participation fees, transportation costs, etc. according to the generated travel plan.

[1017] 8. The server sends the travel plan and costs to the device.

[1018] The server sends the generated travel plan and its cost to the terminal so that the user can check it.

[1019] 9. The device displays trip suggestions and costs to the user.

[1020] The terminal displays the travel proposal and cost information received from the server to the user, who can then check the details of the proposed travel plan (daily activity plan, cost breakdown).

[1021] 10. User reviews trip suggestions and makes booking

[1022] The user checks the proposed travel plans and costs, and if satisfied, makes a reservation by clicking the "Book" button on the screen.

[1023] 11. The device sends the reservation information to the server

[1024] The terminal sends the user's reservation intention to the server using an HTTP request.

[1025] 12. The server sends a reservation request to the partner service.

[1026] The server calls the accommodation reservation API, event reservation API, and transportation reservation API to make each reservation.

[1027] 13. The server checks the reservation result and notifies the device

[1028] The server checks the responses from each reservation service, determines whether the reservation was successful, and notifies the terminal of the result.

[1029] 14. The device displays a notification to the user that the reservation is complete.

[1030] The terminal displays the reservation result received from the server to the user. If successful, it displays the reservation details (such as accommodation information and event ticket information), and if unsuccessful, it displays an appropriate error message.

[1031] Specific example explanation

[1032] For example, if a user inputs "I would like to visit art museums in Tokyo from May 20th to May 23rd," the device sends this information to the server. The server obtains weather information for Tokyo from May 20th to 23rd, then collects information on art events and special exhibitions at art museums being held in Tokyo during the travel period. It also obtains information on accommodations in Tokyo during that period. The server then generates a travel itinerary that includes outdoor art events on sunny days and indoor museum visits on rainy days, and calculates the total cost based on that itinerary.

[1033] This information is sent to the device, where the user can confirm the proposed trip and its cost. Once the user confirms and clicks the "Book" button, the device sends the reservation intent to the server. The server then sends a reservation request to each linked service, and if the reservation is successful, the result is notified to the device and displayed to the user.

[1034] By implementing the present invention, users can efficiently plan their trips without hassle and easily complete the reservation procedure.

[1035] The processing flow will be explained below.

[1036] Step 1:

[1037] A user launches the app and enters their travel dates, destinations, and activities they are interested in. For example, a user might enter, "I want to visit museums in Tokyo from May 20th to May 23rd."

[1038] Step 2:

[1039] The terminal obtains the user's input information and sends the data to the server, usually in the form of an HTTP request.

[1040] Step 3:

[1041] The server calls an external weather API to obtain weather information. The server obtains and analyzes weather data for the relevant period according to the travel period and destination entered by the user. For example, it obtains information that May 20th and 21st will be sunny, and May 22nd and 23rd will be rainy.

[1042] Step 4:

[1043] The server calls an external event API to get event information. The server gets a list of events taking place at the destination during the travel period and filters the event information based on the user's interests. For example, it gets information about special museum exhibitions and art festivals taking place during the period.

[1044] Step 5:

[1045] The server calls an external accommodation API to get accommodation information. The server gets the availability and price information of accommodation during the travel period. For example, it uses OYO service to get the list of accommodations available during the period and their prices.

[1046] Step 6:

[1047] The server generates travel plans by integrating weather, event, and accommodation information. The server creates plans that include outdoor art events on sunny days and indoor museum visits on rainy days. It also selects the most suitable accommodations.

[1048] Step 7:

[1049] The server calculates the travel costs based on the travel plan it generates. The server adds up the cost of accommodation, event participation, and transportation (car rental, taxi, etc.) to calculate the total cost. A detailed breakdown of the costs is also provided.

[1050] Step 8:

[1051] The server sends the generated travel plans and their costs to the device, usually in JSON format, so that the device can easily parse them.

[1052] Step 9:

[1053] The device analyzes the travel proposals and cost information received from the server and displays them to the user, who can then check the details of the proposed travel plan (daily schedule, cost breakdown).

[1054] Step 10:

[1055] The user checks the proposed travel plan and costs, and if satisfied, clicks the "Book" button, thereby indicating their intention to begin the booking process.

[1056] Step 11:

[1057] The terminal sends the user's reservation intention to the server. The reservation data is sent to the server as an HTTP request.

[1058] Step 12:

[1059] The server calls the APIs of each service it connects to and starts the specific reservation procedure. The server makes each reservation using the accommodation reservation API, event reservation API, and transportation reservation API.

[1060] Step 13:

[1061] The server checks the response from each reservation service and determines whether the reservation was successful. The server then sends data to notify the terminal of the reservation result (success / failure).

[1062] Step 14:

[1063] The terminal analyzes the reservation results received from the server and displays them to the user. If the reservation is successful, detailed reservation information (such as the accommodation address and event ticket information) is displayed, and if the reservation is unsuccessful, an error message is displayed to notify the user.

[1064] The above is a specific flow of processing from user input to reservation completion.

[1065] Example 1

[1066] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1067] Traditional travel planning systems require users to manually research and book trip itineraries, destinations, and activities of interest across multiple websites and applications. This manual process is time-consuming, inconvenient, and prone to errors. It also places a burden on users, making it difficult to integrate all aspects of a trip (weather, events, accommodations, etc.) into a single plan. To solve these problems, a system that can capture and integrate all travel information in one place is needed.

[1068] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1069] In this invention, the server includes means for acquiring weather information from an external data providing service, means for acquiring event information from the external data providing service, and means for acquiring accommodation information from the external data providing service. As a result, when a user inputs a travel itinerary, destination, and activities of interest, the necessary weather information, event information, and accommodation information are automatically acquired and integrated, allowing the user to efficiently plan a trip without hassle.

[1070] A "user" is a person who uses the system to make travel plans.

[1071] "Travel itinerary" is information indicating the start and end dates of the period during which the user plans to travel.

[1072] "Destination" is information indicating a specific place or area that the user wants to visit during their trip.

[1073] "Interesting Activities" is information indicating specific activities or events that the user would like to do during the trip.

[1074] A "server" is a computer system that receives travel information sent by a user, obtains necessary data from an external data provider based on that information, and generates travel plans.

[1075] A "terminal" is a device through which a user inputs travel information and displays travel suggestions and costs sent from the server.

[1076] "External data providing services" are various online services that provide weather information, event information, accommodation information, etc.

[1077] "Weather information" is information that indicates weather data for the destination during the user's travel period.

[1078] "Event information" is information about various events that will be held at the destination during the user's travel period.

[1079] "Accommodation information" is data that indicates room availability and price information for accommodation facilities where the user can stay during their travel period.

[1080] A "travel plan" is a specific travel itinerary plan generated by the server based on information input by the user.

[1081] "Travel expenses" is information indicating the overall expenses, such as accommodation expenses, event participation fees, and transportation expenses, based on the generated travel plan.

[1082] "Booking intent" is an indication that the user agrees with the proposed travel plan and wishes to actually make a reservation.

[1083] The "reservation result" is the result of the server sending a reservation request to each linked service, and includes information on whether the reservation was successful or not, as well as detailed information.

[1084] System Overview

[1085] This system is a comprehensive support system that allows users to make travel plans and make reservations based on those plans. The main components are the user's device, a server, and external data provision services (weather API, event API, accommodation API, etc.). When the user enters travel information using their device, the server collects and analyzes the necessary data to generate optimal travel suggestions.

[1086] Hardware and software used

[1087] In this system, users use devices such as smartphones and tablets, and the server is a high-performance computer system installed in a remote data center. The main software tools used are the following services and APIs:

[1088] To obtain weather information, use a weather API (e.g., OpenWeatherMap).

[1089] To get event information, use an event API (e.g., Eventbrite API).

[1090] To obtain accommodation information, use an accommodation API, e.g., Booking.com API.

[1091] Program processing overview

[1092] First, a user opens the application on their device and inputs their travel itinerary, destination, and activities they are interested in. The device then sends this information to the server via an HTTP request. Based on the received data, the server retrieves weather, event, and accommodation information from various external data providers.

[1093] The server integrates the collected data and generates an optimal travel plan based on the user's requests. The generated travel plan includes the schedule for each day and a breakdown of costs. The server sends this information to the terminal and displays it for the user to review.

[1094] When the user is satisfied with the proposed travel plan and clicks the "Book" button, the device sends the reservation intent to the server. The server then calls the accommodation, event, and transportation reservation APIs to automatically complete the reservation process. Once the reservation result is received, the server notifies the device and displays detailed reservation information to the user.

[1095] Specific example explanation

[1096] For example, let's consider what happens when a user enters, "I want to visit art museums in Tokyo from May 20th to May 23rd." The user enters this information on their device and presses the send button. The device then sends this information to the server. The server calls the OpenWeatherMap API to obtain weather information for Tokyo from May 20th to 23rd. At the same time, it uses the Eventbrite API to collect information on art events and special museum exhibitions taking place in Tokyo during the travel period. It then calls the Booking.com API to obtain information on accommodations in Tokyo during that period.

[1097] Based on the information it has acquired, the server generates a travel plan that includes outdoor art events on days when fine weather is predicted, and indoor museum visits on days when rain is predicted. The total cost is calculated based on this plan, and the result is sent to the device. The user checks the travel plan and cost on the device and clicks the "Book" button. The device sends the reservation intent to the server, and the server calls each reservation API to complete the reservation procedure. If the reservation is successful, the result is notified to the device, and detailed reservation information is displayed to the user.

[1098] Prompt Sentence Examples

[1099] Below is an example of a prompt sentence to input to the generative AI model.

[1100] Please explain in detail each step of the system where the program creates travel plans and completes the booking process. In particular, please explain the flow from input to display of results and the APIs used.

[1101] The above is a detailed description of the embodiment of the present invention. This system allows users to efficiently plan their trips and complete reservation procedures without hassle.

[1102] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1103] Step 1:

[1104] User enters travel information

[1105] A user opens a travel planning app and enters their travel dates (e.g., May 20 to May 23, 2023), destination (e.g., Tokyo), and activities of interest (e.g., visiting museums). The information entered is stored in data entry fields on the device.

[1106] Step 2:

[1107] The device sends the input information to the server

[1108] The terminal sends the travel information entered by the user to the server as a single JSON-formatted data. Specifically, it uses an HTTP POST request to send the "travel information entered by the user." This data is prepared to be parsed by the server. The input is the user's travel information, and the output is the server's receipt of the request.

[1109] Step 3:

[1110] The server retrieves weather information

[1111] The server calls a weather API based on the received user travel information. For example, it sends a request to the OpenWeatherMap API saying, "Please get weather information for Tokyo from May 20th to May 23rd, 2023." The weather information (weather, temperature, etc.) returned by the API is received, analyzed, and stored in a database. The input is the travel dates and destination, and the output is the weather information for that period.

[1112] Step 4:

[1113] The server retrieves the event information

[1114] The server calls an event API based on the received weather information. For example, it sends a request to the Eventbrite API saying, "Get information about art events taking place in Tokyo from May 20th to May 23rd, 2023." The event information (event name, location, date and time, etc.) returned by the API is received and stored in a database. The input is the travel dates, destination, and activities of interest, and the output is the event information for that period.

[1115] Step 5:

[1116] The server obtains accommodation information

[1117] The server calls the accommodation information API based on the received event information. For example, it sends a request to the Booking.com API saying, "Please get information about accommodations available in Tokyo from May 20th to May 23rd, 2023." The accommodation information (facility name, availability, price, etc.) returned by the API is received and stored in a database. The input is the travel dates and destination, and the output is accommodation information for that period.

[1118] Step 6:

[1119] The server generates travel suggestions

[1120] The server integrates the acquired weather, event, and accommodation information. It automatically generates an optimal travel plan based on the user's interests. For example, a plan might be created to attend an outdoor art event on a sunny day, and visit a museum on a rainy day. Based on this information, a detailed travel plan is created, including a daily itinerary and transportation methods. The input is weather information, event information, and accommodation information, and the output is a travel plan.

[1121] Step 7:

[1122] The server calculates the trip cost

[1123] The server calculates the total cost of the trip based on the generated travel suggestions. It adds up accommodation costs, event participation fees, transportation costs, etc., and provides the optimal plan within the budget. For example, it calculates "accommodation costs 5,000 yen / night x 3 nights + event participation fee 2,000 yen + transportation costs 1,000 yen = total cost 17,000 yen." The input is the travel suggestions, and the output is the total cost.

[1124] Step 8:

[1125] The server sends travel plans and costs to the terminal

[1126] The server sends the generated travel plan and its cost to the terminal. Specifically, it returns the travel plan and detailed cost information to the terminal as a new HTTP response. It formats the data so that the user can check it and sends it. The input is the travel plan and detailed cost information, and the output is the response to the terminal.

[1127] Step 9:

[1128] The device displays travel suggestions and costs to the user

[1129] The terminal displays the travel plan and cost information received from the server. Details such as "Itinerary for May 20th: Tokyo art events" and "Itinerary for May 21st: Museum tours" are displayed on the screen, along with the total cost to the user. The input is the travel plan and cost details, and the output is an enjoyable travel plan presented to the user.

[1130] Step 10:

[1131] User reviews trip suggestions and makes booking

[1132] The user reviews the proposed itinerary and costs, and if satisfied, clicks the "Book" button. The input is the itinerary details, and the output is a click indicating the intention to book.

[1133] Step 11:

[1134] The device sends the reservation information to the server

[1135] The device then sends the information indicating the user's booking intention back to the server. Specifically, it uses an HTTP POST request to send JSON data containing the content "The user has booked a travel plan." The input is the user's booking intention, and the output is a request sent to the server.

[1136] Step 12:

[1137] The server sends a reservation request to the associated service.

[1138] The server sends each reservation request to the accommodation reservation API, event reservation API, and transportation reservation API. For example, it sends specific request details such as "confirm accommodation reservation," "purchase event tickets," or "reserve transportation tickets." The input is the user's reservation information, and the output is a request sent to each reservation service.

[1139] Step 13:

[1140] The server checks the reservation result and notifies the device

[1141] The server receives responses from each API and checks whether the reservation was successful. If successful, it notifies the terminal of the result "Reservation successful." If unsuccessful, it generates and sends an appropriate error message. The input is the result of the reservation (success or failure), and the output is feedback to the terminal.

[1142] Step 14:

[1143] The device displays a notification to the user that the reservation is complete.

[1144] The terminal displays the reservation results received from the server to the user. If the reservation is successful, it displays detailed information such as "Accommodation: XX Hotel, Event: XX Exhibition, Transportation: Shinkansen", and if the reservation is unsuccessful, it displays an error message such as "Reservation failed. Please try again." The input is the reservation result, and the output is the feedback displayed to the user.

[1145] (Application example 1)

[1146] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1147] Traditional travel planning and booking systems require users to manually gather information from multiple sources and create travel plans, which is time-consuming. They also lack easy access to real-time information during the trip, making it difficult to change or adjust plans. Furthermore, the lack of systems that support specialized visual devices, voice input, or gesture control limits the user experience.

[1148] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1149] In this invention, the server includes: means for a user to input a travel itinerary, destinations, and activities of interest; means for acquiring weather information based on the input information; means for collecting event information; means for acquiring accommodation information; means for generating a travel plan by integrating weather information, event information, and accommodation information; means for calculating travel costs based on the travel plan; means for a user to confirm the travel plan and input a reservation intention; means for making reservations for accommodation, events, and transportation; means for notifying the user of the reservation results; means for a user to confirm and operate the travel information via a visual device; means for inputting travel information by voice input and gesture operation; and means for acquiring and displaying weather information, event information, and accommodation information in real time. This allows a user to efficiently plan a travel plan all at once and obtain and operate information in real time, enabling a flexible and comfortable travel experience.

[1150] "Means for user input of travel itinerary, destinations, and activities of interest" refers to an interface that allows a user to input detailed information about their trip using smart glasses or other visual device.

[1151] "Means for obtaining weather information based on input information" refers to a process for obtaining weather data from an external weather API based on the travel itinerary and destination input by the user.

[1152] "Means of collecting event information" refers to the function for obtaining information about events taking place at the travel destination from an external event calendar API.

[1153] "Means for obtaining accommodation information" refers to the mechanism for obtaining accommodation availability and price information from an external accommodation information API during the user's travel period.

[1154] "Means for generating travel plans by integrating weather information, event information, and accommodation information" refers to algorithms or systems that integrate various collected information and generate optimal travel plans based on the user's interests.

[1155] The "means for calculating travel costs based on the travel plan" refers to a function for calculating the total travel costs based on the content of the generated travel plan.

[1156] The "means for displaying the generated travel proposals and their costs to the user" refers to an interface for displaying the travel plan and cost information so that the user can check it via a visual device.

[1157] "Means for users to confirm travel plans and input their intention to make a reservation" refers to a function that allows users to input their intention to make a reservation for the displayed travel plan using voice commands or gesture operations.

[1158] "Means for booking accommodation, events, and transportation" refers to the process of booking accommodation, event participation, and transportation through an external service based on the user's booking intention.

[1159] "Means for notifying the user of the reservation result" refers to a function for notifying the user whether the reservation was successful or not.

[1160] "Means for users to view and manipulate travel information via a visual device" refers to an interface that allows users to view and manipulate details of their travel plans using a visual device such as smart glasses.

[1161] "Means for inputting travel information by voice input and gesture operation" refers to a function that allows a user to input travel information using voice recognition and gesture recognition.

[1162] "Means for acquiring and displaying weather information, event information, and accommodation information in real time" refers to a system for instantly acquiring and displaying the latest weather information, event information, and accommodation information in response to a user request.

[1163] The travel planning and booking system of the present invention utilizes a user's visual device (e.g., smart glasses) to plan and book a trip all at once. The system is composed of a user terminal, a server, and external data sources (weather API, event API, accommodation API, etc.), and specific embodiments are described in the detailed description below.

[1164] System Configuration

[1165] User terminal

[1166] Users wear the smart glasses and input their travel itinerary, destinations, and activities of interest through voice and gesture control, reducing user effort and enabling more intuitive operation.

[1167] server

[1168] The server includes the following means:

[1169] 1. How to obtain weather information: Weather information is obtained from an external weather API based on the travel itinerary and destination entered by the user.

[1170] 2. Event information collection method: Event information of interest to users is collected from an external event calendar API.

[1171] 3. How to obtain accommodation information: Accommodation availability and pricing information is obtained from an external accommodation information API.

[1172] 4. Travel plan generation method: The acquired weather information, event information, and accommodation information are integrated to generate the optimal travel plan based on the user's interests.

[1173] 5. Travel cost calculation method: Calculate the total travel cost based on the generated travel plans.

[1174] 6. Result display means: The generated travel plans and their cost information are displayed on the user's smart glasses.

[1175] 7. Means for inputting reservation intention: The user inputs the reservation intention using voice commands or gesture operations.

[1176] 8. Reservation Methods: Make reservations for accommodation, events, and transportation through external services.

[1177] 9. Notification means: Notify the user of the reservation results.

[1178] Devices and Software

[1179] The specific hardware and software used are as follows:

[1180] Hardware: Smart glasses (e.g., Google Glass)

[1181] Software: Speech recognition library (Google Cloud Speech API), gesture recognition library, external API (weather API, event API, accommodation API), data analysis library

[1182] Data processing flow

[1183] 1. Data input: The user inputs by voice, "I would like to travel to Kyoto next weekend and am interested in visiting museums and cafes." Information can also be input using gestures, and this information is sent from the user's device to the server.

[1184] 2. Information collection: The server calls the weather API, event API, and accommodation API to collect the necessary weather information, event information, and accommodation information.

[1185] 3. Plan Generation: The system integrates the collected information and generates an optimal travel plan based on the user's interests. For example, it suggests outdoor art events on sunny days and includes a visit to a museum on rainy days.

[1186] 4. Cost calculation and display: The total travel cost is calculated based on the generated plan and the information is overlaid within the field of view of the smart glasses.

[1187] 5. Reservation procedure: If the user is satisfied with the displayed travel plan, he / she can make a reservation by using the voice command "Reserve." The server sends a reservation request to each linked service, collects the reservation results, and notifies the user.

[1188] Examples of concrete examples and prompts

[1189] For example, if a user says, "Plan a trip to Kyoto for next weekend to visit museums and cafes," the system will gather the appropriate information and display something like this:

[1190] "Dates: October 20th to October 23rd"

[1191] "Destination: Kyoto"

[1192] "Activities: Visiting museums and cafes"

[1193] "Travel plan: October 20th: Kyoto International Museum, XX Art Museum (sunny weather forecast)"

[1194] "October 21st: XX Cafe Tour (Rain Forecast)"

[1195] "Total cost:\XXXXX"

[1196] "Would you like to make a reservation?"

[1197] By implementing the present invention in this way, users can plan their trips efficiently and intuitively, and can obtain and manipulate information in real time, allowing them to enjoy a flexible and comfortable travel experience.

[1198] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1199] Step 1:

[1200] The user puts on the smart glasses and uses voice input or gesture control to input their travel itinerary, destinations, and activities they are interested in. For example, the user might say, "I'd like to visit museums and cafes in Kyoto next weekend." This input information is then converted into text data by the device.

[1201] Input: User voice or gesture control

[1202] Output: Text data (travel itinerary, destinations, activities of interest)

[1203] Step 2:

[1204] The device sends the entered text data to the server using a method such as an HTTP request.

[1205] Input: Text data

[1206] Output: HTTP request

[1207] Step 3:

[1208] The server analyzes the received text data and calls an external weather API based on the travel itinerary and destination to obtain weather information for the destination during the travel period. The weather information is obtained in JSON format.

[1209] Input: Text data (travel itinerary, destination)

[1210] Output: Weather information (JSON data)

[1211] Step 4:

[1212] The server calls the event API to retrieve information about events taking place at the travel destination during the trip. The event information is filtered based on the user's interests and retrieved in JSON format.

[1213] Input: Text data (travel itinerary, destination, activities of interest)

[1214] Output: Event information (JSON data)

[1215] Step 5:

[1216] The server calls the accommodation information API to obtain information about accommodations at the destinations during the trip. The accommodation information includes room availability and price information, and is obtained in JSON format.

[1217] Input: Text data (travel itinerary, destination)

[1218] Output: Accommodation information (JSON data)

[1219] Step 6:

[1220] The server integrates the acquired weather, event, and accommodation information to generate an optimal travel plan based on the user's interests. Based on the weather information, it suggests outdoor events on sunny days and indoor activities on rainy days.

[1221] Input: Weather information, event information, accommodation information

[1222] Output: Travel plan

[1223] Step 7:

[1224] The server calculates the travel expenses based on the generated travel plan, including accommodation, event participation fees, and transportation costs.

[1225] Input: Travel Plan

[1226] Output: Trip cost

[1227] Step 8:

[1228] The server sends the generated travel plan and its cost information to the user's smart glasses using an HTTP response.

[1229] Input: Travel plan, travel expenses

[1230] Output: HTTP response

[1231] Step 9:

[1232] The user reviews the travel plan and cost information displayed on the smart glasses, and if satisfied, inputs an intent such as "book" using voice commands or gestures.

[1233] Input: Travel plan, travel expenses

[1234] Output: Reservation intent (voice or gesture operation)

[1235] Step 10:

[1236] The device sends the user's reservation intention to the server, again using a method such as an HTTP request.

[1237] Input: Booking Intent

[1238] Output: HTTP request

[1239] Step 11:

[1240] The server calls an external booking API to make reservations for accommodation, events, and transportation. The booking results are returned in JSON format.

[1241] Input: Booking Intent

[1242] Output: Reservation results (JSON data)

[1243] Step 12:

[1244] The server analyzes the reservation result and generates a success message and reservation details if successful, or an error message if unsuccessful, and sends them to the user's smart glasses.

[1245] Input: Reservation result

[1246] Output: Success or error message

[1247] Step 13:

[1248] The user checks the reservation results displayed on the smart glasses. If successful, the trip details (including accommodation information and event ticket information) are displayed.

[1249] Input: Success message or error message

[1250] Output: Trip details or error message

[1251] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1252] The present invention is a travel planning and booking system incorporating an emotion engine that recognizes the user's emotions. A specific embodiment of this system will be described below.

[1253] System configuration

[1254] The system consists of a user's device, a server, external data sources (weather API, event API, accommodation API, etc.), and an emotion engine. Users operate their devices to make travel plans, and the information is sent to the server. The server collects and analyzes the necessary data to create travel plans. Furthermore, the emotion engine recognizes the user's emotions and adjusts the travel plans accordingly.

[1255] Program processing overview

[1256] 1. User enters travel information

[1257] Users open the app and enter their travel dates, destinations, and activities they are interested in. For example, a user might enter, "I want to visit museums in Tokyo from May 20th to May 23rd."

[1258] 2. The device sends the input information to the server

[1259] The device receives the travel information entered by the user and sends the data to the server, usually in the form of an HTTP request.

[1260] 3. Emotion engine recognizes user emotions

[1261] The device uses an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions, tone of voice, and / or text input to determine the user's emotional state (e.g., joy, sadness, excitement, stress, etc.).

[1262] 4. The server retrieves weather information

[1263] The server calls the weather API to obtain weather information for the destination during the trip. The server analyzes the weather data according to the trip period and destination entered by the user.

[1264] 5. The server retrieves the event information

[1265] The server calls the event API to retrieve event information held at the destination during the trip, and filters the event information based on the user's interests.

[1266] 6. The server obtains the accommodation information

[1267] The server calls the accommodation information API to obtain information on room availability and prices for accommodations during the travel period, and analyzes the obtained data to select the optimal accommodation plan.

[1268] 7. The server generates travel suggestions

[1269] The server integrates the acquired weather, event, and accommodation information and generates travel plans taking into account the user's emotional state. For example, if the user is feeling stressed, it will create a plan that includes many relaxing activities, while if the user is excited, it will create a plan that includes many active activities.

[1270] 8. The server calculates the cost of the trip

[1271] The server calculates the total cost of the trip by adding up accommodation costs, event participation fees, and transportation costs (such as rental cars and taxis) based on the generated travel plan. A detailed breakdown of the costs is also provided.

[1272] 9. The server sends the travel plan and costs to the device.

[1273] The server sends the generated travel plans and their costs to the device, usually in JSON format, which can be easily parsed by the device.

[1274] 10. The device displays trip suggestions and costs to the user.

[1275] The device analyzes the travel proposals and cost information received from the server and displays them to the user, who can then check the details of the proposed travel plan (daily schedule, cost breakdown).

[1276] 11. User reviews trip suggestions and makes booking

[1277] The user checks the proposed travel plans and costs, and if satisfied, clicks the "Book" button, thereby indicating their intention to begin the booking process.

[1278] 12. The device sends the reservation information to the server

[1279] The terminal sends the user's reservation intention to the server. The reservation data is sent to the server as an HTTP request.

[1280] 13. The server calls the API of each service it is connected to and starts the specific reservation procedure.

[1281] The server uses the accommodation reservation API, event reservation API, and transportation reservation API to make each reservation.

[1282] 14. The server checks the response from each reservation service and notifies the device.

[1283] The server checks the responses from each reservation service, determines whether the reservation was successful, and notifies the terminal of the result.

[1284] 15. The terminal displays the reservation results to the user

[1285] The terminal analyzes the reservation results received from the server and displays them to the user. If the reservation is successful, detailed reservation information (such as the accommodation address and event ticket information) is displayed, but if the reservation is unsuccessful, an error message is displayed to notify the user.

[1286] Specific example explanation

[1287] For example, if a user inputs "I want to visit art museums in Tokyo from May 20th to May 23rd," and the emotion engine determines that the user is "feeling stressed" based on their facial expression and tone of voice when they input, the device will send this information to the server. The server will then obtain weather, event, and accommodation information, and create a travel plan that prioritizes relaxing facilities and events based on the user's emotional data.

[1288] In this way, by implementing the present invention, it is possible to provide a travel plan that is customized according to the user's emotional state, thereby improving user satisfaction.

[1289] The processing flow will be explained below.

[1290] Step 1:

[1291] A user launches the app and enters their travel dates, destinations, and activities they are interested in. For example, a user might enter, "I want to visit museums in Tokyo from May 20th to May 23rd."

[1292] Step 2:

[1293] The terminal obtains the user's input information and sends the data to the server, usually using an HTTP request.

[1294] Step 3:

[1295] To recognize the user's emotions, the emotion engine captures and analyzes the user's facial expressions. It also analyzes the user's tone of voice and the content of the text entered by the user to determine their emotional state. For example, if the user enters text with a tired expression, it will determine that they are "feeling stressed."

[1296] Step 4:

[1297] The device sends the emotion data determined by the emotion engine to the server as an HTTP request.

[1298] Step 5:

[1299] The server calls an external weather API to obtain weather information. Based on the travel period and destination entered by the user, the server obtains and analyzes weather data for the relevant period.

[1300] Step 6:

[1301] The server calls an external event API to get event information. The server gets a list of events taking place at the destination during the travel period and filters the event information based on the user's interests. For example, it gets information about special museum exhibitions and art festivals taking place during the period.

[1302] Step 7:

[1303] The server calls an external accommodation API to get accommodation information. The server gets the availability and price information of accommodations during the travel period. For example, it gets the list of accommodations available during the period and their prices.

[1304] Step 8:

[1305] The server generates travel plans by integrating weather information, event information, accommodation information, and emotion data. For example, if the user is feeling stressed, the server creates a plan that includes many relaxing activities and quiet accommodations. If the user is excited, the server creates a plan that includes active activities.

[1306] Step 9:

[1307] The server calculates the travel costs based on the travel plan it generates. The server adds up the cost of accommodation, event participation, and transportation (car rental, taxi, etc.) to calculate the total cost. A detailed breakdown of the costs is also provided.

[1308] Step 10:

[1309] The server sends the generated travel plans and their costs to the device, usually in JSON format, so that the device can easily parse them.

[1310] Step 11:

[1311] The device analyzes the travel proposals and cost information received from the server and displays them to the user, who can then check the details of the proposed travel plan (daily schedule, cost breakdown).

[1312] Step 12:

[1313] The user checks the proposed travel plan and costs, and if satisfied, clicks the "Book" button, thereby indicating their intention to begin the booking process.

[1314] Step 13:

[1315] The terminal sends the user's reservation intention to the server. The reservation data is sent to the server as an HTTP request.

[1316] Step 14:

[1317] The server calls the APIs of each service it connects to and starts the specific reservation procedure. The server makes each reservation using the accommodation reservation API, event reservation API, and transportation reservation API.

[1318] Step 15:

[1319] The server checks the response from each reservation service and determines whether the reservation was successful. The server then sends data to notify the terminal of the reservation result (success / failure).

[1320] Step 16:

[1321] The terminal analyzes the reservation results received from the server and displays them to the user. If the reservation is successful, detailed reservation information (such as the accommodation address and event ticket information) is displayed, and if the reservation is unsuccessful, an error message is displayed to notify the user.

[1322] Step 17:

[1323] The user can confirm the reservation details and complete the trip planning. The user can then prepare for the trip based on the displayed information.

[1324] In this way, the present invention proposes travel plans customized according to the user's emotional state, and can handle everything from travel planning to booking all at once. By being provided with plans that take emotions into consideration, users can have a more satisfying travel experience.

[1325] Example 2

[1326] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1327] Conventional travel planning and booking systems generate itineraries without considering the user's emotions, which means that the user's satisfaction and relaxation level are not fully reflected. Furthermore, even when integrating weather, event, and accommodation information, providing information that ignores the user's emotional state is insufficient to optimize the user experience.

[1328] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for the user to input a travel itinerary, destination, and activities of interest; means for acquiring weather information based on the input information; means for collecting event information; means for acquiring accommodation information; means for recognizing the user's emotional state; means for generating a travel plan by integrating weather information, event information, accommodation information, and the user's emotional state; means for calculating travel costs based on the travel plan; means for displaying the generated travel plan and its costs to the user; means for the user to confirm the travel plan and input their reservation intention; means for making reservations for accommodation, events, and transportation; and means for notifying the user of the reservation results. This makes it possible to provide a travel plan customized according to the user's emotional state, thereby improving user satisfaction.

[1329] A "user" is an individual or entity that uses the travel planning and booking system to enter travel information, obtain travel suggestions, and make reservations.

[1330] A "travel itinerary" is a period consisting of the start and end dates of a trip specified by a user.

[1331] A "destination" is a place or area that a user visits during a trip.

[1332] An "interesting activity" is an event, activity, or interest that a user wishes to participate in while traveling.

[1333] "Weather information" refers to information about the weather conditions at the destination during the travel period, including temperature, probability of precipitation, wind speed, etc.

[1334] "Event information" refers to information about events and activities that will be held at the destination during the trip.

[1335] "Accommodation information" refers to information regarding the availability and prices of accommodations available for stay during the travel period.

[1336] "Emotional state" refers to a user's psychological and emotional state, determined by analyzing the user's facial expressions, tone of voice, text input, etc.

[1337] A "travel suggestion" is a specific travel plan proposed to a user, which is generated based on weather information, event information, accommodation information, and the user's emotional state.

[1338] "Travel Cost" means the total cost of the trip calculated based on the Trip Plan, including accommodation, event participation fees, and transportation costs.

[1339] "Booking intent" refers to the user's willingness to accept the proposed travel plan and to actually begin the booking process.

[1340] "Reservation Result" means the result of the reservation process for accommodation, events, and transportation, including information on the success or failure of the reservation.

[1341] This invention is a travel planning and reservation system incorporating an emotion engine that recognizes user emotions. The system consists of a user's terminal, a server, external data sources (weather data service, event data service, accommodation information service, etc.), and the emotion engine. The user operates their terminal to make travel plans, and the information is sent to the server. The server collects and analyzes the necessary data and creates travel plans. Furthermore, the emotion engine recognizes the user's emotions and adjusts the travel plans based on them.

[1342] System configuration

[1343] This system consists of a user's device, a server, external data sources (weather data service, event data service, accommodation information service, etc.), and an emotion engine. The user enters travel information through the device, and the input information is sent to the server. The server retrieves and analyzes the necessary data from the external data sources, while the emotion engine analyzes the user's emotions.

[1344] Hardware and Software

[1345] The devices used are smartphones or PCs, which must be equipped with a camera and microphone. The server is a system server with the computing power to process data, and has an emotion engine and data analysis engine installed. External data sources include weather data services (e.g., OpenWeatherMap API), event data services (e.g., Eventbrite API), and accommodation information services (e.g., Booking.com API).

[1346] How the system works

[1347] The user enters their travel itinerary, destinations, and activities of interest into the app. For example, they might enter, "I want to visit museums in Tokyo from May 20th to May 23rd." This information is sent to the server by the device. The device sends the information to the server through a REST API in the form of an HTTP request.

[1348] The emotion engine recognizes the user's facial expressions and tone of voice through the camera or microphone and determines their emotional state (e.g., OpenFace or IBM Watson emotion analysis API). For example, if the user is smiling, it will be determined to be "happy."

[1349] The server calls a weather data provider (e.g., OpenWeatherMap API) to obtain weather information for the destination during the travel period, then retrieves event information for the destination from an event data provider (e.g., Eventbrite API) and filters it based on the user's interests, and finally retrieves accommodation availability and price information from an accommodation information provider (e.g., Booking.com API).

[1350] The server integrates all the acquired information (weather, event, accommodation, and the user's emotional state) to generate travel suggestions. For example, if the user is feeling stressed, the server creates a plan that includes many relaxing activities.

[1351] Based on the generated travel plan, the server calculates the travel cost, including accommodation, event participation, and transportation costs, and provides a detailed breakdown. The travel plan and its cost are sent to the device in JSON format, which the device parses and displays to the user.

[1352] The user checks the travel plan and, if satisfied, clicks the "Book" button. The device sends the user's reservation intention to the server, which then calls the accommodation reservation API, event reservation API, and transportation reservation API to carry out the specific reservation procedure. The reservation result is notified to the device from the server, and the final reservation information is displayed to the user.

[1353] Specific example explanation

[1354] For example, if a user inputs "I want to visit art museums in Tokyo from May 20th to May 23rd," and the emotion engine determines that the user is "feeling stressed" based on their facial expression and tone of voice, that information is sent to the server. The server retrieves weather, event, and accommodation information from external data sources and creates a travel plan that prioritizes relaxing facilities and events based on the user's emotional state.

[1355] Examples of specific prompts are:

[1356] A system creates customized travel plans based on the user's emotional state. The user inputs that they want to visit museums in Tokyo from May 20th to May 23rd. The emotion engine determines that the user is stressed based on their facial expressions and tone of voice. Based on this information, create a travel plan that includes many relaxing activities.

[1357] In this way, the present invention provides a customized travel plan according to the user's emotional state, making it possible to improve user satisfaction.

[1358] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1359] Step 1:

[1360] Users input their travel itinerary, destinations, and activities they are interested in. Using a smartphone or PC application, users input travel information such as "I would like to visit art museums in Tokyo from May 20th to May 23rd." This information is stored as data entry on the device.

[1361] Step 2:

[1362] The terminal sends the input information to the server. The travel information entered by the user is sent to the server in the form of an HTTP request. Specifically, the information is transmitted to the server using the REST API and the POST method. The input data is received by the server and processing begins.

[1363] Step 3:

[1364] The emotion engine recognizes the user's emotions. The server analyzes the user's facial expressions and tone of voice data sent from the device to determine their emotional state. Specifically, emotion engines such as OpenFace and IBM Watson's emotion analysis API are used. The result of the data analysis is the user's emotional state (for example, "joy," "sadness," "excitement," "stress," etc.).

[1365] Step 4:

[1366] The server retrieves weather information. The server calls an external weather data provider (for example, OpenWeatherMap API) to retrieve weather information for the destination during the travel period. The input is the user's travel period and destination data, and the output is the corresponding weather data in JSON format.

[1367] Step 5:

[1368] The server obtains event information. The server calls an external event data provider (for example, Eventbrite API) to obtain information about events held at the destination during the travel period. It then filters the information based on the user's interests and extracts useful event information. The input is the user's interests and travel period / destination data, and the output is related event information.

[1369] Step 6:

[1370] The server retrieves accommodation information. The server calls an accommodation information service (e.g., Booking.com API) to retrieve the availability and price information of accommodations during the travel period. The input is the user's travel period and destination data, and the output is accommodation information and price data.

[1371] Step 7:

[1372] The server generates travel suggestions by integrating the acquired weather information, event information, accommodation information, and the user's emotional state. Data processing involves integrating data from different sources to create an optimal travel plan based on the user's emotional state. For example, for a user who is feeling stressed, a plan that includes many relaxing activities is created. The output is design data for specific travel suggestions.

[1373] Step 8:

[1374] The server calculates the travel costs. Based on the generated travel plan, the server adds up accommodation costs, event participation fees, transportation costs, etc. to calculate the total cost of the trip. A detailed breakdown of the costs is also provided. The input is the travel plan data, and the output is detailed travel costs.

[1375] Step 9:

[1376] The server sends the travel plan and cost to the terminal. The server sends the generated travel plan and its cost in JSON format to the terminal. The terminal receives and analyzes it. The input is the travel plan and cost data, and the analyzed data is transferred as the output.

[1377] Step 10:

[1378] The terminal displays the travel plan and costs to the user. The terminal analyzes the travel plan and cost information received from the server and presents it visually to the user. Specifically, the screen displays the daily plan and cost breakdown. The input is the analyzed travel plan and cost data, and the output is a visual display.

[1379] Step 11:

[1380] The user confirms the proposed trip and makes a reservation. The user checks the proposed trip and the cost, and if satisfied, clicks the "Book" button. This indicates the terminal's intention to start the reservation procedure. The input is the user's confirmation operation, and the output is a signal to start the reservation.

[1381] Step 12:

[1382] The terminal sends reservation information to the server. The terminal then sends the user's reservation intention to the server as an HTTP request. Specifically, the POST method is used. The input is the reservation start intention data, and the output is a request sent to the server.

[1383] Step 13:

[1384] The server calls the API of each linked service and starts the specific reservation procedure. The server calls the accommodation reservation API, event reservation API, and transportation reservation API to proceed with each reservation process. For example, it checks accommodation availability and confirms the reservation, secures event tickets, and arranges transportation. The input is reservation information data, and the output is a request to each reservation service.

[1385] Step 14:

[1386] The server checks the response from each reservation service and notifies the terminal. The server checks the response from each reservation service and determines whether the reservation was successful. The input is the response data from each service, and the reservation result is obtained as output and transferred to the terminal.

[1387] Step 15:

[1388] The terminal displays the reservation results to the user. The terminal analyzes the reservation results received from the server and displays them visually to the user. If successful, detailed reservation information (e.g., accommodation address, event ticket information, etc.) is displayed, and if unsuccessful, an error message is displayed. The input is the reservation result data, and the output is a visual display of the reservation result.

[1389] (Application example 2)

[1390] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1391] Conventional travel planning and reservation systems can automatically create itineraries based on user input, but they have the problem of being unable to customize them to take into account the user's emotional state. As a result, it is difficult to provide travel plans that reduce user stress and fatigue, limiting the improvement of user satisfaction. Furthermore, optimizing the work environment in factories while taking into account the emotions of operators is also a challenge.

[1392] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1393] In this invention, the server includes means for generating a travel plan by integrating weather information, event information, and accommodation information using an emotion engine that recognizes the user's emotional state, means for calculating travel costs based on the travel plan, and means for displaying the generated travel plan and its costs to the user, thereby enabling customized travel plans and optimization of the work environment at a factory site that take the user's emotional state into consideration.

[1394] A "user" is a person who operates the system to input information such as travel itinerary and destination.

[1395] "Travel itinerary" is information indicating the start and end dates of a trip planned by a user.

[1396] "Destination" is information indicating a place that the user wants to visit.

[1397] "Interesting Activities" is information indicating activities and events that the user wants to do during the trip.

[1398] "Weather information" is data related to the weather at the destination.

[1399] "Event information" is data about events and activities held at a destination.

[1400] "Accommodation information" is data related to accommodation facilities at the destination.

[1401] "Emotional state" refers to the user's psychological state (e.g., joy, sadness, stress, etc.).

[1402] An "emotion engine" is software or hardware for recognizing and analyzing a user's emotions.

[1403] A "travel suggestion" is a travel plan that is generated based on the user's input information and emotional state.

[1404] "Trip Cost" means the total cost calculated based on the proposed trip.

[1405] "Server" is a central computer system that processes information sent from user terminals, retrieves data, and generates travel proposals.

[1406] A "user terminal" is a device (e.g., a smartphone, tablet, etc.) that a user uses to input information and exchange information with a server.

[1407] The present invention is a travel planning and booking system incorporating an emotion engine that recognizes the user's emotions. The details of this system are described below.

[1408] System configuration

[1409] This system consists of a user device, a server, external data sources (weather API, event API, accommodation API, etc.), and an emotion engine. Users operate their devices to make travel plans, and the information is sent to the server. The server collects and analyzes the necessary data and creates travel plans. Furthermore, the emotion engine recognizes the user's emotions and adjusts the travel plans accordingly.

[1410] Hardware and software used

[1411] Emotion recognition hardware: camera, microphone (captures the user's facial expressions and tone of voice)

[1412] Server: A central computer system for data analysis and travel proposal generation

[1413] Emotion engine software: Microsoft Azure Cognitive Services and Google Cloud Vision API

[1414] User devices: smartphones, tablets, etc.

[1415] Program processing

[1416] The server processes the data using the following methods:

[1417] 1. Data collection: Collect travel information from user devices and data from weather API, event API, and accommodation API.

[1418] 2. Emotion Recognition: Analyze the user's emotional state in real time using an emotion engine.

[1419] 3. Data integration and analysis: Integrate the collected information and sentiment data and perform data analysis.

[1420] 4. Travel plan generation: Generate optimal travel plans based on the user's emotional state.

[1421] 5. Displaying the results: The generated travel plans and costs are sent to the user's terminal and displayed to the user.

[1422] Specific example explanation

[1423] For example, if a user inputs "I want to visit art museums in Tokyo from May 20th to May 23rd," and the emotion engine determines that the user is "feeling stressed" based on their facial expression and tone of voice when they input, the device will send this information to the server. The server will then obtain weather information, event information, and accommodation information, and create a travel plan that prioritizes relaxing facilities and events based on the user's emotional data. In this way, a travel plan customized to the user's emotional state can be provided, improving user satisfaction.

[1424] Generative AI model prompt example

[1425] "Draw up a stress reduction work plan based on the following data:

[1426] Facial expressions

[1427] Tone of voice

[1428] Past work history

[1429] Work environment data (temperature, humidity, light, sound level)

[1430] Work Contents

[1431] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1432] Step 1:

[1433] Users input their travel itinerary, destinations, and activities they are interested in. Specifically, they use a device such as a smartphone or tablet to enter travel information into the application, such as "I would like to visit art museums in Tokyo from May 20th to May 23rd." This information is captured as input data.

[1434] Step 2:

[1435] The terminal sends the entered travel information to the server. Specifically, it uses an HTTP request to transfer the input data (travel itinerary, destination, and activities of interest) to the server. The input of this step is the travel information from the user, and the output is the data sent to the server.

[1436] Step 3:

[1437] The device uses an emotion engine to recognize the user's emotions. Specifically, it uses a camera and microphone to capture the user's facial expressions and tone of voice, which are then analyzed by emotion engine software (e.g., Microsoft Azure Cognitive Services or Google Cloud Vision API). The input for this step is the user's real-time facial and voice data, and the output is a recognized emotional state (e.g., joy, sadness, stress, etc.).

[1438] Step 4:

[1439] The server retrieves weather information. Specifically, it calls an external weather API to retrieve weather data based on the user's travel itinerary and destination. The input for this step is the travel itinerary and destination, and the output is the weather information for the corresponding period.

[1440] Step 5:

[1441] The server retrieves event information. Specifically, it calls the event API to retrieve event information held at the destination during the trip period. It also filters based on the user's interests. The inputs for this step are the trip period, destination, and user interests, and the output is related event information.

[1442] Step 6:

[1443] The server obtains accommodation information. Specifically, it uses the accommodation API to collect and analyze information on room availability and prices for accommodations during the travel period. The input for this step is the travel dates and destination, and the output is the optimal accommodation plan.

[1444] Step 7:

[1445] The server integrates weather information, event information, accommodation information, and the user's recognized emotional state to generate optimal travel recommendations. Specifically, it uses a data analysis algorithm to create a plan that prioritizes relaxation facilities and events. The inputs for this step are weather information, event information, accommodation information, and emotional data, and the output is customized travel recommendations.

[1446] Step 8:

[1447] The server calculates the cost of the trip based on the trip suggestions. Specifically, it calculates the total cost by adding up accommodation costs, event participation fees, transportation costs, etc. A detailed breakdown of the costs is also provided. The input of this step is the generated trip suggestions, and the output is the total cost of the trip.

[1448] Step 9:

[1449] The server sends the generated travel plan and its cost to the user's device. Specifically, it converts the travel plan and cost into JSON format and sends it to the device. The input of this step is the travel plan and cost, and the output is the data sent to the device.

[1450] Step 10:

[1451] The device analyzes the trip suggestions and cost information received from the server and displays them to the user. Specifically, it provides the user with a daily plan and a breakdown of costs in an easy-to-understand format. The input for this step is the trip suggestions and cost data from the server, and the output is the information displayed to the user.

[1452] Step 11:

[1453] The user checks the proposed travel plan and costs, and if satisfied, clicks the "Book" button. This indicates the user's intention to begin the reservation process. The input to this step is confirmation of the travel plan and costs, and the output is the user's intention to make a reservation.

[1454] Step 12:

[1455] The terminal sends the user's reservation intention to the server. Specifically, it sends the reservation data to the server as an HTTP request. The input of this step is the reservation intention from the user, and the output is the reservation data sent to the server.

[1456] Step 13:

[1457] The server uses the accommodation reservation API, event reservation API, and transportation reservation API to make each reservation. Specifically, it calls each API to execute the reservation procedure. The input of this step is the reservation data, and the output is reservation confirmation data.

[1458] Step 14:

[1459] The server checks the response from each reservation service and determines whether the reservation was successful. It notifies the terminal of the result. The input of this step is the reservation confirmation data, and the output is the reservation result sent to the terminal.

[1460] Step 15:

[1461] The terminal analyzes the reservation results received from the server and displays them to the user. Specifically, if the reservation is successful, detailed reservation information (e.g., accommodation address, event ticket information, etc.) is displayed, and if the reservation is unsuccessful, an error message is displayed. The input to this step is the reservation result from the server, and the output is the reservation information displayed to the user.

[1462] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1463] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1464] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1465] [Fourth embodiment]

[1466] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1467] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1468] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1469] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1470] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1471] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1472] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1473] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1474] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1475] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1476] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1477] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1478] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1479] The present invention is a system that allows a user to plan and book a trip all at once. A specific embodiment of this system will be described below.

[1480] System configuration

[1481] This system consists of a user's device, a server, and external data sources (weather API, event API, accommodation API, etc.). Users operate their own devices to make travel plans, and the information is sent to the server, which then collects and analyzes the necessary data to create travel plans.

[1482] Program processing overview

[1483] 1. User enters travel information

[1484] Users open the app and enter their travel dates, destinations, and activities of interest.

[1485] 2. The device sends the input information to the server

[1486] The device sends the entered travel information to the server using a method such as an HTTP request.

[1487] 3. The server retrieves weather information

[1488] The server calls the weather API to obtain weather information for the destination during the trip, using an external weather information provider.

[1489] 4. The server retrieves the event information

[1490] The server calls the event API to obtain information about events held at the travel destination during the travel period, by collecting data from an event information service.

[1491] 5. The server obtains the accommodation information

[1492] The server calls the accommodation information API to obtain the availability and price information of accommodations at the destination during the travel period. For this purpose, the accommodation information provider service is used.

[1493] 6. The server generates travel suggestions

[1494] The server integrates the acquired weather, event, and accommodation information to generate an optimal travel plan based on the user's interests. For example, it suggests outdoor activities on sunny days and indoor activities on rainy days.

[1495] 7. The server calculates the cost of the trip

[1496] The server calculates the total cost of the trip by adding up accommodation costs, event participation fees, transportation costs, etc. according to the generated travel plan.

[1497] 8. The server sends the travel plan and costs to the device.

[1498] The server sends the generated travel plan and its cost to the terminal so that the user can check it.

[1499] 9. The device displays trip suggestions and costs to the user.

[1500] The terminal displays the travel proposal and cost information received from the server to the user, who can then check the details of the proposed travel plan (daily activity plan, cost breakdown).

[1501] 10. User reviews trip suggestions and makes booking

[1502] The user checks the proposed travel plans and costs, and if satisfied, makes a reservation by clicking the "Book" button on the screen.

[1503] 11. The device sends the reservation information to the server

[1504] The terminal sends the user's reservation intention to the server using an HTTP request.

[1505] 12. The server sends a reservation request to the partner service.

[1506] The server calls the accommodation reservation API, event reservation API, and transportation reservation API to make each reservation.

[1507] 13. The server checks the reservation result and notifies the device

[1508] The server checks the responses from each reservation service, determines whether the reservation was successful, and notifies the terminal of the result.

[1509] 14. The device displays a notification to the user that the reservation is complete.

[1510] The terminal displays the reservation result received from the server to the user. If successful, it displays the reservation details (such as accommodation information and event ticket information), and if unsuccessful, it displays an appropriate error message.

[1511] Specific example explanation

[1512] For example, if a user inputs "I would like to visit art museums in Tokyo from May 20th to May 23rd," the device sends this information to the server. The server obtains weather information for Tokyo from May 20th to 23rd, then collects information on art events and special exhibitions at art museums being held in Tokyo during the travel period. It also obtains information on accommodations in Tokyo during that period. The server then generates a travel itinerary that includes outdoor art events on sunny days and indoor museum visits on rainy days, and calculates the total cost based on that itinerary.

[1513] This information is sent to the device, where the user can confirm the proposed trip and its cost. Once the user confirms and clicks the "Book" button, the device sends the reservation intent to the server. The server then sends a reservation request to each linked service, and if the reservation is successful, the result is notified to the device and displayed to the user.

[1514] By implementing the present invention, users can efficiently plan their trips without hassle and easily complete the reservation procedure.

[1515] The processing flow will be explained below.

[1516] Step 1:

[1517] A user launches the app and enters their travel dates, destinations, and activities they are interested in. For example, a user might enter, "I want to visit museums in Tokyo from May 20th to May 23rd."

[1518] Step 2:

[1519] The terminal obtains the user's input information and sends the data to the server, usually in the form of an HTTP request.

[1520] Step 3:

[1521] The server calls an external weather API to obtain weather information. The server obtains and analyzes weather data for the relevant period according to the travel period and destination entered by the user. For example, it obtains information that May 20th and 21st will be sunny, and May 22nd and 23rd will be rainy.

[1522] Step 4:

[1523] The server calls an external event API to get event information. The server gets a list of events taking place at the destination during the travel period and filters the event information based on the user's interests. For example, it gets information about special museum exhibitions and art festivals taking place during the period.

[1524] Step 5:

[1525] The server calls an external accommodation API to get accommodation information. The server gets the availability and price information of accommodation during the travel period. For example, it uses OYO service to get the list of accommodations available during the period and their prices.

[1526] Step 6:

[1527] The server generates travel plans by integrating weather, event, and accommodation information. The server creates plans that include outdoor art events on sunny days and indoor museum visits on rainy days. It also selects the most suitable accommodations.

[1528] Step 7:

[1529] The server calculates the travel costs based on the travel plan it generates. The server adds up the cost of accommodation, event participation, and transportation (car rental, taxi, etc.) to calculate the total cost. A detailed breakdown of the costs is also provided.

[1530] Step 8:

[1531] The server sends the generated travel plans and their costs to the device, usually in JSON format, so that the device can easily parse them.

[1532] Step 9:

[1533] The device analyzes the travel proposals and cost information received from the server and displays them to the user, who can then check the details of the proposed travel plan (daily schedule, cost breakdown).

[1534] Step 10:

[1535] The user checks the proposed travel plan and costs, and if satisfied, clicks the "Book" button, thereby indicating their intention to begin the booking process.

[1536] Step 11:

[1537] The terminal sends the user's reservation intention to the server. The reservation data is sent to the server as an HTTP request.

[1538] Step 12:

[1539] The server calls the APIs of each service it connects to and starts the specific reservation procedure. The server makes each reservation using the accommodation reservation API, event reservation API, and transportation reservation API.

[1540] Step 13:

[1541] The server checks the response from each reservation service and determines whether the reservation was successful. The server then sends data to notify the terminal of the reservation result (success / failure).

[1542] Step 14:

[1543] The terminal analyzes the reservation results received from the server and displays them to the user. If the reservation is successful, detailed reservation information (such as the accommodation address and event ticket information) is displayed, and if the reservation is unsuccessful, an error message is displayed to notify the user.

[1544] The above is a specific flow of processing from user input to reservation completion.

[1545] Example 1

[1546] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1547] Traditional travel planning systems require users to manually research and book trip itineraries, destinations, and activities of interest across multiple websites and applications. This manual process is time-consuming, inconvenient, and prone to errors. It also places a burden on users, making it difficult to integrate all aspects of a trip (weather, events, accommodations, etc.) into a single plan. To solve these problems, a system that can capture and integrate all travel information in one place is needed.

[1548] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1549] In this invention, the server includes means for acquiring weather information from an external data providing service, means for acquiring event information from the external data providing service, and means for acquiring accommodation information from the external data providing service. As a result, when a user inputs a travel itinerary, destination, and activities of interest, the necessary weather information, event information, and accommodation information are automatically acquired and integrated, allowing the user to efficiently plan a trip without hassle.

[1550] A "user" is a person who uses the system to make travel plans.

[1551] "Travel itinerary" is information indicating the start and end dates of the period during which the user plans to travel.

[1552] "Destination" is information indicating a specific place or area that the user wants to visit during their trip.

[1553] "Interesting Activities" is information indicating specific activities or events that the user would like to do during the trip.

[1554] A "server" is a computer system that receives travel information sent by a user, obtains necessary data from an external data provider based on that information, and generates travel plans.

[1555] A "terminal" is a device through which a user inputs travel information and displays travel suggestions and costs sent from the server.

[1556] "External data providing services" are various online services that provide weather information, event information, accommodation information, etc.

[1557] "Weather information" is information that indicates weather data for the destination during the user's travel period.

[1558] "Event information" is information about various events that will be held at the destination during the user's travel period.

[1559] "Accommodation information" is data that indicates room availability and price information for accommodation facilities where the user can stay during their travel period.

[1560] A "travel plan" is a specific travel itinerary plan generated by the server based on information input by the user.

[1561] "Travel expenses" is information indicating the overall expenses, such as accommodation expenses, event participation fees, and transportation expenses, based on the generated travel plan.

[1562] "Booking intent" is an indication that the user agrees with the proposed travel plan and wishes to actually make a reservation.

[1563] The "reservation result" is the result of the server sending a reservation request to each linked service, and includes information on whether the reservation was successful or not, as well as detailed information.

[1564] System Overview

[1565] This system is a comprehensive support system that allows users to make travel plans and make reservations based on those plans. The main components are the user's device, a server, and external data provision services (weather API, event API, accommodation API, etc.). When the user enters travel information using their device, the server collects and analyzes the necessary data to generate optimal travel suggestions.

[1566] Hardware and software used

[1567] In this system, users use devices such as smartphones and tablets, and the server is a high-performance computer system installed in a remote data center. The main software tools used are the following services and APIs:

[1568] To obtain weather information, use a weather API (e.g., OpenWeatherMap).

[1569] To get event information, use an event API (e.g., Eventbrite API).

[1570] To obtain accommodation information, use an accommodation API, e.g., Booking.com API.

[1571] Program processing overview

[1572] First, a user opens the application on their device and inputs their travel itinerary, destination, and activities they are interested in. The device then sends this information to the server via an HTTP request. Based on the received data, the server retrieves weather, event, and accommodation information from various external data providers.

[1573] The server integrates the collected data and generates an optimal travel plan based on the user's requests. The generated travel plan includes the schedule for each day and a breakdown of costs. The server sends this information to the terminal and displays it for the user to review.

[1574] When the user is satisfied with the proposed travel plan and clicks the "Book" button, the device sends the reservation intent to the server. The server then calls the accommodation, event, and transportation reservation APIs to automatically complete the reservation process. Once the reservation result is received, the server notifies the device and displays detailed reservation information to the user.

[1575] Specific example explanation

[1576] For example, let's consider what happens when a user enters, "I want to visit art museums in Tokyo from May 20th to May 23rd." The user enters this information on their device and presses the send button. The device then sends this information to the server. The server calls the OpenWeatherMap API to obtain weather information for Tokyo from May 20th to 23rd. At the same time, it uses the Eventbrite API to collect information on art events and special museum exhibitions taking place in Tokyo during the travel period. It then calls the Booking.com API to obtain information on accommodations in Tokyo during that period.

[1577] Based on the information it has acquired, the server generates a travel plan that includes outdoor art events on days when fine weather is predicted, and indoor museum visits on days when rain is predicted. The total cost is calculated based on this plan, and the result is sent to the device. The user checks the travel plan and cost on the device and clicks the "Book" button. The device sends the reservation intent to the server, and the server calls each reservation API to complete the reservation procedure. If the reservation is successful, the result is notified to the device, and detailed reservation information is displayed to the user.

[1578] Prompt Sentence Examples

[1579] Below is an example of a prompt sentence to input to the generative AI model.

[1580] Please explain in detail each step of the system where the program creates travel plans and completes the booking process. In particular, please explain the flow from input to display of results and the APIs used.

[1581] The above is a detailed description of the embodiment of the present invention. This system allows users to efficiently plan their trips and complete reservation procedures without hassle.

[1582] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1583] Step 1:

[1584] User enters travel information

[1585] A user opens a travel planning app and enters their travel dates (e.g., May 20 to May 23, 2023), destination (e.g., Tokyo), and activities of interest (e.g., visiting museums). The information entered is stored in data entry fields on the device.

[1586] Step 2:

[1587] The device sends the input information to the server

[1588] The terminal sends the travel information entered by the user to the server as a single JSON-formatted data. Specifically, it uses an HTTP POST request to send the "travel information entered by the user." This data is prepared to be parsed by the server. The input is the user's travel information, and the output is the server's receipt of the request.

[1589] Step 3:

[1590] The server retrieves weather information

[1591] The server calls a weather API based on the received user travel information. For example, it sends a request to the OpenWeatherMap API saying, "Please get weather information for Tokyo from May 20th to May 23rd, 2023." The weather information (weather, temperature, etc.) returned by the API is received, analyzed, and stored in a database. The input is the travel dates and destination, and the output is the weather information for that period.

[1592] Step 4:

[1593] The server retrieves the event information

[1594] The server calls an event API based on the received weather information. For example, it sends a request to the Eventbrite API saying, "Get information about art events taking place in Tokyo from May 20th to May 23rd, 2023." The event information (event name, location, date and time, etc.) returned by the API is received and stored in a database. The input is the travel dates, destination, and activities of interest, and the output is the event information for that period.

[1595] Step 5:

[1596] The server obtains accommodation information

[1597] The server calls the accommodation information API based on the received event information. For example, it sends a request to the Booking.com API saying, "Please get information about accommodations available in Tokyo from May 20th to May 23rd, 2023." The accommodation information (facility name, availability, price, etc.) returned by the API is received and stored in a database. The input is the travel dates and destination, and the output is accommodation information for that period.

[1598] Step 6:

[1599] The server generates travel suggestions

[1600] The server integrates the acquired weather, event, and accommodation information. It automatically generates an optimal travel plan based on the user's interests. For example, a plan might be created to attend an outdoor art event on a sunny day, and visit a museum on a rainy day. Based on this information, a detailed travel plan is created, including a daily itinerary and transportation methods. The input is weather information, event information, and accommodation information, and the output is a travel plan.

[1601] Step 7:

[1602] The server calculates the trip cost

[1603] The server calculates the total cost of the trip based on the generated travel suggestions. It adds up accommodation costs, event participation fees, transportation costs, etc., and provides the optimal plan within the budget. For example, it calculates "accommodation costs 5,000 yen / night x 3 nights + event participation fee 2,000 yen + transportation costs 1,000 yen = total cost 17,000 yen." The input is the travel suggestions, and the output is the total cost.

[1604] Step 8:

[1605] The server sends travel plans and costs to the terminal

[1606] The server sends the generated travel plan and its cost to the terminal. Specifically, it returns the travel plan and detailed cost information to the terminal as a new HTTP response. It formats the data so that the user can check it and sends it. The input is the travel plan and detailed cost information, and the output is the response to the terminal.

[1607] Step 9:

[1608] The device displays travel suggestions and costs to the user

[1609] The terminal displays the travel plan and cost information received from the server. Details such as "Itinerary for May 20th: Tokyo art events" and "Itinerary for May 21st: Museum tours" are displayed on the screen, along with the total cost to the user. The input is the travel plan and cost details, and the output is an enjoyable travel plan presented to the user.

[1610] Step 10:

[1611] User reviews trip suggestions and makes booking

[1612] The user reviews the proposed itinerary and costs, and if satisfied, clicks the "Book" button. The input is the itinerary details, and the output is a click indicating the intention to book.

[1613] Step 11:

[1614] The device sends the reservation information to the server

[1615] The device then sends the information indicating the user's booking intention back to the server. Specifically, it uses an HTTP POST request to send JSON data containing the content "The user has booked a travel plan." The input is the user's booking intention, and the output is a request sent to the server.

[1616] Step 12:

[1617] The server sends a reservation request to the associated service.

[1618] The server sends each reservation request to the accommodation reservation API, event reservation API, and transportation reservation API. For example, it sends specific request details such as "confirm accommodation reservation," "purchase event tickets," or "reserve transportation tickets." The input is the user's reservation information, and the output is a request sent to each reservation service.

[1619] Step 13:

[1620] The server checks the reservation result and notifies the device

[1621] The server receives responses from each API and checks whether the reservation was successful. If successful, it notifies the terminal of the result "Reservation successful." If unsuccessful, it generates and sends an appropriate error message. The input is the result of the reservation (success or failure), and the output is feedback to the terminal.

[1622] Step 14:

[1623] The device displays a notification to the user that the reservation is complete.

[1624] The terminal displays the reservation results received from the server to the user. If the reservation is successful, it displays detailed information such as "Accommodation: XX Hotel, Event: XX Exhibition, Transportation: Shinkansen", and if the reservation is unsuccessful, it displays an error message such as "Reservation failed. Please try again." The input is the reservation result, and the output is the feedback displayed to the user.

[1625] (Application example 1)

[1626] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1627] Traditional travel planning and booking systems require users to manually gather information from multiple sources and create travel plans, which is time-consuming. They also lack easy access to real-time information during the trip, making it difficult to change or adjust plans. Furthermore, the lack of systems that support specialized visual devices, voice input, or gesture control limits the user experience.

[1628] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1629] In this invention, the server includes: means for a user to input a travel itinerary, destinations, and activities of interest; means for acquiring weather information based on the input information; means for collecting event information; means for acquiring accommodation information; means for generating a travel plan by integrating weather information, event information, and accommodation information; means for calculating travel costs based on the travel plan; means for a user to confirm the travel plan and input a reservation intention; means for making reservations for accommodation, events, and transportation; means for notifying the user of the reservation results; means for a user to confirm and operate the travel information via a visual device; means for inputting travel information by voice input and gesture operation; and means for acquiring and displaying weather information, event information, and accommodation information in real time. This allows a user to efficiently plan a travel plan all at once and obtain and operate information in real time, enabling a flexible and comfortable travel experience.

[1630] "Means for user input of travel itinerary, destinations, and activities of interest" refers to an interface that allows a user to input detailed information about their trip using smart glasses or other visual device.

[1631] "Means for obtaining weather information based on input information" refers to a process for obtaining weather data from an external weather API based on the travel itinerary and destination input by the user.

[1632] "Means of collecting event information" refers to the function for obtaining information about events taking place at the travel destination from an external event calendar API.

[1633] "Means for obtaining accommodation information" refers to the mechanism for obtaining accommodation availability and price information from an external accommodation information API during the user's travel period.

[1634] "Means for generating travel plans by integrating weather information, event information, and accommodation information" refers to algorithms or systems that integrate various collected information and generate optimal travel plans based on the user's interests.

[1635] The "means for calculating travel costs based on the travel plan" refers to a function for calculating the total travel costs based on the content of the generated travel plan.

[1636] The "means for displaying the generated travel proposals and their costs to the user" refers to an interface for displaying the travel plan and cost information so that the user can check it via a visual device.

[1637] "Means for users to confirm travel plans and input their intention to make a reservation" refers to a function that allows users to input their intention to make a reservation for the displayed travel plan using voice commands or gesture operations.

[1638] "Means for booking accommodation, events, and transportation" refers to the process of booking accommodation, event participation, and transportation through an external service based on the user's booking intention.

[1639] "Means for notifying the user of the reservation result" refers to a function for notifying the user whether the reservation was successful or not.

[1640] "Means for users to view and manipulate travel information via a visual device" refers to an interface that allows users to view and manipulate details of their travel plans using a visual device such as smart glasses.

[1641] "Means for inputting travel information by voice input and gesture operation" refers to a function that allows a user to input travel information using voice recognition and gesture recognition.

[1642] "Means for acquiring and displaying weather information, event information, and accommodation information in real time" refers to a system for instantly acquiring and displaying the latest weather information, event information, and accommodation information in response to a user request.

[1643] The travel planning and booking system of the present invention utilizes a user's visual device (e.g., smart glasses) to plan and book a trip all at once. The system is composed of a user terminal, a server, and external data sources (weather API, event API, accommodation API, etc.), and specific embodiments are described in the detailed description below.

[1644] System Configuration

[1645] User terminal

[1646] Users wear the smart glasses and input their travel itinerary, destinations, and activities of interest through voice and gesture control, reducing user effort and enabling more intuitive operation.

[1647] server

[1648] The server includes the following means:

[1649] 1. How to obtain weather information: Weather information is obtained from an external weather API based on the travel itinerary and destination entered by the user.

[1650] 2. Event information collection method: Event information of interest to users is collected from an external event calendar API.

[1651] 3. How to obtain accommodation information: Accommodation availability and pricing information is obtained from an external accommodation information API.

[1652] 4. Travel plan generation method: The acquired weather information, event information, and accommodation information are integrated to generate the optimal travel plan based on the user's interests.

[1653] 5. Travel cost calculation method: Calculate the total travel cost based on the generated travel plans.

[1654] 6. Result display means: The generated travel plans and their cost information are displayed on the user's smart glasses.

[1655] 7. Means for inputting reservation intention: The user inputs the reservation intention using voice commands or gesture operations.

[1656] 8. Reservation Methods: Make reservations for accommodation, events, and transportation through external services.

[1657] 9. Notification means: Notify the user of the reservation results.

[1658] Devices and Software

[1659] The specific hardware and software used are as follows:

[1660] Hardware: Smart glasses (e.g., Google Glass)

[1661] Software: Speech recognition library (Google Cloud Speech API), gesture recognition library, external API (weather API, event API, accommodation API), data analysis library

[1662] Data processing flow

[1663] 1. Data input: The user inputs by voice, "I would like to travel to Kyoto next weekend and am interested in visiting museums and cafes." Information can also be input using gestures, and this information is sent from the user's device to the server.

[1664] 2. Information collection: The server calls the weather API, event API, and accommodation API to collect the necessary weather information, event information, and accommodation information.

[1665] 3. Plan Generation: The system integrates the collected information and generates an optimal travel plan based on the user's interests. For example, it suggests outdoor art events on sunny days and includes a visit to a museum on rainy days.

[1666] 4. Cost calculation and display: The total travel cost is calculated based on the generated plan and the information is overlaid within the field of view of the smart glasses.

[1667] 5. Reservation procedure: If the user is satisfied with the displayed travel plan, he / she can make a reservation by using the voice command "Reserve." The server sends a reservation request to each linked service, collects the reservation results, and notifies the user.

[1668] Examples of concrete examples and prompts

[1669] For example, if a user says, "Plan a trip to Kyoto for next weekend to visit museums and cafes," the system will gather the appropriate information and display something like this:

[1670] "Dates: October 20th to October 23rd"

[1671] "Destination: Kyoto"

[1672] "Activities: Visiting museums and cafes"

[1673] "Travel plan: October 20th: Kyoto International Museum, XX Art Museum (sunny weather forecast)"

[1674] "October 21st: XX Cafe Tour (Rain Forecast)"

[1675] "Total cost:\XXXXX"

[1676] "Would you like to make a reservation?"

[1677] By implementing the present invention in this way, users can plan their trips efficiently and intuitively, and can obtain and manipulate information in real time, allowing them to enjoy a flexible and comfortable travel experience.

[1678] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1679] Step 1:

[1680] The user puts on the smart glasses and uses voice input or gesture control to input their travel itinerary, destinations, and activities they are interested in. For example, the user might say, "I'd like to visit museums and cafes in Kyoto next weekend." This input information is then converted into text data by the device.

[1681] Input: User voice or gesture control

[1682] Output: Text data (travel itinerary, destinations, activities of interest)

[1683] Step 2:

[1684] The device sends the entered text data to the server using a method such as an HTTP request.

[1685] Input: Text data

[1686] Output: HTTP request

[1687] Step 3:

[1688] The server analyzes the received text data and calls an external weather API based on the travel itinerary and destination to obtain weather information for the destination during the travel period. The weather information is obtained in JSON format.

[1689] Input: Text data (travel itinerary, destination)

[1690] Output: Weather information (JSON data)

[1691] Step 4:

[1692] The server calls the event API to retrieve information about events taking place at the travel destination during the trip. The event information is filtered based on the user's interests and retrieved in JSON format.

[1693] Input: Text data (travel itinerary, destination, activities of interest)

[1694] Output: Event information (JSON data)

[1695] Step 5:

[1696] The server calls the accommodation information API to obtain information about accommodations at the destinations during the trip. The accommodation information includes room availability and price information, and is obtained in JSON format.

[1697] Input: Text data (travel itinerary, destination)

[1698] Output: Accommodation information (JSON data)

[1699] Step 6:

[1700] The server integrates the acquired weather, event, and accommodation information to generate an optimal travel plan based on the user's interests. Based on the weather information, it suggests outdoor events on sunny days and indoor activities on rainy days.

[1701] Input: Weather information, event information, accommodation information

[1702] Output: Travel plan

[1703] Step 7:

[1704] The server calculates the travel expenses based on the generated travel plan, including accommodation, event participation fees, and transportation costs.

[1705] Input: Travel Plan

[1706] Output: Trip cost

[1707] Step 8:

[1708] The server sends the generated travel plan and its cost information to the user's smart glasses using an HTTP response.

[1709] Input: Travel plan, travel expenses

[1710] Output: HTTP response

[1711] Step 9:

[1712] The user reviews the travel plan and cost information displayed on the smart glasses, and if satisfied, inputs an intent such as "book" using voice commands or gestures.

[1713] Input: Travel plan, travel expenses

[1714] Output: Reservation intent (voice or gesture operation)

[1715] Step 10:

[1716] The device sends the user's reservation intention to the server, again using a method such as an HTTP request.

[1717] Input: Booking Intent

[1718] Output: HTTP request

[1719] Step 11:

[1720] The server calls an external booking API to make reservations for accommodation, events, and transportation. The booking results are returned in JSON format.

[1721] Input: Booking Intent

[1722] Output: Reservation results (JSON data)

[1723] Step 12:

[1724] The server analyzes the reservation result and generates a success message and reservation details if successful, or an error message if unsuccessful, and sends them to the user's smart glasses.

[1725] Input: Reservation result

[1726] Output: Success or error message

[1727] Step 13:

[1728] The user checks the reservation results displayed on the smart glasses. If successful, the trip details (including accommodation information and event ticket information) are displayed.

[1729] Input: Success message or error message

[1730] Output: Trip details or error message

[1731] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1732] The present invention is a travel planning and booking system incorporating an emotion engine that recognizes the user's emotions. A specific embodiment of this system will be described below.

[1733] System configuration

[1734] The system consists of a user's device, a server, external data sources (weather API, event API, accommodation API, etc.), and an emotion engine. Users operate their devices to make travel plans, and the information is sent to the server. The server collects and analyzes the necessary data to create travel plans. Furthermore, the emotion engine recognizes the user's emotions and adjusts the travel plans accordingly.

[1735] Program processing overview

[1736] 1. User enters travel information

[1737] Users open the app and enter their travel dates, destinations, and activities they are interested in. For example, a user might enter, "I want to visit museums in Tokyo from May 20th to May 23rd."

[1738] 2. The device sends the input information to the server

[1739] The device receives the travel information entered by the user and sends the data to the server, usually in the form of an HTTP request.

[1740] 3. Emotion engine recognizes user emotions

[1741] The device uses an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions, tone of voice, and / or text input to determine the user's emotional state (e.g., joy, sadness, excitement, stress, etc.).

[1742] 4. The server retrieves weather information

[1743] The server calls the weather API to obtain weather information for the destination during the trip. The server analyzes the weather data according to the trip period and destination entered by the user.

[1744] 5. The server retrieves the event information

[1745] The server calls the event API to retrieve event information held at the destination during the trip, and filters the event information based on the user's interests.

[1746] 6. The server obtains the accommodation information

[1747] The server calls the accommodation information API to obtain information on room availability and prices for accommodations during the travel period, and analyzes the obtained data to select the optimal accommodation plan.

[1748] 7. The server generates travel suggestions

[1749] The server integrates the acquired weather, event, and accommodation information and generates travel plans taking into account the user's emotional state. For example, if the user is feeling stressed, it will create a plan that includes many relaxing activities, while if the user is excited, it will create a plan that includes many active activities.

[1750] 8. The server calculates the cost of the trip

[1751] The server calculates the total cost of the trip by adding up accommodation costs, event participation fees, and transportation costs (such as rental cars and taxis) based on the generated travel plan. A detailed breakdown of the costs is also provided.

[1752] 9. The server sends the travel plan and costs to the device.

[1753] The server sends the generated travel plans and their costs to the device, usually in JSON format, which can be easily parsed by the device.

[1754] 10. The device displays trip suggestions and costs to the user.

[1755] The device analyzes the travel proposals and cost information received from the server and displays them to the user, who can then check the details of the proposed travel plan (daily schedule, cost breakdown).

[1756] 11. User reviews trip suggestions and makes booking

[1757] The user checks the proposed travel plans and costs, and if satisfied, clicks the "Book" button, thereby indicating their intention to begin the booking process.

[1758] 12. The device sends the reservation information to the server

[1759] The terminal sends the user's reservation intention to the server. The reservation data is sent to the server as an HTTP request.

[1760] 13. The server calls the API of each service it is connected to and starts the specific reservation procedure.

[1761] The server uses the accommodation reservation API, event reservation API, and transportation reservation API to make each reservation.

[1762] 14. The server checks the response from each reservation service and notifies the device.

[1763] The server checks the responses from each reservation service, determines whether the reservation was successful, and notifies the terminal of the result.

[1764] 15. The terminal displays the reservation results to the user

[1765] The terminal analyzes the reservation results received from the server and displays them to the user. If the reservation is successful, detailed reservation information (such as the accommodation address and event ticket information) is displayed, but if the reservation is unsuccessful, an error message is displayed to notify the user.

[1766] Specific example explanation

[1767] For example, if a user inputs "I want to visit art museums in Tokyo from May 20th to May 23rd," and the emotion engine determines that the user is "feeling stressed" based on their facial expression and tone of voice when they input, the device will send this information to the server. The server will then obtain weather, event, and accommodation information, and create a travel plan that prioritizes relaxing facilities and events based on the user's emotional data.

[1768] In this way, by implementing the present invention, it is possible to provide a travel plan that is customized according to the user's emotional state, thereby improving user satisfaction.

[1769] The processing flow will be explained below.

[1770] Step 1:

[1771] A user launches the app and enters their travel dates, destinations, and activities they are interested in. For example, a user might enter, "I want to visit museums in Tokyo...

Claims

1. a means for the user to input travel itineraries, destinations, and activities of interest; means for obtaining weather information based on input information; a means for collecting event information; A means for obtaining accommodation information; means for integrating weather information, event information, and accommodation information to generate travel suggestions; means for calculating travel costs based on the travel proposal; means for displaying the generated travel plans and their costs to the user; a means for the user to review the travel suggestions and input their booking intent; A way to make reservations for accommodation, events, and transportation; means for notifying the user of the reservation result; A system including:

2. 2. The system according to claim 1, wherein the means for acquiring weather information based on input information is means for acquiring weather data from an external weather API.

3. 2. The system of claim 1, wherein the means for collecting event information is means for obtaining event data from an event calendar API.

Images