System

Abstract

A system is provided.SOLUTION: This system includes a means for storing past use tendency data, a means for storing travel plan information inputted by a user, a means for generating a reservation candidate on the basis of the past use tendency data and the travel plan information, a means for presenting the reservation candidate to the user, and a means for determining the reservation candidate selected by the user and performing reservation cooperation.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional travel reservation systems require users to make detailed plans and make individual reservations for each trip, which is time-consuming and labor-intensive. Furthermore, they lack recommendations based on users' preferences and past travel history, making it difficult for users to create optimal travel plans. Another issue is the lack of a flexible reservation process, such as when users want to confirm only part of a reservation. [Means for solving the problem]

[0005] To solve this problem, we provide a system that includes a means for saving past usage trend data and a means for saving travel plan information entered by the user.We also build a system that includes a means for generating reservation candidates based on the past usage trend data and travel plan information, a means for presenting reservation candidates to the user, and a means for confirming the reservation candidates selected by the user and coordinating the reservation.Furthermore, by adding a means for inputting and saving user profile information and a means for notifying the user of the reservation results, we enable users to make travel plans efficiently and flexibly.

[0006] "Past usage trend data" refers to historical information about travel plans and reservations that a user has made in the past, and analyzing this data is intended to understand the user's preferences and behavioral patterns.

[0007] "Travel planning information" refers to detailed information such as the destination, desired tourist spots, activities, budget, and schedule that a user inputs when planning a new trip.

[0008] "Reservation candidates" refers to a list of travel destinations, tourist spots, and activities that should be suggested to the user, which is generated based on the user's past usage trend data and travel plan information.

[0009] "Reservation integration" refers to the process of confirming the reservation options selected by the user, sending reservation information via the API of the reservation site or app to carry out the actual reservation procedure based on that, and ultimately completing the reservation.

[0010] "User profile information" refers to basic personal information (such as name, age, gender, and email address) that a user enters when creating an account.

[0011] The "reservation result" refers to information obtained after the reservation link is executed, indicating whether the reservation was successful or unsuccessful, and is ultimately notified to the user. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0013] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0016] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0017] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0018] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0020] [First embodiment]

[0021] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0022] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0025] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0028] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0032] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0033] This invention relates to an advance travel reservation assistance system that allows users to plan their trips efficiently and flexibly. This system has the function of automatically generating and suggesting optimal reservation options by storing user profile information and past reservation history and learning the user's preferences and travel patterns.

[0034] System configuration

[0035] The system consists of the following main components:

[0036] 1. Server

[0037] 2. Terminal

[0038] 3. Users

[0039] Program processing and natural language explanation

[0040] Registering user profile information

[0041] A user installs the app and creates an account.

[0042] The device sends the basic information entered by the user (such as name, age, gender, and email address) to the server.

[0043] The server stores the received information in a database.

[0044] Learn from past usage trends

[0045] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[0046] For example, it is determined that a user who has visited many resort areas in the past will tend to prefer resort areas in the future.

[0047] Enter your travel plans

[0048] The user opens the travel planning screen and inputs the travel destination, desired tourist spots, activities, budget, dates, etc.

[0049] The terminal transmits this information to the server.

[0050] Generation and suggestion of reservation candidates

[0051] The server selects appropriate tourist destinations and activities based on past usage trend data and newly entered travel plan information.

[0052] The server converts the selected reservation candidates into JSON format and sends them to the terminal.

[0053] The terminal analyzes the transmitted information and displays reservation candidates to the user.

[0054] Confirmation and notification of reservations

[0055] The user selects the desired reservation from the displayed reservation candidates and confirms the reservation.

[0056] The terminal transmits the user's selection to the server.

[0057] The server then links the reservation information to the corresponding reservation site or app to confirm the reservation.

[0058] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a final confirmation notice to the user.

[0059] Specific examples

[0060] As a specific example, consider the case where User A uses an app to plan their next trip. User A has previously visited Kyoto and Okinawa and tends to enjoy sightseeing and relaxation. For their next trip, User A wants to visit Hokkaido, and plans to visit Furano and the Kushiro Wetlands and enjoy snowboarding.

[0061] User A creates an account and enters their travel plans into the app.

[0062] The terminal sends this information to the server, which then generates appropriate reservation candidates based on past usage trend data and new travel plan information.

[0063] For example, a plan is generated that includes accommodation at Furano Ski Resort and a sightseeing tour of the Kushiro Wetlands.

[0064] User A selects accommodation and snowboarding lessons from the displayed reservation options and confirms the reservation.

[0065] The server then connects this information to the reservation site to complete the reservation.

[0066] Finally, the terminal notifies user A of the reservation result, allowing him to check the detailed reservation contents.

[0067] Such a configuration and operation allows users to more efficiently and conveniently plan their trips and smoothly proceed with reservations.

[0068] The processing flow will be explained below.

[0069] Step 1:

[0070] The user installs the app, opens the account creation screen, enters basic information such as name, age, gender, and email address, and presses the "Register" button.

[0071] Step 2:

[0072] The terminal converts the information entered by the user into JSON format and sends it to the server via HTTPS.

[0073] Step 3:

[0074] The server analyzes the received JSON data, saves the user information in the database, generates a save completion status, and returns it to the device.

[0075] Step 4:

[0076] The user opens the travel planning screen, enters details such as the travel destination, desired tourist spots, activities, budget, and dates, and presses the "Save Travel Plan" button.

[0077] Step 5:

[0078] The device converts the travel planning information into JSON format and sends it to the server via HTTPS.

[0079] Step 6:

[0080] The server parses the received JSON data, saves the travel plan information to the database, generates a save completion status, and returns it to the device.

[0081] Step 7:

[0082] The server runs a recommendation algorithm based on past usage trends in the database and newly entered travel plan information, collecting data from similar cases of other users to select matching tourist spots and activities.

[0083] Step 8:

[0084] The server lists the selected reservation candidates, converts the list into JSON format, and sends it to the terminal.

[0085] Step 9:

[0086] The device analyzes the received JSON data and displays a list of reservation candidates to the user.

[0087] Step 10:

[0088] The user selects the desired tourist spot or activity from the displayed reservation options and presses the "Confirm reservation" button.

[0089] Step 11:

[0090] The terminal converts the reservation candidate information selected by the user into JSON format and sends it to the server via HTTPS.

[0091] Step 12:

[0092] The server analyzes the received JSON data and executes the reservation procedure using the API of the relevant reservation site or app. When the reservation process is complete, it records the result and generates a status.

[0093] Step 13:

[0094] The server converts the reservation completion status and detailed information into JSON format and sends it to the terminal.

[0095] Step 14:

[0096] The terminal analyzes the received reservation completion notification, displays the reservation completion notification to the user, and provides an interface for checking the details of the reservation.

[0097] Step 15:

[0098] Users can check reservation details and make changes or cancellations as necessary from the "My Page" or "Reservation History" within the app.

[0099] Example 1

[0100] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0101] Conventional travel reservation systems often make blanket recommendations without fully considering a user's past booking history or preferences. This results in inefficiency, as users have to spend a lot of time searching for a travel plan that suits their needs. Furthermore, the process of retrieving information from different booking sites and confirming and notifying reservations is cumbersome. These issues make it difficult for users to easily create optimal travel plans based on their preferences and past usage history.

[0102] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0103] In this invention, the server includes means for saving past usage trend data, means for saving travel plan information entered by the user, means for generating reservation candidates based on the past usage trend data and the travel plan information, means for creating prompt sentences for generating reservation candidates using a generative AI model, means for converting the generated reservation candidates into JSON format and transmitting them, means for presenting reservation candidates to the user, means for confirming the reservation candidates selected by the user and coordinating reservations, and means for notifying the user of confirmed reservation information. This enables users to efficiently create optimal travel plans based on their past usage trends and preferences.

[0104] "Past usage trend data" refers to historical information about reservations made by a user, including travel destinations, travel dates, and information about accommodations and activities used.

[0105] "Travel planning information" is information that a user inputs when planning a new trip, and includes the travel destination, desired tourist spots, activities, budget, schedule, and the like.

[0106] A "reservation candidate" is a proposed plan generated by the server based on the user's past usage trend data and travel plan information, and includes suggestions for specific accommodations and activities.

[0107] "Generative AI model" refers to artificial intelligence technology that analyzes users' past usage trends and travel planning information to generate optimal reservation options.

[0108] A "prompt sentence" is an instruction sentence input to the generative AI model, and is generated based on the user's travel plan information.

[0109] "JSON format" is an abbreviation for JavaScript Object Notation, and is a format for expressing data in text format.

[0110] "Reservation linkage" is a procedure in which the server communicates with an external reservation site or application to confirm the reservation candidates selected by the user and make the reservation.

[0111] "Notification" is a function that allows the server to notify the user of confirmed reservation information, and includes methods such as email and in-app notifications.

[0112] This invention relates to an advance travel reservation assistance system that allows users to make travel plans efficiently and flexibly. This system is composed of multiple components, the main components of which include a server, a terminal, and a user.

[0113] Registering user profile information

[0114] Users first install the application and create an account, which involves entering basic profile information such as name, age, gender, and email address on the device.

[0115] The terminal transmits the input profile information to the server via an HTTP request.

[0116] The server stores the received profile information in a database (e.g., MySQL), which serves to systematically manage the profile information for each user.

[0117] Learn from past usage trends

[0118] The server periodically retrieves and analyzes the user's past booking history stored in the database, using machine learning algorithms (e.g., K-means clustering) to identify the user's preferences and travel patterns.

[0119] The server's analysis results clarify the user's preferred travel destinations and activity patterns. For example, it learns that a user who has visited many resort areas in the past tends to continue to prefer resort areas in the future.

[0120] Enter your travel plans

[0121] The user opens the travel plan input screen within the application and enters detailed information such as the travel destination, desired tourist spots, activities, budget, and schedule.

[0122] The device sends this input information to the server in JSON format, specifically using a REST API.

[0123] Generate reservation candidates

[0124] The server uses a generative AI model (e.g., GPT-4) to generate suitable reservation suggestions based on past usage trend data and newly entered travel plan information, using the following prompt:

[0125] Suggest suitable booking suggestions based on your next travel plans.

[0126] Travel destination: Hokkaido

[0127] Preferred tourist destinations: Furano, Kushiro Marsh

[0128] Activity: Snowboarding

[0129] Budget: Under 200,000 yen

[0130] Dates: February 15, 2024 to February 20, 2024

[0131] Past travel history: Kyoto, Okinawa (sightseeing, relaxation)

[0132] This prompt is then fed into an AI model to generate an optimal travel plan.

[0133] Presenting reservation candidates and confirming reservations

[0134] The server converts the generated reservation candidates into JSON format and sends them to the terminal.

[0135] The terminal analyzes the received reservation candidates and displays them in a visually easy-to-understand format for the user.

[0136] The user selects the desired reservation from the displayed reservation candidates and confirms the reservation.

[0137] The terminal transmits the user's selection to the server, and the server cooperates with an external reservation site or application to confirm the reservation.

[0138] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a final confirmation notice to the user.

[0139] As a concrete example, if User A uses this system to plan and book a trip to Hokkaido, the system generates an optimal plan based on the user's past travel history and current preferences, and the reservation is completed smoothly. This series of operations allows the user to plan an efficient trip without any hassle.

[0140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0141] Step 1: Register your user profile information

[0142] The user installs the application and creates a new account.

[0143] The user enters basic information such as name, age, gender, and email address and submits it.

[0144] The terminal sends the entered basic information to the server via an HTTP request.

[0145] The server saves the received profile information in a database (e.g., MySQL). At this time, the server validates the input data to maintain data consistency. It responds with a success message to the terminal.

[0146] input:

[0147] Basic information (name, age, gender, email address)

[0148] output:

[0149] Save to database, success message

[0150] Step 2: Learn from past usage trends

[0151] The server periodically retrieves the stored past usage trend data from the database.

[0152] The server analyzes the acquired usage trend data and runs machine learning algorithms (e.g., K-means clustering) to learn user preferences and travel patterns.

[0153] The server uses the analysis results to classify the user's preferences and travel patterns and create basic information for a new travel plan.

[0154] input:

[0155] Past usage trend data

[0156] output:

[0157] User preference classification, travel pattern data

[0158] Step 3: Enter your travel plans

[0159] The user opens the travel plan input screen within the application and enters detailed information such as the travel destination, desired tourist spots, activities, budget, and schedule.

[0160] The terminal converts the input travel planning information into JSON format and sends it to the server via an HTTP request.

[0161] input:

[0162] Travel planning information (destination, sightseeing spots, activities, budget, schedule)

[0163] output:

[0164] JSON formatted travel plan information

[0165] Step 4: Generate reservation candidates

[0166] The server generates a prompt sentence based on past usage trend data and newly entered travel plan information.

[0167] The generated prompt sentence is input into a generative AI model (e.g., GPT-4) to generate appropriate reservation candidates. The example prompt sentence used is as follows:

[0168] Suggest suitable booking suggestions based on your next travel plans.

[0169] Travel destination: Hokkaido

[0170] Preferred tourist destinations: Furano, Kushiro Marsh

[0171] Activity: Snowboarding

[0172] Budget: Under 200,000 yen

[0173] Dates: February 15, 2024 to February 20, 2024

[0174] Past travel history: Kyoto, Okinawa (sightseeing, relaxation)

[0175] The server converts the reservation suggestions obtained from the generative AI model into JSON format.

[0176] input:

[0177] Prompt text, historical usage trend data, travel planning information

[0178] output:

[0179] JSON format data of reservation candidates

[0180] Step 5: Present reservation options

[0181] The server transmits the generated reservation candidates to the terminal.

[0182] The terminal analyzes the received JSON-formatted reservation candidates and visually displays them to the user. Specifically, the terminal uses GUI components to display the reservation candidates in a list or calendar format.

[0183] input:

[0184] JSON format data of reservation candidates

[0185] output:

[0186] Displaying reservation options

[0187] Step 6: Confirmation and notification of reservation

[0188] The user selects the desired reservation from the displayed reservation candidates and confirms the reservation.

[0189] The terminal transmits the selected reservation candidates to the server.

[0190] The server confirms the received reservation information by connecting with an external reservation site or application, and executes the reservation using an external API.

[0191] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a notification to the user.

[0192] input:

[0193] Selected reservation candidate

[0194] output:

[0195] Confirmation of reservation information, notification to users

[0196] Through these steps, users can efficiently plan their trip and make reservations smoothly.

[0197] (Application example 1)

[0198] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0199] Food delivery services have rapidly grown in popularity in recent years, but there is a lack of personalized menu suggestions for users, making it difficult for users to find an appropriate menu that meets their food preferences and restrictions. Furthermore, past order history cannot be effectively utilized, which can lead to a decrease in user satisfaction. The present invention aims to solve these problems by providing a food delivery assistant system that suggests optimal menus to users.

[0200] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0201] In this invention, the server includes means for saving past usage trend data, means for saving meal order information entered by the user, means for generating menu candidates based on the past usage trend data and the meal order information, means for presenting the menu candidates to the user, and means for confirming the menu candidates selected by the user and linking the orders, thereby enabling personalized menu suggestions that take into account the user's food preferences and past ordering history.

[0202] "Past usage trend data" is data that records the history of orders and selections made by a user in the past.

[0203] "Meal order information" refers to details of the meal preferences and orders currently input by the user.

[0204] "Menu candidates" is a list of menus that the server may generate and suggest based on past usage trend data and meal order information.

[0205] "Means for presenting menu candidates to the user" refers to a method for displaying and notifying the generated menu candidates on the user's device.

[0206] The "means for coordinating orders" refers to the process of sending the menu options selected by the user to the corresponding restaurant or provider and finalizing the order.

[0207] "Profile information" is data that indicates basic information such as the user's name, age, gender, and food preferences.

[0208] "Order result" is information indicating the status of the menu candidate selected by the user when it is actually ordered.

[0209] This invention relates to a food delivery assistant system that proposes personalized menus based on a user's food preferences and past ordering history. This system is composed of a server, a terminal, and a user.

[0210] Registering user profile information

[0211] First, the user installs the app and creates an account. The device sends basic information entered by the user, such as name, age, gender, food preferences, and allergy information, to the server, which then stores this information in a database.

[0212] Learn from past usage trends

[0213] The server analyzes the user's past order history stored in a database to learn the user's food preferences and patterns. For example, it determines that a user who frequently orders vegetarian dishes will likely continue to prefer vegetarian dishes. This analysis is performed using machine learning algorithms (for example, models using Scikit-Learn or TensorFlow).

[0214] Entering a meal order

[0215] The user opens the app's order screen and inputs their current mood and desired meal type (e.g., dessert, main dish, etc.). The device sends this information to the server. In this case, an example of a prompt when the user places a new order is "What would you like to eat today? Please choose a category (e.g., dessert, main dish, salad)."

[0216] Menu candidate generation and suggestions

[0217] The server selects an appropriate restaurant and menu based on past usage trend data and newly entered order information. The selected menu candidates are converted into JSON format and sent to the device. The device analyzes the sent information and displays recommended menu items to the user. At this stage, an example of a prompt for the machine learning model is sent to the generative AI model: "Consider past order history and predict the menu item that the user is likely to like next."

[0218] Order confirmation and notification

[0219] The user selects the desired item from the displayed menu options and confirms the order. The device sends the user's selection to the server. The server then connects the order information to the corresponding restaurant and confirms the order. Finally, if the order is successful, the server notifies the device of the result, and the device displays a final confirmation notice to the user.

[0220] Hardware and software used

[0221] Device: User's smartphone

[0222] Server: Cloud server (e.g., Amazon Web Services (AWS))

[0223] Database: MySQL

[0224] Analysis algorithm: Machine learning model (e.g., model using Scikit-Learn or TensorFlow)

[0225] API: RESTful API (transmits and receives data in JSON format)

[0226] In this way, it becomes possible to propose a personalized menu that takes into account the user's food preferences and past ordering history, thereby improving user satisfaction.

[0227] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0228] Step 1: Registering user profile information

[0229] Input: The user installs the app and enters basic information such as name, age, gender, food preferences, and allergy information.

[0230] Specific operation: The terminal processes the information entered by the user and sends it to the server.

[0231] Data processing or data calculation: The server organizes the received information and stores it in a database.

[0232] Output: The profile information is saved in the database.

[0233] Step 2: Learn from past usage trends

[0234] Input: The user's past order history stored in a database.

[0235] How it works: The server analyzes past order data using machine learning algorithms (e.g., Scikit-Learn or TensorFlow models).

[0236] Data processing or data calculation: Through analysis, the system learns the user's food preferences and patterns.

[0237] Output: The learning results are saved and serve as the basis for future data-based recommendations.

[0238] Step 3: Enter your meal order

[0239] Input: The user inputs their current mood and desired meal type (e.g., dessert, main dish, etc.) into the app.

[0240] Specific operation: The terminal sends the order information entered by the user to the server, and a prompt such as "What would you like to eat today? Please select a category (e.g., dessert, main dish, salad)" is presented to the user.

[0241] Data processing or data calculation: The order information is converted into a data format and sent to the server.

[0242] Output: The order information is saved on the server.

[0243] Step 4: Generate and suggest menu options

[0244] Input: Historical usage trend data and newly entered order information.

[0245] How it works: The server uses the generative AI model to select an appropriate restaurant and menu. At this time, it sends a prompt to the generative AI model saying, "Consider the user's past order history and predict the menu item that the user is likely to like next."

[0246] Data processing or data calculation: The menu candidates generated by the server are converted into JSON format and sent to the terminal.

[0247] Output: Menu suggestions are displayed on the terminal.

[0248] Step 5: Order confirmation and notification

[0249] Input: The user selects the desired menu option on the device.

[0250] Specific operation: The device sends the user's selection to the server, which then sends the order information to the corresponding restaurant and confirms the order.

[0251] Data processing or data calculation: The order information is sent to the restaurant's system and the order is confirmed.

[0252] Output: The server notifies the terminal of the order result and displays a final confirmation notice to the user.

[0253] This process flow allows the user to efficiently order food and receive suggested menu options that suit their food preferences.

[0254] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0255] This invention relates to an advance travel reservation assistant system that combines an emotion engine that recognizes the user's emotions. This system not only automatically generates and suggests optimal reservation options based on the user's past usage trend data and travel plan information, but also has the function of adjusting the reservation options based on the user's emotions. This allows users to create more personalized and optimized travel plans.

[0256] System configuration

[0257] The system consists of the following main components:

[0258] 1. Server

[0259] 2. Terminal

[0260] 3. Users

[0261] 4. Emotion Engine

[0262] Program processing and natural language explanation

[0263] Registering user profile information

[0264] A user installs the app and creates an account.

[0265] The device sends the basic information entered by the user (such as name, age, gender, and email address) to the server.

[0266] The server stores the received information in a database.

[0267] Learn from past usage trends

[0268] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[0269] Enter your travel plans

[0270] The user opens the travel planning screen and inputs the travel destination, desired tourist spots, activities, budget, dates, etc.

[0271] The terminal transmits this information to the server.

[0272] User Emotion Recognition

[0273] The emotion engine analyzes the user's emotions in real time based on their facial expressions, tone of voice, typing speed, etc.

[0274] The terminal transmits this emotion data to the server.

[0275] Emotion-based reservation candidate generation and adjustment

[0276] The server selects appropriate tourist spots and activities based on past usage trend data, travel plan information, and emotional data.

[0277] If the user's emotions are judged to be positive, the system will prioritize suggestions that reflect their preferences. On the other hand, if negative emotions are detected, the system will suggest relaxing plans and less stressful options.

[0278] The server converts the selected reservation candidates into JSON format and sends them to the terminal.

[0279] The terminal analyzes the transmitted information and displays a list of reservation candidates to the user.

[0280] Confirmation and notification of reservations

[0281] The user selects the desired tourist spot and activity from the displayed reservation options and confirms the reservation.

[0282] The terminal transmits the user's selection to the server.

[0283] The server then links the reservation information to the corresponding reservation site or app to confirm the reservation.

[0284] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a notification of reservation completion to the user.

[0285] Specific examples

[0286] As a specific example, consider the case where User A uses an app to plan their next trip. User A has previously visited Kyoto and Okinawa and tends to enjoy sightseeing and relaxation. For their next trip, User A wants to visit Hokkaido, and plans to visit Furano and the Kushiro Wetlands and enjoy snowboarding.

[0287] User A creates an account and enters their travel plans into the app.

[0288] The emotion engine detects positive emotions filled with anticipation from User A's facial expressions and tone of voice while he is typing.

[0289] The server generates a plan based on past usage trend data, new travel plans, and emotional data, including accommodation at Furano Ski Resort and a sightseeing tour of the Kushiro Wetlands.

[0290] User A selects accommodation and snowboarding lessons from the suggested reservation options and confirms the reservation.

[0291] The server then connects this information to the reservation site to complete the reservation.

[0292] Finally, the terminal notifies user A of the reservation result, allowing him to check the detailed reservation contents.

[0293] Such a configuration and operation allows users to create more personalized travel plans, and the introduction of an emotion engine improves the user experience.

[0294] The processing flow will be explained below.

[0295] Step 1:

[0296] The user installs the app, opens the account creation screen, enters basic information such as name, age, gender, and email address, and presses the "Register" button.

[0297] Step 2:

[0298] The terminal converts the information entered by the user into JSON format and sends it to the server via HTTPS.

[0299] Step 3:

[0300] The server analyzes the received JSON data, saves the user information in the database, generates a save completion status, and returns it to the device.

[0301] Step 4:

[0302] The user opens the travel planning screen, enters details such as the travel destination, desired tourist spots, activities, budget, and dates, and presses the "Save Travel Plan" button.

[0303] Step 5:

[0304] The device converts the travel planning information into JSON format and sends it to the server via HTTPS.

[0305] Step 6:

[0306] The server parses the received JSON data, saves the travel plan information to the database, generates a save completion status, and returns it to the device.

[0307] Step 7:

[0308] The emotion engine analyzes the user's emotions in real time based on their facial expressions, tone of voice, typing speed, etc. This emotion data is collected by the device.

[0309] Step 8:

[0310] The device converts the collected emotion data into JSON format and sends it to the server via HTTPS.

[0311] Step 9:

[0312] The server runs a recommendation algorithm based on past usage trends in the database, newly entered travel plan information, and emotional data.

[0313] Step 10:

[0314] The server collects data from similar cases of other users and selects the most suitable sightseeing spots and activities based on the user's emotions. For example, if positive emotions are detected, it will prioritize active plans, and if negative emotions are detected, it will suggest relaxing plans.

[0315] Step 11:

[0316] The server lists the selected reservation candidates, converts the list into JSON format, and sends it to the terminal.

[0317] Step 12:

[0318] The device analyzes the received JSON data and displays a list of reservation candidates to the user.

[0319] Step 13:

[0320] The user selects the desired tourist spot or activity from the displayed reservation options and presses the "Confirm reservation" button.

[0321] Step 14:

[0322] The terminal converts the reservation candidate information selected by the user into JSON format and sends it to the server via HTTPS.

[0323] Step 15:

[0324] The server analyzes the received JSON data and executes the reservation procedure using the API of the relevant reservation site or app. When the reservation process is complete, it records the result and generates a status.

[0325] Step 16:

[0326] The server converts the reservation completion status and detailed information into JSON format and sends it to the terminal.

[0327] Step 17:

[0328] The terminal analyzes the received reservation completion notification, displays the reservation completion notification to the user, and provides an interface for checking the details of the reservation.

[0329] Step 18:

[0330] Users can check reservation details and make changes or cancellations as necessary from the "My Page" or "Reservation History" within the app.

[0331] Example 2

[0332] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0333] Conventional travel reservation systems generate reservation candidates based on a user's past usage trends and travel plan information, but do not take into account the user's real-time emotional state, which has the problem of not being able to sufficiently increase user satisfaction. The present invention aims to further improve the user experience by proposing personalized reservation candidates based on the user's emotions.

[0334] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for saving past usage trend data, means for saving travel plan information entered by the user, means for generating reservation candidates based on the past usage trend data and the travel plan information, means for presenting reservation candidates to the user, means for confirming the reservation candidates selected by the user and coordinating reservations, means for analyzing user emotion data, and means for adjusting reservation candidates based on the user emotion data. This makes it possible to propose personalized travel plans based on the user's emotions.

[0335] "Past usage trend data" is data relating to trips taken by the user in the past, and is information indicating the user's preferences and patterns.

[0336] "Travel planning information" is data related to a trip that the user is planning, and includes detailed information such as the travel destination, desired tourist spots, activities, budget, and schedule.

[0337] "Reservation candidates" are travel plans and options suggested to the user, generated based on the user's past usage trend data and travel plan information.

[0338] "Emotion data" is data that indicates the user's real-time emotional state, and is information that is analyzed based on facial expressions, tone of voice, input speed, and the like.

[0339] "Reservation linkage" refers to the process of linking the reservation candidates selected by the user with an external reservation system to actually confirm the reservation.

[0340] MODE FOR CARRYING OUT THE INVENTION

[0341] This invention relates to an advance travel reservation assistant system that combines an emotion engine that recognizes the user's emotions. This system not only automatically generates and suggests optimal reservation options based on the user's past usage trend data and travel plan information, but also has the function of adjusting the reservation options based on the user's emotions. This allows users to create more personalized and optimized travel plans.

[0342] System configuration

[0343] The system consists of the following main components:

[0344] 1. Server

[0345] 2. Terminal

[0346] 3. Users

[0347] 4. Emotion Engine

[0348] Registering user profile information

[0349] A user installs the app and creates an account by entering basic information such as a username and email address. The device acquires the entered basic information and sends it to the server. The server stores the received information in a database. Specifically, data is sent via an HTTP POST request, and a common RDBMS (e.g., PostgreSQL, MySQL) is used as the database.

[0350] Learn from past usage trends

[0351] The server periodically analyzes the past booking history data in the database. Using Python and Scikit-learn, it uses clustering algorithms (e.g., K-means) to learn user preferences and travel patterns. The server adds the learning results to the user profile and reflects them in future recommendations.

[0352] Enter your travel plans

[0353] The user opens the travel planning screen and inputs information such as the areas and tourist spots they want to visit, activities, budget, schedule, etc. The device then sends this input information to the server.

[0354] User Emotion Recognition

[0355] The emotion engine uses the camera and microphone to analyze the user's facial expressions and tone of voice in real time, for example, using OpenCV and the Google Cloud Speech-to-Text API. The device then sends the emotion analysis results to the server.

[0356] Emotion-based reservation candidate generation and adjustment

[0357] The server generates suitable sightseeing spots and activity candidates based on the received travel plan information, past usage trend data, and emotional data. If the user's emotion is positive, it prioritizes suggestions that will satisfy excitement and expectations, and if it is negative, it selects a relaxing plan. The server sends the generated reservation candidates to the terminal in JSON format.

[0358] Confirmation and notification of reservations

[0359] The user selects the plan they want to reserve from the displayed list of reservation candidates and presses the Confirm button. The device sends the user's selection to the server. The server calls the API of the relevant reservation site to confirm the reservation. The server sends a reservation success message to the device, and the device displays a confirmation notification to the user.

[0360] Specific examples

[0361] As a specific example, consider the case where User A uses an app to plan their next trip. User A has previously visited Kyoto and Okinawa and tends to enjoy sightseeing and relaxation. For their next trip, User A wants to visit Hokkaido, and plans to visit Furano and the Kushiro Wetlands and enjoy snowboarding.

[0362] User A creates an account and enters their travel plans into the app.

[0363] The emotion engine detects positive emotions filled with anticipation from User A's facial expressions and tone of voice while he is typing.

[0364] The server generates a plan based on past usage trend data, new travel plans, and emotional data, including accommodation at Furano Ski Resort and a sightseeing tour of the Kushiro Wetlands.

[0365] User A selects accommodation and snowboarding lessons from the suggested reservation options and confirms the reservation.

[0366] The server then connects this information to the reservation site to complete the reservation.

[0367] Finally, the terminal notifies user A of the reservation result, allowing him to check the detailed reservation contents.

[0368] Examples of prompts

[0369] "Develop a system that suggests next travel plans based on the user's booking history and adjusts the optimal plan by analyzing the user's emotions in real time. Incorporate a mechanism that uses an emotion recognition engine to analyze the user's facial expressions, tone of voice, typing speed, etc. and reflects this in the travel plans."

[0370] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0371] Step 1:

[0372] A user installs the app and creates an account by entering basic information such as a username and email address. When the user presses the "Register" button, the basic information is generated as input data. The device obtains this input data and sends it to the server via an HTTP POST request. The server saves the received basic information in a database. Specifically, it uses PostgreSQL or MySQL to insert the user information into a table. It returns a success message as a save result.

[0373] Step 2:

[0374] The server periodically analyzes past reservation history data in the database. It extracts past usage trend data using the user ID as a key. The server uses Python and Scikit-learn to apply a clustering algorithm (e.g., K-means). This allows it to learn the user's preferences and travel patterns, and generates the results as clustering data. The generated clustering data is added to the user profile.

[0375] Step 3:

[0376] The user opens the travel planning screen and enters information such as the areas and tourist spots they want to visit, activities, budget, and dates. This information is generated as input data. The device compiles the entered travel planning information and sends it to the server via an HTTP request. The server saves the received travel planning information in a database and returns a success message as a result of saving.

[0377] Step 4:

[0378] The emotion engine uses the user's camera and microphone to analyze facial expressions and tone of voice in real time. Captured images and audio data are generated as input data. Specifically, facial expressions are analyzed using OpenCV, and audio is analyzed using the Google Cloud Speech-to-Text API. Emotion data is generated as the analysis result. The device acquires the emotion data and sends it to the server via an HTTP request.

[0379] Step 5:

[0380] The server generates suitable sightseeing spots and activity candidates based on the received travel plan information, past usage trend data, and emotion data. Input data includes travel plan information, past usage trend data, and emotion data. The server runs a Python script to generate reservation candidates based on this data. Based on the emotion analysis results, active plans are prioritized if the emotion is positive, and relaxing plans are prioritized if the emotion is negative. The generated reservation candidates are converted into JSON format and sent to the device.

[0381] Step 6:

[0382] The user selects a plan from the displayed list of reservation candidates and presses the confirm button. The device sends the selected reservation details to the server via an HTTP request. The server receives the selection and calls the API of the relevant reservation site to confirm the reservation. The reservation information includes accommodation, activities, and dates. The server notifies the device that the reservation is complete, and the device displays a confirmation notification to the user.

[0383] (Application example 2)

[0384] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0385] Many conventional travel reservation systems generate reservation candidates based on a user's past usage data and travel plan information, but because they cannot adjust reservation candidates taking into account the user's emotional state, it is difficult to propose travel plans optimized for each individual user.In addition, there are limited ways to notify users of reservation results, which reduces user convenience.

[0386] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0387] In this invention, the server includes means for saving past usage trend data, means for saving travel plan information entered by the user, means for generating reservation candidates based on the past usage trend data and travel plan information, means for analyzing the user's facial expressions and tone of voice to recognize emotions, means for adjusting and generating reservation candidates based on the emotion data, means for presenting reservation candidates to the user, means for confirming the reservation candidates selected by the user and coordinating reservations, and means for notifying the user of the reservation results. This makes it possible to propose an optimal travel plan that takes the user's emotional state into consideration. It also enables faster and more convenient notification of reservation results.

[0388] "Past usage trend data" refers to data including travel plans, reservation history, and preference information that the user has used in the past.

[0389] "Travel planning information" refers to information such as the destination, itinerary, budget, and desired activities that the user has entered in advance regarding their next trip.

[0390] "Reservation candidates" are candidates for travel plans, accommodations, activities, etc. that are generated based on the user's usage trend data and travel plan information and that the user can select.

[0391] "Emotion recognition" is the process of analyzing a user's emotions from their facial expressions, tone of voice, etc., and determining whether their emotional state is positive or negative.

[0392] "Emotion data" is the result of the user's emotion recognition, and is data that includes information such as the type and intensity of the emotion.

[0393] "Adjustment" is the process of modifying and optimizing the generated reservation candidates based on the acquired data to suit the user's emotional state.

[0394] "Smart glasses" are wearable devices that use a camera to analyze a user's facial expressions.

[0395] A "head-mounted display" is a wearable display device that inputs information from the user's vision and voice and analyzes emotions.

[0396] This invention relates to a system that utilizes a "smart travel assistant" system to recognize a user's emotions and generate and present an optimized travel plan. The system mainly consists of a server, a terminal (smartphone), and an emotion engine (including smart glasses and head-mounted displays) for emotion recognition.

[0397] When a user inputs their travel plans using the application, the device sends this information to the server, which then uses past usage trend data and the newly input travel plan information to generate initial reservation candidates.

[0398] The emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions in real time. This emotion data is sent to the server via the device. The server adjusts and optimizes reservation options based on the acquired emotion data. Specifically, if a positive emotion is recognized, it will prioritize generating travel plans that match the user's preferences, and if a negative emotion is recognized, it will suggest relaxing activities and low-stress plans.

[0399] The final reservation candidates are sent to the terminal and presented to the user. When the user selects and confirms a reservation, the information is sent back to the server, completing the reservation link. Finally, the reservation results are notified to the user via the terminal.

[0400] The operation of the system can be specifically explained as follows:

[0401] 1. Registering user profile information

[0402] The user installs the application and creates an account. The device sends the basic information entered by the user (such as name, age, gender, and email address) to the server and stores it in a database.

[0403] 2. Learning from past usage trends

[0404] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[0405] 3. Enter your travel plans

[0406] The user opens the travel planning screen and inputs the travel destination, desired sightseeing spots, activities, budget, schedule, etc. The device then sends this information to the server.

[0407] 4. User Emotion Recognition

[0408] The emotion engine (including smart glasses and head-mounted displays) analyzes the user's emotions in real time based on their facial expressions, tone of voice, etc. The device then transmits this emotion data to a server.

[0409] 5. Emotion-based reservation candidate generation and adjustment

[0410] The server selects appropriate sightseeing spots and activities based on past usage trend data, travel plan information, and emotional data. If the user's emotions are judged to be positive, it will prioritize suggestions that reflect their preferences, but if negative emotions are detected, it will suggest relaxing plans and activities that reduce stress.

[0411] 6. Booking confirmation and notification

[0412] The user selects the desired sightseeing spot and activity from the displayed reservation options and confirms the reservation. The device sends the user's selection to the server, which then links the reservation information to the corresponding reservation site or app to confirm the reservation. If the reservation is successful, the server notifies the device of the result, and the device displays a notification to the user that the reservation is complete.

[0413] This allows users to create more personalized travel plans, and the introduction of an emotion engine improves the user experience.

[0414] Example: Prompt sentences to input to a generative AI model

[0415] plaintext

[0416] User emotion recognition prompts

[0417] Analyze the following text data and determine the user's sentiment.

[0418] Facial image: {image_data}

[0419] Voice data: {voice_data}

[0420] Prompt sentences for generating reservation candidates based on emotions

[0421] Generate the best travel plan based on the following user and sentiment data:

[0422] User ID: {user_id}

[0423] emotion: {emotion}

[0424] By implementing the above-described embodiment, more advanced travel planning can be performed taking into account the user's emotional state.

[0425] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0426] Step 1:

[0427] A user installs the application and creates an account.

[0428] Input: Basic information entered by the user (name, age, gender, email address, etc.)

[0429] Operation: The device collects basic information and sends it to the server.

[0430] Output: User basic information is saved in a server-side database.

[0431] Step 2:

[0432] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[0433] Input: Past usage trend data stored on the server

[0434] How it works: The server uses machine learning algorithms to analyze the data and learn user preference patterns.

[0435] Output: The user's preference patterns are stored in a database.

[0436] Step 3:

[0437] The user opens the travel planning screen and inputs the travel destination, desired tourist spots, activities, budget, dates, etc.

[0438] Input: Information about your travel destination, desired sightseeing spots, activities, budget, dates, etc.

[0439] Operation: The device sends this information to the server.

[0440] Output: The travel plan information is saved on the server.

[0441] Step 4:

[0442] Emotion engines (including smart glasses and head-mounted displays) analyze users' emotions in real time based on their facial expressions, tone of voice, etc.

[0443] Input: User's facial expressions and tone of voice

[0444] How it works: The emotion engine uses facial expression analysis algorithms and voice analysis algorithms to analyze emotions and send them to the device.

[0445] Output: Emotion data is sent to the server via the device.

[0446] Step 5:

[0447] The server selects appropriate tourist spots and activities based on past usage trend data, travel plan information, and emotional data.

[0448] Input: Past usage trend data, travel plan information, emotional data

[0449] How it works: The server aggregates the data and runs algorithms to select suitable tourist destinations and activities.

[0450] Output: The generated reservation candidates are sent to the terminal in JSON format.

[0451] Step 6:

[0452] The user selects the desired tourist spot and activity from the generated reservation candidates and confirms the reservation.

[0453] Input: User selected reservation candidate

[0454] How it works: The device sends the selection to the server, and the server links the reservation information to the reservation site or app.

[0455] Output: The reservation confirmation result is sent from the server to the terminal.

[0456] Step 7:

[0457] If the reservation is successful, the result is notified to the terminal and displayed to the user.

[0458] Input: Reservation confirmation result

[0459] Operation: The server notifies the terminal of the result, and the terminal displays the notification to the user.

[0460] Output: The user is notified that the reservation is complete.

[0461] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0462] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0463] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0464] [Second embodiment]

[0465] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0466] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0467] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0468] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0469] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0470] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0471] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0472] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0473] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0474] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0475] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0476] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0477] This invention relates to an advance travel reservation assistance system that allows users to plan their trips efficiently and flexibly. This system has the function of automatically generating and suggesting optimal reservation options by storing user profile information and past reservation history and learning the user's preferences and travel patterns.

[0478] System configuration

[0479] The system consists of the following main components:

[0480] 1. Server

[0481] 2. Terminal

[0482] 3. Users

[0483] Program processing and natural language explanation

[0484] Registering user profile information

[0485] A user installs the app and creates an account.

[0486] The device sends the basic information entered by the user (such as name, age, gender, and email address) to the server.

[0487] The server stores the received information in a database.

[0488] Learn from past usage trends

[0489] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[0490] For example, it is determined that a user who has visited many resort areas in the past will tend to prefer resort areas in the future.

[0491] Enter your travel plans

[0492] The user opens the travel planning screen and inputs the travel destination, desired tourist spots, activities, budget, dates, etc.

[0493] The terminal transmits this information to the server.

[0494] Generation and suggestion of reservation candidates

[0495] The server selects appropriate tourist destinations and activities based on past usage trend data and newly entered travel plan information.

[0496] The server converts the selected reservation candidates into JSON format and sends them to the terminal.

[0497] The terminal analyzes the transmitted information and displays reservation candidates to the user.

[0498] Confirmation and notification of reservations

[0499] The user selects the desired reservation from the displayed reservation candidates and confirms the reservation.

[0500] The terminal transmits the user's selection to the server.

[0501] The server then links the reservation information to the corresponding reservation site or app to confirm the reservation.

[0502] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a final confirmation notice to the user.

[0503] Specific examples

[0504] As a specific example, consider the case where User A uses an app to plan their next trip. User A has previously visited Kyoto and Okinawa and tends to enjoy sightseeing and relaxation. For their next trip, User A wants to visit Hokkaido, and plans to visit Furano and the Kushiro Wetlands and enjoy snowboarding.

[0505] User A creates an account and enters their travel plans into the app.

[0506] The terminal sends this information to the server, which then generates appropriate reservation candidates based on past usage trend data and new travel plan information.

[0507] For example, a plan is generated that includes accommodation at Furano Ski Resort and a sightseeing tour of the Kushiro Wetlands.

[0508] User A selects accommodation and snowboarding lessons from the displayed reservation options and confirms the reservation.

[0509] The server then connects this information to the reservation site to complete the reservation.

[0510] Finally, the terminal notifies user A of the reservation result, allowing him to check the detailed reservation contents.

[0511] Such a configuration and operation allows users to more efficiently and conveniently plan their trips and smoothly proceed with reservations.

[0512] The processing flow will be explained below.

[0513] Step 1:

[0514] The user installs the app, opens the account creation screen, enters basic information such as name, age, gender, and email address, and presses the "Register" button.

[0515] Step 2:

[0516] The terminal converts the information entered by the user into JSON format and sends it to the server via HTTPS.

[0517] Step 3:

[0518] The server analyzes the received JSON data, saves the user information in the database, generates a save completion status, and returns it to the device.

[0519] Step 4:

[0520] The user opens the travel planning screen, enters details such as the travel destination, desired tourist spots, activities, budget, and dates, and presses the "Save Travel Plan" button.

[0521] Step 5:

[0522] The device converts the travel planning information into JSON format and sends it to the server via HTTPS.

[0523] Step 6:

[0524] The server parses the received JSON data, saves the travel plan information to the database, generates a save completion status, and returns it to the device.

[0525] Step 7:

[0526] The server runs a recommendation algorithm based on past usage trends in the database and newly entered travel plan information, collecting data from similar cases of other users to select matching tourist spots and activities.

[0527] Step 8:

[0528] The server lists the selected reservation candidates, converts the list into JSON format, and sends it to the terminal.

[0529] Step 9:

[0530] The device analyzes the received JSON data and displays a list of reservation candidates to the user.

[0531] Step 10:

[0532] The user selects the desired tourist spot or activity from the displayed reservation options and presses the "Confirm reservation" button.

[0533] Step 11:

[0534] The terminal converts the reservation candidate information selected by the user into JSON format and sends it to the server via HTTPS.

[0535] Step 12:

[0536] The server analyzes the received JSON data and executes the reservation procedure using the API of the relevant reservation site or app. When the reservation process is complete, it records the result and generates a status.

[0537] Step 13:

[0538] The server converts the reservation completion status and detailed information into JSON format and sends it to the terminal.

[0539] Step 14:

[0540] The terminal analyzes the received reservation completion notification, displays the reservation completion notification to the user, and provides an interface for checking the details of the reservation.

[0541] Step 15:

[0542] Users can check reservation details and make changes or cancellations as necessary from the "My Page" or "Reservation History" within the app.

[0543] Example 1

[0544] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0545] Conventional travel reservation systems often make blanket recommendations without fully considering a user's past booking history or preferences. This results in inefficiency, as users have to spend a lot of time searching for a travel plan that suits their needs. Furthermore, the process of retrieving information from different booking sites and confirming and notifying reservations is cumbersome. These issues make it difficult for users to easily create optimal travel plans based on their preferences and past usage history.

[0546] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0547] In this invention, the server includes means for saving past usage trend data, means for saving travel plan information entered by the user, means for generating reservation candidates based on the past usage trend data and the travel plan information, means for creating prompt sentences for generating reservation candidates using a generative AI model, means for converting the generated reservation candidates into JSON format and transmitting them, means for presenting reservation candidates to the user, means for confirming the reservation candidates selected by the user and coordinating reservations, and means for notifying the user of confirmed reservation information. This enables users to efficiently create optimal travel plans based on their past usage trends and preferences.

[0548] "Past usage trend data" refers to historical information about reservations made by a user, including travel destinations, travel dates, and information about accommodations and activities used.

[0549] "Travel planning information" is information that a user inputs when planning a new trip, and includes the travel destination, desired tourist spots, activities, budget, schedule, and the like.

[0550] A "reservation candidate" is a proposed plan generated by the server based on the user's past usage trend data and travel plan information, and includes suggestions for specific accommodations and activities.

[0551] "Generative AI model" refers to artificial intelligence technology that analyzes users' past usage trends and travel planning information to generate optimal reservation options.

[0552] A "prompt sentence" is an instruction sentence input to the generative AI model, and is generated based on the user's travel plan information.

[0553] "JSON format" is an abbreviation for JavaScript Object Notation, and is a format for expressing data in text format.

[0554] "Reservation linkage" is a procedure in which the server communicates with an external reservation site or application to confirm the reservation candidates selected by the user and make the reservation.

[0555] "Notification" is a function that allows the server to notify the user of confirmed reservation information, and includes methods such as email and in-app notifications.

[0556] This invention relates to an advance travel reservation assistance system that allows users to make travel plans efficiently and flexibly. This system is composed of multiple components, the main components of which include a server, a terminal, and a user.

[0557] Registering user profile information

[0558] Users first install the application and create an account, which involves entering basic profile information such as name, age, gender, and email address on the device.

[0559] The terminal transmits the input profile information to the server via an HTTP request.

[0560] The server stores the received profile information in a database (e.g., MySQL), which serves to systematically manage the profile information for each user.

[0561] Learn from past usage trends

[0562] The server periodically retrieves and analyzes the user's past booking history stored in the database, using machine learning algorithms (e.g., K-means clustering) to identify the user's preferences and travel patterns.

[0563] The server's analysis results clarify the user's preferred travel destinations and activity patterns. For example, it learns that a user who has visited many resort areas in the past tends to continue to prefer resort areas in the future.

[0564] Enter your travel plans

[0565] The user opens the travel plan input screen within the application and enters detailed information such as the travel destination, desired tourist spots, activities, budget, and schedule.

[0566] The device sends this input information to the server in JSON format, specifically using a REST API.

[0567] Generate reservation candidates

[0568] The server uses a generative AI model (e.g., GPT-4) to generate suitable reservation suggestions based on past usage trend data and newly entered travel plan information, using the following prompt:

[0569] Suggest suitable booking suggestions based on your next travel plans.

[0570] Travel destination: Hokkaido

[0571] Preferred tourist destinations: Furano, Kushiro Marsh

[0572] Activity: Snowboarding

[0573] Budget: Under 200,000 yen

[0574] Dates: February 15, 2024 to February 20, 2024

[0575] Past travel history: Kyoto, Okinawa (sightseeing, relaxation)

[0576] This prompt is then fed into an AI model to generate an optimal travel plan.

[0577] Presenting reservation candidates and confirming reservations

[0578] The server converts the generated reservation candidates into JSON format and sends them to the terminal.

[0579] The terminal analyzes the received reservation candidates and displays them in a visually easy-to-understand format for the user.

[0580] The user selects the desired reservation from the displayed reservation candidates and confirms the reservation.

[0581] The terminal transmits the user's selection to the server, and the server cooperates with an external reservation site or application to confirm the reservation.

[0582] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a final confirmation notice to the user.

[0583] As a concrete example, if User A uses this system to plan and book a trip to Hokkaido, the system generates an optimal plan based on the user's past travel history and current preferences, and the reservation is completed smoothly. This series of operations allows the user to plan an efficient trip without any hassle.

[0584] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0585] Step 1: Register your user profile information

[0586] The user installs the application and creates a new account.

[0587] The user enters basic information such as name, age, gender, and email address and submits it.

[0588] The terminal sends the entered basic information to the server via an HTTP request.

[0589] The server saves the received profile information in a database (e.g., MySQL). At this time, the server validates the input data to maintain data consistency. It responds with a success message to the terminal.

[0590] input:

[0591] Basic information (name, age, gender, email address)

[0592] output:

[0593] Save to database, success message

[0594] Step 2: Learn from past usage trends

[0595] The server periodically retrieves the stored past usage trend data from the database.

[0596] The server analyzes the acquired usage trend data and runs machine learning algorithms (e.g., K-means clustering) to learn user preferences and travel patterns.

[0597] The server uses the analysis results to classify the user's preferences and travel patterns and create basic information for a new travel plan.

[0598] input:

[0599] Past usage trend data

[0600] output:

[0601] User preference classification, travel pattern data

[0602] Step 3: Enter your travel plans

[0603] The user opens the travel plan input screen within the application and enters detailed information such as the travel destination, desired tourist spots, activities, budget, and schedule.

[0604] The terminal converts the input travel planning information into JSON format and sends it to the server via an HTTP request.

[0605] input:

[0606] Travel planning information (destination, sightseeing spots, activities, budget, schedule)

[0607] output:

[0608] JSON formatted travel plan information

[0609] Step 4: Generate reservation candidates

[0610] The server generates a prompt sentence based on past usage trend data and newly entered travel plan information.

[0611] The generated prompt sentence is input into a generative AI model (e.g., GPT-4) to generate appropriate reservation candidates. The example prompt sentence used is as follows:

[0612] Suggest suitable booking suggestions based on your next travel plans.

[0613] Travel destination: Hokkaido

[0614] Preferred tourist destinations: Furano, Kushiro Marsh

[0615] Activity: Snowboarding

[0616] Budget: Under 200,000 yen

[0617] Dates: February 15, 2024 to February 20, 2024

[0618] Past travel history: Kyoto, Okinawa (sightseeing, relaxation)

[0619] The server converts the reservation suggestions obtained from the generative AI model into JSON format.

[0620] input:

[0621] Prompt text, historical usage trend data, travel planning information

[0622] output:

[0623] JSON format data of reservation candidates

[0624] Step 5: Present reservation options

[0625] The server transmits the generated reservation candidates to the terminal.

[0626] The terminal analyzes the received JSON-formatted reservation candidates and visually displays them to the user. Specifically, the terminal uses GUI components to display the reservation candidates in a list or calendar format.

[0627] input:

[0628] JSON format data of reservation candidates

[0629] output:

[0630] Displaying reservation options

[0631] Step 6: Confirmation and notification of reservation

[0632] The user selects the desired reservation from the displayed reservation candidates and confirms the reservation.

[0633] The terminal transmits the selected reservation candidates to the server.

[0634] The server confirms the received reservation information by connecting with an external reservation site or application, and executes the reservation using an external API.

[0635] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a notification to the user.

[0636] input:

[0637] Selected reservation candidate

[0638] output:

[0639] Confirmation of reservation information, notification to users

[0640] Through these steps, users can efficiently plan their trip and make reservations smoothly.

[0641] (Application example 1)

[0642] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0643] Food delivery services have rapidly grown in popularity in recent years, but there is a lack of personalized menu suggestions for users, making it difficult for users to find an appropriate menu that meets their food preferences and restrictions. Furthermore, past order history cannot be effectively utilized, which can lead to a decrease in user satisfaction. The present invention aims to solve these problems by providing a food delivery assistant system that suggests optimal menus to users.

[0644] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0645] In this invention, the server includes means for saving past usage trend data, means for saving meal order information entered by the user, means for generating menu candidates based on the past usage trend data and the meal order information, means for presenting the menu candidates to the user, and means for confirming the menu candidates selected by the user and linking the orders, thereby enabling personalized menu suggestions that take into account the user's food preferences and past ordering history.

[0646] "Past usage trend data" is data that records the history of orders and selections made by a user in the past.

[0647] "Meal order information" refers to details of the meal preferences and orders currently input by the user.

[0648] "Menu candidates" is a list of menus that the server may generate and suggest based on past usage trend data and meal order information.

[0649] "Means for presenting menu candidates to the user" refers to a method for displaying and notifying the generated menu candidates on the user's device.

[0650] The "means for coordinating orders" refers to the process of sending the menu options selected by the user to the corresponding restaurant or provider and finalizing the order.

[0651] "Profile information" is data that indicates basic information such as the user's name, age, gender, and food preferences.

[0652] "Order result" is information indicating the status of the menu candidate selected by the user when it is actually ordered.

[0653] This invention relates to a food delivery assistant system that proposes personalized menus based on a user's food preferences and past ordering history. This system is composed of a server, a terminal, and a user.

[0654] Registering user profile information

[0655] First, the user installs the app and creates an account. The device sends basic information entered by the user, such as name, age, gender, food preferences, and allergy information, to the server, which then stores this information in a database.

[0656] Learn from past usage trends

[0657] The server analyzes the user's past order history stored in a database to learn the user's food preferences and patterns. For example, it determines that a user who frequently orders vegetarian dishes will likely continue to prefer vegetarian dishes. This analysis is performed using machine learning algorithms (for example, models using Scikit-Learn or TensorFlow).

[0658] Entering a meal order

[0659] The user opens the app's order screen and inputs their current mood and desired meal type (e.g., dessert, main dish, etc.). The device sends this information to the server. In this case, an example of a prompt when the user places a new order is "What would you like to eat today? Please choose a category (e.g., dessert, main dish, salad)."

[0660] Menu candidate generation and suggestions

[0661] The server selects an appropriate restaurant and menu based on past usage trend data and newly entered order information. The selected menu candidates are converted into JSON format and sent to the device. The device analyzes the sent information and displays recommended menu items to the user. At this stage, an example of a prompt for the machine learning model is sent to the generative AI model: "Consider past order history and predict the menu item that the user is likely to like next."

[0662] Order confirmation and notification

[0663] The user selects the desired item from the displayed menu options and confirms the order. The device sends the user's selection to the server. The server then connects the order information to the corresponding restaurant and confirms the order. Finally, if the order is successful, the server notifies the device of the result, and the device displays a final confirmation notice to the user.

[0664] Hardware and software used

[0665] Device: User's smartphone

[0666] Server: Cloud server (e.g., Amazon Web Services (AWS))

[0667] Database: MySQL

[0668] Analysis algorithm: Machine learning model (e.g., model using Scikit-Learn or TensorFlow)

[0669] API: RESTful API (transmits and receives data in JSON format)

[0670] In this way, it becomes possible to propose a personalized menu that takes into account the user's food preferences and past ordering history, thereby improving user satisfaction.

[0671] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0672] Step 1: Registering user profile information

[0673] Input: The user installs the app and enters basic information such as name, age, gender, food preferences, and allergy information.

[0674] Specific operation: The terminal processes the information entered by the user and sends it to the server.

[0675] Data processing or data calculation: The server organizes the received information and stores it in a database.

[0676] Output: The profile information is saved in the database.

[0677] Step 2: Learn from past usage trends

[0678] Input: The user's past order history stored in a database.

[0679] How it works: The server analyzes past order data using machine learning algorithms (e.g., Scikit-Learn or TensorFlow models).

[0680] Data processing or data calculation: Through analysis, the system learns the user's food preferences and patterns.

[0681] Output: The learning results are saved and serve as the basis for future data-based recommendations.

[0682] Step 3: Enter your meal order

[0683] Input: The user inputs their current mood and desired meal type (e.g., dessert, main dish, etc.) into the app.

[0684] Specific operation: The terminal sends the order information entered by the user to the server, and a prompt such as "What would you like to eat today? Please select a category (e.g., dessert, main dish, salad)" is presented to the user.

[0685] Data processing or data calculation: The order information is converted into a data format and sent to the server.

[0686] Output: The order information is saved on the server.

[0687] Step 4: Generate and suggest menu options

[0688] Input: Historical usage trend data and newly entered order information.

[0689] How it works: The server uses the generative AI model to select an appropriate restaurant and menu. At this time, it sends a prompt to the generative AI model saying, "Consider the user's past order history and predict the menu item that the user is likely to like next."

[0690] Data processing or data calculation: The menu candidates generated by the server are converted into JSON format and sent to the terminal.

[0691] Output: Menu suggestions are displayed on the terminal.

[0692] Step 5: Order confirmation and notification

[0693] Input: The user selects the desired menu option on the device.

[0694] Specific operation: The device sends the user's selection to the server, which then sends the order information to the corresponding restaurant and confirms the order.

[0695] Data processing or data calculation: The order information is sent to the restaurant's system and the order is confirmed.

[0696] Output: The server notifies the terminal of the order result and displays a final confirmation notice to the user.

[0697] This process flow allows the user to efficiently order food and receive suggested menu options that suit their food preferences.

[0698] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0699] This invention relates to an advance travel reservation assistant system that combines an emotion engine that recognizes the user's emotions. This system not only automatically generates and suggests optimal reservation options based on the user's past usage trend data and travel plan information, but also has the function of adjusting the reservation options based on the user's emotions. This allows users to create more personalized and optimized travel plans.

[0700] System configuration

[0701] The system consists of the following main components:

[0702] 1. Server

[0703] 2. Terminal

[0704] 3. Users

[0705] 4. Emotion Engine

[0706] Program processing and natural language explanation

[0707] Registering user profile information

[0708] A user installs the app and creates an account.

[0709] The device sends the basic information entered by the user (such as name, age, gender, and email address) to the server.

[0710] The server stores the received information in a database.

[0711] Learn from past usage trends

[0712] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[0713] Enter your travel plans

[0714] The user opens the travel planning screen and inputs the travel destination, desired tourist spots, activities, budget, dates, etc.

[0715] The terminal transmits this information to the server.

[0716] User Emotion Recognition

[0717] The emotion engine analyzes the user's emotions in real time based on their facial expressions, tone of voice, typing speed, etc.

[0718] The terminal transmits this emotion data to the server.

[0719] Emotion-based reservation candidate generation and adjustment

[0720] The server selects appropriate tourist spots and activities based on past usage trend data, travel plan information, and emotional data.

[0721] If the user's emotions are judged to be positive, the system will prioritize suggestions that reflect their preferences. On the other hand, if negative emotions are detected, the system will suggest relaxing plans and less stressful options.

[0722] The server converts the selected reservation candidates into JSON format and sends them to the terminal.

[0723] The terminal analyzes the transmitted information and displays a list of reservation candidates to the user.

[0724] Confirmation and notification of reservations

[0725] The user selects the desired tourist spot and activity from the displayed reservation options and confirms the reservation.

[0726] The terminal transmits the user's selection to the server.

[0727] The server then links the reservation information to the corresponding reservation site or app to confirm the reservation.

[0728] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a notification of reservation completion to the user.

[0729] Specific examples

[0730] As a specific example, consider the case where User A uses an app to plan their next trip. User A has previously visited Kyoto and Okinawa and tends to enjoy sightseeing and relaxation. For their next trip, User A wants to visit Hokkaido, and plans to visit Furano and the Kushiro Wetlands and enjoy snowboarding.

[0731] User A creates an account and enters their travel plans into the app.

[0732] The emotion engine detects positive emotions filled with anticipation from User A's facial expressions and tone of voice while he is typing.

[0733] The server generates a plan based on past usage trend data, new travel plans, and emotional data, including accommodation at Furano Ski Resort and a sightseeing tour of the Kushiro Wetlands.

[0734] User A selects accommodation and snowboarding lessons from the suggested reservation options and confirms the reservation.

[0735] The server then connects this information to the reservation site to complete the reservation.

[0736] Finally, the terminal notifies user A of the reservation result, allowing him to check the detailed reservation contents.

[0737] Such a configuration and operation allows users to create more personalized travel plans, and the introduction of an emotion engine improves the user experience.

[0738] The processing flow will be explained below.

[0739] Step 1:

[0740] The user installs the app, opens the account creation screen, enters basic information such as name, age, gender, and email address, and presses the "Register" button.

[0741] Step 2:

[0742] The terminal converts the information entered by the user into JSON format and sends it to the server via HTTPS.

[0743] Step 3:

[0744] The server analyzes the received JSON data, saves the user information in the database, generates a save completion status, and returns it to the device.

[0745] Step 4:

[0746] The user opens the travel planning screen, enters details such as the travel destination, desired tourist spots, activities, budget, and dates, and presses the "Save Travel Plan" button.

[0747] Step 5:

[0748] The device converts the travel planning information into JSON format and sends it to the server via HTTPS.

[0749] Step 6:

[0750] The server parses the received JSON data, saves the travel plan information to the database, generates a save completion status, and returns it to the device.

[0751] Step 7:

[0752] The emotion engine analyzes the user's emotions in real time based on their facial expressions, tone of voice, typing speed, etc. This emotion data is collected by the device.

[0753] Step 8:

[0754] The device converts the collected emotion data into JSON format and sends it to the server via HTTPS.

[0755] Step 9:

[0756] The server runs a recommendation algorithm based on past usage trends in the database, newly entered travel plan information, and emotional data.

[0757] Step 10:

[0758] The server collects data from similar cases of other users and selects the most suitable sightseeing spots and activities based on the user's emotions. For example, if positive emotions are detected, it will prioritize active plans, and if negative emotions are detected, it will suggest relaxing plans.

[0759] Step 11:

[0760] The server lists the selected reservation candidates, converts the list into JSON format, and sends it to the terminal.

[0761] Step 12:

[0762] The device analyzes the received JSON data and displays a list of reservation candidates to the user.

[0763] Step 13:

[0764] The user selects the desired tourist spot or activity from the displayed reservation options and presses the "Confirm reservation" button.

[0765] Step 14:

[0766] The terminal converts the reservation candidate information selected by the user into JSON format and sends it to the server via HTTPS.

[0767] Step 15:

[0768] The server analyzes the received JSON data and executes the reservation procedure using the API of the relevant reservation site or app. When the reservation process is complete, it records the result and generates a status.

[0769] Step 16:

[0770] The server converts the reservation completion status and detailed information into JSON format and sends it to the terminal.

[0771] Step 17:

[0772] The terminal analyzes the received reservation completion notification, displays the reservation completion notification to the user, and provides an interface for checking the details of the reservation.

[0773] Step 18:

[0774] Users can check reservation details and make changes or cancellations as necessary from the "My Page" or "Reservation History" within the app.

[0775] Example 2

[0776] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0777] Conventional travel reservation systems generate reservation candidates based on a user's past usage trends and travel plan information, but do not take into account the user's real-time emotional state, which has the problem of not being able to sufficiently increase user satisfaction. The present invention aims to further improve the user experience by proposing personalized reservation candidates based on the user's emotions.

[0778] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for saving past usage trend data, means for saving travel plan information entered by the user, means for generating reservation candidates based on the past usage trend data and the travel plan information, means for presenting reservation candidates to the user, means for confirming the reservation candidates selected by the user and coordinating reservations, means for analyzing user emotion data, and means for adjusting reservation candidates based on the user emotion data. This makes it possible to propose personalized travel plans based on the user's emotions.

[0779] "Past usage trend data" is data relating to trips taken by the user in the past, and is information indicating the user's preferences and patterns.

[0780] "Travel planning information" is data related to a trip that the user is planning, and includes detailed information such as the travel destination, desired tourist spots, activities, budget, and schedule.

[0781] "Reservation candidates" are travel plans and options suggested to the user, generated based on the user's past usage trend data and travel plan information.

[0782] "Emotion data" is data that indicates the user's real-time emotional state, and is information that is analyzed based on facial expressions, tone of voice, input speed, and the like.

[0783] "Reservation linkage" refers to the process of linking the reservation candidates selected by the user with an external reservation system to actually confirm the reservation.

[0784] MODE FOR CARRYING OUT THE INVENTION

[0785] This invention relates to an advance travel reservation assistant system that combines an emotion engine that recognizes the user's emotions. This system not only automatically generates and suggests optimal reservation options based on the user's past usage trend data and travel plan information, but also has the function of adjusting the reservation options based on the user's emotions. This allows users to create more personalized and optimized travel plans.

[0786] System configuration

[0787] The system consists of the following main components:

[0788] 1. Server

[0789] 2. Terminal

[0790] 3. Users

[0791] 4. Emotion Engine

[0792] Registering user profile information

[0793] A user installs the app and creates an account by entering basic information such as a username and email address. The device acquires the entered basic information and sends it to the server. The server stores the received information in a database. Specifically, data is sent via an HTTP POST request, and a common RDBMS (e.g., PostgreSQL, MySQL) is used as the database.

[0794] Learn from past usage trends

[0795] The server periodically analyzes the past booking history data in the database. Using Python and Scikit-learn, it uses clustering algorithms (e.g., K-means) to learn user preferences and travel patterns. The server adds the learning results to the user profile and reflects them in future recommendations.

[0796] Enter your travel plans

[0797] The user opens the travel planning screen and inputs information such as the areas and tourist spots they want to visit, activities, budget, schedule, etc. The device then sends this input information to the server.

[0798] User Emotion Recognition

[0799] The emotion engine uses the camera and microphone to analyze the user's facial expressions and tone of voice in real time, for example, using OpenCV and the Google Cloud Speech-to-Text API. The device then sends the emotion analysis results to the server.

[0800] Emotion-based reservation candidate generation and adjustment

[0801] The server generates suitable sightseeing spots and activity candidates based on the received travel plan information, past usage trend data, and emotional data. If the user's emotion is positive, it prioritizes suggestions that will satisfy excitement and expectations, and if it is negative, it selects a relaxing plan. The server sends the generated reservation candidates to the terminal in JSON format.

[0802] Confirmation and notification of reservations

[0803] The user selects the plan they want to reserve from the displayed list of reservation candidates and presses the Confirm button. The device sends the user's selection to the server. The server calls the API of the relevant reservation site to confirm the reservation. The server sends a reservation success message to the device, and the device displays a confirmation notification to the user.

[0804] Specific examples

[0805] As a specific example, consider the case where User A uses an app to plan their next trip. User A has previously visited Kyoto and Okinawa and tends to enjoy sightseeing and relaxation. For their next trip, User A wants to visit Hokkaido, and plans to visit Furano and the Kushiro Wetlands and enjoy snowboarding.

[0806] User A creates an account and enters their travel plans into the app.

[0807] The emotion engine detects positive emotions filled with anticipation from User A's facial expressions and tone of voice while he is typing.

[0808] The server generates a plan based on past usage trend data, new travel plans, and emotional data, including accommodation at Furano Ski Resort and a sightseeing tour of the Kushiro Wetlands.

[0809] User A selects accommodation and snowboarding lessons from the suggested reservation options and confirms the reservation.

[0810] The server then connects this information to the reservation site to complete the reservation.

[0811] Finally, the terminal notifies user A of the reservation result, allowing him to check the detailed reservation contents.

[0812] Examples of prompts

[0813] "Develop a system that suggests next travel plans based on the user's booking history and adjusts the optimal plan by analyzing the user's emotions in real time. Incorporate a mechanism that uses an emotion recognition engine to analyze the user's facial expressions, tone of voice, typing speed, etc. and reflects this in the travel plans."

[0814] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0815] Step 1:

[0816] A user installs the app and creates an account by entering basic information such as a username and email address. When the user presses the "Register" button, the basic information is generated as input data. The device obtains this input data and sends it to the server via an HTTP POST request. The server saves the received basic information in a database. Specifically, it uses PostgreSQL or MySQL to insert the user information into a table. It returns a success message as a save result.

[0817] Step 2:

[0818] The server periodically analyzes past reservation history data in the database. It extracts past usage trend data using the user ID as a key. The server uses Python and Scikit-learn to apply a clustering algorithm (e.g., K-means). This allows it to learn the user's preferences and travel patterns, and generates the results as clustering data. The generated clustering data is added to the user profile.

[0819] Step 3:

[0820] The user opens the travel planning screen and enters information such as the areas and tourist spots they want to visit, activities, budget, and dates. This information is generated as input data. The device compiles the entered travel planning information and sends it to the server via an HTTP request. The server saves the received travel planning information in a database and returns a success message as a result of saving.

[0821] Step 4:

[0822] The emotion engine uses the user's camera and microphone to analyze facial expressions and tone of voice in real time. Captured images and audio data are generated as input data. Specifically, facial expressions are analyzed using OpenCV, and audio is analyzed using the Google Cloud Speech-to-Text API. Emotion data is generated as the analysis result. The device acquires the emotion data and sends it to the server via an HTTP request.

[0823] Step 5:

[0824] The server generates suitable sightseeing spots and activity candidates based on the received travel plan information, past usage trend data, and emotion data. Input data includes travel plan information, past usage trend data, and emotion data. The server runs a Python script to generate reservation candidates based on this data. Based on the emotion analysis results, active plans are prioritized if the emotion is positive, and relaxing plans are prioritized if the emotion is negative. The generated reservation candidates are converted into JSON format and sent to the device.

[0825] Step 6:

[0826] The user selects a plan from the displayed list of reservation candidates and presses the confirm button. The device sends the selected reservation details to the server via an HTTP request. The server receives the selection and calls the API of the relevant reservation site to confirm the reservation. The reservation information includes accommodation, activities, and dates. The server notifies the device that the reservation is complete, and the device displays a confirmation notification to the user.

[0827] (Application example 2)

[0828] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0829] Many conventional travel reservation systems generate reservation candidates based on a user's past usage data and travel plan information, but because they cannot adjust reservation candidates taking into account the user's emotional state, it is difficult to propose travel plans optimized for each individual user.In addition, there are limited ways to notify users of reservation results, which reduces user convenience.

[0830] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0831] In this invention, the server includes means for saving past usage trend data, means for saving travel plan information entered by the user, means for generating reservation candidates based on the past usage trend data and travel plan information, means for analyzing the user's facial expressions and tone of voice to recognize emotions, means for adjusting and generating reservation candidates based on the emotion data, means for presenting reservation candidates to the user, means for confirming the reservation candidates selected by the user and coordinating reservations, and means for notifying the user of the reservation results. This makes it possible to propose an optimal travel plan that takes the user's emotional state into consideration. It also enables faster and more convenient notification of reservation results.

[0832] "Past usage trend data" refers to data including travel plans, reservation history, and preference information that the user has used in the past.

[0833] "Travel planning information" refers to information such as the destination, itinerary, budget, and desired activities that the user has entered in advance regarding their next trip.

[0834] "Reservation candidates" are candidates for travel plans, accommodations, activities, etc. that are generated based on the user's usage trend data and travel plan information and that the user can select.

[0835] "Emotion recognition" is the process of analyzing a user's emotions from their facial expressions, tone of voice, etc., and determining whether their emotional state is positive or negative.

[0836] "Emotion data" is the result of the user's emotion recognition, and is data that includes information such as the type and intensity of the emotion.

[0837] "Adjustment" is the process of modifying and optimizing the generated reservation candidates based on the acquired data to suit the user's emotional state.

[0838] "Smart glasses" are wearable devices that use a camera to analyze a user's facial expressions.

[0839] A "head-mounted display" is a wearable display device that inputs information from the user's vision and voice and analyzes emotions.

[0840] This invention relates to a system that utilizes a "smart travel assistant" system to recognize a user's emotions and generate and present an optimized travel plan. The system mainly consists of a server, a terminal (smartphone), and an emotion engine (including smart glasses and head-mounted displays) for emotion recognition.

[0841] When a user inputs their travel plans using the application, the device sends this information to the server, which then uses past usage trend data and the newly input travel plan information to generate initial reservation candidates.

[0842] The emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions in real time. This emotion data is sent to the server via the device. The server adjusts and optimizes reservation options based on the acquired emotion data. Specifically, if a positive emotion is recognized, it will prioritize generating travel plans that match the user's preferences, and if a negative emotion is recognized, it will suggest relaxing activities and low-stress plans.

[0843] The final reservation candidates are sent to the terminal and presented to the user. When the user selects and confirms a reservation, the information is sent back to the server, completing the reservation link. Finally, the reservation results are notified to the user via the terminal.

[0844] The operation of the system can be specifically explained as follows:

[0845] 1. Registering user profile information

[0846] The user installs the application and creates an account. The device sends the basic information entered by the user (such as name, age, gender, and email address) to the server and stores it in a database.

[0847] 2. Learning from past usage trends

[0848] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[0849] 3. Enter your travel plans

[0850] The user opens the travel planning screen and inputs the travel destination, desired sightseeing spots, activities, budget, schedule, etc. The device then sends this information to the server.

[0851] 4. User Emotion Recognition

[0852] The emotion engine (including smart glasses and head-mounted displays) analyzes the user's emotions in real time based on their facial expressions, tone of voice, etc. The device then transmits this emotion data to a server.

[0853] 5. Emotion-based reservation candidate generation and adjustment

[0854] The server selects appropriate sightseeing spots and activities based on past usage trend data, travel plan information, and emotional data. If the user's emotions are judged to be positive, it will prioritize suggestions that reflect their preferences, but if negative emotions are detected, it will suggest relaxing plans and activities that reduce stress.

[0855] 6. Booking confirmation and notification

[0856] The user selects the desired sightseeing spot and activity from the displayed reservation options and confirms the reservation. The device sends the user's selection to the server, which then links the reservation information to the corresponding reservation site or app to confirm the reservation. If the reservation is successful, the server notifies the device of the result, and the device displays a notification to the user that the reservation is complete.

[0857] This allows users to create more personalized travel plans, and the introduction of an emotion engine improves the user experience.

[0858] Example: Prompt sentences to input to a generative AI model

[0859] plaintext

[0860] User emotion recognition prompts

[0861] Analyze the following text data and determine the user's sentiment.

[0862] Facial image: {image_data}

[0863] Voice data: {voice_data}

[0864] Prompt sentences for generating reservation candidates based on emotions

[0865] Generate the best travel plan based on the following user and sentiment data:

[0866] User ID: {user_id}

[0867] emotion: {emotion}

[0868] By implementing the above-described embodiment, more advanced travel planning can be performed taking into account the user's emotional state.

[0869] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0870] Step 1:

[0871] A user installs the application and creates an account.

[0872] Input: Basic information entered by the user (name, age, gender, email address, etc.)

[0873] Operation: The device collects basic information and sends it to the server.

[0874] Output: User basic information is saved in a server-side database.

[0875] Step 2:

[0876] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[0877] Input: Past usage trend data stored on the server

[0878] How it works: The server uses machine learning algorithms to analyze the data and learn user preference patterns.

[0879] Output: The user's preference patterns are stored in a database.

[0880] Step 3:

[0881] The user opens the travel planning screen and inputs the travel destination, desired tourist spots, activities, budget, dates, etc.

[0882] Input: Information about your travel destination, desired sightseeing spots, activities, budget, dates, etc.

[0883] Operation: The device sends this information to the server.

[0884] Output: The travel plan information is saved on the server.

[0885] Step 4:

[0886] Emotion engines (including smart glasses and head-mounted displays) analyze users' emotions in real time based on their facial expressions, tone of voice, etc.

[0887] Input: User's facial expressions and tone of voice

[0888] How it works: The emotion engine uses facial expression analysis algorithms and voice analysis algorithms to analyze emotions and send them to the device.

[0889] Output: Emotion data is sent to the server via the device.

[0890] Step 5:

[0891] The server selects appropriate tourist spots and activities based on past usage trend data, travel plan information, and emotional data.

[0892] Input: Past usage trend data, travel plan information, emotional data

[0893] How it works: The server aggregates the data and runs algorithms to select suitable tourist destinations and activities.

[0894] Output: The generated reservation candidates are sent to the terminal in JSON format.

[0895] Step 6:

[0896] The user selects the desired tourist spot and activity from the generated reservation candidates and confirms the reservation.

[0897] Input: User selected reservation candidate

[0898] How it works: The device sends the selection to the server, and the server links the reservation information to the reservation site or app.

[0899] Output: The reservation confirmation result is sent from the server to the terminal.

[0900] Step 7:

[0901] If the reservation is successful, the result is notified to the terminal and displayed to the user.

[0902] Input: Reservation confirmation result

[0903] Operation: The server notifies the terminal of the result, and the terminal displays the notification to the user.

[0904] Output: The user is notified that the reservation is complete.

[0905] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0906] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0907] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0908] [Third embodiment]

[0909] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0910] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0911] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0912] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0913] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0914] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0915] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0916] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0917] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0918] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0919] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0920] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0921] This invention relates to an advance travel reservation assistance system that allows users to plan their trips efficiently and flexibly. This system has the function of automatically generating and suggesting optimal reservation options by storing user profile information and past reservation history and learning the user's preferences and travel patterns.

[0922] System configuration

[0923] The system consists of the following main components:

[0924] 1. Server

[0925] 2. Terminal

[0926] 3. Users

[0927] Program processing and natural language explanation

[0928] Registering user profile information

[0929] A user installs the app and creates an account.

[0930] The device sends the basic information entered by the user (such as name, age, gender, and email address) to the server.

[0931] The server stores the received information in a database.

[0932] Learn from past usage trends

[0933] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[0934] For example, it is determined that a user who has visited many resort areas in the past will tend to prefer resort areas in the future.

[0935] Enter your travel plans

[0936] The user opens the travel planning screen and inputs the travel destination, desired tourist spots, activities, budget, dates, etc.

[0937] The terminal transmits this information to the server.

[0938] Generation and suggestion of reservation candidates

[0939] The server selects appropriate tourist destinations and activities based on past usage trend data and newly entered travel plan information.

[0940] The server converts the selected reservation candidates into JSON format and sends them to the terminal.

[0941] The terminal analyzes the transmitted information and displays reservation candidates to the user.

[0942] Confirmation and notification of reservations

[0943] The user selects the desired reservation from the displayed reservation candidates and confirms the reservation.

[0944] The terminal transmits the user's selection to the server.

[0945] The server then links the reservation information to the corresponding reservation site or app to confirm the reservation.

[0946] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a final confirmation notice to the user.

[0947] Specific examples

[0948] As a specific example, consider the case where User A uses an app to plan their next trip. User A has previously visited Kyoto and Okinawa and tends to enjoy sightseeing and relaxation. For their next trip, User A wants to visit Hokkaido, and plans to visit Furano and the Kushiro Wetlands and enjoy snowboarding.

[0949] User A creates an account and enters their travel plans into the app.

[0950] The terminal sends this information to the server, which then generates appropriate reservation candidates based on past usage trend data and new travel plan information.

[0951] For example, a plan is generated that includes accommodation at Furano Ski Resort and a sightseeing tour of the Kushiro Wetlands.

[0952] User A selects accommodation and snowboarding lessons from the displayed reservation options and confirms the reservation.

[0953] The server then connects this information to the reservation site to complete the reservation.

[0954] Finally, the terminal notifies user A of the reservation result, allowing him to check the detailed reservation contents.

[0955] Such a configuration and operation allows users to more efficiently and conveniently plan their trips and smoothly proceed with reservations.

[0956] The processing flow will be explained below.

[0957] Step 1:

[0958] The user installs the app, opens the account creation screen, enters basic information such as name, age, gender, and email address, and presses the "Register" button.

[0959] Step 2:

[0960] The terminal converts the information entered by the user into JSON format and sends it to the server via HTTPS.

[0961] Step 3:

[0962] The server analyzes the received JSON data, saves the user information in the database, generates a save completion status, and returns it to the device.

[0963] Step 4:

[0964] The user opens the travel planning screen, enters details such as the travel destination, desired tourist spots, activities, budget, and dates, and presses the "Save Travel Plan" button.

[0965] Step 5:

[0966] The device converts the travel planning information into JSON format and sends it to the server via HTTPS.

[0967] Step 6:

[0968] The server parses the received JSON data, saves the travel plan information to the database, generates a save completion status, and returns it to the device.

[0969] Step 7:

[0970] The server runs a recommendation algorithm based on past usage trends in the database and newly entered travel plan information, collecting data from similar cases of other users to select matching tourist spots and activities.

[0971] Step 8:

[0972] The server lists the selected reservation candidates, converts the list into JSON format, and sends it to the terminal.

[0973] Step 9:

[0974] The device analyzes the received JSON data and displays a list of reservation candidates to the user.

[0975] Step 10:

[0976] The user selects the desired tourist spot or activity from the displayed reservation options and presses the "Confirm reservation" button.

[0977] Step 11:

[0978] The terminal converts the reservation candidate information selected by the user into JSON format and sends it to the server via HTTPS.

[0979] Step 12:

[0980] The server analyzes the received JSON data and executes the reservation procedure using the API of the relevant reservation site or app. When the reservation process is complete, it records the result and generates a status.

[0981] Step 13:

[0982] The server converts the reservation completion status and detailed information into JSON format and sends it to the terminal.

[0983] Step 14:

[0984] The terminal analyzes the received reservation completion notification, displays the reservation completion notification to the user, and provides an interface for checking the details of the reservation.

[0985] Step 15:

[0986] Users can check reservation details and make changes or cancellations as necessary from the "My Page" or "Reservation History" within the app.

[0987] Example 1

[0988] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0989] Conventional travel reservation systems often make blanket recommendations without fully considering a user's past booking history or preferences. This results in inefficiency, as users have to spend a lot of time searching for a travel plan that suits their needs. Furthermore, the process of retrieving information from different booking sites and confirming and notifying reservations is cumbersome. These issues make it difficult for users to easily create optimal travel plans based on their preferences and past usage history.

[0990] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0991] In this invention, the server includes means for saving past usage trend data, means for saving travel plan information entered by the user, means for generating reservation candidates based on the past usage trend data and the travel plan information, means for creating prompt sentences for generating reservation candidates using a generative AI model, means for converting the generated reservation candidates into JSON format and transmitting them, means for presenting reservation candidates to the user, means for confirming the reservation candidates selected by the user and coordinating reservations, and means for notifying the user of confirmed reservation information. This enables users to efficiently create optimal travel plans based on their past usage trends and preferences.

[0992] "Past usage trend data" refers to historical information about reservations made by a user, including travel destinations, travel dates, and information about accommodations and activities used.

[0993] "Travel planning information" is information that a user inputs when planning a new trip, and includes the travel destination, desired tourist spots, activities, budget, schedule, and the like.

[0994] A "reservation candidate" is a proposed plan generated by the server based on the user's past usage trend data and travel plan information, and includes suggestions for specific accommodations and activities.

[0995] "Generative AI model" refers to artificial intelligence technology that analyzes users' past usage trends and travel planning information to generate optimal reservation options.

[0996] A "prompt sentence" is an instruction sentence input to the generative AI model, and is generated based on the user's travel plan information.

[0997] "JSON format" is an abbreviation for JavaScript Object Notation, and is a format for expressing data in text format.

[0998] "Reservation linkage" is a procedure in which the server communicates with an external reservation site or application to confirm the reservation candidates selected by the user and make the reservation.

[0999] "Notification" is a function that allows the server to notify the user of confirmed reservation information, and includes methods such as email and in-app notifications.

[1000] This invention relates to an advance travel reservation assistance system that allows users to make travel plans efficiently and flexibly. This system is composed of multiple components, the main components of which include a server, a terminal, and a user.

[1001] Registering user profile information

[1002] Users first install the application and create an account, which involves entering basic profile information such as name, age, gender, and email address on the device.

[1003] The terminal transmits the input profile information to the server via an HTTP request.

[1004] The server stores the received profile information in a database (e.g., MySQL), which serves to systematically manage the profile information for each user.

[1005] Learn from past usage trends

[1006] The server periodically retrieves and analyzes the user's past booking history stored in the database, using machine learning algorithms (e.g., K-means clustering) to identify the user's preferences and travel patterns.

[1007] The server's analysis results clarify the user's preferred travel destinations and activity patterns. For example, it learns that a user who has visited many resort areas in the past tends to continue to prefer resort areas in the future.

[1008] Enter your travel plans

[1009] The user opens the travel plan input screen within the application and enters detailed information such as the travel destination, desired tourist spots, activities, budget, and schedule.

[1010] The device sends this input information to the server in JSON format, specifically using a REST API.

[1011] Generate reservation candidates

[1012] The server uses a generative AI model (e.g., GPT-4) to generate suitable reservation suggestions based on past usage trend data and newly entered travel plan information, using the following prompt:

[1013] Suggest suitable booking suggestions based on your next travel plans.

[1014] Travel destination: Hokkaido

[1015] Preferred tourist destinations: Furano, Kushiro Marsh

[1016] Activity: Snowboarding

[1017] Budget: Under 200,000 yen

[1018] Dates: February 15, 2024 to February 20, 2024

[1019] Past travel history: Kyoto, Okinawa (sightseeing, relaxation)

[1020] This prompt is then fed into an AI model to generate an optimal travel plan.

[1021] Presenting reservation candidates and confirming reservations

[1022] The server converts the generated reservation candidates into JSON format and sends them to the terminal.

[1023] The terminal analyzes the received reservation candidates and displays them in a visually easy-to-understand format for the user.

[1024] The user selects the desired reservation from the displayed reservation candidates and confirms the reservation.

[1025] The terminal transmits the user's selection to the server, and the server cooperates with an external reservation site or application to confirm the reservation.

[1026] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a final confirmation notice to the user.

[1027] As a concrete example, if User A uses this system to plan and book a trip to Hokkaido, the system generates an optimal plan based on the user's past travel history and current preferences, and the reservation is completed smoothly. This series of operations allows the user to plan an efficient trip without any hassle.

[1028] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1029] Step 1: Register your user profile information

[1030] The user installs the application and creates a new account.

[1031] The user enters basic information such as name, age, gender, and email address and submits it.

[1032] The terminal sends the entered basic information to the server via an HTTP request.

[1033] The server saves the received profile information in a database (e.g., MySQL). At this time, the server validates the input data to maintain data consistency. It responds with a success message to the terminal.

[1034] input:

[1035] Basic information (name, age, gender, email address)

[1036] output:

[1037] Save to database, success message

[1038] Step 2: Learn from past usage trends

[1039] The server periodically retrieves the stored past usage trend data from the database.

[1040] The server analyzes the acquired usage trend data and runs machine learning algorithms (e.g., K-means clustering) to learn user preferences and travel patterns.

[1041] The server uses the analysis results to classify the user's preferences and travel patterns and create basic information for a new travel plan.

[1042] input:

[1043] Past usage trend data

[1044] output:

[1045] User preference classification, travel pattern data

[1046] Step 3: Enter your travel plans

[1047] The user opens the travel plan input screen within the application and enters detailed information such as the travel destination, desired tourist spots, activities, budget, and schedule.

[1048] The terminal converts the input travel planning information into JSON format and sends it to the server via an HTTP request.

[1049] input:

[1050] Travel planning information (destination, sightseeing spots, activities, budget, schedule)

[1051] output:

[1052] JSON formatted travel plan information

[1053] Step 4: Generate reservation candidates

[1054] The server generates a prompt sentence based on past usage trend data and newly entered travel plan information.

[1055] The generated prompt sentence is input into a generative AI model (e.g., GPT-4) to generate appropriate reservation candidates. The example prompt sentence used is as follows:

[1056] Suggest suitable booking suggestions based on your next travel plans.

[1057] Travel destination: Hokkaido

[1058] Preferred tourist destinations: Furano, Kushiro Marsh

[1059] Activity: Snowboarding

[1060] Budget: Under 200,000 yen

[1061] Dates: February 15, 2024 to February 20, 2024

[1062] Past travel history: Kyoto, Okinawa (sightseeing, relaxation)

[1063] The server converts the reservation suggestions obtained from the generative AI model into JSON format.

[1064] input:

[1065] Prompt text, historical usage trend data, travel planning information

[1066] output:

[1067] JSON format data of reservation candidates

[1068] Step 5: Present reservation options

[1069] The server transmits the generated reservation candidates to the terminal.

[1070] The terminal analyzes the received JSON-formatted reservation candidates and visually displays them to the user. Specifically, the terminal uses GUI components to display the reservation candidates in a list or calendar format.

[1071] input:

[1072] JSON format data of reservation candidates

[1073] output:

[1074] Displaying reservation options

[1075] Step 6: Confirmation and notification of reservation

[1076] The user selects the desired reservation from the displayed reservation candidates and confirms the reservation.

[1077] The terminal transmits the selected reservation candidates to the server.

[1078] The server confirms the received reservation information by connecting with an external reservation site or application, and executes the reservation using an external API.

[1079] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a notification to the user.

[1080] input:

[1081] Selected reservation candidate

[1082] output:

[1083] Confirmation of reservation information, notification to users

[1084] Through these steps, users can efficiently plan their trip and make reservations smoothly.

[1085] (Application example 1)

[1086] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1087] Food delivery services have rapidly grown in popularity in recent years, but there is a lack of personalized menu suggestions for users, making it difficult for users to find an appropriate menu that meets their food preferences and restrictions. Furthermore, past order history cannot be effectively utilized, which can lead to a decrease in user satisfaction. The present invention aims to solve these problems by providing a food delivery assistant system that suggests optimal menus to users.

[1088] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1089] In this invention, the server includes means for saving past usage trend data, means for saving meal order information entered by the user, means for generating menu candidates based on the past usage trend data and the meal order information, means for presenting the menu candidates to the user, and means for confirming the menu candidates selected by the user and linking the orders, thereby enabling personalized menu suggestions that take into account the user's food preferences and past ordering history.

[1090] "Past usage trend data" is data that records the history of orders and selections made by a user in the past.

[1091] "Meal order information" refers to details of the meal preferences and orders currently input by the user.

[1092] "Menu candidates" is a list of menus that the server may generate and suggest based on past usage trend data and meal order information.

[1093] "Means for presenting menu candidates to the user" refers to a method for displaying and notifying the generated menu candidates on the user's device.

[1094] The "means for coordinating orders" refers to the process of sending the menu options selected by the user to the corresponding restaurant or provider and finalizing the order.

[1095] "Profile information" is data that indicates basic information such as the user's name, age, gender, and food preferences.

[1096] "Order result" is information indicating the status of the menu candidate selected by the user when it is actually ordered.

[1097] This invention relates to a food delivery assistant system that proposes personalized menus based on a user's food preferences and past ordering history. This system is composed of a server, a terminal, and a user.

[1098] Registering user profile information

[1099] First, the user installs the app and creates an account. The device sends basic information entered by the user, such as name, age, gender, food preferences, and allergy information, to the server, which then stores this information in a database.

[1100] Learn from past usage trends

[1101] The server analyzes the user's past order history stored in a database to learn the user's food preferences and patterns. For example, it determines that a user who frequently orders vegetarian dishes will likely continue to prefer vegetarian dishes. This analysis is performed using machine learning algorithms (for example, models using Scikit-Learn or TensorFlow).

[1102] Entering a meal order

[1103] The user opens the app's order screen and inputs their current mood and desired meal type (e.g., dessert, main dish, etc.). The device sends this information to the server. In this case, an example of a prompt when the user places a new order is "What would you like to eat today? Please choose a category (e.g., dessert, main dish, salad)."

[1104] Menu candidate generation and suggestions

[1105] The server selects an appropriate restaurant and menu based on past usage trend data and newly entered order information. The selected menu candidates are converted into JSON format and sent to the device. The device analyzes the sent information and displays recommended menu items to the user. At this stage, an example of a prompt for the machine learning model is sent to the generative AI model: "Consider past order history and predict the menu item that the user is likely to like next."

[1106] Order confirmation and notification

[1107] The user selects the desired item from the displayed menu options and confirms the order. The device sends the user's selection to the server. The server then connects the order information to the corresponding restaurant and confirms the order. Finally, if the order is successful, the server notifies the device of the result, and the device displays a final confirmation notice to the user.

[1108] Hardware and software used

[1109] Device: User's smartphone

[1110] Server: Cloud server (e.g., Amazon Web Services (AWS))

[1111] Database: MySQL

[1112] Analysis algorithm: Machine learning model (e.g., model using Scikit-Learn or TensorFlow)

[1113] API: RESTful API (transmits and receives data in JSON format)

[1114] In this way, it becomes possible to propose a personalized menu that takes into account the user's food preferences and past ordering history, thereby improving user satisfaction.

[1115] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1116] Step 1: Registering user profile information

[1117] Input: The user installs the app and enters basic information such as name, age, gender, food preferences, and allergy information.

[1118] Specific operation: The terminal processes the information entered by the user and sends it to the server.

[1119] Data processing or data calculation: The server organizes the received information and stores it in a database.

[1120] Output: The profile information is saved in the database.

[1121] Step 2: Learn from past usage trends

[1122] Input: The user's past order history stored in a database.

[1123] How it works: The server analyzes past order data using machine learning algorithms (e.g., Scikit-Learn or TensorFlow models).

[1124] Data processing or data calculation: Through analysis, the system learns the user's food preferences and patterns.

[1125] Output: The learning results are saved and serve as the basis for future data-based recommendations.

[1126] Step 3: Enter your meal order

[1127] Input: The user inputs their current mood and desired meal type (e.g., dessert, main dish, etc.) into the app.

[1128] Specific operation: The terminal sends the order information entered by the user to the server, and a prompt such as "What would you like to eat today? Please select a category (e.g., dessert, main dish, salad)" is presented to the user.

[1129] Data processing or data calculation: The order information is converted into a data format and sent to the server.

[1130] Output: The order information is saved on the server.

[1131] Step 4: Generate and suggest menu options

[1132] Input: Historical usage trend data and newly entered order information.

[1133] How it works: The server uses the generative AI model to select an appropriate restaurant and menu. At this time, it sends a prompt to the generative AI model saying, "Consider the user's past order history and predict the menu item that the user is likely to like next."

[1134] Data processing or data calculation: The menu candidates generated by the server are converted into JSON format and sent to the terminal.

[1135] Output: Menu suggestions are displayed on the terminal.

[1136] Step 5: Order confirmation and notification

[1137] Input: The user selects the desired menu option on the device.

[1138] Specific operation: The device sends the user's selection to the server, which then sends the order information to the corresponding restaurant and confirms the order.

[1139] Data processing or data calculation: The order information is sent to the restaurant's system and the order is confirmed.

[1140] Output: The server notifies the terminal of the order result and displays a final confirmation notice to the user.

[1141] This process flow allows the user to efficiently order food and receive suggested menu options that suit their food preferences.

[1142] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1143] This invention relates to an advance travel reservation assistant system that combines an emotion engine that recognizes the user's emotions. This system not only automatically generates and suggests optimal reservation options based on the user's past usage trend data and travel plan information, but also has the function of adjusting the reservation options based on the user's emotions. This allows users to create more personalized and optimized travel plans.

[1144] System configuration

[1145] The system consists of the following main components:

[1146] 1. Server

[1147] 2. Terminal

[1148] 3. Users

[1149] 4. Emotion Engine

[1150] Program processing and natural language explanation

[1151] Registering user profile information

[1152] A user installs the app and creates an account.

[1153] The device sends the basic information entered by the user (such as name, age, gender, and email address) to the server.

[1154] The server stores the received information in a database.

[1155] Learn from past usage trends

[1156] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[1157] Enter your travel plans

[1158] The user opens the travel planning screen and inputs the travel destination, desired tourist spots, activities, budget, dates, etc.

[1159] The terminal transmits this information to the server.

[1160] User Emotion Recognition

[1161] The emotion engine analyzes the user's emotions in real time based on their facial expressions, tone of voice, typing speed, etc.

[1162] The terminal transmits this emotion data to the server.

[1163] Emotion-based reservation candidate generation and adjustment

[1164] The server selects appropriate tourist spots and activities based on past usage trend data, travel plan information, and emotional data.

[1165] If the user's emotions are judged to be positive, the system will prioritize suggestions that reflect their preferences. On the other hand, if negative emotions are detected, the system will suggest relaxing plans and less stressful options.

[1166] The server converts the selected reservation candidates into JSON format and sends them to the terminal.

[1167] The terminal analyzes the transmitted information and displays a list of reservation candidates to the user.

[1168] Confirmation and notification of reservations

[1169] The user selects the desired tourist spot and activity from the displayed reservation options and confirms the reservation.

[1170] The terminal transmits the user's selection to the server.

[1171] The server then links the reservation information to the corresponding reservation site or app to confirm the reservation.

[1172] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a notification of reservation completion to the user.

[1173] Specific examples

[1174] As a specific example, consider the case where User A uses an app to plan their next trip. User A has previously visited Kyoto and Okinawa and tends to enjoy sightseeing and relaxation. For their next trip, User A wants to visit Hokkaido, and plans to visit Furano and the Kushiro Wetlands and enjoy snowboarding.

[1175] User A creates an account and enters their travel plans into the app.

[1176] The emotion engine detects positive emotions filled with anticipation from User A's facial expressions and tone of voice while he is typing.

[1177] The server generates a plan based on past usage trend data, new travel plans, and emotional data, including accommodation at Furano Ski Resort and a sightseeing tour of the Kushiro Wetlands.

[1178] User A selects accommodation and snowboarding lessons from the suggested reservation options and confirms the reservation.

[1179] The server then connects this information to the reservation site to complete the reservation.

[1180] Finally, the terminal notifies user A of the reservation result, allowing him to check the detailed reservation contents.

[1181] Such a configuration and operation allows users to create more personalized travel plans, and the introduction of an emotion engine improves the user experience.

[1182] The processing flow will be explained below.

[1183] Step 1:

[1184] The user installs the app, opens the account creation screen, enters basic information such as name, age, gender, and email address, and presses the "Register" button.

[1185] Step 2:

[1186] The terminal converts the information entered by the user into JSON format and sends it to the server via HTTPS.

[1187] Step 3:

[1188] The server analyzes the received JSON data, saves the user information in the database, generates a save completion status, and returns it to the device.

[1189] Step 4:

[1190] The user opens the travel planning screen, enters details such as the travel destination, desired tourist spots, activities, budget, and dates, and presses the "Save Travel Plan" button.

[1191] Step 5:

[1192] The device converts the travel planning information into JSON format and sends it to the server via HTTPS.

[1193] Step 6:

[1194] The server parses the received JSON data, saves the travel plan information to the database, generates a save completion status, and returns it to the device.

[1195] Step 7:

[1196] The emotion engine analyzes the user's emotions in real time based on their facial expressions, tone of voice, typing speed, etc. This emotion data is collected by the device.

[1197] Step 8:

[1198] The device converts the collected emotion data into JSON format and sends it to the server via HTTPS.

[1199] Step 9:

[1200] The server runs a recommendation algorithm based on past usage trends in the database, newly entered travel plan information, and emotional data.

[1201] Step 10:

[1202] The server collects data from similar cases of other users and selects the most suitable sightseeing spots and activities based on the user's emotions. For example, if positive emotions are detected, it will prioritize active plans, and if negative emotions are detected, it will suggest relaxing plans.

[1203] Step 11:

[1204] The server lists the selected reservation candidates, converts the list into JSON format, and sends it to the terminal.

[1205] Step 12:

[1206] The device analyzes the received JSON data and displays a list of reservation candidates to the user.

[1207] Step 13:

[1208] The user selects the desired tourist spot or activity from the displayed reservation options and presses the "Confirm reservation" button.

[1209] Step 14:

[1210] The terminal converts the reservation candidate information selected by the user into JSON format and sends it to the server via HTTPS.

[1211] Step 15:

[1212] The server analyzes the received JSON data and executes the reservation procedure using the API of the relevant reservation site or app. When the reservation process is complete, it records the result and generates a status.

[1213] Step 16:

[1214] The server converts the reservation completion status and detailed information into JSON format and sends it to the terminal.

[1215] Step 17:

[1216] The terminal analyzes the received reservation completion notification, displays the reservation completion notification to the user, and provides an interface for checking the details of the reservation.

[1217] Step 18:

[1218] Users can check reservation details and make changes or cancellations as necessary from the "My Page" or "Reservation History" within the app.

[1219] Example 2

[1220] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1221] Conventional travel reservation systems generate reservation candidates based on a user's past usage trends and travel plan information, but do not take into account the user's real-time emotional state, which has the problem of not being able to sufficiently increase user satisfaction. The present invention aims to further improve the user experience by proposing personalized reservation candidates based on the user's emotions.

[1222] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for saving past usage trend data, means for saving travel plan information entered by the user, means for generating reservation candidates based on the past usage trend data and the travel plan information, means for presenting reservation candidates to the user, means for confirming the reservation candidates selected by the user and coordinating reservations, means for analyzing user emotion data, and means for adjusting reservation candidates based on the user emotion data. This makes it possible to propose personalized travel plans based on the user's emotions.

[1223] "Past usage trend data" is data relating to trips taken by the user in the past, and is information indicating the user's preferences and patterns.

[1224] "Travel planning information" is data related to a trip that the user is planning, and includes detailed information such as the travel destination, desired tourist spots, activities, budget, and schedule.

[1225] "Reservation candidates" are travel plans and options suggested to the user, generated based on the user's past usage trend data and travel plan information.

[1226] "Emotion data" is data that indicates the user's real-time emotional state, and is information that is analyzed based on facial expressions, tone of voice, input speed, and the like.

[1227] "Reservation linkage" refers to the process of linking the reservation candidates selected by the user with an external reservation system to actually confirm the reservation.

[1228] MODE FOR CARRYING OUT THE INVENTION

[1229] This invention relates to an advance travel reservation assistant system that combines an emotion engine that recognizes the user's emotions. This system not only automatically generates and suggests optimal reservation options based on the user's past usage trend data and travel plan information, but also has the function of adjusting the reservation options based on the user's emotions. This allows users to create more personalized and optimized travel plans.

[1230] System configuration

[1231] The system consists of the following main components:

[1232] 1. Server

[1233] 2. Terminal

[1234] 3. Users

[1235] 4. Emotion Engine

[1236] Registering user profile information

[1237] A user installs the app and creates an account by entering basic information such as a username and email address. The device acquires the entered basic information and sends it to the server. The server stores the received information in a database. Specifically, data is sent via an HTTP POST request, and a common RDBMS (e.g., PostgreSQL, MySQL) is used as the database.

[1238] Learn from past usage trends

[1239] The server periodically analyzes the past booking history data in the database. Using Python and Scikit-learn, it uses clustering algorithms (e.g., K-means) to learn user preferences and travel patterns. The server adds the learning results to the user profile and reflects them in future recommendations.

[1240] Enter your travel plans

[1241] The user opens the travel planning screen and inputs information such as the areas and tourist spots they want to visit, activities, budget, schedule, etc. The device then sends this input information to the server.

[1242] User Emotion Recognition

[1243] The emotion engine uses the camera and microphone to analyze the user's facial expressions and tone of voice in real time, for example, using OpenCV and the Google Cloud Speech-to-Text API. The device then sends the emotion analysis results to the server.

[1244] Emotion-based reservation candidate generation and adjustment

[1245] The server generates suitable sightseeing spots and activity candidates based on the received travel plan information, past usage trend data, and emotional data. If the user's emotion is positive, it prioritizes suggestions that will satisfy excitement and expectations, and if it is negative, it selects a relaxing plan. The server sends the generated reservation candidates to the terminal in JSON format.

[1246] Confirmation and notification of reservations

[1247] The user selects the plan they want to reserve from the displayed list of reservation candidates and presses the Confirm button. The device sends the user's selection to the server. The server calls the API of the relevant reservation site to confirm the reservation. The server sends a reservation success message to the device, and the device displays a confirmation notification to the user.

[1248] Specific examples

[1249] As a specific example, consider the case where User A uses an app to plan their next trip. User A has previously visited Kyoto and Okinawa and tends to enjoy sightseeing and relaxation. For their next trip, User A wants to visit Hokkaido, and plans to visit Furano and the Kushiro Wetlands and enjoy snowboarding.

[1250] User A creates an account and enters their travel plans into the app.

[1251] The emotion engine detects positive emotions filled with anticipation from User A's facial expressions and tone of voice while he is typing.

[1252] The server generates a plan based on past usage trend data, new travel plans, and emotional data, including accommodation at Furano Ski Resort and a sightseeing tour of the Kushiro Wetlands.

[1253] User A selects accommodation and snowboarding lessons from the suggested reservation options and confirms the reservation.

[1254] The server then connects this information to the reservation site to complete the reservation.

[1255] Finally, the terminal notifies user A of the reservation result, allowing him to check the detailed reservation contents.

[1256] Examples of prompts

[1257] "Develop a system that suggests next travel plans based on the user's booking history and adjusts the optimal plan by analyzing the user's emotions in real time. Incorporate a mechanism that uses an emotion recognition engine to analyze the user's facial expressions, tone of voice, typing speed, etc. and reflects this in the travel plans."

[1258] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1259] Step 1:

[1260] A user installs the app and creates an account by entering basic information such as a username and email address. When the user presses the "Register" button, the basic information is generated as input data. The device obtains this input data and sends it to the server via an HTTP POST request. The server saves the received basic information in a database. Specifically, it uses PostgreSQL or MySQL to insert the user information into a table. It returns a success message as a save result.

[1261] Step 2:

[1262] The server periodically analyzes past reservation history data in the database. It extracts past usage trend data using the user ID as a key. The server uses Python and Scikit-learn to apply a clustering algorithm (e.g., K-means). This allows it to learn the user's preferences and travel patterns, and generates the results as clustering data. The generated clustering data is added to the user profile.

[1263] Step 3:

[1264] The user opens the travel planning screen and enters information such as the areas and tourist spots they want to visit, activities, budget, and dates. This information is generated as input data. The device compiles the entered travel planning information and sends it to the server via an HTTP request. The server saves the received travel planning information in a database and returns a success message as a result of saving.

[1265] Step 4:

[1266] The emotion engine uses the user's camera and microphone to analyze facial expressions and tone of voice in real time. Captured images and audio data are generated as input data. Specifically, facial expressions are analyzed using OpenCV, and audio is analyzed using the Google Cloud Speech-to-Text API. Emotion data is generated as the analysis result. The device acquires the emotion data and sends it to the server via an HTTP request.

[1267] Step 5:

[1268] The server generates suitable sightseeing spots and activity candidates based on the received travel plan information, past usage trend data, and emotion data. Input data includes travel plan information, past usage trend data, and emotion data. The server runs a Python script to generate reservation candidates based on this data. Based on the emotion analysis results, active plans are prioritized if the emotion is positive, and relaxing plans are prioritized if the emotion is negative. The generated reservation candidates are converted into JSON format and sent to the device.

[1269] Step 6:

[1270] The user selects a plan from the displayed list of reservation candidates and presses the confirm button. The device sends the selected reservation details to the server via an HTTP request. The server receives the selection and calls the API of the relevant reservation site to confirm the reservation. The reservation information includes accommodation, activities, and dates. The server notifies the device that the reservation is complete, and the device displays a confirmation notification to the user.

[1271] (Application example 2)

[1272] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1273] Many conventional travel reservation systems generate reservation candidates based on a user's past usage data and travel plan information, but because they cannot adjust reservation candidates taking into account the user's emotional state, it is difficult to propose travel plans optimized for each individual user.In addition, there are limited ways to notify users of reservation results, which reduces user convenience.

[1274] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1275] In this invention, the server includes means for saving past usage trend data, means for saving travel plan information entered by the user, means for generating reservation candidates based on the past usage trend data and travel plan information, means for analyzing the user's facial expressions and tone of voice to recognize emotions, means for adjusting and generating reservation candidates based on the emotion data, means for presenting reservation candidates to the user, means for confirming the reservation candidates selected by the user and coordinating reservations, and means for notifying the user of the reservation results. This makes it possible to propose an optimal travel plan that takes the user's emotional state into consideration. It also enables faster and more convenient notification of reservation results.

[1276] "Past usage trend data" refers to data including travel plans, reservation history, and preference information that the user has used in the past.

[1277] "Travel planning information" refers to information such as the destination, itinerary, budget, and desired activities that the user has entered in advance regarding their next trip.

[1278] "Reservation candidates" are candidates for travel plans, accommodations, activities, etc. that are generated based on the user's usage trend data and travel plan information and that the user can select.

[1279] "Emotion recognition" is the process of analyzing a user's emotions from their facial expressions, tone of voice, etc., and determining whether their emotional state is positive or negative.

[1280] "Emotion data" is the result of the user's emotion recognition, and is data that includes information such as the type and intensity of the emotion.

[1281] "Adjustment" is the process of modifying and optimizing the generated reservation candidates based on the acquired data to suit the user's emotional state.

[1282] "Smart glasses" are wearable devices that use a camera to analyze a user's facial expressions.

[1283] A "head-mounted display" is a wearable display device that inputs information from the user's vision and voice and analyzes emotions.

[1284] This invention relates to a system that utilizes a "smart travel assistant" system to recognize a user's emotions and generate and present an optimized travel plan. The system mainly consists of a server, a terminal (smartphone), and an emotion engine (including smart glasses and head-mounted displays) for emotion recognition.

[1285] When a user inputs their travel plans using the application, the device sends this information to the server, which then uses past usage trend data and the newly input travel plan information to generate initial reservation candidates.

[1286] The emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions in real time. This emotion data is sent to the server via the device. The server adjusts and optimizes reservation options based on the acquired emotion data. Specifically, if a positive emotion is recognized, it will prioritize generating travel plans that match the user's preferences, and if a negative emotion is recognized, it will suggest relaxing activities and low-stress plans.

[1287] The final reservation candidates are sent to the terminal and presented to the user. When the user selects and confirms a reservation, the information is sent back to the server, completing the reservation link. Finally, the reservation results are notified to the user via the terminal.

[1288] The operation of the system can be specifically explained as follows:

[1289] 1. Registering user profile information

[1290] The user installs the application and creates an account. The device sends the basic information entered by the user (such as name, age, gender, and email address) to the server and stores it in a database.

[1291] 2. Learning from past usage trends

[1292] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[1293] 3. Enter your travel plans

[1294] The user opens the travel planning screen and inputs the travel destination, desired sightseeing spots, activities, budget, schedule, etc. The device then sends this information to the server.

[1295] 4. User Emotion Recognition

[1296] The emotion engine (including smart glasses and head-mounted displays) analyzes the user's emotions in real time based on their facial expressions, tone of voice, etc. The device then transmits this emotion data to a server.

[1297] 5. Emotion-based reservation candidate generation and adjustment

[1298] The server selects appropriate sightseeing spots and activities based on past usage trend data, travel plan information, and emotional data. If the user's emotions are judged to be positive, it will prioritize suggestions that reflect their preferences, but if negative emotions are detected, it will suggest relaxing plans and activities that reduce stress.

[1299] 6. Booking confirmation and notification

[1300] The user selects the desired sightseeing spot and activity from the displayed reservation options and confirms the reservation. The device sends the user's selection to the server, which then links the reservation information to the corresponding reservation site or app to confirm the reservation. If the reservation is successful, the server notifies the device of the result, and the device displays a notification to the user that the reservation is complete.

[1301] This allows users to create more personalized travel plans, and the introduction of an emotion engine improves the user experience.

[1302] Example: Prompt sentences to input to a generative AI model

[1303] plaintext

[1304] User emotion recognition prompts

[1305] Analyze the following text data and determine the user's sentiment.

[1306] Facial image: {image_data}

[1307] Voice data: {voice_data}

[1308] Prompt sentences for generating reservation candidates based on emotions

[1309] Generate the best travel plan based on the following user and sentiment data:

[1310] User ID: {user_id}

[1311] emotion: {emotion}

[1312] By implementing the above-described embodiment, more advanced travel planning can be performed taking into account the user's emotional state.

[1313] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1314] Step 1:

[1315] A user installs the application and creates an account.

[1316] Input: Basic information entered by the user (name, age, gender, email address, etc.)

[1317] Operation: The device collects basic information and sends it to the server.

[1318] Output: User basic information is saved in a server-side database.

[1319] Step 2:

[1320] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[1321] Input: Past usage trend data stored on the server

[1322] How it works: The server uses machine learning algorithms to analyze the data and learn user preference patterns.

[1323] Output: The user's preference patterns are stored in a database.

[1324] Step 3:

[1325] The user opens the travel planning screen and inputs the travel destination, desired tourist spots, activities, budget, dates, etc.

[1326] Input: Information about your travel destination, desired sightseeing spots, activities, budget, dates, etc.

[1327] Operation: The device sends this information to the server.

[1328] Output: The travel plan information is saved on the server.

[1329] Step 4:

[1330] Emotion engines (including smart glasses and head-mounted displays) analyze users' emotions in real time based on their facial expressions, tone of voice, etc.

[1331] Input: User's facial expressions and tone of voice

[1332] How it works: The emotion engine uses facial expression analysis algorithms and voice analysis algorithms to analyze emotions and send them to the device.

[1333] Output: Emotion data is sent to the server via the device.

[1334] Step 5:

[1335] The server selects appropriate tourist spots and activities based on past usage trend data, travel plan information, and emotional data.

[1336] Input: Past usage trend data, travel plan information, emotional data

[1337] How it works: The server aggregates the data and runs algorithms to select suitable tourist destinations and activities.

[1338] Output: The generated reservation candidates are sent to the terminal in JSON format.

[1339] Step 6:

[1340] The user selects the desired tourist spot and activity from the generated reservation candidates and confirms the reservation.

[1341] Input: User selected reservation candidate

[1342] How it works: The device sends the selection to the server, and the server links the reservation information to the reservation site or app.

[1343] Output: The reservation confirmation result is sent from the server to the terminal.

[1344] Step 7:

[1345] If the reservation is successful, the result is notified to the terminal and displayed to the user.

[1346] Input: Reservation confirmation result

[1347] Operation: The server notifies the terminal of the result, and the terminal displays the notification to the user.

[1348] Output: The user is notified that the reservation is complete.

[1349] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1350] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1351] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1352] [Fourth embodiment]

[1353] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1354] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1355] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1356] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1357] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1358] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1359] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1360] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1361] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1362] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1363] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1364] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1365] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1366] This invention relates to an advance travel reservation assistance system that allows users to plan their trips efficiently and flexibly. This system has the function of automatically generating and suggesting optimal reservation options by storing user profile information and past reservation history and learning the user's preferences and travel patterns.

[1367] System configuration

[1368] The system consists of the following main components:

[1369] 1. Server

[1370] 2. Terminal

[1371] 3. Users

[1372] Program processing and natural language explanation

[1373] Registering user profile information

[1374] A user installs the app and creates an account.

[1375] The device sends the basic information entered by the user (such as name, age, gender, and email address) to the server.

[1376] The server stores the received information in a database.

[1377] Learn from past usage trends

[1378] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[1379] For example, it is determined that a user who has visited many resort areas in the past will tend to prefer resort areas in the future.

[1380] Enter your travel plans

[1381] The user opens the travel planning screen and inputs the travel destination, desired tourist spots, activities, budget, dates, etc.

[1382] The terminal transmits this information to the server.

[1383] Generation and suggestion of reservation candidates

[1384] The server selects appropriate tourist destinations and activities based on past usage trend data and newly entered travel plan information.

[1385] The server converts the selected reservation candidates into JSON format and sends them to the terminal.

[1386] The terminal analyzes the transmitted information and displays reservation candidates to the user.

[1387] Confirmation and notification of reservations

[1388] The user selects the desired reservation from the displayed reservation candidates and confirms the reservation.

[1389] The terminal transmits the user's selection to the server.

[1390] The server then links the reservation information to the corresponding reservation site or app to confirm the reservation.

[1391] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a final confirmation notice to the user.

[1392] Specific examples

[1393] As a specific example, consider the case where User A uses an app to plan their next trip. User A has previously visited Kyoto and Okinawa and tends to enjoy sightseeing and relaxation. For their next trip, User A wants to visit Hokkaido, and plans to visit Furano and the Kushiro Wetlands and enjoy snowboarding.

[1394] User A creates an account and enters their travel plans into the app.

[1395] The terminal sends this information to the server, which then generates appropriate reservation candidates based on past usage trend data and new travel plan information.

[1396] For example, a plan is generated that includes accommodation at Furano Ski Resort and a sightseeing tour of the Kushiro Wetlands.

[1397] User A selects accommodation and snowboarding lessons from the displayed reservation options and confirms the reservation.

[1398] The server then connects this information to the reservation site to complete the reservation.

[1399] Finally, the terminal notifies user A of the reservation result, allowing him to check the detailed reservation contents.

[1400] Such a configuration and operation allows users to more efficiently and conveniently plan their trips and smoothly proceed with reservations.

[1401] The processing flow will be explained below.

[1402] Step 1:

[1403] The user installs the app, opens the account creation screen, enters basic information such as name, age, gender, and email address, and presses the "Register" button.

[1404] Step 2:

[1405] The terminal converts the information entered by the user into JSON format and sends it to the server via HTTPS.

[1406] Step 3:

[1407] The server analyzes the received JSON data, saves the user information in the database, generates a save completion status, and returns it to the device.

[1408] Step 4:

[1409] The user opens the travel planning screen, enters details such as the travel destination, desired tourist spots, activities, budget, and dates, and presses the "Save Travel Plan" button.

[1410] Step 5:

[1411] The device converts the travel planning information into JSON format and sends it to the server via HTTPS.

[1412] Step 6:

[1413] The server parses the received JSON data, saves the travel plan information to the database, generates a save completion status, and returns it to the device.

[1414] Step 7:

[1415] The server runs a recommendation algorithm based on past usage trends in the database and newly entered travel plan information, collecting data from similar cases of other users to select matching tourist spots and activities.

[1416] Step 8:

[1417] The server lists the selected reservation candidates, converts the list into JSON format, and sends it to the terminal.

[1418] Step 9:

[1419] The device analyzes the received JSON data and displays a list of reservation candidates to the user.

[1420] Step 10:

[1421] The user selects the desired tourist spot or activity from the displayed reservation options and presses the "Confirm reservation" button.

[1422] Step 11:

[1423] The terminal converts the reservation candidate information selected by the user into JSON format and sends it to the server via HTTPS.

[1424] Step 12:

[1425] The server analyzes the received JSON data and executes the reservation procedure using the API of the relevant reservation site or app. When the reservation process is complete, it records the result and generates a status.

[1426] Step 13:

[1427] The server converts the reservation completion status and detailed information into JSON format and sends it to the terminal.

[1428] Step 14:

[1429] The terminal analyzes the received reservation completion notification, displays the reservation completion notification to the user, and provides an interface for checking the details of the reservation.

[1430] Step 15:

[1431] Users can check reservation details and make changes or cancellations as necessary from the "My Page" or "Reservation History" within the app.

[1432] Example 1

[1433] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1434] Conventional travel reservation systems often make blanket recommendations without fully considering a user's past booking history or preferences. This results in inefficiency, as users have to spend a lot of time searching for a travel plan that suits their needs. Furthermore, the process of retrieving information from different booking sites and confirming and notifying reservations is cumbersome. These issues make it difficult for users to easily create optimal travel plans based on their preferences and past usage history.

[1435] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1436] In this invention, the server includes means for saving past usage trend data, means for saving travel plan information entered by the user, means for generating reservation candidates based on the past usage trend data and the travel plan information, means for creating prompt sentences for generating reservation candidates using a generative AI model, means for converting the generated reservation candidates into JSON format and transmitting them, means for presenting reservation candidates to the user, means for confirming the reservation candidates selected by the user and coordinating reservations, and means for notifying the user of confirmed reservation information. This enables users to efficiently create optimal travel plans based on their past usage trends and preferences.

[1437] "Past usage trend data" refers to historical information about reservations made by a user, including travel destinations, travel dates, and information about accommodations and activities used.

[1438] "Travel planning information" is information that a user inputs when planning a new trip, and includes the travel destination, desired tourist spots, activities, budget, schedule, and the like.

[1439] A "reservation candidate" is a proposed plan generated by the server based on the user's past usage trend data and travel plan information, and includes suggestions for specific accommodations and activities.

[1440] "Generative AI model" refers to artificial intelligence technology that analyzes users' past usage trends and travel planning information to generate optimal reservation options.

[1441] A "prompt sentence" is an instruction sentence input to the generative AI model, and is generated based on the user's travel plan information.

[1442] "JSON format" is an abbreviation for JavaScript Object Notation, and is a format for expressing data in text format.

[1443] "Reservation linkage" is a procedure in which the server communicates with an external reservation site or application to confirm the reservation candidates selected by the user and make the reservation.

[1444] "Notification" is a function that allows the server to notify the user of confirmed reservation information, and includes methods such as email and in-app notifications.

[1445] This invention relates to an advance travel reservation assistance system that allows users to make travel plans efficiently and flexibly. This system is composed of multiple components, the main components of which include a server, a terminal, and a user.

[1446] Registering user profile information

[1447] Users first install the application and create an account, which involves entering basic profile information such as name, age, gender, and email address on the device.

[1448] The terminal transmits the input profile information to the server via an HTTP request.

[1449] The server stores the received profile information in a database (e.g., MySQL), which serves to systematically manage the profile information for each user.

[1450] Learn from past usage trends

[1451] The server periodically retrieves and analyzes the user's past booking history stored in the database, using machine learning algorithms (e.g., K-means clustering) to identify the user's preferences and travel patterns.

[1452] The server's analysis results clarify the user's preferred travel destinations and activity patterns. For example, it learns that a user who has visited many resort areas in the past tends to continue to prefer resort areas in the future.

[1453] Enter your travel plans

[1454] The user opens the travel plan input screen within the application and enters detailed information such as the travel destination, desired tourist spots, activities, budget, and schedule.

[1455] The device sends this input information to the server in JSON format, specifically using a REST API.

[1456] Generate reservation candidates

[1457] The server uses a generative AI model (e.g., GPT-4) to generate suitable reservation suggestions based on past usage trend data and newly entered travel plan information, using the following prompt:

[1458] Suggest suitable booking suggestions based on your next travel plans.

[1459] Travel destination: Hokkaido

[1460] Preferred tourist destinations: Furano, Kushiro Marsh

[1461] Activity: Snowboarding

[1462] Budget: Under 200,000 yen

[1463] Dates: February 15, 2024 to February 20, 2024

[1464] Past travel history: Kyoto, Okinawa (sightseeing, relaxation)

[1465] This prompt is then fed into an AI model to generate an optimal travel plan.

[1466] Presenting reservation candidates and confirming reservations

[1467] The server converts the generated reservation candidates into JSON format and sends them to the terminal.

[1468] The terminal analyzes the received reservation candidates and displays them in a visually easy-to-understand format for the user.

[1469] The user selects the desired reservation from the displayed reservation candidates and confirms the reservation.

[1470] The terminal transmits the user's selection to the server, and the server cooperates with an external reservation site or application to confirm the reservation.

[1471] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a final confirmation notice to the user.

[1472] As a concrete example, if User A uses this system to plan and book a trip to Hokkaido, the system generates an optimal plan based on the user's past travel history and current preferences, and the reservation is completed smoothly. This series of operations allows the user to plan an efficient trip without any hassle.

[1473] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1474] Step 1: Register your user profile information

[1475] The user installs the application and creates a new account.

[1476] The user enters basic information such as name, age, gender, and email address and submits it.

[1477] The terminal sends the entered basic information to the server via an HTTP request.

[1478] The server saves the received profile information in a database (e.g., MySQL). At this time, the server validates the input data to maintain data consistency. It responds with a success message to the terminal.

[1479] input:

[1480] Basic information (name, age, gender, email address)

[1481] output:

[1482] Save to database, success message

[1483] Step 2: Learn from past usage trends

[1484] The server periodically retrieves the stored past usage trend data from the database.

[1485] The server analyzes the acquired usage trend data and runs machine learning algorithms (e.g., K-means clustering) to learn user preferences and travel patterns.

[1486] The server uses the analysis results to classify the user's preferences and travel patterns and create basic information for a new travel plan.

[1487] input:

[1488] Past usage trend data

[1489] output:

[1490] User preference classification, travel pattern data

[1491] Step 3: Enter your travel plans

[1492] The user opens the travel plan input screen within the application and enters detailed information such as the travel destination, desired tourist spots, activities, budget, and schedule.

[1493] The terminal converts the input travel planning information into JSON format and sends it to the server via an HTTP request.

[1494] input:

[1495] Travel planning information (destination, sightseeing spots, activities, budget, schedule)

[1496] output:

[1497] JSON formatted travel plan information

[1498] Step 4: Generate reservation candidates

[1499] The server generates a prompt sentence based on past usage trend data and newly entered travel plan information.

[1500] The generated prompt sentence is input into a generative AI model (e.g., GPT-4) to generate appropriate reservation candidates. The example prompt sentence used is as follows:

[1501] Suggest suitable booking suggestions based on your next travel plans.

[1502] Travel destination: Hokkaido

[1503] Preferred tourist destinations: Furano, Kushiro Marsh

[1504] Activity: Snowboarding

[1505] Budget: Under 200,000 yen

[1506] Dates: February 15, 2024 to February 20, 2024

[1507] Past travel history: Kyoto, Okinawa (sightseeing, relaxation)

[1508] The server converts the reservation suggestions obtained from the generative AI model into JSON format.

[1509] input:

[1510] Prompt text, historical usage trend data, travel planning information

[1511] output:

[1512] JSON format data of reservation candidates

[1513] Step 5: Present reservation options

[1514] The server transmits the generated reservation candidates to the terminal.

[1515] The terminal analyzes the received JSON-formatted reservation candidates and visually displays them to the user. Specifically, the terminal uses GUI components to display the reservation candidates in a list or calendar format.

[1516] input:

[1517] JSON format data of reservation candidates

[1518] output:

[1519] Displaying reservation options

[1520] Step 6: Confirmation and notification of reservation

[1521] The user selects the desired reservation from the displayed reservation candidates and confirms the reservation.

[1522] The terminal transmits the selected reservation candidates to the server.

[1523] The server confirms the received reservation information by connecting with an external reservation site or application, and executes the reservation using an external API.

[1524] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a notification to the user.

[1525] input:

[1526] Selected reservation candidate

[1527] output:

[1528] Confirmation of reservation information, notification to users

[1529] Through these steps, users can efficiently plan their trip and make reservations smoothly.

[1530] (Application example 1)

[1531] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1532] Food delivery services have rapidly grown in popularity in recent years, but there is a lack of personalized menu suggestions for users, making it difficult for users to find an appropriate menu that meets their food preferences and restrictions. Furthermore, past order history cannot be effectively utilized, which can lead to a decrease in user satisfaction. The present invention aims to solve these problems by providing a food delivery assistant system that suggests optimal menus to users.

[1533] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1534] In this invention, the server includes means for saving past usage trend data, means for saving meal order information entered by the user, means for generating menu candidates based on the past usage trend data and the meal order information, means for presenting the menu candidates to the user, and means for confirming the menu candidates selected by the user and linking the orders, thereby enabling personalized menu suggestions that take into account the user's food preferences and past ordering history.

[1535] "Past usage trend data" is data that records the history of orders and selections made by a user in the past.

[1536] "Meal order information" refers to details of the meal preferences and orders currently input by the user.

[1537] "Menu candidates" is a list of menus that the server may generate and suggest based on past usage trend data and meal order information.

[1538] "Means for presenting menu candidates to the user" refers to a method for displaying and notifying the generated menu candidates on the user's device.

[1539] The "means for coordinating orders" refers to the process of sending the menu options selected by the user to the corresponding restaurant or provider and finalizing the order.

[1540] "Profile information" is data that indicates basic information such as the user's name, age, gender, and food preferences.

[1541] "Order result" is information indicating the status of the menu candidate selected by the user when it is actually ordered.

[1542] This invention relates to a food delivery assistant system that proposes personalized menus based on a user's food preferences and past ordering history. This system is composed of a server, a terminal, and a user.

[1543] Registering user profile information

[1544] First, the user installs the app and creates an account. The device sends basic information entered by the user, such as name, age, gender, food preferences, and allergy information, to the server, which then stores this information in a database.

[1545] Learn from past usage trends

[1546] The server analyzes the user's past order history stored in a database to learn the user's food preferences and patterns. For example, it determines that a user who frequently orders vegetarian dishes will likely continue to prefer vegetarian dishes. This analysis is performed using machine learning algorithms (for example, models using Scikit-Learn or TensorFlow).

[1547] Entering a meal order

[1548] The user opens the app's order screen and inputs their current mood and desired meal type (e.g., dessert, main dish, etc.). The device sends this information to the server. In this case, an example of a prompt when the user places a new order is "What would you like to eat today? Please choose a category (e.g., dessert, main dish, salad)."

[1549] Menu candidate generation and suggestions

[1550] The server selects an appropriate restaurant and menu based on past usage trend data and newly entered order information. The selected menu candidates are converted into JSON format and sent to the device. The device analyzes the sent information and displays recommended menu items to the user. At this stage, an example of a prompt for the machine learning model is sent to the generative AI model: "Consider past order history and predict the menu item that the user is likely to like next."

[1551] Order confirmation and notification

[1552] The user selects the desired item from the displayed menu options and confirms the order. The device sends the user's selection to the server. The server then connects the order information to the corresponding restaurant and confirms the order. Finally, if the order is successful, the server notifies the device of the result, and the device displays a final confirmation notice to the user.

[1553] Hardware and software used

[1554] Device: User's smartphone

[1555] Server: Cloud server (e.g., Amazon Web Services (AWS))

[1556] Database: MySQL

[1557] Analysis algorithm: Machine learning model (e.g., model using Scikit-Learn or TensorFlow)

[1558] API: RESTful API (transmits and receives data in JSON format)

[1559] In this way, it becomes possible to propose a personalized menu that takes into account the user's food preferences and past ordering history, thereby improving user satisfaction.

[1560] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1561] Step 1: Registering user profile information

[1562] Input: The user installs the app and enters basic information such as name, age, gender, food preferences, and allergy information.

[1563] Specific operation: The terminal processes the information entered by the user and sends it to the server.

[1564] Data processing or data calculation: The server organizes the received information and stores it in a database.

[1565] Output: The profile information is saved in the database.

[1566] Step 2: Learn from past usage trends

[1567] Input: The user's past order history stored in a database.

[1568] How it works: The server analyzes past order data using machine learning algorithms (e.g., Scikit-Learn or TensorFlow models).

[1569] Data processing or data calculation: Through analysis, the system learns the user's food preferences and patterns.

[1570] Output: The learning results are saved and serve as the basis for future data-based recommendations.

[1571] Step 3: Enter your meal order

[1572] Input: The user inputs their current mood and desired meal type (e.g., dessert, main dish, etc.) into the app.

[1573] Specific operation: The terminal sends the order information entered by the user to the server, and a prompt such as "What would you like to eat today? Please select a category (e.g., dessert, main dish, salad)" is presented to the user.

[1574] Data processing or data calculation: The order information is converted into a data format and sent to the server.

[1575] Output: The order information is saved on the server.

[1576] Step 4: Generate and suggest menu options

[1577] Input: Historical usage trend data and newly entered order information.

[1578] How it works: The server uses the generative AI model to select an appropriate restaurant and menu. At this time, it sends a prompt to the generative AI model saying, "Consider the user's past order history and predict the menu item that the user is likely to like next."

[1579] Data processing or data calculation: The menu candidates generated by the server are converted into JSON format and sent to the terminal.

[1580] Output: Menu suggestions are displayed on the terminal.

[1581] Step 5: Order confirmation and notification

[1582] Input: The user selects the desired menu option on the device.

[1583] Specific operation: The device sends the user's selection to the server, which then sends the order information to the corresponding restaurant and confirms the order.

[1584] Data processing or data calculation: The order information is sent to the restaurant's system and the order is confirmed.

[1585] Output: The server notifies the terminal of the order result and displays a final confirmation notice to the user.

[1586] This process flow allows the user to efficiently order food and receive suggested menu options that suit their food preferences.

[1587] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1588] This invention relates to an advance travel reservation assistant system that combines an emotion engine that recognizes the user's emotions. This system not only automatically generates and suggests optimal reservation options based on the user's past usage trend data and travel plan information, but also has the function of adjusting the reservation options based on the user's emotions. This allows users to create more personalized and optimized travel plans.

[1589] System configuration

[1590] The system consists of the following main components:

[1591] 1. Server

[1592] 2. Terminal

[1593] 3. Users

[1594] 4. Emotion Engine

[1595] Program processing and natural language explanation

[1596] Registering user profile information

[1597] A user installs the app and creates an account.

[1598] The device sends the basic information entered by the user (such as name, age, gender, and email address) to the server.

[1599] The server stores the received information in a database.

[1600] Learn from past usage trends

[1601] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[1602] Enter your travel plans

[1603] The user opens the travel planning screen and inputs the travel destination, desired tourist spots, activities, budget, dates, etc.

[1604] The terminal transmits this information to the server.

[1605] User Emotion Recognition

[1606] The emotion engine analyzes the user's emotions in real time based on their facial expressions, tone of voice, typing speed, etc.

[1607] The terminal transmits this emotion data to the server.

[1608] Emotion-based reservation candidate generation and adjustment

[1609] The server selects appropriate tourist spots and activities based on past usage trend data, travel plan information, and emotional data.

[1610] If the user's emotions are judged to be positive, the system will prioritize suggestions that reflect their preferences. On the other hand, if negative emotions are detected, the system will suggest relaxing plans and less stressful options.

[1611] The server converts the selected reservation candidates into JSON format and sends them to the terminal.

[1612] The terminal analyzes the transmitted information and displays a list of reservation candidates to the user.

[1613] Confirmation and notification of reservations

[1614] The user selects the desired tourist spot and activity from the displayed reservation options and confirms the reservation.

[1615] The terminal transmits the user's selection to the server.

[1616] The server then links the reservation information to the corresponding reservation site or app to confirm the reservation.

[1617] If the reservation is successful, the server notifies the terminal of the result, and the terminal displays a notification of reservation completion to the user.

[1618] Specific examples

[1619] As a specific example, consider the case where User A uses an app to plan their next trip. User A has previously visited Kyoto and Okinawa and tends to enjoy sightseeing and relaxation. For their next trip, User A wants to visit Hokkaido, and plans to visit Furano and the Kushiro Wetlands and enjoy snowboarding.

[1620] User A creates an account and enters their travel plans into the app.

[1621] The emotion engine detects positive emotions filled with anticipation from User A's facial expressions and tone of voice while he is typing.

[1622] The server generates a plan based on past usage trend data, new travel plans, and emotional data, including accommodation at Furano Ski Resort and a sightseeing tour of the Kushiro Wetlands.

[1623] User A selects accommodation and snowboarding lessons from the suggested reservation options and confirms the reservation.

[1624] The server then connects this information to the reservation site to complete the reservation.

[1625] Finally, the terminal notifies user A of the reservation result, allowing him to check the detailed reservation contents.

[1626] Such a configuration and operation allows users to create more personalized travel plans, and the introduction of an emotion engine improves the user experience.

[1627] The processing flow will be explained below.

[1628] Step 1:

[1629] The user installs the app, opens the account creation screen, enters basic information such as name, age, gender, and email address, and presses the "Register" button.

[1630] Step 2:

[1631] The terminal converts the information entered by the user into JSON format and sends it to the server via HTTPS.

[1632] Step 3:

[1633] The server analyzes the received JSON data, saves the user information in the database, generates a save completion status, and returns it to the device.

[1634] Step 4:

[1635] The user opens the travel planning screen, enters details such as the travel destination, desired tourist spots, activities, budget, and dates, and presses the "Save Travel Plan" button.

[1636] Step 5:

[1637] The device converts the travel planning information into JSON format and sends it to the server via HTTPS.

[1638] Step 6:

[1639] The server parses the received JSON data, saves the travel plan information to the database, generates a save completion status, and returns it to the device.

[1640] Step 7:

[1641] The emotion engine analyzes the user's emotions in real time based on their facial expressions, tone of voice, typing speed, etc. This emotion data is collected by the device.

[1642] Step 8:

[1643] The device converts the collected emotion data into JSON format and sends it to the server via HTTPS.

[1644] Step 9:

[1645] The server runs a recommendation algorithm based on past usage trends in the database, newly entered travel plan information, and emotional data.

[1646] Step 10:

[1647] The server collects data from similar cases of other users and selects the most suitable sightseeing spots and activities based on the user's emotions. For example, if positive emotions are detected, it will prioritize active plans, and if negative emotions are detected, it will suggest relaxing plans.

[1648] Step 11:

[1649] The server lists the selected reservation candidates, converts the list into JSON format, and sends it to the terminal.

[1650] Step 12:

[1651] The device analyzes the received JSON data and displays a list of reservation candidates to the user.

[1652] Step 13:

[1653] The user selects the desired tourist spot or activity from the displayed reservation options and presses the "Confirm reservation" button.

[1654] Step 14:

[1655] The terminal converts the reservation candidate information selected by the user into JSON format and sends it to the server via HTTPS.

[1656] Step 15:

[1657] The server analyzes the received JSON data and executes the reservation procedure using the API of the relevant reservation site or app. When the reservation process is complete, it records the result and generates a status.

[1658] Step 16:

[1659] The server converts the reservation completion status and detailed information into JSON format and sends it to the terminal.

[1660] Step 17:

[1661] The terminal analyzes the received reservation completion notification, displays the reservation completion notification to the user, and provides an interface for checking the details of the reservation.

[1662] Step 18:

[1663] Users can check reservation details and make changes or cancellations as necessary from the "My Page" or "Reservation History" within the app.

[1664] Example 2

[1665] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1666] Conventional travel reservation systems generate reservation candidates based on a user's past usage trends and travel plan information, but do not take into account the user's real-time emotional state, which has the problem of not being able to sufficiently increase user satisfaction. The present invention aims to further improve the user experience by proposing personalized reservation candidates based on the user's emotions.

[1667] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for saving past usage trend data, means for saving travel plan information entered by the user, means for generating reservation candidates based on the past usage trend data and the travel plan information, means for presenting reservation candidates to the user, means for confirming the reservation candidates selected by the user and coordinating reservations, means for analyzing user emotion data, and means for adjusting reservation candidates based on the user emotion data. This makes it possible to propose personalized travel plans based on the user's emotions.

[1668] "Past usage trend data" is data relating to trips taken by the user in the past, and is information indicating the user's preferences and patterns.

[1669] "Travel planning information" is data related to a trip that the user is planning, and includes detailed information such as the travel destination, desired tourist spots, activities, budget, and schedule.

[1670] "Reservation candidates" are travel plans and options suggested to the user, generated based on the user's past usage trend data and travel plan information.

[1671] "Emotion data" is data that indicates the user's real-time emotional state, and is information that is analyzed based on facial expressions, tone of voice, input speed, and the like.

[1672] "Reservation linkage" refers to the process of linking the reservation candidates selected by the user with an external reservation system to actually confirm the reservation.

[1673] MODE FOR CARRYING OUT THE INVENTION

[1674] This invention relates to an advance travel reservation assistant system that combines an emotion engine that recognizes the user's emotions. This system not only automatically generates and suggests optimal reservation options based on the user's past usage trend data and travel plan information, but also has the function of adjusting the reservation options based on the user's emotions. This allows users to create more personalized and optimized travel plans.

[1675] System configuration

[1676] The system consists of the following main components:

[1677] 1. Server

[1678] 2. Terminal

[1679] 3. Users

[1680] 4. Emotion Engine

[1681] Registering user profile information

[1682] A user installs the app and creates an account by entering basic information such as a username and email address. The device acquires the entered basic information and sends it to the server. The server stores the received information in a database. Specifically, data is sent via an HTTP POST request, and a common RDBMS (e.g., PostgreSQL, MySQL) is used as the database.

[1683] Learn from past usage trends

[1684] The server periodically analyzes the past booking history data in the database. Using Python and Scikit-learn, it uses clustering algorithms (e.g., K-means) to learn user preferences and travel patterns. The server adds the learning results to the user profile and reflects them in future recommendations.

[1685] Enter your travel plans

[1686] The user opens the travel planning screen and inputs information such as the areas and tourist spots they want to visit, activities, budget, schedule, etc. The device then sends this input information to the server.

[1687] User Emotion Recognition

[1688] The emotion engine uses the camera and microphone to analyze the user's facial expressions and tone of voice in real time, for example, using OpenCV and the Google Cloud Speech-to-Text API. The device then sends the emotion analysis results to the server.

[1689] Emotion-based reservation candidate generation and adjustment

[1690] The server generates suitable sightseeing spots and activity candidates based on the received travel plan information, past usage trend data, and emotional data. If the user's emotion is positive, it prioritizes suggestions that will satisfy excitement and expectations, and if it is negative, it selects a relaxing plan. The server sends the generated reservation candidates to the terminal in JSON format.

[1691] Confirmation and notification of reservations

[1692] The user selects the plan they want to reserve from the displayed list of reservation candidates and presses the Confirm button. The device sends the user's selection to the server. The server calls the API of the relevant reservation site to confirm the reservation. The server sends a reservation success message to the device, and the device displays a confirmation notification to the user.

[1693] Specific examples

[1694] As a specific example, consider the case where User A uses an app to plan their next trip. User A has previously visited Kyoto and Okinawa and tends to enjoy sightseeing and relaxation. For their next trip, User A wants to visit Hokkaido, and plans to visit Furano and the Kushiro Wetlands and enjoy snowboarding.

[1695] User A creates an account and enters their travel plans into the app.

[1696] The emotion engine detects positive emotions filled with anticipation from User A's facial expressions and tone of voice while he is typing.

[1697] The server generates a plan based on past usage trend data, new travel plans, and emotional data, including accommodation at Furano Ski Resort and a sightseeing tour of the Kushiro Wetlands.

[1698] User A selects accommodation and snowboarding lessons from the suggested reservation options and confirms the reservation.

[1699] The server then connects this information to the reservation site to complete the reservation.

[1700] Finally, the terminal notifies user A of the reservation result, allowing him to check the detailed reservation contents.

[1701] Examples of prompts

[1702] "Develop a system that suggests next travel plans based on the user's booking history and adjusts the optimal plan by analyzing the user's emotions in real time. Incorporate a mechanism that uses an emotion recognition engine to analyze the user's facial expressions, tone of voice, typing speed, etc. and reflects this in the travel plans."

[1703] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1704] Step 1:

[1705] A user installs the app and creates an account by entering basic information such as a username and email address. When the user presses the "Register" button, the basic information is generated as input data. The device obtains this input data and sends it to the server via an HTTP POST request. The server saves the received basic information in a database. Specifically, it uses PostgreSQL or MySQL to insert the user information into a table. It returns a success message as a save result.

[1706] Step 2:

[1707] The server periodically analyzes past reservation history data in the database. It extracts past usage trend data using the user ID as a key. The server uses Python and Scikit-learn to apply a clustering algorithm (e.g., K-means). This allows it to learn the user's preferences and travel patterns, and generates the results as clustering data. The generated clustering data is added to the user profile.

[1708] Step 3:

[1709] The user opens the travel planning screen and enters information such as the areas and tourist spots they want to visit, activities, budget, and dates. This information is generated as input data. The device compiles the entered travel planning information and sends it to the server via an HTTP request. The server saves the received travel planning information in a database and returns a success message as a result of saving.

[1710] Step 4:

[1711] The emotion engine uses the user's camera and microphone to analyze facial expressions and tone of voice in real time. Captured images and audio data are generated as input data. Specifically, facial expressions are analyzed using OpenCV, and audio is analyzed using the Google Cloud Speech-to-Text API. Emotion data is generated as the analysis result. The device acquires the emotion data and sends it to the server via an HTTP request.

[1712] Step 5:

[1713] The server generates suitable sightseeing spots and activity candidates based on the received travel plan information, past usage trend data, and emotion data. Input data includes travel plan information, past usage trend data, and emotion data. The server runs a Python script to generate reservation candidates based on this data. Based on the emotion analysis results, active plans are prioritized if the emotion is positive, and relaxing plans are prioritized if the emotion is negative. The generated reservation candidates are converted into JSON format and sent to the device.

[1714] Step 6:

[1715] The user selects a plan from the displayed list of reservation candidates and presses the confirm button. The device sends the selected reservation details to the server via an HTTP request. The server receives the selection and calls the API of the relevant reservation site to confirm the reservation. The reservation information includes accommodation, activities, and dates. The server notifies the device that the reservation is complete, and the device displays a confirmation notification to the user.

[1716] (Application example 2)

[1717] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1718] Many conventional travel reservation systems generate reservation candidates based on a user's past usage data and travel plan information, but because they cannot adjust reservation candidates taking into account the user's emotional state, it is difficult to propose travel plans optimized for each individual user.In addition, there are limited ways to notify users of reservation results, which reduces user convenience.

[1719] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1720] In this invention, the server includes means for saving past usage trend data, means for saving travel plan information entered by the user, means for generating reservation candidates based on the past usage trend data and travel plan information, means for analyzing the user's facial expressions and tone of voice to recognize emotions, means for adjusting and generating reservation candidates based on the emotion data, means for presenting reservation candidates to the user, means for confirming the reservation candidates selected by the user and coordinating reservations, and means for notifying the user of the reservation results. This makes it possible to propose an optimal travel plan that takes the user's emotional state into consideration. It also enables faster and more convenient notification of reservation results.

[1721] "Past usage trend data" refers to data including travel plans, reservation history, and preference information that the user has used in the past.

[1722] "Travel planning information" refers to information such as the destination, itinerary, budget, and desired activities that the user has entered in advance regarding their next trip.

[1723] "Reservation candidates" are candidates for travel plans, accommodations, activities, etc. that are generated based on the user's usage trend data and travel plan information and that the user can select.

[1724] "Emotion recognition" is the process of analyzing a user's emotions from their facial expressions, tone of voice, etc., and determining whether their emotional state is positive or negative.

[1725] "Emotion data" is the result of the user's emotion recognition, and is data that includes information such as the type and intensity of the emotion.

[1726] "Adjustment" is the process of modifying and optimizing the generated reservation candidates based on the acquired data to suit the user's emotional state.

[1727] "Smart glasses" are wearable devices that use a camera to analyze a user's facial expressions.

[1728] A "head-mounted display" is a wearable display device that inputs information from the user's vision and voice and analyzes emotions.

[1729] This invention relates to a system that utilizes a "smart travel assistant" system to recognize a user's emotions and generate and present an optimized travel plan. The system mainly consists of a server, a terminal (smartphone), and an emotion engine (including smart glasses and head-mounted displays) for emotion recognition.

[1730] When a user inputs their travel plans using the application, the device sends this information to the server, which then uses past usage trend data and the newly input travel plan information to generate initial reservation candidates.

[1731] The emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions in real time. This emotion data is sent to the server via the device. The server adjusts and optimizes reservation options based on the acquired emotion data. Specifically, if a positive emotion is recognized, it will prioritize generating travel plans that match the user's preferences, and if a negative emotion is recognized, it will suggest relaxing activities and low-stress plans.

[1732] The final reservation candidates are sent to the terminal and presented to the user. When the user selects and confirms a reservation, the information is sent back to the server, completing the reservation link. Finally, the reservation results are notified to the user via the terminal.

[1733] The operation of the system can be specifically explained as follows:

[1734] 1. Registering user profile information

[1735] The user installs the application and creates an account. The device sends the basic information entered by the user (such as name, age, gender, and email address) to the server and stores it in a database.

[1736] 2. Learning from past usage trends

[1737] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[1738] 3. Enter your travel plans

[1739] The user opens the travel planning screen and inputs the travel destination, desired sightseeing spots, activities, budget, schedule, etc. The device then sends this information to the server.

[1740] 4. User Emotion Recognition

[1741] The emotion engine (including smart glasses and head-mounted displays) analyzes the user's emotions in real time based on their facial expressions, tone of voice, etc. The device then transmits this emotion data to a server.

[1742] 5. Emotion-based reservation candidate generation and adjustment

[1743] The server selects appropriate sightseeing spots and activities based on past usage trend data, travel plan information, and emotional data. If the user's emotions are judged to be positive, it will prioritize suggestions that reflect their preferences, but if negative emotions are detected, it will suggest relaxing plans and activities that reduce stress.

[1744] 6. Booking confirmation and notification

[1745] The user selects the desired sightseeing spot and activity from the displayed reservation options and confirms the reservation. The device sends the user's selection to the server, which then links the reservation information to the corresponding reservation site or app to confirm the reservation. If the reservation is successful, the server notifies the device of the result, and the device displays a notification to the user that the reservation is complete.

[1746] This allows users to create more personalized travel plans, and the introduction of an emotion engine improves the user experience.

[1747] Example: Prompt sentences to input to a generative AI model

[1748] plaintext

[1749] User emotion recognition prompts

[1750] Analyze the following text data and determine the user's sentiment.

[1751] Facial image: {image_data}

[1752] Voice data: {voice_data}

[1753] Prompt sentences for generating reservation candidates based on emotions

[1754] Generate the best travel plan based on the following user and sentiment data:

[1755] User ID: {user_id}

[1756] emotion: {emotion}

[1757] By implementing the above-described embodiment, more advanced travel planning can be performed taking into account the user's emotional state.

[1758] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1759] Step 1:

[1760] A user installs the application and creates an account.

[1761] Input: Basic information entered by the user (name, age, gender, email address, etc.)

[1762] Operation: The device collects basic information and sends it to the server.

[1763] Output: User basic information is saved in a server-side database.

[1764] Step 2:

[1765] The server analyzes the user's past reservation history stored in a database and learns the user's preferences and travel patterns.

[1766] Input: Past usage trend data stored on the server

[1767] How it works: The server uses machine learning algorithms to analyze the data and learn user preference patterns.

[1768] Output: The user's preference patterns are stored in a database.

[1769] Step 3:

[1770] The user opens the travel planning screen and inputs the travel destination, desired tourist spots, activities, budget, dates, etc.

[1771] Input: Information about your travel destination, desired sightseeing spots, activities, budget, dates, etc.

[1772] Operation: The device sends this information to the server.

[1773] Output: The travel plan information is saved on the server.

[1774] Step 4:

[1775] Emotion engines (including smart glasses and head-mounted displays) analyze users' emotions in real time based on their facial expressions, tone of voice, etc.

[1776] Input: User's facial expressions and tone of voice

[1777] How it works: The emotion engine uses facial expression analysis algorithms and voice analysis algorithms to analyze emotions and send them to the device.

[1778] Output: Emotion data is sent to the server via the device.

[1779] Step 5:

[1780] The server selects appropriate tourist spots and activities based on past usage trend data, travel plan information, and emotional data.

[1781] Input: Past usage trend data, travel plan information, emotional data

[1782] How it works: The server aggregates the data and runs algorithms to select suitable tourist destinations and activities.

[1783] Output: The generated reservation candidates are sent to the terminal in JSON format.

[1784] Step 6:

[1785] The user selects the desired tourist spot and activity from the generated reservation candidates and confirms the reservation.

[1786] Input: User selected reservation candidate

[1787] How it works: The device sends the selection to the server, and the server links the reservation information to the reservation site or app.

[1788] Output: The reservation confirmation result is sent from the server to the terminal.

[1789] Step 7:

[1790] If the reservation is successful, the result is notified to the terminal and displayed to the user.

[1791] Input: Reservation confirmation result

[1792] Operation: The server notifies the terminal of the result, and the terminal displays the notification to the user.

[1793] Output: The user is notified that the reservation is complete.

[1794] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1795] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1796] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1797] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1798] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1799] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1800] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1801] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1802] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1803] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1804] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1805] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1806] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1807] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1808] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1809] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1810] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1811] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1812] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1813] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1814] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1815] The following is further disclosed regarding the above embodiment.

[1816] (Claim 1)

[1817] a means for storing historical usage trend data;

[1818] means for storing travel planning information entered by a user;

[1819] A means for generating reservation candidates based on past usage trend data and travel plan information;

[1820] means for presenting reservation candidates to a user;

[1821] A means for confirming the reservation candidates selected by the user and linking the reservations;

[1822] A system including:

[1823] (Claim 2)

[1824] 10. The system of claim 1, wherein user profile information is entered and stored.

[1825] (Claim 3)

[1826] 2. The system according to claim 1, wherein the system notifies the user of the reservation result.

[1827] "Example 1"

[1828] (Claim 1)

[1829] a means for storing historical usage trend data;

[1830] means for storing travel planning information entered by a user;

[1831] A means for generating reservation candidates based on past usage trend data and travel plan information;

[1832] A means for generating prompt sentences for generating reservation candidates using a generative AI model;

[1833] A means to convert the generated reservation candidates into JSON format and send them;

[1834] means for presenting reservation candidates to a user;

[1835] A means for confirming the reservation candidates selected by the user and linking the reservations;

[1836] A means of notifying the reservation confirmation information;

[1837] A system including:

[1838] (Claim 2)

[1839] 10. The system of claim 1, wherein user profile information is entered and stored.

[1840] (Claim 3)

[1841] 2. The system according to claim 1, wherein the system notifies the user of the reservation result.

[1842] "Application Example 1"

[1843] (Claim 1)

[1844] a means for storing historical usage trend data;

[1845] means for storing meal order information entered by a user;

[1846] A means for generating menu options based on past usage trend data and meal order information;

[1847] means for presenting menu options to a user;

[1848] A means for confirming the menu candidates selected by the user and linking the order;

[1849] A system including:

[1850] (Claim 2)

[1851] 10. The system of claim 1, wherein user profile information is entered and stored.

[1852] (Claim 3)

[1853] The system according to claim 1, which notifies the user of the order results.

[1854] "Example 2: Combining Emotion Engines"

[1855] Claims

[1856] (Claim 1)

[1857] a means for storing historical usage trend data;

[1858] means for storing travel planning information entered by a user;

[1859] A means for generating reservation candidates based on past usage trend data and travel plan information;

[1860] means for presenting reservation candidates to a user;

[1861] A means for confirming the reservation candidates selected by the user and linking the reservations;

[1862] means for analyzing user emotion data;

[1863] A means for adjusting reservation candidates based on user emotion data;

[1864] A system including:

[1865] (Claim 2)

[1866] 10. The system of claim 1, wherein user profile information is entered and stored.

[1867] (Claim 3)

[1868] 2. The system according to claim 1, wherein the system notifies the user of the reservation result.

[1869] "Application example 2 when combining emotion engines"

[1870] (Claim 1)

[1871] a means for storing historical usage trend data;

[1872] means for storing travel planning information entered by a user;

[1873] A means for generating reservation candidates based on past usage trend data and travel plan information;

[1874] A means for recognizing emotions by analyzing the user's facial expressions and tone of voice;

[1875] means for adjusting and generating reservation suggestions based on emotion data;

[1876] means for presenting reservation candidates to a user;

[1877] A means for confirming the reservation candidates selected by the user and linking the reservations;

[1878] A means for notifying the reservation result;

[1879] A system including:

[1880] (Claim 2)

[1881] 10. The system of claim 1, wherein user profile information is entered and stored.

[1882] (Claim 3)

[1883] 10. The system of claim 1, wherein a head-mounted display or smart glasses is used for collecting and analyzing emotion data. [Explanation of symbols]

[1884] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for storing historical usage trend data; means for storing travel planning information entered by a user; A means for generating reservation candidates based on past usage trend data and travel plan information; means for presenting reservation candidates to a user; A means for confirming the reservation candidates selected by the user and linking the reservations; A system including:

2. 10. The system of claim 1, wherein user profile information is entered and stored.

3. 2. The system according to claim 1, wherein the system notifies the user of the reservation result.

Images