System

Abstract

A system is provided.SOLUTION: This system is provided with an interface means for inputting a travel condition by a user, a processing means for automatically generating plural travel plans based on the travel condition, a display means for displaying the plural travel plans, a reservation procedure means for reserving the travel plan selected by the user and a reporting means for reporting the result of the reservation procedure.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional travel planning methods require users to book transportation and accommodations separately through separate websites or services, resulting in a significant amount of time and effort. This has led many people to find travel planning difficult and to hesitate to travel at all. This process is particularly cumbersome for older people and young people who are not accustomed to planning trips. The present invention aims to solve these problems by providing a system that allows users to easily and consistently perform the entire process from planning to booking a trip. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing the following means. Specifically, it provides a system including an interface means for a user to input travel conditions, a processing means for automatically generating multiple travel plans based on the travel conditions, a display means for displaying the multiple travel plans, a reservation procedure means for reserving the travel plan selected by the user, and a notification means for notifying the user of the results of the reservation procedure. The system further includes an information collection means for collecting transportation information and accommodation information via an external information provider according to the travel conditions. The system also includes a comparison means for comparing multiple travel plans based on cost and time criteria. In this way, the system automatically generates the optimal travel plan based on the travel conditions entered by the user, and allows for easy comparison, selection, reservation, and notification, thereby simplifying the entire process from travel planning to booking.

[0006] "User" means a person who uses the System to enter travel requirements, select and book a travel plan.

[0007] "Travel conditions" refers to information specified by users when making travel plans, such as budget, travel schedule, means of transportation, and preferred accommodations.

[0008] "Interface means" refers to the input device or screen that the user uses to input travel conditions.

[0009] The "processing means" refers to a computer program or algorithm that automatically generates multiple travel plans based on travel conditions input through the interface means.

[0010] "Display means" refers to a display or screen for visually presenting the generated travel plan to the user.

[0011] "Reservation procedure means" refers to the functions and systems used to make reservations for the travel plan selected by the user.

[0012] "Notification means" refers to the mechanism or method for notifying the user of the results of the reservation procedure.

[0013] "External information providers" refer to external databases and services that provide transportation and accommodation information.

[0014] "Information gathering means" refers to functions and systems for gathering necessary travel-related information via external information providing means.

[0015] The "comparison means" refers to a function or system that allows the user to compare the generated travel plans based on cost and time criteria and select the most suitable plan. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention relates to a system that handles everything from travel planning to reservations all at once, and in particular to a system that automatically generates an optimal travel plan and smoothly processes reservations by the user simply by inputting conditions such as budget, schedule, desired means of transportation, and accommodations.

[0038] This system is configured as follows:

[0039] 1. User Input Phase

[0040] Users first access a website or application and enter details of their travel plans, including budget, travel dates, mode of transportation (e.g., plane, car, train), and accommodation preferences (e.g., with hot springs, near a specific station, etc.). This data is collected by the device and sent to the server.

[0041] 2. Data Collection Phase

[0042] The server receives the travel condition data sent by the user. The server collects the necessary information through external information providers (e.g., transportation information providers, accommodation information providers, etc.). Specifically, it obtains the availability and prices of Shinkansen trains and hotels with hot springs.

[0043] 3. Travel plan generation phase

[0044] The server creates a travel plan based on the collected information. It automatically generates multiple plans and evaluates each plan based on factors such as cost, travel time, and accommodation ratings. In this step, the server compares multiple travel plans that best fit the user's criteria.

[0045] 4. Plan presentation phase

[0046] The server sends the generated itineraries to the terminal, which then visually displays the itineraries to the user. The displayed itineraries are easy for users to compare, and each plan's cost, travel time, and accommodation details are displayed in a list.

[0047] 5. Booking process phase

[0048] Once the user selects the desired travel plan, the terminal transmits the selection to the server, which then calls the external information provider again to confirm the selected Shinkansen and hotel reservations.

[0049] 6. Reservation confirmation and completion phase

[0050] The server retrieves the reservation confirmation and sends it to the terminal, which displays it to the user, informing them that their travel plans have been successfully completed.

[0051] Specific examples

[0052] For example, suppose a user logs into a website and enters the following information:

[0053] Budget: 100,000 yen

[0054] Travel dates: December 1, 2023 to December 5, 2023

[0055] Transportation: Shinkansen

[0056] Accommodation: Hot Spring Hotel

[0057] The device collects this information and sends it to a server, which then collects data such as bullet train schedules and availability of hotels with hot springs, and generates multiple travel plans.

[0058] for example,

[0059] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[0060] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[0061] The server sends these plans to the terminal, which displays them to the user. If the user selects Plan A, the terminal sends the information to the server, which then confirms the Shinkansen and hotel reservations. Finally, the reservation confirmation information is displayed on the terminal, letting the user know that their travel plan has been successfully completed.

[0062] The system configuration described above allows for consistent and efficient travel planning and booking.

[0063] The processing flow will be explained below.

[0064] Step 1:

[0065] A user opens a website or application and accesses a form to enter travel planning details.

[0066] Step 2:

[0067] The user inputs travel conditions such as budget, travel dates, transportation, desired accommodation, etc. The input data is collected by the terminal.

[0068] Step 3:

[0069] The device sends the collected travel condition data to the server in JSON format.

[0070] Step 4:

[0071] The server analyzes the received travel condition data, and based on the analyzed data, prepares to call an external information providing means for collecting information that matches the travel conditions.

[0072] Step 5:

[0073] The server uses external information providers (e.g., transportation information providers, accommodation information providers) to collect necessary travel-related data, such as the availability and prices of Shinkansen trains and hotels with hot springs.

[0074] Step 6:

[0075] The server integrates the collected data and automatically generates travel plans. Multiple travel plans are created and each plan is evaluated based on criteria such as cost, travel time, and accommodation ratings.

[0076] Step 7:

[0077] The server compares the generated travel plans and selects the optimal travel plan based on the comparison results.

[0078] Step 8:

[0079] The server sends the optimal travel plan to the device. The plan information is sent to the device in JSON format.

[0080] Step 9:

[0081] The terminal displays the received travel plans to the user. Details of multiple plans are listed and presented in a format that makes it easy for the user to compare them.

[0082] Step 10:

[0083] The user selects a desired travel plan, and the selected plan information is sent to the server by the terminal.

[0084] Step 11:

[0085] The server starts the reservation procedure for the selected travel plan, and confirms the reservation of the selected Shinkansen and hotel through an external information providing means.

[0086] Step 12:

[0087] The server acquires the reservation confirmation information, which is then sent to the terminal.

[0088] Step 13:

[0089] The terminal displays the reservation confirmation information to the user, who confirms that the travel plan has been successfully completed.

[0090] Through the above processing steps, a series of processes from travel planning to reservation completion can be efficiently executed.

[0091] Example 1

[0092] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0093] In conventional travel planning systems, after users input their travel requirements, they had to manually compare many options to find the best plan, which was time-consuming and labor-intensive. Furthermore, when comparing several plans, the detailed information was not visually centrally managed, which reduced user convenience. To solve these problems, a new system was needed.

[0094] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0095] In this invention, the server includes an interface means for users to input travel conditions, a processing means for automatically generating multiple travel plans based on the travel conditions, a display means for displaying the multiple travel plans, a reservation procedure means for reserving the travel plan selected by the user, a notification means for notifying the results of the reservation procedure, an information collection means for collecting transportation information and accommodation information via an external information providing means, a comparison means for comparing the generated travel plans based on cost and time criteria, and an optimization means for optimizing the multiple travel plans based on an evolutionary algorithm. This allows users to easily input travel conditions and compare, select, and reserve from a wide range of options in a unified manner.

[0096] The "interface means" is a means by which a user inputs travel conditions into the system.

[0097] The "processing means" is a means for automatically generating a plurality of travel plans based on travel conditions input by the user.

[0098] The "display means" is a means for visually displaying the generated travel plans to the user.

[0099] The "reservation procedure means" is a means by which a user confirms the travel plan selected and completes the reservation procedure.

[0100] "Notification means" is a means for notifying the user of the results of the reservation procedure.

[0101] "External information providing means" refers to a means for receiving data such as transportation information and accommodation information from outside.

[0102] "Information gathering means" refers to a means for gathering necessary transportation information and accommodation information via external information providing means.

[0103] The "comparison means" is a means for comparing the generated travel plans based on cost and time.

[0104] The "optimization means" is a means for optimizing multiple travel plans based on an evolutionary algorithm and providing the optimal plan to the user.

[0105] The "graphical user interface means" is a means for visually displaying multiple travel plans in an easy-to-understand manner, allowing users to operate the system intuitively.

[0106] The "update means" is a means for automatically updating the reservation information confirmed by the reservation procedure means.

[0107] The present invention relates to a system that handles everything from travel planning to reservations all at once, and in particular to a system that automatically generates an optimal travel plan and smoothly processes reservations by the user simply by inputting conditions such as budget, schedule, desired means of transportation, and accommodations.

[0108] User Input Phase

[0109] Users first access a website or application and enter details of their travel plans, including budget, travel dates, mode of transportation (e.g., plane, car, train), and accommodation preferences (e.g., with hot springs, near a specific station, etc.). This data is collected by the device and sent to the server.

[0110] Data Collection Phase

[0111] The server receives the travel condition data sent by the user. The server collects the necessary information through external information providers (e.g., transportation information providers, accommodation information providers, etc.). Specifically, it obtains the availability and prices of Shinkansen trains and hotels with hot springs.

[0112] Travel plan generation phase

[0113] The server creates a travel plan based on the collected information. It automatically generates multiple plans and evaluates each plan based on factors such as cost, travel time, and accommodation ratings. In this step, the server uses a generative AI model to compare multiple travel plans that best fit the user's requirements.

[0114] Plan presentation phase

[0115] The server sends the generated itineraries to the terminal, which then visually displays the itineraries to the user. The displayed itineraries are easy for users to compare, and each plan's cost, travel time, and accommodation details are displayed in a list.

[0116] Booking process phase

[0117] Once the user selects the desired travel plan, the terminal transmits the selection to the server, which then calls the external information provider again to confirm the selected Shinkansen and hotel reservations.

[0118] Reservation confirmation and completion phase

[0119] The server retrieves the reservation confirmation and sends it to the terminal, which displays it to the user, informing them that their travel plans have been successfully completed.

[0120] Specific examples

[0121] For example, say a user logs into a website and enters the following information:

[0122] Budget: 100,000 yen

[0123] Travel dates: December 1, 2023 to December 5, 2023

[0124] Transportation: Shinkansen

[0125] Accommodation: Hot Spring Hotel

[0126] The device collects this information and sends it to a server, which then collects data such as bullet train schedules and availability of hotels with hot springs, and generates multiple travel plans.

[0127] for example,

[0128] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[0129] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[0130] The server sends these plans to the terminal, which displays them to the user. If the user selects Plan A, the terminal sends the information to the server, which then confirms the Shinkansen and hotel reservations. Finally, the reservation confirmation information is displayed on the terminal, letting the user know that their travel plan has been successfully completed.

[0131] Example prompts to be input to the generative AI model

[0132] "Generate the best travel plan based on your budget, mode of transportation, and other factors."

[0133] "Please suggest multiple travel plans based on the following criteria: budget \100,000, travel dates December 1, 2023 to December 5, 2023, Shinkansen, hotel with hot spring."

[0134] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0135] Step 1: User Input Phase

[0136] Input: User's travel conditions (budget, travel dates, transportation, accommodation preferences)

[0137] Output: Collected travel condition data

[0138] Specific behavior:

[0139] A user accesses a website or application and enters details of their travel plans. They select their travel dates in a calendar format and enter their budget in text boxes. They select transportation and accommodation options using drop-down lists. After completing all the input fields, the user clicks the "Submit" button. This action causes the device to collect the data and send it to the server.

[0140] Step 2: Data collection phase

[0141] Input: Travel condition data

[0142] Output: Transportation information, accommodation information

[0143] Specific behavior:

[0144] The server receives the travel condition data that has been sent. It then collects the necessary information through external information providers (e.g., transportation information providers, accommodation information providers). The server sends an API request to obtain the availability and prices of Shinkansen trains and hotels with hot springs. The API response is returned to the server, where the collected information is integrated.

[0145] Step 3: Travel plan generation phase

[0146] Input: Collected transportation information, accommodation information, and travel condition data

[0147] Output: Multiple itineraries

[0148] Specific behavior:

[0149] The server uses a generative AI model to generate multiple travel plans based on the collected data and the user's travel conditions. Each plan is evaluated taking into account factors such as cost, travel time, and accommodation ratings. The generated plans are temporarily stored in the server's internal database.

[0150] Step 4: Plan presentation phase

[0151] Input: Multiple itineraries

[0152] Output: Travel plan information for display

[0153] Specific behavior:

[0154] The server sends the generated travel plans to the device, which receives data for visually displaying these plans. The user can compare the details of the plans through the device, and the plan costs, travel times, and accommodation details are displayed in a list.

[0155] Step 5: Booking process phase

[0156] Input: User selected itinerary

[0157] Output: Confirmed reservation information

[0158] Specific behavior:

[0159] Once the user selects their desired travel plan, the device sends the selection information to the server. The server then calls the external information provider again to confirm the selected Shinkansen and hotel reservations. The necessary reservation information is sent and received via the API.

[0160] Step 6: Reservation confirmation and completion phase

[0161] Input: Confirmed reservation information

[0162] Output: Reservation confirmation information, notifications

[0163] Specific behavior:

[0164] The server retrieves the reservation confirmation and sends it to the terminal, which receives it and displays it to the user. The reservation confirmation includes details such as the reservation number, dates, and hotel address, and also displays the message "Your reservation is complete."

[0165] (Application example 1)

[0166] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0167] Conventional food delivery planning systems have the drawback of requiring users to input their preferences through a complex interface, making it difficult to compare multiple options and select the optimal plan. When users have specific requests, there is a need for a system that can generate the optimal plan and quickly complete the ordering process.

[0168] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0169] In this invention, the server includes input means for the user to input conditions, processing means for automatically generating multiple delivery plans based on the conditions, display means for displaying the multiple delivery plans, ordering means for ordering the delivery plan selected by the user, and notification means for notifying the result of the ordering process. This allows the optimal delivery plan based on the conditions desired by the user to be generated quickly and efficiently, and makes the ordering process even easier.

[0170] "Input means" refers to a device or interface for inputting desired conditions or information by a user.

[0171] The "processing means" is a device or software for executing a specific process based on data received from the input means.

[0172] The "display means" is a device or interface for visually presenting the generated plans and information to the user.

[0173] "Order processing means" refers to a device or software that allows a user to confirm the plan selected and execute an order.

[0174] "Notification means" refers to a device or interface for notifying users of the results of the order process and updated information.

[0175] "Information collection means" refers to a device or software for acquiring necessary data from an external information providing service.

[0176] A "comparison tool" is a device or software used to evaluate and compare multiple plans based on specific criteria.

[0177] This invention is a system that automatically generates an optimal delivery plan based on the conditions entered by the user and performs the entire ordering process. This system uses the following hardware and software.

[0178] Hardware and software used:

[0179] Server: AWS EC2 (data processing)

[0180] Database: AWS RDS (data storage)

[0181] API: Google Places API, Yelp API (collecting restaurant information)

[0182] Frontend: React Native (smartphone app)

[0183] Overall system processing overview:

[0184] 1. Data entry method

[0185] Users enter their order requirements through a smartphone app, including budget, desired delivery time, type of food, and specific requests (e.g., vegetarian). This information is sent from the user's device to the server.

[0186] 2. Information gathering methods

[0187] The server uses external information providers (such as Google Places API and Yelp API) to collect information about restaurants and delivery services based on the entered conditions, including restaurant menus, prices, ratings, and available delivery times.

[0188] 3. Plan Generation Method

[0189] The server automatically generates multiple delivery plans based on the collected information, taking into account criteria such as cost, delivery time, and review ratings.

[0190] 4. Plan display method

[0191] The generated multiple delivery plans are sent to the device and displayed on the user's smartphone app, where they are presented in a format that makes it easy to compare plans by cost and delivery time.

[0192] 5. Order Processing Methods

[0193] Once the user selects the desired plan, the selection information is sent to the server, which then confirms the order with the delivery service and completes the order process.

[0194] 6. Means of notification

[0195] The server receives order confirmation information and estimated delivery time and sends it to the user's device. The user can then check the order confirmation and estimated delivery time on their smartphone app.

[0196] Examples:

[0197] For example, suppose a user enters information into a smartphone app under the following conditions:

[0198] Budget: 2,000 yen

[0199] Delivery time: 18:00~19:00

[0200] Cuisine type: Japanese

[0201] Specific requests: Vegetarian

[0202] Based on this information, the server uses the Google Places API and Yelp API to gather relevant restaurant information and generate a delivery plan like this:

[0203] Plan A: Japanese restaurant (¥1,800) + delivery fee (¥200) + delivery time: 18:30

[0204] Plan B: New Japanese restaurant (¥1,500) + Delivery fee (¥300) + Delivery time: 18:45

[0205] These plans are displayed on the user's smartphone app, and once the user selects the desired plan and confirms the order, the results are notified.

[0206] Example prompts to be input to the generative AI model:

[0207] "I'd like Japanese food delivered by 8 PM for under 2,000 yen. I'd like to choose a vegetarian option. Please prioritize restaurants with high reviews."

[0208] This allows users to quickly and efficiently generate the optimal delivery plan based on their specific preferences and easily complete the ordering process.

[0209] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0210] Step 1:

[0211] The user launches the smartphone app and inputs conditions such as budget, delivery time, desired type of food, specific requests, etc. The following information is then sent to the server via the input means:

[0212] Input: Budget (e.g., ¥2,000), Delivery time (e.g., 18:00-19:00), Cuisine type (e.g., Japanese), Specific requests (e.g., Vegetarian)

[0213] Output: The condition is sent to the server

[0214] Step 2:

[0215] The server calls external information providers (e.g., Google Places API, Yelp API) based on the received condition data to obtain related restaurant information and delivery service information. Specifically, it makes an API call and collects data that matches the conditions.

[0216] Input: User condition data

[0217] Output: Restaurant information (e.g., menu, prices, ratings) and delivery service information (e.g., delivery times, delivery fees)

[0218] Step 3:

[0219] The server automatically generates multiple delivery plans based on the collected restaurant and delivery service information. Specifically, it evaluates each combination of restaurant and delivery service based on evaluation criteria and creates plans.

[0220] Input: Collected restaurant and delivery service information

[0221] Output: Multiple delivery plans (e.g., Plan A: Japanese restaurant (¥1,800) + delivery fee (¥200), Plan B: New Japanese restaurant (¥1,500) + delivery fee (¥300))

[0222] Step 4:

[0223] The server sends the generated multiple delivery plans to the user's device and displays them on the user's smartphone app. Specifically, it formats the data to display the plans in a format that makes it easy to visually compare them.

[0224] Input: Multiple delivery plans

[0225] Output: A formatted list of plans

[0226] Step 5:

[0227] The user selects the desired delivery plan on the smartphone app. Once selected, the information is sent to the server. Specifically, data is sent to reflect the user's selection on the server.

[0228] Input: User's selected plan

[0229] Output: Selection information is sent to the server

[0230] Step 6:

[0231] The server confirms the order with the delivery service based on the received selection information. Specifically, it calls the delivery service's API and performs order confirmation processing.

[0232] Input: User selection information

[0233] Output: Order confirmation information

[0234] Step 7:

[0235] The server receives the order confirmation information and estimated delivery time and sends it to the user's device. The user can then confirm the order confirmation and estimated delivery time on the smartphone app. Specifically, the confirmation information is sent via a notification means.

[0236] Input: Order confirmation information and estimated delivery time

[0237] Output: A confirmation message is displayed on the user's terminal.

[0238] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0239] This invention relates to a system that combines a system that handles everything from travel planning to booking with an emotion engine that recognizes the user's emotions, and is able to propose optimal travel plans taking into account the user's emotional state.By analyzing the user's emotional information in real time and adjusting and proposing travel plans based on this, it is possible to provide travel plans that will provide a higher level of satisfaction.

[0240] This system is configured as follows:

[0241] 1. User Input Phase

[0242] A user opens a website or application and enters details of their travel plans, including budget, travel dates, mode of transportation (e.g., plane, car, train), and accommodation preferences (e.g., with hot springs, near a specific station), and this data is collected by the device and sent to the server.

[0243] 2. Data Collection Phase

[0244] The server receives the travel condition data sent by the user. The server collects the necessary travel-related data through external information providers (e.g., transportation information providers, accommodation information providers). For example, it obtains information such as the availability and price of Shinkansen trains and the availability and price of hotels with hot springs.

[0245] 3. Travel plan generation phase

[0246] The server integrates the collected data and automatically generates travel plans. Multiple travel plans are created and each plan is evaluated based on criteria such as cost, travel time, and accommodation ratings.

[0247] 4. Sentiment Analysis Phase

[0248] The emotion engine analyzes the user's input data and emotional information such as the user's facial expressions and tone of voice. This analyzed emotional data is reflected in the travel plan generation, and the plan is adjusted to match the user's emotional state.

[0249] 5. Plan presentation phase

[0250] The server generates multiple travel plans and sends them to the device. The plan information, which also takes into account the results of sentiment analysis, is displayed to the user on the device. The details of the multiple plans are displayed in a list format that makes it easy for the user to compare and consider them.

[0251] 6. Real-time feedback phase

[0252] The emotion engine monitors the user's emotional state in real time and provides immediate feedback when selecting a desired travel plan. For example, if the engine detects anxiety or dissatisfaction with the user's chosen plan, it will present other options.

[0253] 7. Booking Process Phase

[0254] After the user selects the desired travel plan, the selected information is sent from the terminal to the server, which then calls the external information providing means again to confirm the selected Shinkansen and hotel reservations.

[0255] 8. Reservation confirmation and completion phase

[0256] The server retrieves the reservation confirmation information and sends it to the terminal, which displays it to the user, indicating that the travel plan has been successfully completed.

[0257] Specific examples

[0258] For example, suppose a user logs into a website and enters the following information:

[0259] Budget: 100,000 yen

[0260] Travel dates: December 1, 2023 to December 5, 2023

[0261] Transportation: Shinkansen

[0262] Accommodation: Hot Spring Hotel

[0263] The device collects this information and sends it to a server, which then collects data such as bullet train schedules and availability of hotels with hot springs, and generates multiple travel plans.

[0264] for example,

[0265] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[0266] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[0267] At the same time, the emotion engine analyzes the user's emotional state from their facial expressions and tone of voice, and if the user is looking to relax, it makes adjustments such as adding hotels with hot springs to the recommendations.

[0268] The server sends these adjusted plans to the device, which displays them to the user. Real-time feedback suggests alternative plans if the user expresses anxiety or dissatisfaction when selecting Plan A.

[0269] Finally, the user selects the best plan and the reservation is confirmed. The reservation confirmation information is displayed on the terminal, and the user can confirm that the travel plan has been successfully completed.

[0270] The system configuration described above makes it possible to consistently and efficiently carry out travel planning and booking while taking emotional information into consideration.

[0271] The processing flow will be explained below.

[0272] Step 1:

[0273] A user opens a website or application and accesses a form to enter travel planning details.

[0274] Step 2:

[0275] The user inputs travel conditions such as budget, travel schedule, transportation, desired accommodation, etc. The input data is collected by the terminal and sent to the server.

[0276] Step 3:

[0277] The server analyzes the received travel condition data, and based on the analyzed data, prepares to call an external information providing means for collecting information that matches the travel conditions.

[0278] Step 4:

[0279] The server uses external information providers (e.g., transportation information providers, accommodation information providers) to collect necessary travel-related data, such as the availability and prices of Shinkansen trains and hotels with hot springs.

[0280] Step 5:

[0281] The server integrates the collected data and automatically generates travel plans. Multiple travel plans are created and each plan is evaluated based on criteria such as cost, travel time, and accommodation ratings.

[0282] Step 6:

[0283] The emotion engine collects user input data and emotion information such as the user's facial expressions and tone of voice, and analyzes the user's emotional state.

[0284] Step 7:

[0285] The server then adjusts the travel plan based on the results of the sentiment analysis. For example, if the user wants a relaxing trip, it will prioritize accommodations with quiet environments.

[0286] Step 8:

[0287] The server sends the adjusted travel plans to the terminal, which displays them to the user. The plan details are displayed in a list format that makes it easy for the user to compare them.

[0288] Step 9:

[0289] The emotion engine monitors the user's emotional state in real time, analyzing the emotional state and providing immediate feedback when the user selects a desired travel plan from the presented plans.

[0290] Step 10:

[0291] The user selects a desired travel plan, and the selected plan information is sent to the server by the terminal.

[0292] Step 11:

[0293] The server starts the reservation procedure for the selected travel plan, and confirms the reservation of the selected Shinkansen and hotel through an external information providing means.

[0294] Step 12:

[0295] The server receives the reservation confirmation information and sends it to the terminal, which displays it to the user.

[0296] Step 13:

[0297] The user checks the reservation confirmation information and finds out that the travel plan has been successfully completed. Through the above processing steps, a series of processes from travel planning to reservation completion is efficiently executed, taking into account the user's emotional information.

[0298] Example 2

[0299] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0300] Conventional travel planning systems only automatically generate travel plans based on user-entered conditions, but are unable to consider the user's emotional state, making it difficult to propose highly satisfying travel plans. Furthermore, there have been few systems that can analyze the user's emotions in real time and adjust or propose travel plans based on those emotions.

[0301] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0302] In this invention, the server includes an input means for a user to input travel conditions, a generation means for automatically generating multiple travel plans based on the travel conditions, and an emotion analysis means for adjusting the travel plans based on the user's emotional state, thereby making it possible to propose highly satisfying travel plans that reflect the user's emotional state in real time.

[0303] "Input means" refers to a means by which a user inputs travel conditions.

[0304] The "generation means" is a means for automatically generating multiple itineraries based on input travel conditions.

[0305] The "emotion analysis means" is a means for analyzing the user's emotional state and adjusting the travel plan based on that.

[0306] The "display means" is a means for displaying the generated itineraries to the user.

[0307] The "reservation procedure means" refers to the procedure means for reserving the travel plan selected by the user.

[0308] The "notification means" is a means for notifying the user of the results of the reservation procedure.

[0309] "Data collection means" refers to a means for collecting transportation information and accommodation information via external data providing means.

[0310] A "comparison tool" is a tool for comparing multiple travel plans based on cost and time criteria.

[0311] This invention is a system that handles everything from travel planning to booking, and is particularly equipped with an emotion engine that recognizes the user's emotions. This system can propose optimal travel plans taking into account the user's emotional state.

[0312] Specific configuration

[0313] Hardware and Software

[0314] The system uses the following hardware and software:

[0315] Server: Data processing, travel plan generation, and sentiment analysis.

[0316] Terminal: A PC or smartphone that performs user input and displays plans.

[0317] Emotion engine: Analyzes user emotions in real time and adjusts and suggests plans.

[0318] The software used includes:

[0319] Web application: Provides an interface for users to input their travel plans.

[0320] External API: Used to collect data to obtain information on transportation and accommodation.

[0321] Generative AI models: Used for itinerary generation and sentiment analysis.

[0322] How to use

[0323] 1. User Input

[0324] A user opens a website or application and enters the following information:

[0325] budget

[0326] travel itinerary

[0327] Transportation (e.g., bullet train, car, airplane, etc.)

[0328] Accommodation preferences (e.g. with hot springs, specific location, etc.)

[0329] The terminal collects this information and sends it to the server.

[0330] 2. Data Collection

[0331] The server receives the input data and collects information on Shinkansen trains and accommodations through external data providers, specifically information on Shinkansen availability and prices, availability and prices of hotels with hot springs, etc.

[0332] 3. Travel plan generation

[0333] The server generates multiple travel plans based on the collected data, and the plans are scored based on criteria such as cost, travel time, and accommodation ratings.

[0334] 4. Emotion analysis

[0335] The emotion engine analyzes the user's emotional state based on input data, facial expressions, tone of voice, etc. The travel plan is then adjusted based on the analyzed data. For example, if the user is looking for relaxation, hotels with hot springs will be suggested first.

[0336] 5. Plan presentation

[0337] The server sends the generated travel plan to the terminal, which then displays it to the user. The plan information is presented in an easy-to-compare format.

[0338] 6. Real-time feedback

[0339] The emotion engine monitors users' emotions in real time and provides immediate feedback when selecting travel plans, suggesting alternative plans if anxiety or dissatisfaction is detected.

[0340] 7. Reservation Procedure

[0341] After the user selects the desired travel plan, the information is sent to the server, which then calls the external data providing means again to confirm the selected Shinkansen and hotel reservations.

[0342] 8. Reservation Confirmation

[0343] The server receives the reservation confirmation information and sends it to the terminal, which displays the reservation completion information to the user.

[0344] Specific examples

[0345] For example, if a user logs into a website and enters the following information:

[0346] Budget: 100,000 yen

[0347] Travel dates: December 1, 2023 to December 5, 2023

[0348] Transportation: Shinkansen

[0349] Accommodation: Hot Spring Hotel

[0350] The device collects information and sends it to the server. The server collects data on Shinkansen trains and hotels with hot springs, and generates multiple travel plans:

[0351] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[0352] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[0353] The emotion engine analyzes the user's emotions and recommends hotels with hot springs if they are looking for relaxation. The server sends the adjusted plan to the device and displays it to the user. Real-time feedback allows the user to select the best plan.

[0354] Finally, the user selects a plan and the reservation is confirmed. Reservation confirmation information is displayed on the terminal, and the user confirms that the travel plan has been successfully completed.

[0355] Generative AI model prompt example

[0356] "Please tell us about your next trip. Explain your budget, travel dates, transportation, and accommodation preferences. Also, tell us how you want to feel during your trip."

[0357] The system configuration described above makes it possible to consistently and efficiently carry out travel planning and booking while taking emotional information into consideration.

[0358] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0359] Step 1:

[0360] User Input Phase

[0361] A user opens a website or application and enters details of their travel plans (budget, travel dates, transportation options, and preferred accommodations). The data entered by the user is collected by the device and sent to the server. The server then receives the travel conditions data. As a specific example, a user enters information such as a budget of 100,000 yen, travel dates from December 1st to 5th, 2023, travel by Shinkansen, and a hotel with a hot spring.

[0362] Input: Travel conditions entered by the user (budget, travel dates, transportation, accommodation preferences)

[0363] Output: Travel condition data sent to the server

[0364] Step 2:

[0365] Data Collection Phase

[0366] After the server receives the travel condition data, it collects transportation information and accommodation information via external data provision means. As a specific example, the server calls the APIs of a Shinkansen train operation information service and an accommodation information service to obtain the availability and prices of Shinkansen trains and hotels with hot springs.

[0367] Input: Travel condition data

[0368] Output: Transportation and accommodation information

[0369] Step 3:

[0370] Travel plan generation phase

[0371] The server automatically generates multiple travel plans based on the data collected. The server scores the generated plans based on criteria such as cost, travel time, and accommodation ratings. Specifically, the server combines Shinkansen prices and hotel prices to calculate the cost of the travel plan.

[0372] Input: Transportation and accommodation information

[0373] Output: Multiple itineraries

[0374] Step 4:

[0375] Sentiment Analysis Phase

[0376] The emotion engine analyzes the user's input data, facial expressions, and tone of voice to extract the user's emotional state. Based on this emotional data, the server adjusts the travel plan. For example, if the user is looking for relaxation, it will emphasize and suggest hotels with hot springs.

[0377] Input: User's emotional information

[0378] Output: Adjusted itinerary

[0379] Step 5:

[0380] Plan presentation phase

[0381] The server sends the generated travel plan to the terminal, which displays it to the user. The terminal displays a list of details of multiple travel plans so that the user can easily compare them.

[0382] Input: Adjusted travel plans

[0383] Output: The itinerary displayed to the user

[0384] Step 6:

[0385] Real-time feedback phase

[0386] The emotion engine monitors the user's emotional state in real time and provides immediate feedback when selecting a travel plan, for example suggesting other options if the user expresses anxiety or dissatisfaction with the plan they have chosen.

[0387] Input: Real-time user emotion information

[0388] Output: Alternative suggestions

[0389] Step 7:

[0390] Booking process phase

[0391] When the user selects the desired travel plan, the information is sent from the terminal to the server, which then calls the external data providing means again to confirm the reservation of the selected Shinkansen and hotel.

[0392] Input: User selected travel plan

[0393] Output: Confirmed reservation information

[0394] Step 8:

[0395] Reservation confirmation and completion phase

[0396] The server retrieves the reservation confirmation information and sends it to the terminal, which displays the reservation confirmation information to the user, informing them that their travel plans have been successfully completed.

[0397] Input: Confirmed reservation information

[0398] Output: Booking confirmation information displayed to the user

[0399] (Application example 2)

[0400] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0401] In conventional autonomous vehicles, operation plans are determined without taking the driver's emotions into consideration, which increases driver stress and anxiety and reduces driving comfort. Furthermore, there was no system that provided real-time feedback to the driver based on their emotions when proposing a plan or during operation. Therefore, in order to provide a safer and more comfortable driving environment, a system that analyzes the driver's emotions in real time and adjusts operation plans based on that information is needed.

[0402] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes emotion analysis means for analyzing the user's emotion information, plan adjustment means for adjusting the travel plan based on the emotion information, and feedback means for presenting the adjusted plan. This makes it possible to analyze the driver's emotion information in real time and adjust or propose a trip plan based on it.

[0403] "User" refers to the general public who uses the system to input travel conditions.

[0404] "Interface means" refers to an input device or software that allows a user to input travel conditions.

[0405] "Processing means" refers to a device or software that automatically generates multiple travel plans based on input travel conditions.

[0406] "Display means" refers to a device or software that visually presents the generated travel plans to the user.

[0407] "Reservation procedure means" refers to the device or software used to actually reserve the travel plan selected by the user.

[0408] "Notification means" refers to devices or software for notifying users of the results of their reservation procedures.

[0409] "Emotion analysis means" refers to devices or software for analyzing users' emotional information.

[0410] "Plan adjustment means" refers to a device or software for adjusting a travel plan based on the analyzed emotional information.

[0411] "Feedback means" refers to a device or software for presenting the adjusted plan to the user and providing feedback in real time.

[0412] "External information providing means" refers to devices and services for collecting external information such as transportation information and accommodation information.

[0413] This invention combines an emotion engine with a system that handles everything from user travel planning to reservations, and is a specific form for providing safe and comfortable driving plans for drivers of self-driving vehicles. The details are provided below.

[0414] 1. User input phase

[0415] Users use an application installed on their smartphone or head-mounted display to input travel conditions such as destination, travel time, desired route, etc. This allows the user's desired travel plan to be clearly collected.

[0416] 2. Data Collection Phase

[0417] The server collects information on traffic, weather, and congestion at destinations through external information providers, obtaining the latest data in real time via a REST API.

[0418] 3. Operation plan generation phase

[0419] Based on the collected data, the server generates multiple operation plans using Python scripts and machine learning algorithms (e.g., optimization algorithms) and evaluates each one.

[0420] 4. Sentiment Analysis Phase

[0421] Using the camera and microphone installed on the smartphone or head-mounted display, and using face recognition API and voice recognition API, emotional information is analyzed in real time from the user's facial expressions and tone of voice, thereby obtaining emotional information about the user.

[0422] 5. Plan presentation phase

[0423] The server presents the user with multiple operation plans that take emotion information into consideration through a display means, thereby providing the user with an intuitive interface for selecting the optimal plan based on emotion.

[0424] 6. Real-time feedback phase

[0425] Based on the analyzed emotional information, the server adjusts and re-presents the operation plan according to the user's real-time emotions through feedback means. For example, if the server detects anxiety or dissatisfaction with the plan selected by the user, it will present an alternative plan.

[0426] 7. Operational Phase

[0427] After the user selects the optimal plan, the app sends driving instructions to the autonomous vehicle's onboard computer, starting a safe and comfortable journey.

[0428] Specific examples

[0429] User input: Destination "Tokyo", travel time "within 2 hours"

[0430] Prompt (when inputting to a generative AI model):

[0431] Suggest a route that will make the driver feel safe and relaxed. Emotions are a little tense right now. Choose a safe and efficient route within the desired timeframe.

[0432] As described above, this invention is a system that analyzes user emotions in real time and adjusts and proposes operation plans based on that analysis, providing a safe and comfortable operating environment for self-driving vehicles.

[0433] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0434] Step 1:

[0435] The terminal provides an interface where users can input travel conditions (destination, travel time, desired route, etc.) using a smartphone or head-mounted display. The input information is sent from the terminal to the server. The input is the user's desired conditions, and the output is data sent to the server.

[0436] Step 2:

[0437] The server collects traffic information, weather information, and congestion information at the destination via external information providing means based on the travel conditions sent from the terminal. The collected information is stored in the server. The input is the travel conditions and external information, and the output is the storage of the collected data.

[0438] Step 3:

[0439] The server generates a trip plan using the collected data. It uses Python scripts and machine learning algorithms to apply optimization algorithms to create multiple trip plans. The generated trip plans are stored on the server. The input is the collected data and the algorithm, and the output is the trip plan.

[0440] Step 4:

[0441] The device uses the camera and microphone of the smartphone or head-mounted display to analyze the user's face and voice, and analyzes facial expressions and tone of voice in real time using an emotion analysis API. The analysis results are sent from the device to a server. The input is facial expression and voice data, and the output is the emotion analysis results.

[0442] Step 5:

[0443] The server adjusts the previously created operation plan based on the emotion analysis results. Using the plan adjustment means, it selects the operation plan that best suits the user's emotional state and creates an adjusted operation plan. The input is the emotion analysis results and the operation plan, and the output is the adjusted operation plan.

[0444] Step 6:

[0445] The terminal presents the adjusted operation plans from the server to the user. The plans are displayed in a visually easy-to-compare format to make it easier for the user to select a plan. The input is the adjusted operation plan, and the output is the plan presented to the user.

[0446] Step 7:

[0447] The server provides real-time feedback on the trip plan selected by the user, detects the user's anxiety and dissatisfaction, and presents alternative plans as necessary. The input is the user's preferences and emotional state, and the output is the presentation of alternative plans.

[0448] Step 8:

[0449] The terminal confirms the user's final selected trip plan and sends instructions to the autonomous vehicle's onboard computer, which then executes the selected trip plan. The input is the final selected trip plan, and the output is instructions to the onboard computer.

[0450] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0451] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0452] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0453] [Second embodiment]

[0454] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0455] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0456] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0457] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0458] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0459] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0460] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0461] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0462] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0463] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0464] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0465] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0466] The present invention relates to a system that handles everything from travel planning to reservations all at once, and in particular to a system that automatically generates an optimal travel plan and smoothly processes reservations by the user simply by inputting conditions such as budget, schedule, desired means of transportation, and accommodations.

[0467] This system is configured as follows:

[0468] 1. User Input Phase

[0469] Users first access a website or application and enter details of their travel plans, including budget, travel dates, mode of transportation (e.g., plane, car, train), and accommodation preferences (e.g., with hot springs, near a specific station, etc.). This data is collected by the device and sent to the server.

[0470] 2. Data Collection Phase

[0471] The server receives the travel condition data sent by the user. The server collects the necessary information through external information providers (e.g., transportation information providers, accommodation information providers, etc.). Specifically, it obtains the availability and prices of Shinkansen trains and hotels with hot springs.

[0472] 3. Travel plan generation phase

[0473] The server creates a travel plan based on the collected information. It automatically generates multiple plans and evaluates each plan based on factors such as cost, travel time, and accommodation ratings. In this step, the server compares multiple travel plans that best fit the user's criteria.

[0474] 4. Plan presentation phase

[0475] The server sends the generated itineraries to the terminal, which then visually displays the itineraries to the user. The displayed itineraries are easy for users to compare, and each plan's cost, travel time, and accommodation details are displayed in a list.

[0476] 5. Booking process phase

[0477] Once the user selects the desired travel plan, the terminal transmits the selection to the server, which then calls the external information provider again to confirm the selected Shinkansen and hotel reservations.

[0478] 6. Reservation confirmation and completion phase

[0479] The server retrieves the reservation confirmation and sends it to the terminal, which displays it to the user, informing them that their travel plans have been successfully completed.

[0480] Specific examples

[0481] For example, suppose a user logs into a website and enters the following information:

[0482] Budget: 100,000 yen

[0483] Travel dates: December 1, 2023 to December 5, 2023

[0484] Transportation: Shinkansen

[0485] Accommodation: Hot Spring Hotel

[0486] The device collects this information and sends it to a server, which then collects data such as bullet train schedules and availability of hotels with hot springs, and generates multiple travel plans.

[0487] for example,

[0488] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[0489] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[0490] The server sends these plans to the terminal, which displays them to the user. If the user selects Plan A, the terminal sends the information to the server, which then confirms the Shinkansen and hotel reservations. Finally, the reservation confirmation information is displayed on the terminal, letting the user know that their travel plan has been successfully completed.

[0491] The system configuration described above allows for consistent and efficient travel planning and booking.

[0492] The processing flow will be explained below.

[0493] Step 1:

[0494] A user opens a website or application and accesses a form to enter travel planning details.

[0495] Step 2:

[0496] The user inputs travel conditions such as budget, travel dates, transportation, desired accommodation, etc. The input data is collected by the terminal.

[0497] Step 3:

[0498] The device sends the collected travel condition data to the server in JSON format.

[0499] Step 4:

[0500] The server analyzes the received travel condition data, and based on the analyzed data, prepares to call an external information providing means for collecting information that matches the travel conditions.

[0501] Step 5:

[0502] The server uses external information providers (e.g., transportation information providers, accommodation information providers) to collect necessary travel-related data, such as the availability and prices of Shinkansen trains and hotels with hot springs.

[0503] Step 6:

[0504] The server integrates the collected data and automatically generates travel plans. Multiple travel plans are created and each plan is evaluated based on criteria such as cost, travel time, and accommodation ratings.

[0505] Step 7:

[0506] The server compares the generated travel plans and selects the optimal travel plan based on the comparison results.

[0507] Step 8:

[0508] The server sends the optimal travel plan to the device. The plan information is sent to the device in JSON format.

[0509] Step 9:

[0510] The terminal displays the received travel plans to the user. Details of multiple plans are listed and presented in a format that makes it easy for the user to compare them.

[0511] Step 10:

[0512] The user selects a desired travel plan, and the selected plan information is sent to the server by the terminal.

[0513] Step 11:

[0514] The server starts the reservation procedure for the selected travel plan, and confirms the reservation of the selected Shinkansen and hotel through an external information providing means.

[0515] Step 12:

[0516] The server acquires the reservation confirmation information, which is then sent to the terminal.

[0517] Step 13:

[0518] The terminal displays the reservation confirmation information to the user, who confirms that the travel plan has been successfully completed.

[0519] Through the above processing steps, a series of processes from travel planning to reservation completion can be efficiently executed.

[0520] Example 1

[0521] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0522] In conventional travel planning systems, after users input their travel requirements, they had to manually compare many options to find the best plan, which was time-consuming and labor-intensive. Furthermore, when comparing several plans, the detailed information was not visually centrally managed, which reduced user convenience. To solve these problems, a new system was needed.

[0523] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0524] In this invention, the server includes an interface means for users to input travel conditions, a processing means for automatically generating multiple travel plans based on the travel conditions, a display means for displaying the multiple travel plans, a reservation procedure means for reserving the travel plan selected by the user, a notification means for notifying the results of the reservation procedure, an information collection means for collecting transportation information and accommodation information via an external information providing means, a comparison means for comparing the generated travel plans based on cost and time criteria, and an optimization means for optimizing the multiple travel plans based on an evolutionary algorithm. This allows users to easily input travel conditions and compare, select, and reserve from a wide range of options in a unified manner.

[0525] The "interface means" is a means by which a user inputs travel conditions into the system.

[0526] The "processing means" is a means for automatically generating a plurality of travel plans based on travel conditions input by the user.

[0527] The "display means" is a means for visually displaying the generated travel plans to the user.

[0528] The "reservation procedure means" is a means by which a user confirms the travel plan selected and completes the reservation procedure.

[0529] "Notification means" is a means for notifying the user of the results of the reservation procedure.

[0530] "External information providing means" refers to a means for receiving data such as transportation information and accommodation information from outside.

[0531] "Information gathering means" refers to a means for gathering necessary transportation information and accommodation information via external information providing means.

[0532] The "comparison means" is a means for comparing the generated travel plans based on cost and time.

[0533] The "optimization means" is a means for optimizing multiple travel plans based on an evolutionary algorithm and providing the optimal plan to the user.

[0534] The "graphical user interface means" is a means for visually displaying multiple travel plans in an easy-to-understand manner, allowing users to operate the system intuitively.

[0535] The "update means" is a means for automatically updating the reservation information confirmed by the reservation procedure means.

[0536] The present invention relates to a system that handles everything from travel planning to reservations all at once, and in particular to a system that automatically generates an optimal travel plan and smoothly processes reservations by the user simply by inputting conditions such as budget, schedule, desired means of transportation, and accommodations.

[0537] User Input Phase

[0538] Users first access a website or application and enter details of their travel plans, including budget, travel dates, mode of transportation (e.g., plane, car, train), and accommodation preferences (e.g., with hot springs, near a specific station, etc.). This data is collected by the device and sent to the server.

[0539] Data Collection Phase

[0540] The server receives the travel condition data sent by the user. The server collects the necessary information through external information providers (e.g., transportation information providers, accommodation information providers, etc.). Specifically, it obtains the availability and prices of Shinkansen trains and hotels with hot springs.

[0541] Travel plan generation phase

[0542] The server creates a travel plan based on the collected information. It automatically generates multiple plans and evaluates each plan based on factors such as cost, travel time, and accommodation ratings. In this step, the server uses a generative AI model to compare multiple travel plans that best fit the user's requirements.

[0543] Plan presentation phase

[0544] The server sends the generated itineraries to the terminal, which then visually displays the itineraries to the user. The displayed itineraries are easy for users to compare, and each plan's cost, travel time, and accommodation details are displayed in a list.

[0545] Booking process phase

[0546] Once the user selects the desired travel plan, the terminal transmits the selection to the server, which then calls the external information provider again to confirm the selected Shinkansen and hotel reservations.

[0547] Reservation confirmation and completion phase

[0548] The server retrieves the reservation confirmation and sends it to the terminal, which displays it to the user, informing them that their travel plans have been successfully completed.

[0549] Specific examples

[0550] For example, say a user logs into a website and enters the following information:

[0551] Budget: 100,000 yen

[0552] Travel dates: December 1, 2023 to December 5, 2023

[0553] Transportation: Shinkansen

[0554] Accommodation: Hot Spring Hotel

[0555] The device collects this information and sends it to a server, which then collects data such as bullet train schedules and availability of hotels with hot springs, and generates multiple travel plans.

[0556] for example,

[0557] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[0558] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[0559] The server sends these plans to the terminal, which displays them to the user. If the user selects Plan A, the terminal sends the information to the server, which then confirms the Shinkansen and hotel reservations. Finally, the reservation confirmation information is displayed on the terminal, letting the user know that their travel plan has been successfully completed.

[0560] Example prompts to be input to the generative AI model

[0561] "Generate the best travel plan based on your budget, mode of transportation, and other factors."

[0562] "Please suggest multiple travel plans based on the following criteria: budget \100,000, travel dates December 1, 2023 to December 5, 2023, Shinkansen, hotel with hot spring."

[0563] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0564] Step 1: User Input Phase

[0565] Input: User's travel conditions (budget, travel dates, transportation, accommodation preferences)

[0566] Output: Collected travel condition data

[0567] Specific behavior:

[0568] A user accesses a website or application and enters details of their travel plans. They select their travel dates in a calendar format and enter their budget in text boxes. They select transportation and accommodation options using drop-down lists. After completing all the input fields, the user clicks the "Submit" button. This action causes the device to collect the data and send it to the server.

[0569] Step 2: Data collection phase

[0570] Input: Travel condition data

[0571] Output: Transportation information, accommodation information

[0572] Specific behavior:

[0573] The server receives the travel condition data that has been sent. It then collects the necessary information through external information providers (e.g., transportation information providers, accommodation information providers). The server sends an API request to obtain the availability and prices of Shinkansen trains and hotels with hot springs. The API response is returned to the server, where the collected information is integrated.

[0574] Step 3: Travel plan generation phase

[0575] Input: Collected transportation information, accommodation information, and travel condition data

[0576] Output: Multiple itineraries

[0577] Specific behavior:

[0578] The server uses a generative AI model to generate multiple travel plans based on the collected data and the user's travel conditions. Each plan is evaluated taking into account factors such as cost, travel time, and accommodation ratings. The generated plans are temporarily stored in the server's internal database.

[0579] Step 4: Plan presentation phase

[0580] Input: Multiple itineraries

[0581] Output: Travel plan information for display

[0582] Specific behavior:

[0583] The server sends the generated travel plans to the device, which receives data for visually displaying these plans. The user can compare the details of the plans through the device, and the plan costs, travel times, and accommodation details are displayed in a list.

[0584] Step 5: Booking process phase

[0585] Input: User selected itinerary

[0586] Output: Confirmed reservation information

[0587] Specific behavior:

[0588] Once the user selects their desired travel plan, the device sends the selection information to the server. The server then calls the external information provider again to confirm the selected Shinkansen and hotel reservations. The necessary reservation information is sent and received via the API.

[0589] Step 6: Reservation confirmation and completion phase

[0590] Input: Confirmed reservation information

[0591] Output: Reservation confirmation information, notifications

[0592] Specific behavior:

[0593] The server retrieves the reservation confirmation and sends it to the terminal, which receives it and displays it to the user. The reservation confirmation includes details such as the reservation number, dates, and hotel address, and also displays the message "Your reservation is complete."

[0594] (Application example 1)

[0595] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0596] Conventional food delivery planning systems have the drawback of requiring users to input their preferences through a complex interface, making it difficult to compare multiple options and select the optimal plan. When users have specific requests, there is a need for a system that can generate the optimal plan and quickly complete the ordering process.

[0597] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0598] In this invention, the server includes input means for the user to input conditions, processing means for automatically generating multiple delivery plans based on the conditions, display means for displaying the multiple delivery plans, ordering means for ordering the delivery plan selected by the user, and notification means for notifying the result of the ordering process. This allows the optimal delivery plan based on the conditions desired by the user to be generated quickly and efficiently, and makes the ordering process even easier.

[0599] "Input means" refers to a device or interface for inputting desired conditions or information by a user.

[0600] The "processing means" is a device or software for executing a specific process based on data received from the input means.

[0601] The "display means" is a device or interface for visually presenting the generated plans and information to the user.

[0602] "Order processing means" refers to a device or software that allows a user to confirm the plan selected and execute an order.

[0603] "Notification means" refers to a device or interface for notifying users of the results of the order process and updated information.

[0604] "Information collection means" refers to a device or software for acquiring necessary data from an external information providing service.

[0605] A "comparison tool" is a device or software used to evaluate and compare multiple plans based on specific criteria.

[0606] This invention is a system that automatically generates an optimal delivery plan based on the conditions entered by the user and performs the entire ordering process. This system uses the following hardware and software.

[0607] Hardware and software used:

[0608] Server: AWS EC2 (data processing)

[0609] Database: AWS RDS (data storage)

[0610] API: Google Places API, Yelp API (collecting restaurant information)

[0611] Frontend: React Native (smartphone app)

[0612] Overall system processing overview:

[0613] 1. Data entry method

[0614] Users enter their order requirements through a smartphone app, including budget, desired delivery time, type of food, and specific requests (e.g., vegetarian). This information is sent from the user's device to the server.

[0615] 2. Information gathering methods

[0616] The server uses external information providers (such as Google Places API and Yelp API) to collect information about restaurants and delivery services based on the entered conditions, including restaurant menus, prices, ratings, and available delivery times.

[0617] 3. Plan Generation Method

[0618] The server automatically generates multiple delivery plans based on the collected information, taking into account criteria such as cost, delivery time, and review ratings.

[0619] 4. Plan display method

[0620] The generated multiple delivery plans are sent to the device and displayed on the user's smartphone app, where they are presented in a format that makes it easy to compare plans by cost and delivery time.

[0621] 5. Order Processing Methods

[0622] Once the user selects the desired plan, the selection information is sent to the server, which then confirms the order with the delivery service and completes the order process.

[0623] 6. Means of notification

[0624] The server receives order confirmation information and estimated delivery time and sends it to the user's device. The user can then check the order confirmation and estimated delivery time on their smartphone app.

[0625] Examples:

[0626] For example, suppose a user enters information into a smartphone app under the following conditions:

[0627] Budget: 2,000 yen

[0628] Delivery time: 18:00~19:00

[0629] Cuisine type: Japanese

[0630] Specific requests: Vegetarian

[0631] Based on this information, the server uses the Google Places API and Yelp API to gather relevant restaurant information and generate a delivery plan like this:

[0632] Plan A: Japanese restaurant (¥1,800) + delivery fee (¥200) + delivery time: 18:30

[0633] Plan B: New Japanese restaurant (¥1,500) + Delivery fee (¥300) + Delivery time: 18:45

[0634] These plans are displayed on the user's smartphone app, and once the user selects the desired plan and confirms the order, the results are notified.

[0635] Example prompts to be input to the generative AI model:

[0636] "I'd like Japanese food delivered by 8 PM for under 2,000 yen. I'd like to choose a vegetarian option. Please prioritize restaurants with high reviews."

[0637] This allows users to quickly and efficiently generate the optimal delivery plan based on their specific preferences and easily complete the ordering process.

[0638] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0639] Step 1:

[0640] The user launches the smartphone app and inputs conditions such as budget, delivery time, desired type of food, specific requests, etc. The following information is then sent to the server via the input means:

[0641] Input: Budget (e.g., ¥2,000), Delivery time (e.g., 18:00-19:00), Cuisine type (e.g., Japanese), Specific requests (e.g., Vegetarian)

[0642] Output: The condition is sent to the server

[0643] Step 2:

[0644] The server calls external information providers (e.g., Google Places API, Yelp API) based on the received condition data to obtain related restaurant information and delivery service information. Specifically, it makes an API call and collects data that matches the conditions.

[0645] Input: User condition data

[0646] Output: Restaurant information (e.g., menu, prices, ratings) and delivery service information (e.g., delivery times, delivery fees)

[0647] Step 3:

[0648] The server automatically generates multiple delivery plans based on the collected restaurant and delivery service information. Specifically, it evaluates each combination of restaurant and delivery service based on evaluation criteria and creates plans.

[0649] Input: Collected restaurant and delivery service information

[0650] Output: Multiple delivery plans (e.g., Plan A: Japanese restaurant (¥1,800) + delivery fee (¥200), Plan B: New Japanese restaurant (¥1,500) + delivery fee (¥300))

[0651] Step 4:

[0652] The server sends the generated multiple delivery plans to the user's device and displays them on the user's smartphone app. Specifically, it formats the data to display the plans in a format that makes it easy to visually compare them.

[0653] Input: Multiple delivery plans

[0654] Output: A formatted list of plans

[0655] Step 5:

[0656] The user selects the desired delivery plan on the smartphone app. Once selected, the information is sent to the server. Specifically, data is sent to reflect the user's selection on the server.

[0657] Input: User's selected plan

[0658] Output: Selection information is sent to the server

[0659] Step 6:

[0660] The server confirms the order with the delivery service based on the received selection information. Specifically, it calls the delivery service's API and performs order confirmation processing.

[0661] Input: User selection information

[0662] Output: Order confirmation information

[0663] Step 7:

[0664] The server receives the order confirmation information and estimated delivery time and sends it to the user's device. The user can then confirm the order confirmation and estimated delivery time on the smartphone app. Specifically, the confirmation information is sent via a notification means.

[0665] Input: Order confirmation information and estimated delivery time

[0666] Output: A confirmation message is displayed on the user's terminal.

[0667] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0668] This invention relates to a system that combines a system that handles everything from travel planning to booking with an emotion engine that recognizes the user's emotions, and is able to propose optimal travel plans taking into account the user's emotional state.By analyzing the user's emotional information in real time and adjusting and proposing travel plans based on this, it is possible to provide travel plans that will provide a higher level of satisfaction.

[0669] This system is configured as follows:

[0670] 1. User Input Phase

[0671] A user opens a website or application and enters details of their travel plans, including budget, travel dates, mode of transportation (e.g., plane, car, train), and accommodation preferences (e.g., with hot springs, near a specific station), and this data is collected by the device and sent to the server.

[0672] 2. Data Collection Phase

[0673] The server receives the travel condition data sent by the user. The server collects the necessary travel-related data through external information providers (e.g., transportation information providers, accommodation information providers). For example, it obtains information such as the availability and price of Shinkansen trains and the availability and price of hotels with hot springs.

[0674] 3. Travel plan generation phase

[0675] The server integrates the collected data and automatically generates travel plans. Multiple travel plans are created and each plan is evaluated based on criteria such as cost, travel time, and accommodation ratings.

[0676] 4. Sentiment Analysis Phase

[0677] The emotion engine analyzes the user's input data and emotional information such as the user's facial expressions and tone of voice. This analyzed emotional data is reflected in the travel plan generation, and the plan is adjusted to match the user's emotional state.

[0678] 5. Plan presentation phase

[0679] The server generates multiple travel plans and sends them to the device. The plan information, which also takes into account the results of sentiment analysis, is displayed to the user on the device. The details of the multiple plans are displayed in a list format that makes it easy for the user to compare and consider them.

[0680] 6. Real-time feedback phase

[0681] The emotion engine monitors the user's emotional state in real time and provides immediate feedback when selecting a desired travel plan. For example, if the engine detects anxiety or dissatisfaction with the user's chosen plan, it will present other options.

[0682] 7. Booking Process Phase

[0683] After the user selects the desired travel plan, the selected information is sent from the terminal to the server, which then calls the external information providing means again to confirm the selected Shinkansen and hotel reservations.

[0684] 8. Reservation confirmation and completion phase

[0685] The server retrieves the reservation confirmation information and sends it to the terminal, which displays it to the user, indicating that the travel plan has been successfully completed.

[0686] Specific examples

[0687] For example, suppose a user logs into a website and enters the following information:

[0688] Budget: 100,000 yen

[0689] Travel dates: December 1, 2023 to December 5, 2023

[0690] Transportation: Shinkansen

[0691] Accommodation: Hot Spring Hotel

[0692] The device collects this information and sends it to a server, which then collects data such as bullet train schedules and availability of hotels with hot springs, and generates multiple travel plans.

[0693] for example,

[0694] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[0695] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[0696] At the same time, the emotion engine analyzes the user's emotional state from their facial expressions and tone of voice, and if the user is looking to relax, it makes adjustments such as adding hotels with hot springs to the recommendations.

[0697] The server sends these adjusted plans to the device, which displays them to the user. Real-time feedback suggests alternative plans if the user expresses anxiety or dissatisfaction when selecting Plan A.

[0698] Finally, the user selects the best plan and the reservation is confirmed. The reservation confirmation information is displayed on the terminal, and the user can confirm that the travel plan has been successfully completed.

[0699] The system configuration described above makes it possible to consistently and efficiently carry out travel planning and booking while taking emotional information into consideration.

[0700] The processing flow will be explained below.

[0701] Step 1:

[0702] A user opens a website or application and accesses a form to enter travel planning details.

[0703] Step 2:

[0704] The user inputs travel conditions such as budget, travel schedule, transportation, desired accommodation, etc. The input data is collected by the terminal and sent to the server.

[0705] Step 3:

[0706] The server analyzes the received travel condition data, and based on the analyzed data, prepares to call an external information providing means for collecting information that matches the travel conditions.

[0707] Step 4:

[0708] The server uses external information providers (e.g., transportation information providers, accommodation information providers) to collect necessary travel-related data, such as the availability and prices of Shinkansen trains and hotels with hot springs.

[0709] Step 5:

[0710] The server integrates the collected data and automatically generates travel plans. Multiple travel plans are created and each plan is evaluated based on criteria such as cost, travel time, and accommodation ratings.

[0711] Step 6:

[0712] The emotion engine collects user input data and emotion information such as the user's facial expressions and tone of voice, and analyzes the user's emotional state.

[0713] Step 7:

[0714] The server then adjusts the travel plan based on the results of the sentiment analysis. For example, if the user wants a relaxing trip, it will prioritize accommodations with quiet environments.

[0715] Step 8:

[0716] The server sends the adjusted travel plans to the terminal, which displays them to the user. The plan details are displayed in a list format that makes it easy for the user to compare them.

[0717] Step 9:

[0718] The emotion engine monitors the user's emotional state in real time, analyzing the emotional state and providing immediate feedback when the user selects a desired travel plan from the presented plans.

[0719] Step 10:

[0720] The user selects a desired travel plan, and the selected plan information is sent to the server by the terminal.

[0721] Step 11:

[0722] The server starts the reservation procedure for the selected travel plan, and confirms the reservation of the selected Shinkansen and hotel through an external information providing means.

[0723] Step 12:

[0724] The server receives the reservation confirmation information and sends it to the terminal, which displays it to the user.

[0725] Step 13:

[0726] The user checks the reservation confirmation information and finds out that the travel plan has been successfully completed. Through the above processing steps, a series of processes from travel planning to reservation completion is efficiently executed, taking into account the user's emotional information.

[0727] Example 2

[0728] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0729] Conventional travel planning systems only automatically generate travel plans based on user-entered conditions, but are unable to consider the user's emotional state, making it difficult to propose highly satisfying travel plans. Furthermore, there have been few systems that can analyze the user's emotions in real time and adjust or propose travel plans based on those emotions.

[0730] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0731] In this invention, the server includes an input means for a user to input travel conditions, a generation means for automatically generating multiple travel plans based on the travel conditions, and an emotion analysis means for adjusting the travel plans based on the user's emotional state, thereby making it possible to propose highly satisfying travel plans that reflect the user's emotional state in real time.

[0732] "Input means" refers to a means by which a user inputs travel conditions.

[0733] The "generation means" is a means for automatically generating multiple itineraries based on input travel conditions.

[0734] The "emotion analysis means" is a means for analyzing the user's emotional state and adjusting the travel plan based on that.

[0735] The "display means" is a means for displaying the generated itineraries to the user.

[0736] The "reservation procedure means" refers to the procedure means for reserving the travel plan selected by the user.

[0737] The "notification means" is a means for notifying the user of the results of the reservation procedure.

[0738] "Data collection means" refers to a means for collecting transportation information and accommodation information via external data providing means.

[0739] A "comparison tool" is a tool for comparing multiple travel plans based on cost and time criteria.

[0740] This invention is a system that handles everything from travel planning to booking, and is particularly equipped with an emotion engine that recognizes the user's emotions. This system can propose optimal travel plans taking into account the user's emotional state.

[0741] Specific configuration

[0742] Hardware and Software

[0743] The system uses the following hardware and software:

[0744] Server: Data processing, travel plan generation, and sentiment analysis.

[0745] Terminal: A PC or smartphone that performs user input and displays plans.

[0746] Emotion engine: Analyzes user emotions in real time and adjusts and suggests plans.

[0747] The software used includes:

[0748] Web application: Provides an interface for users to input their travel plans.

[0749] External API: Used to collect data to obtain information on transportation and accommodation.

[0750] Generative AI models: Used for itinerary generation and sentiment analysis.

[0751] How to use

[0752] 1. User Input

[0753] A user opens a website or application and enters the following information:

[0754] budget

[0755] travel itinerary

[0756] Transportation (e.g., bullet train, car, airplane, etc.)

[0757] Accommodation preferences (e.g. with hot springs, specific location, etc.)

[0758] The terminal collects this information and sends it to the server.

[0759] 2. Data Collection

[0760] The server receives the input data and collects information on Shinkansen trains and accommodations through external data providers, specifically information on Shinkansen availability and prices, availability and prices of hotels with hot springs, etc.

[0761] 3. Travel plan generation

[0762] The server generates multiple travel plans based on the collected data, and the plans are scored based on criteria such as cost, travel time, and accommodation ratings.

[0763] 4. Emotion analysis

[0764] The emotion engine analyzes the user's emotional state based on input data, facial expressions, tone of voice, etc. The travel plan is then adjusted based on the analyzed data. For example, if the user is looking for relaxation, hotels with hot springs will be suggested first.

[0765] 5. Plan presentation

[0766] The server sends the generated travel plan to the terminal, which then displays it to the user. The plan information is presented in an easy-to-compare format.

[0767] 6. Real-time feedback

[0768] The emotion engine monitors users' emotions in real time and provides immediate feedback when selecting travel plans, suggesting alternative plans if anxiety or dissatisfaction is detected.

[0769] 7. Reservation Procedure

[0770] After the user selects the desired travel plan, the information is sent to the server, which then calls the external data providing means again to confirm the selected Shinkansen and hotel reservations.

[0771] 8. Reservation Confirmation

[0772] The server receives the reservation confirmation information and sends it to the terminal, which displays the reservation completion information to the user.

[0773] Specific examples

[0774] For example, if a user logs into a website and enters the following information:

[0775] Budget: 100,000 yen

[0776] Travel dates: December 1, 2023 to December 5, 2023

[0777] Transportation: Shinkansen

[0778] Accommodation: Hot Spring Hotel

[0779] The device collects information and sends it to the server. The server collects data on Shinkansen trains and hotels with hot springs, and generates multiple travel plans:

[0780] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[0781] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[0782] The emotion engine analyzes the user's emotions and recommends hotels with hot springs if they are looking for relaxation. The server sends the adjusted plan to the device and displays it to the user. Real-time feedback allows the user to select the best plan.

[0783] Finally, the user selects a plan and the reservation is confirmed. Reservation confirmation information is displayed on the terminal, and the user confirms that the travel plan has been successfully completed.

[0784] Generative AI model prompt example

[0785] "Please tell us about your next trip. Explain your budget, travel dates, transportation, and accommodation preferences. Also, tell us how you want to feel during your trip."

[0786] The system configuration described above makes it possible to consistently and efficiently carry out travel planning and booking while taking emotional information into consideration.

[0787] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0788] Step 1:

[0789] User Input Phase

[0790] A user opens a website or application and enters details of their travel plans (budget, travel dates, transportation options, and preferred accommodations). The data entered by the user is collected by the device and sent to the server. The server then receives the travel conditions data. As a specific example, a user enters information such as a budget of 100,000 yen, travel dates from December 1st to 5th, 2023, travel by Shinkansen, and a hotel with a hot spring.

[0791] Input: Travel conditions entered by the user (budget, travel dates, transportation, accommodation preferences)

[0792] Output: Travel condition data sent to the server

[0793] Step 2:

[0794] Data Collection Phase

[0795] After the server receives the travel condition data, it collects transportation information and accommodation information via external data provision means. As a specific example, the server calls the APIs of a Shinkansen train operation information service and an accommodation information service to obtain the availability and prices of Shinkansen trains and hotels with hot springs.

[0796] Input: Travel condition data

[0797] Output: Transportation and accommodation information

[0798] Step 3:

[0799] Travel plan generation phase

[0800] The server automatically generates multiple travel plans based on the data collected. The server scores the generated plans based on criteria such as cost, travel time, and accommodation ratings. Specifically, the server combines Shinkansen prices and hotel prices to calculate the cost of the travel plan.

[0801] Input: Transportation and accommodation information

[0802] Output: Multiple itineraries

[0803] Step 4:

[0804] Sentiment Analysis Phase

[0805] The emotion engine analyzes the user's input data, facial expressions, and tone of voice to extract the user's emotional state. Based on this emotional data, the server adjusts the travel plan. For example, if the user is looking for relaxation, it will emphasize and suggest hotels with hot springs.

[0806] Input: User's emotional information

[0807] Output: Adjusted itinerary

[0808] Step 5:

[0809] Plan presentation phase

[0810] The server sends the generated travel plan to the terminal, which displays it to the user. The terminal displays a list of details of multiple travel plans so that the user can easily compare them.

[0811] Input: Adjusted travel plans

[0812] Output: The itinerary displayed to the user

[0813] Step 6:

[0814] Real-time feedback phase

[0815] The emotion engine monitors the user's emotional state in real time and provides immediate feedback when selecting a travel plan, for example suggesting other options if the user expresses anxiety or dissatisfaction with the plan they have chosen.

[0816] Input: Real-time user emotion information

[0817] Output: Alternative suggestions

[0818] Step 7:

[0819] Booking process phase

[0820] When the user selects the desired travel plan, the information is sent from the terminal to the server, which then calls the external data providing means again to confirm the reservation of the selected Shinkansen and hotel.

[0821] Input: User selected travel plan

[0822] Output: Confirmed reservation information

[0823] Step 8:

[0824] Reservation confirmation and completion phase

[0825] The server retrieves the reservation confirmation information and sends it to the terminal, which displays the reservation confirmation information to the user, informing them that their travel plans have been successfully completed.

[0826] Input: Confirmed reservation information

[0827] Output: Booking confirmation information displayed to the user

[0828] (Application example 2)

[0829] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0830] In conventional autonomous vehicles, operation plans are determined without taking the driver's emotions into consideration, which increases driver stress and anxiety and reduces driving comfort. Furthermore, there was no system that provided real-time feedback to the driver based on their emotions when proposing a plan or during operation. Therefore, in order to provide a safer and more comfortable driving environment, a system that analyzes the driver's emotions in real time and adjusts operation plans based on that information is needed.

[0831] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes emotion analysis means for analyzing the user's emotion information, plan adjustment means for adjusting the travel plan based on the emotion information, and feedback means for presenting the adjusted plan. This makes it possible to analyze the driver's emotion information in real time and adjust or propose a trip plan based on it.

[0832] "User" refers to the general public who uses the system to input travel conditions.

[0833] "Interface means" refers to an input device or software that allows a user to input travel conditions.

[0834] "Processing means" refers to a device or software that automatically generates multiple travel plans based on input travel conditions.

[0835] "Display means" refers to a device or software that visually presents the generated travel plans to the user.

[0836] "Reservation procedure means" refers to the device or software used to actually reserve the travel plan selected by the user.

[0837] "Notification means" refers to devices or software for notifying users of the results of their reservation procedures.

[0838] "Emotion analysis means" refers to devices or software for analyzing users' emotional information.

[0839] "Plan adjustment means" refers to a device or software for adjusting a travel plan based on the analyzed emotional information.

[0840] "Feedback means" refers to a device or software for presenting the adjusted plan to the user and providing feedback in real time.

[0841] "External information providing means" refers to devices and services for collecting external information such as transportation information and accommodation information.

[0842] This invention combines an emotion engine with a system that handles everything from user travel planning to reservations, and is a specific form for providing safe and comfortable driving plans for drivers of self-driving vehicles. The details are provided below.

[0843] 1. User input phase

[0844] Users use an application installed on their smartphone or head-mounted display to input travel conditions such as destination, travel time, desired route, etc. This allows the user's desired travel plan to be clearly collected.

[0845] 2. Data Collection Phase

[0846] The server collects information on traffic, weather, and congestion at destinations through external information providers, obtaining the latest data in real time via a REST API.

[0847] 3. Operation plan generation phase

[0848] Based on the collected data, the server generates multiple operation plans using Python scripts and machine learning algorithms (e.g., optimization algorithms) and evaluates each one.

[0849] 4. Sentiment Analysis Phase

[0850] Using the camera and microphone installed on the smartphone or head-mounted display, and using face recognition API and voice recognition API, emotional information is analyzed in real time from the user's facial expressions and tone of voice, thereby obtaining emotional information about the user.

[0851] 5. Plan presentation phase

[0852] The server presents the user with multiple operation plans that take emotion information into consideration through a display means, thereby providing the user with an intuitive interface for selecting the optimal plan based on emotion.

[0853] 6. Real-time feedback phase

[0854] Based on the analyzed emotional information, the server adjusts and re-presents the operation plan according to the user's real-time emotions through feedback means. For example, if the server detects anxiety or dissatisfaction with the plan selected by the user, it will present an alternative plan.

[0855] 7. Operational Phase

[0856] After the user selects the optimal plan, the app sends driving instructions to the autonomous vehicle's onboard computer, starting a safe and comfortable journey.

[0857] Specific examples

[0858] User input: Destination "Tokyo", travel time "within 2 hours"

[0859] Prompt (when inputting to a generative AI model):

[0860] Suggest a route that will make the driver feel safe and relaxed. Emotions are a little tense right now. Choose a safe and efficient route within the desired timeframe.

[0861] As described above, this invention is a system that analyzes user emotions in real time and adjusts and proposes operation plans based on that analysis, providing a safe and comfortable operating environment for self-driving vehicles.

[0862] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0863] Step 1:

[0864] The terminal provides an interface where users can input travel conditions (destination, travel time, desired route, etc.) using a smartphone or head-mounted display. The input information is sent from the terminal to the server. The input is the user's desired conditions, and the output is data sent to the server.

[0865] Step 2:

[0866] The server collects traffic information, weather information, and congestion information at the destination via external information providing means based on the travel conditions sent from the terminal. The collected information is stored in the server. The input is the travel conditions and external information, and the output is the storage of the collected data.

[0867] Step 3:

[0868] The server generates a trip plan using the collected data. It uses Python scripts and machine learning algorithms to apply optimization algorithms to create multiple trip plans. The generated trip plans are stored on the server. The input is the collected data and the algorithm, and the output is the trip plan.

[0869] Step 4:

[0870] The device uses the camera and microphone of the smartphone or head-mounted display to analyze the user's face and voice, and analyzes facial expressions and tone of voice in real time using an emotion analysis API. The analysis results are sent from the device to a server. The input is facial expression and voice data, and the output is the emotion analysis results.

[0871] Step 5:

[0872] The server adjusts the previously created operation plan based on the emotion analysis results. Using the plan adjustment means, it selects the operation plan that best suits the user's emotional state and creates an adjusted operation plan. The input is the emotion analysis results and the operation plan, and the output is the adjusted operation plan.

[0873] Step 6:

[0874] The terminal presents the adjusted operation plans from the server to the user. The plans are displayed in a visually easy-to-compare format to make it easier for the user to select a plan. The input is the adjusted operation plan, and the output is the plan presented to the user.

[0875] Step 7:

[0876] The server provides real-time feedback on the trip plan selected by the user, detects the user's anxiety and dissatisfaction, and presents alternative plans as necessary. The input is the user's preferences and emotional state, and the output is the presentation of alternative plans.

[0877] Step 8:

[0878] The terminal confirms the user's final selected trip plan and sends instructions to the autonomous vehicle's onboard computer, which then executes the selected trip plan. The input is the final selected trip plan, and the output is instructions to the onboard computer.

[0879] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0880] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0881] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0882] [Third embodiment]

[0883] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0884] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0885] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0886] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0887] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0888] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0889] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0890] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0891] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0892] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0893] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0894] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0895] The present invention relates to a system that handles everything from travel planning to reservations all at once, and in particular to a system that automatically generates an optimal travel plan and smoothly processes reservations by the user simply by inputting conditions such as budget, schedule, desired means of transportation, and accommodations.

[0896] This system is configured as follows:

[0897] 1. User Input Phase

[0898] Users first access a website or application and enter details of their travel plans, including budget, travel dates, mode of transportation (e.g., plane, car, train), and accommodation preferences (e.g., with hot springs, near a specific station, etc.). This data is collected by the device and sent to the server.

[0899] 2. Data Collection Phase

[0900] The server receives the travel condition data sent by the user. The server collects the necessary information through external information providers (e.g., transportation information providers, accommodation information providers, etc.). Specifically, it obtains the availability and prices of Shinkansen trains and hotels with hot springs.

[0901] 3. Travel plan generation phase

[0902] The server creates a travel plan based on the collected information. It automatically generates multiple plans and evaluates each plan based on factors such as cost, travel time, and accommodation ratings. In this step, the server compares multiple travel plans that best fit the user's criteria.

[0903] 4. Plan presentation phase

[0904] The server sends the generated itineraries to the terminal, which then visually displays the itineraries to the user. The displayed itineraries are easy for users to compare, and each plan's cost, travel time, and accommodation details are displayed in a list.

[0905] 5. Booking process phase

[0906] Once the user selects the desired travel plan, the terminal transmits the selection to the server, which then calls the external information provider again to confirm the selected Shinkansen and hotel reservations.

[0907] 6. Reservation confirmation and completion phase

[0908] The server retrieves the reservation confirmation and sends it to the terminal, which displays it to the user, informing them that their travel plans have been successfully completed.

[0909] Specific examples

[0910] For example, suppose a user logs into a website and enters the following information:

[0911] Budget: 100,000 yen

[0912] Travel dates: December 1, 2023 to December 5, 2023

[0913] Transportation: Shinkansen

[0914] Accommodation: Hot Spring Hotel

[0915] The device collects this information and sends it to a server, which then collects data such as bullet train schedules and availability of hotels with hot springs, and generates multiple travel plans.

[0916] for example,

[0917] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[0918] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[0919] The server sends these plans to the terminal, which displays them to the user. If the user selects Plan A, the terminal sends the information to the server, which then confirms the Shinkansen and hotel reservations. Finally, the reservation confirmation information is displayed on the terminal, letting the user know that their travel plan has been successfully completed.

[0920] The system configuration described above allows for consistent and efficient travel planning and booking.

[0921] The processing flow will be explained below.

[0922] Step 1:

[0923] A user opens a website or application and accesses a form to enter travel planning details.

[0924] Step 2:

[0925] The user inputs travel conditions such as budget, travel dates, transportation, desired accommodation, etc. The input data is collected by the terminal.

[0926] Step 3:

[0927] The device sends the collected travel condition data to the server in JSON format.

[0928] Step 4:

[0929] The server analyzes the received travel condition data, and based on the analyzed data, prepares to call an external information providing means for collecting information that matches the travel conditions.

[0930] Step 5:

[0931] The server uses external information providers (e.g., transportation information providers, accommodation information providers) to collect necessary travel-related data, such as the availability and prices of Shinkansen trains and hotels with hot springs.

[0932] Step 6:

[0933] The server integrates the collected data and automatically generates travel plans. Multiple travel plans are created and each plan is evaluated based on criteria such as cost, travel time, and accommodation ratings.

[0934] Step 7:

[0935] The server compares the generated travel plans and selects the optimal travel plan based on the comparison results.

[0936] Step 8:

[0937] The server sends the optimal travel plan to the device. The plan information is sent to the device in JSON format.

[0938] Step 9:

[0939] The terminal displays the received travel plans to the user. Details of multiple plans are listed and presented in a format that makes it easy for the user to compare them.

[0940] Step 10:

[0941] The user selects a desired travel plan, and the selected plan information is sent to the server by the terminal.

[0942] Step 11:

[0943] The server starts the reservation procedure for the selected travel plan, and confirms the reservation of the selected Shinkansen and hotel through an external information providing means.

[0944] Step 12:

[0945] The server acquires the reservation confirmation information, which is then sent to the terminal.

[0946] Step 13:

[0947] The terminal displays the reservation confirmation information to the user, who confirms that the travel plan has been successfully completed.

[0948] Through the above processing steps, a series of processes from travel planning to reservation completion can be efficiently executed.

[0949] Example 1

[0950] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0951] In conventional travel planning systems, after users input their travel requirements, they had to manually compare many options to find the best plan, which was time-consuming and labor-intensive. Furthermore, when comparing several plans, the detailed information was not visually centrally managed, which reduced user convenience. To solve these problems, a new system was needed.

[0952] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0953] In this invention, the server includes an interface means for users to input travel conditions, a processing means for automatically generating multiple travel plans based on the travel conditions, a display means for displaying the multiple travel plans, a reservation procedure means for reserving the travel plan selected by the user, a notification means for notifying the results of the reservation procedure, an information collection means for collecting transportation information and accommodation information via an external information providing means, a comparison means for comparing the generated travel plans based on cost and time criteria, and an optimization means for optimizing the multiple travel plans based on an evolutionary algorithm. This allows users to easily input travel conditions and compare, select, and reserve from a wide range of options in a unified manner.

[0954] The "interface means" is a means by which a user inputs travel conditions into the system.

[0955] The "processing means" is a means for automatically generating a plurality of travel plans based on travel conditions input by the user.

[0956] The "display means" is a means for visually displaying the generated travel plans to the user.

[0957] The "reservation procedure means" is a means by which a user confirms the travel plan selected and completes the reservation procedure.

[0958] "Notification means" is a means for notifying the user of the results of the reservation procedure.

[0959] "External information providing means" refers to a means for receiving data such as transportation information and accommodation information from outside.

[0960] "Information gathering means" refers to a means for gathering necessary transportation information and accommodation information via external information providing means.

[0961] The "comparison means" is a means for comparing the generated travel plans based on cost and time.

[0962] The "optimization means" is a means for optimizing multiple travel plans based on an evolutionary algorithm and providing the optimal plan to the user.

[0963] The "graphical user interface means" is a means for visually displaying multiple travel plans in an easy-to-understand manner, allowing users to operate the system intuitively.

[0964] The "update means" is a means for automatically updating the reservation information confirmed by the reservation procedure means.

[0965] The present invention relates to a system that handles everything from travel planning to reservations all at once, and in particular to a system that automatically generates an optimal travel plan and smoothly processes reservations by the user simply by inputting conditions such as budget, schedule, desired means of transportation, and accommodations.

[0966] User Input Phase

[0967] Users first access a website or application and enter details of their travel plans, including budget, travel dates, mode of transportation (e.g., plane, car, train), and accommodation preferences (e.g., with hot springs, near a specific station, etc.). This data is collected by the device and sent to the server.

[0968] Data Collection Phase

[0969] The server receives the travel condition data sent by the user. The server collects the necessary information through external information providers (e.g., transportation information providers, accommodation information providers, etc.). Specifically, it obtains the availability and prices of Shinkansen trains and hotels with hot springs.

[0970] Travel plan generation phase

[0971] The server creates a travel plan based on the collected information. It automatically generates multiple plans and evaluates each plan based on factors such as cost, travel time, and accommodation ratings. In this step, the server uses a generative AI model to compare multiple travel plans that best fit the user's requirements.

[0972] Plan presentation phase

[0973] The server sends the generated itineraries to the terminal, which then visually displays the itineraries to the user. The displayed itineraries are easy for users to compare, and each plan's cost, travel time, and accommodation details are displayed in a list.

[0974] Booking process phase

[0975] Once the user selects the desired travel plan, the terminal transmits the selection to the server, which then calls the external information provider again to confirm the selected Shinkansen and hotel reservations.

[0976] Reservation confirmation and completion phase

[0977] The server retrieves the reservation confirmation and sends it to the terminal, which displays it to the user, informing them that their travel plans have been successfully completed.

[0978] Specific examples

[0979] For example, say a user logs into a website and enters the following information:

[0980] Budget: 100,000 yen

[0981] Travel dates: December 1, 2023 to December 5, 2023

[0982] Transportation: Shinkansen

[0983] Accommodation: Hot Spring Hotel

[0984] The device collects this information and sends it to a server, which then collects data such as bullet train schedules and availability of hotels with hot springs, and generates multiple travel plans.

[0985] for example,

[0986] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[0987] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[0988] The server sends these plans to the terminal, which displays them to the user. If the user selects Plan A, the terminal sends the information to the server, which then confirms the Shinkansen and hotel reservations. Finally, the reservation confirmation information is displayed on the terminal, letting the user know that their travel plan has been successfully completed.

[0989] Example prompts to be input to the generative AI model

[0990] "Generate the best travel plan based on your budget, mode of transportation, and other factors."

[0991] "Please suggest multiple travel plans based on the following criteria: budget \100,000, travel dates December 1, 2023 to December 5, 2023, Shinkansen, hotel with hot spring."

[0992] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0993] Step 1: User Input Phase

[0994] Input: User's travel conditions (budget, travel dates, transportation, accommodation preferences)

[0995] Output: Collected travel condition data

[0996] Specific behavior:

[0997] A user accesses a website or application and enters details of their travel plans. They select their travel dates in a calendar format and enter their budget in text boxes. They select transportation and accommodation options using drop-down lists. After completing all the input fields, the user clicks the "Submit" button. This action causes the device to collect the data and send it to the server.

[0998] Step 2: Data collection phase

[0999] Input: Travel condition data

[1000] Output: Transportation information, accommodation information

[1001] Specific behavior:

[1002] The server receives the travel condition data that has been sent. It then collects the necessary information through external information providers (e.g., transportation information providers, accommodation information providers). The server sends an API request to obtain the availability and prices of Shinkansen trains and hotels with hot springs. The API response is returned to the server, where the collected information is integrated.

[1003] Step 3: Travel plan generation phase

[1004] Input: Collected transportation information, accommodation information, and travel condition data

[1005] Output: Multiple itineraries

[1006] Specific behavior:

[1007] The server uses a generative AI model to generate multiple travel plans based on the collected data and the user's travel conditions. Each plan is evaluated taking into account factors such as cost, travel time, and accommodation ratings. The generated plans are temporarily stored in the server's internal database.

[1008] Step 4: Plan presentation phase

[1009] Input: Multiple itineraries

[1010] Output: Travel plan information for display

[1011] Specific behavior:

[1012] The server sends the generated travel plans to the device, which receives data for visually displaying these plans. The user can compare the details of the plans through the device, and the plan costs, travel times, and accommodation details are displayed in a list.

[1013] Step 5: Booking process phase

[1014] Input: User selected itinerary

[1015] Output: Confirmed reservation information

[1016] Specific behavior:

[1017] Once the user selects their desired travel plan, the device sends the selection information to the server. The server then calls the external information provider again to confirm the selected Shinkansen and hotel reservations. The necessary reservation information is sent and received via the API.

[1018] Step 6: Reservation confirmation and completion phase

[1019] Input: Confirmed reservation information

[1020] Output: Reservation confirmation information, notifications

[1021] Specific behavior:

[1022] The server retrieves the reservation confirmation and sends it to the terminal, which receives it and displays it to the user. The reservation confirmation includes details such as the reservation number, dates, and hotel address, and also displays the message "Your reservation is complete."

[1023] (Application example 1)

[1024] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1025] Conventional food delivery planning systems have the drawback of requiring users to input their preferences through a complex interface, making it difficult to compare multiple options and select the optimal plan. When users have specific requests, there is a need for a system that can generate the optimal plan and quickly complete the ordering process.

[1026] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1027] In this invention, the server includes input means for the user to input conditions, processing means for automatically generating multiple delivery plans based on the conditions, display means for displaying the multiple delivery plans, ordering means for ordering the delivery plan selected by the user, and notification means for notifying the result of the ordering process. This allows the optimal delivery plan based on the conditions desired by the user to be generated quickly and efficiently, and makes the ordering process even easier.

[1028] "Input means" refers to a device or interface for inputting desired conditions or information by a user.

[1029] The "processing means" is a device or software for executing a specific process based on data received from the input means.

[1030] The "display means" is a device or interface for visually presenting the generated plans and information to the user.

[1031] "Order processing means" refers to a device or software that allows a user to confirm the plan selected and execute an order.

[1032] "Notification means" refers to a device or interface for notifying users of the results of the order process and updated information.

[1033] "Information collection means" refers to a device or software for acquiring necessary data from an external information providing service.

[1034] A "comparison tool" is a device or software used to evaluate and compare multiple plans based on specific criteria.

[1035] This invention is a system that automatically generates an optimal delivery plan based on the conditions entered by the user and performs the entire ordering process. This system uses the following hardware and software.

[1036] Hardware and software used:

[1037] Server: AWS EC2 (data processing)

[1038] Database: AWS RDS (data storage)

[1039] API: Google Places API, Yelp API (collecting restaurant information)

[1040] Frontend: React Native (smartphone app)

[1041] Overall system processing overview:

[1042] 1. Data entry method

[1043] Users enter their order requirements through a smartphone app, including budget, desired delivery time, type of food, and specific requests (e.g., vegetarian). This information is sent from the user's device to the server.

[1044] 2. Information gathering methods

[1045] The server uses external information providers (such as Google Places API and Yelp API) to collect information about restaurants and delivery services based on the entered conditions, including restaurant menus, prices, ratings, and available delivery times.

[1046] 3. Plan Generation Method

[1047] The server automatically generates multiple delivery plans based on the collected information, taking into account criteria such as cost, delivery time, and review ratings.

[1048] 4. Plan display method

[1049] The generated multiple delivery plans are sent to the device and displayed on the user's smartphone app, where they are presented in a format that makes it easy to compare plans by cost and delivery time.

[1050] 5. Order Processing Methods

[1051] Once the user selects the desired plan, the selection information is sent to the server, which then confirms the order with the delivery service and completes the order process.

[1052] 6. Means of notification

[1053] The server receives order confirmation information and estimated delivery time and sends it to the user's device. The user can then check the order confirmation and estimated delivery time on their smartphone app.

[1054] Examples:

[1055] For example, suppose a user enters information into a smartphone app under the following conditions:

[1056] Budget: 2,000 yen

[1057] Delivery time: 18:00~19:00

[1058] Cuisine type: Japanese

[1059] Specific requests: Vegetarian

[1060] Based on this information, the server uses the Google Places API and Yelp API to gather relevant restaurant information and generate a delivery plan like this:

[1061] Plan A: Japanese restaurant (¥1,800) + delivery fee (¥200) + delivery time: 18:30

[1062] Plan B: New Japanese restaurant (¥1,500) + Delivery fee (¥300) + Delivery time: 18:45

[1063] These plans are displayed on the user's smartphone app, and once the user selects the desired plan and confirms the order, the results are notified.

[1064] Example prompts to be input to the generative AI model:

[1065] "I'd like Japanese food delivered by 8 PM for under 2,000 yen. I'd like to choose a vegetarian option. Please prioritize restaurants with high reviews."

[1066] This allows users to quickly and efficiently generate the optimal delivery plan based on their specific preferences and easily complete the ordering process.

[1067] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1068] Step 1:

[1069] The user launches the smartphone app and inputs conditions such as budget, delivery time, desired type of food, specific requests, etc. The following information is then sent to the server via the input means:

[1070] Input: Budget (e.g., ¥2,000), Delivery time (e.g., 18:00-19:00), Cuisine type (e.g., Japanese), Specific requests (e.g., Vegetarian)

[1071] Output: The condition is sent to the server

[1072] Step 2:

[1073] The server calls external information providers (e.g., Google Places API, Yelp API) based on the received condition data to obtain related restaurant information and delivery service information. Specifically, it makes an API call and collects data that matches the conditions.

[1074] Input: User condition data

[1075] Output: Restaurant information (e.g., menu, prices, ratings) and delivery service information (e.g., delivery times, delivery fees)

[1076] Step 3:

[1077] The server automatically generates multiple delivery plans based on the collected restaurant and delivery service information. Specifically, it evaluates each combination of restaurant and delivery service based on evaluation criteria and creates plans.

[1078] Input: Collected restaurant and delivery service information

[1079] Output: Multiple delivery plans (e.g., Plan A: Japanese restaurant (¥1,800) + delivery fee (¥200), Plan B: New Japanese restaurant (¥1,500) + delivery fee (¥300))

[1080] Step 4:

[1081] The server sends the generated multiple delivery plans to the user's device and displays them on the user's smartphone app. Specifically, it formats the data to display the plans in a format that makes it easy to visually compare them.

[1082] Input: Multiple delivery plans

[1083] Output: A formatted list of plans

[1084] Step 5:

[1085] The user selects the desired delivery plan on the smartphone app. Once selected, the information is sent to the server. Specifically, data is sent to reflect the user's selection on the server.

[1086] Input: User's selected plan

[1087] Output: Selection information is sent to the server

[1088] Step 6:

[1089] The server confirms the order with the delivery service based on the received selection information. Specifically, it calls the delivery service's API and performs order confirmation processing.

[1090] Input: User selection information

[1091] Output: Order confirmation information

[1092] Step 7:

[1093] The server receives the order confirmation information and estimated delivery time and sends it to the user's device. The user can then confirm the order confirmation and estimated delivery time on the smartphone app. Specifically, the confirmation information is sent via a notification means.

[1094] Input: Order confirmation information and estimated delivery time

[1095] Output: A confirmation message is displayed on the user's terminal.

[1096] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1097] This invention relates to a system that combines a system that handles everything from travel planning to booking with an emotion engine that recognizes the user's emotions, and is able to propose optimal travel plans taking into account the user's emotional state.By analyzing the user's emotional information in real time and adjusting and proposing travel plans based on this, it is possible to provide travel plans that will provide a higher level of satisfaction.

[1098] This system is configured as follows:

[1099] 1. User Input Phase

[1100] A user opens a website or application and enters details of their travel plans, including budget, travel dates, mode of transportation (e.g., plane, car, train), and accommodation preferences (e.g., with hot springs, near a specific station), and this data is collected by the device and sent to the server.

[1101] 2. Data Collection Phase

[1102] The server receives the travel condition data sent by the user. The server collects the necessary travel-related data through external information providers (e.g., transportation information providers, accommodation information providers). For example, it obtains information such as the availability and price of Shinkansen trains and the availability and price of hotels with hot springs.

[1103] 3. Travel plan generation phase

[1104] The server integrates the collected data and automatically generates travel plans. Multiple travel plans are created and each plan is evaluated based on criteria such as cost, travel time, and accommodation ratings.

[1105] 4. Sentiment Analysis Phase

[1106] The emotion engine analyzes the user's input data and emotional information such as the user's facial expressions and tone of voice. This analyzed emotional data is reflected in the travel plan generation, and the plan is adjusted to match the user's emotional state.

[1107] 5. Plan presentation phase

[1108] The server generates multiple travel plans and sends them to the device. The plan information, which also takes into account the results of sentiment analysis, is displayed to the user on the device. The details of the multiple plans are displayed in a list format that makes it easy for the user to compare and consider them.

[1109] 6. Real-time feedback phase

[1110] The emotion engine monitors the user's emotional state in real time and provides immediate feedback when selecting a desired travel plan. For example, if the engine detects anxiety or dissatisfaction with the user's chosen plan, it will present other options.

[1111] 7. Booking Process Phase

[1112] After the user selects the desired travel plan, the selected information is sent from the terminal to the server, which then calls the external information providing means again to confirm the selected Shinkansen and hotel reservations.

[1113] 8. Reservation confirmation and completion phase

[1114] The server retrieves the reservation confirmation information and sends it to the terminal, which displays it to the user, indicating that the travel plan has been successfully completed.

[1115] Specific examples

[1116] For example, suppose a user logs into a website and enters the following information:

[1117] Budget: 100,000 yen

[1118] Travel dates: December 1, 2023 to December 5, 2023

[1119] Transportation: Shinkansen

[1120] Accommodation: Hot Spring Hotel

[1121] The device collects this information and sends it to a server, which then collects data such as bullet train schedules and availability of hotels with hot springs, and generates multiple travel plans.

[1122] for example,

[1123] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[1124] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[1125] At the same time, the emotion engine analyzes the user's emotional state from their facial expressions and tone of voice, and if the user is looking to relax, it makes adjustments such as adding hotels with hot springs to the recommendations.

[1126] The server sends these adjusted plans to the device, which displays them to the user. Real-time feedback suggests alternative plans if the user expresses anxiety or dissatisfaction when selecting Plan A.

[1127] Finally, the user selects the best plan and the reservation is confirmed. The reservation confirmation information is displayed on the terminal, and the user can confirm that the travel plan has been successfully completed.

[1128] The system configuration described above makes it possible to consistently and efficiently carry out travel planning and booking while taking emotional information into consideration.

[1129] The processing flow will be explained below.

[1130] Step 1:

[1131] A user opens a website or application and accesses a form to enter travel planning details.

[1132] Step 2:

[1133] The user inputs travel conditions such as budget, travel schedule, transportation, desired accommodation, etc. The input data is collected by the terminal and sent to the server.

[1134] Step 3:

[1135] The server analyzes the received travel condition data, and based on the analyzed data, prepares to call an external information providing means for collecting information that matches the travel conditions.

[1136] Step 4:

[1137] The server uses external information providers (e.g., transportation information providers, accommodation information providers) to collect necessary travel-related data, such as the availability and prices of Shinkansen trains and hotels with hot springs.

[1138] Step 5:

[1139] The server integrates the collected data and automatically generates travel plans. Multiple travel plans are created and each plan is evaluated based on criteria such as cost, travel time, and accommodation ratings.

[1140] Step 6:

[1141] The emotion engine collects user input data and emotion information such as the user's facial expressions and tone of voice, and analyzes the user's emotional state.

[1142] Step 7:

[1143] The server then adjusts the travel plan based on the results of the sentiment analysis. For example, if the user wants a relaxing trip, it will prioritize accommodations with quiet environments.

[1144] Step 8:

[1145] The server sends the adjusted travel plans to the terminal, which displays them to the user. The plan details are displayed in a list format that makes it easy for the user to compare them.

[1146] Step 9:

[1147] The emotion engine monitors the user's emotional state in real time, analyzing the emotional state and providing immediate feedback when the user selects a desired travel plan from the presented plans.

[1148] Step 10:

[1149] The user selects a desired travel plan, and the selected plan information is sent to the server by the terminal.

[1150] Step 11:

[1151] The server starts the reservation procedure for the selected travel plan, and confirms the reservation of the selected Shinkansen and hotel through an external information providing means.

[1152] Step 12:

[1153] The server receives the reservation confirmation information and sends it to the terminal, which displays it to the user.

[1154] Step 13:

[1155] The user checks the reservation confirmation information and finds out that the travel plan has been successfully completed. Through the above processing steps, a series of processes from travel planning to reservation completion is efficiently executed, taking into account the user's emotional information.

[1156] Example 2

[1157] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1158] Conventional travel planning systems only automatically generate travel plans based on user-entered conditions, but are unable to consider the user's emotional state, making it difficult to propose highly satisfying travel plans. Furthermore, there have been few systems that can analyze the user's emotions in real time and adjust or propose travel plans based on those emotions.

[1159] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1160] In this invention, the server includes an input means for a user to input travel conditions, a generation means for automatically generating multiple travel plans based on the travel conditions, and an emotion analysis means for adjusting the travel plans based on the user's emotional state, thereby making it possible to propose highly satisfying travel plans that reflect the user's emotional state in real time.

[1161] "Input means" refers to a means by which a user inputs travel conditions.

[1162] The "generation means" is a means for automatically generating multiple itineraries based on input travel conditions.

[1163] The "emotion analysis means" is a means for analyzing the user's emotional state and adjusting the travel plan based on that.

[1164] The "display means" is a means for displaying the generated itineraries to the user.

[1165] The "reservation procedure means" refers to the procedure means for reserving the travel plan selected by the user.

[1166] The "notification means" is a means for notifying the user of the results of the reservation procedure.

[1167] "Data collection means" refers to a means for collecting transportation information and accommodation information via external data providing means.

[1168] A "comparison tool" is a tool for comparing multiple travel plans based on cost and time criteria.

[1169] This invention is a system that handles everything from travel planning to booking, and is particularly equipped with an emotion engine that recognizes the user's emotions. This system can propose optimal travel plans taking into account the user's emotional state.

[1170] Specific configuration

[1171] Hardware and Software

[1172] The system uses the following hardware and software:

[1173] Server: Data processing, travel plan generation, and sentiment analysis.

[1174] Terminal: A PC or smartphone that performs user input and displays plans.

[1175] Emotion engine: Analyzes user emotions in real time and adjusts and suggests plans.

[1176] The software used includes:

[1177] Web application: Provides an interface for users to input their travel plans.

[1178] External API: Used to collect data to obtain information on transportation and accommodation.

[1179] Generative AI models: Used for itinerary generation and sentiment analysis.

[1180] How to use

[1181] 1. User Input

[1182] A user opens a website or application and enters the following information:

[1183] budget

[1184] travel itinerary

[1185] Transportation (e.g., bullet train, car, airplane, etc.)

[1186] Accommodation preferences (e.g. with hot springs, specific location, etc.)

[1187] The terminal collects this information and sends it to the server.

[1188] 2. Data Collection

[1189] The server receives the input data and collects information on Shinkansen trains and accommodations through external data providers, specifically information on Shinkansen availability and prices, availability and prices of hotels with hot springs, etc.

[1190] 3. Travel plan generation

[1191] The server generates multiple travel plans based on the collected data, and the plans are scored based on criteria such as cost, travel time, and accommodation ratings.

[1192] 4. Emotion analysis

[1193] The emotion engine analyzes the user's emotional state based on input data, facial expressions, tone of voice, etc. The travel plan is then adjusted based on the analyzed data. For example, if the user is looking for relaxation, hotels with hot springs will be suggested first.

[1194] 5. Plan presentation

[1195] The server sends the generated travel plan to the terminal, which then displays it to the user. The plan information is presented in an easy-to-compare format.

[1196] 6. Real-time feedback

[1197] The emotion engine monitors users' emotions in real time and provides immediate feedback when selecting travel plans, suggesting alternative plans if anxiety or dissatisfaction is detected.

[1198] 7. Reservation Procedure

[1199] After the user selects the desired travel plan, the information is sent to the server, which then calls the external data providing means again to confirm the selected Shinkansen and hotel reservations.

[1200] 8. Reservation Confirmation

[1201] The server receives the reservation confirmation information and sends it to the terminal, which displays the reservation completion information to the user.

[1202] Specific examples

[1203] For example, if a user logs into a website and enters the following information:

[1204] Budget: 100,000 yen

[1205] Travel dates: December 1, 2023 to December 5, 2023

[1206] Transportation: Shinkansen

[1207] Accommodation: Hot Spring Hotel

[1208] The device collects information and sends it to the server. The server collects data on Shinkansen trains and hotels with hot springs, and generates multiple travel plans:

[1209] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[1210] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[1211] The emotion engine analyzes the user's emotions and recommends hotels with hot springs if they are looking for relaxation. The server sends the adjusted plan to the device and displays it to the user. Real-time feedback allows the user to select the best plan.

[1212] Finally, the user selects a plan and the reservation is confirmed. Reservation confirmation information is displayed on the terminal, and the user confirms that the travel plan has been successfully completed.

[1213] Generative AI model prompt example

[1214] "Please tell us about your next trip. Explain your budget, travel dates, transportation, and accommodation preferences. Also, tell us how you want to feel during your trip."

[1215] The system configuration described above makes it possible to consistently and efficiently carry out travel planning and booking while taking emotional information into consideration.

[1216] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1217] Step 1:

[1218] User Input Phase

[1219] A user opens a website or application and enters details of their travel plans (budget, travel dates, transportation options, and preferred accommodations). The data entered by the user is collected by the device and sent to the server. The server then receives the travel conditions data. As a specific example, a user enters information such as a budget of 100,000 yen, travel dates from December 1st to 5th, 2023, travel by Shinkansen, and a hotel with a hot spring.

[1220] Input: Travel conditions entered by the user (budget, travel dates, transportation, accommodation preferences)

[1221] Output: Travel condition data sent to the server

[1222] Step 2:

[1223] Data Collection Phase

[1224] After the server receives the travel condition data, it collects transportation information and accommodation information via external data provision means. As a specific example, the server calls the APIs of a Shinkansen train operation information service and an accommodation information service to obtain the availability and prices of Shinkansen trains and hotels with hot springs.

[1225] Input: Travel condition data

[1226] Output: Transportation and accommodation information

[1227] Step 3:

[1228] Travel plan generation phase

[1229] The server automatically generates multiple travel plans based on the data collected. The server scores the generated plans based on criteria such as cost, travel time, and accommodation ratings. Specifically, the server combines Shinkansen prices and hotel prices to calculate the cost of the travel plan.

[1230] Input: Transportation and accommodation information

[1231] Output: Multiple itineraries

[1232] Step 4:

[1233] Sentiment Analysis Phase

[1234] The emotion engine analyzes the user's input data, facial expressions, and tone of voice to extract the user's emotional state. Based on this emotional data, the server adjusts the travel plan. For example, if the user is looking for relaxation, it will emphasize and suggest hotels with hot springs.

[1235] Input: User's emotional information

[1236] Output: Adjusted itinerary

[1237] Step 5:

[1238] Plan presentation phase

[1239] The server sends the generated travel plan to the terminal, which displays it to the user. The terminal displays a list of details of multiple travel plans so that the user can easily compare them.

[1240] Input: Adjusted travel plans

[1241] Output: The itinerary displayed to the user

[1242] Step 6:

[1243] Real-time feedback phase

[1244] The emotion engine monitors the user's emotional state in real time and provides immediate feedback when selecting a travel plan, for example suggesting other options if the user expresses anxiety or dissatisfaction with the plan they have chosen.

[1245] Input: Real-time user emotion information

[1246] Output: Alternative suggestions

[1247] Step 7:

[1248] Booking process phase

[1249] When the user selects the desired travel plan, the information is sent from the terminal to the server, which then calls the external data providing means again to confirm the reservation of the selected Shinkansen and hotel.

[1250] Input: User selected travel plan

[1251] Output: Confirmed reservation information

[1252] Step 8:

[1253] Reservation confirmation and completion phase

[1254] The server retrieves the reservation confirmation information and sends it to the terminal, which displays the reservation confirmation information to the user, informing them that their travel plans have been successfully completed.

[1255] Input: Confirmed reservation information

[1256] Output: Booking confirmation information displayed to the user

[1257] (Application example 2)

[1258] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1259] In conventional autonomous vehicles, operation plans are determined without taking the driver's emotions into consideration, which increases driver stress and anxiety and reduces driving comfort. Furthermore, there was no system that provided real-time feedback to the driver based on their emotions when proposing a plan or during operation. Therefore, in order to provide a safer and more comfortable driving environment, a system that analyzes the driver's emotions in real time and adjusts operation plans based on that information is needed.

[1260] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes emotion analysis means for analyzing the user's emotion information, plan adjustment means for adjusting the travel plan based on the emotion information, and feedback means for presenting the adjusted plan. This makes it possible to analyze the driver's emotion information in real time and adjust or propose a trip plan based on it.

[1261] "User" refers to the general public who uses the system to input travel conditions.

[1262] "Interface means" refers to an input device or software that allows a user to input travel conditions.

[1263] "Processing means" refers to a device or software that automatically generates multiple travel plans based on input travel conditions.

[1264] "Display means" refers to a device or software that visually presents the generated travel plans to the user.

[1265] "Reservation procedure means" refers to the device or software used to actually reserve the travel plan selected by the user.

[1266] "Notification means" refers to devices or software for notifying users of the results of their reservation procedures.

[1267] "Emotion analysis means" refers to devices or software for analyzing users' emotional information.

[1268] "Plan adjustment means" refers to a device or software for adjusting a travel plan based on the analyzed emotional information.

[1269] "Feedback means" refers to a device or software for presenting the adjusted plan to the user and providing feedback in real time.

[1270] "External information providing means" refers to devices and services for collecting external information such as transportation information and accommodation information.

[1271] This invention combines an emotion engine with a system that handles everything from user travel planning to reservations, and is a specific form for providing safe and comfortable driving plans for drivers of self-driving vehicles. The details are provided below.

[1272] 1. User input phase

[1273] Users use an application installed on their smartphone or head-mounted display to input travel conditions such as destination, travel time, desired route, etc. This allows the user's desired travel plan to be clearly collected.

[1274] 2. Data Collection Phase

[1275] The server collects information on traffic, weather, and congestion at destinations through external information providers, obtaining the latest data in real time via a REST API.

[1276] 3. Operation plan generation phase

[1277] Based on the collected data, the server generates multiple operation plans using Python scripts and machine learning algorithms (e.g., optimization algorithms) and evaluates each one.

[1278] 4. Sentiment Analysis Phase

[1279] Using the camera and microphone installed on the smartphone or head-mounted display, and using face recognition API and voice recognition API, emotional information is analyzed in real time from the user's facial expressions and tone of voice, thereby obtaining emotional information about the user.

[1280] 5. Plan presentation phase

[1281] The server presents the user with multiple operation plans that take emotion information into consideration through a display means, thereby providing the user with an intuitive interface for selecting the optimal plan based on emotion.

[1282] 6. Real-time feedback phase

[1283] Based on the analyzed emotional information, the server adjusts and re-presents the operation plan according to the user's real-time emotions through feedback means. For example, if the server detects anxiety or dissatisfaction with the plan selected by the user, it will present an alternative plan.

[1284] 7. Operational Phase

[1285] After the user selects the optimal plan, the app sends driving instructions to the autonomous vehicle's onboard computer, starting a safe and comfortable journey.

[1286] Specific examples

[1287] User input: Destination "Tokyo", travel time "within 2 hours"

[1288] Prompt (when inputting to a generative AI model):

[1289] Suggest a route that will make the driver feel safe and relaxed. Emotions are a little tense right now. Choose a safe and efficient route within the desired timeframe.

[1290] As described above, this invention is a system that analyzes user emotions in real time and adjusts and proposes operation plans based on that analysis, providing a safe and comfortable operating environment for self-driving vehicles.

[1291] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1292] Step 1:

[1293] The terminal provides an interface where users can input travel conditions (destination, travel time, desired route, etc.) using a smartphone or head-mounted display. The input information is sent from the terminal to the server. The input is the user's desired conditions, and the output is data sent to the server.

[1294] Step 2:

[1295] The server collects traffic information, weather information, and congestion information at the destination via external information providing means based on the travel conditions sent from the terminal. The collected information is stored in the server. The input is the travel conditions and external information, and the output is the storage of the collected data.

[1296] Step 3:

[1297] The server generates a trip plan using the collected data. It uses Python scripts and machine learning algorithms to apply optimization algorithms to create multiple trip plans. The generated trip plans are stored on the server. The input is the collected data and the algorithm, and the output is the trip plan.

[1298] Step 4:

[1299] The device uses the camera and microphone of the smartphone or head-mounted display to analyze the user's face and voice, and analyzes facial expressions and tone of voice in real time using an emotion analysis API. The analysis results are sent from the device to a server. The input is facial expression and voice data, and the output is the emotion analysis results.

[1300] Step 5:

[1301] The server adjusts the previously created operation plan based on the emotion analysis results. Using the plan adjustment means, it selects the operation plan that best suits the user's emotional state and creates an adjusted operation plan. The input is the emotion analysis results and the operation plan, and the output is the adjusted operation plan.

[1302] Step 6:

[1303] The terminal presents the adjusted operation plans from the server to the user. The plans are displayed in a visually easy-to-compare format to make it easier for the user to select a plan. The input is the adjusted operation plan, and the output is the plan presented to the user.

[1304] Step 7:

[1305] The server provides real-time feedback on the trip plan selected by the user, detects the user's anxiety and dissatisfaction, and presents alternative plans as necessary. The input is the user's preferences and emotional state, and the output is the presentation of alternative plans.

[1306] Step 8:

[1307] The terminal confirms the user's final selected trip plan and sends instructions to the autonomous vehicle's onboard computer, which then executes the selected trip plan. The input is the final selected trip plan, and the output is instructions to the onboard computer.

[1308] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1309] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1310] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1311] [Fourth embodiment]

[1312] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1313] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1314] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1315] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1316] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1317] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1318] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1319] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1320] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1321] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1322] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1323] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1324] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1325] The present invention relates to a system that handles everything from travel planning to reservations all at once, and in particular to a system that automatically generates an optimal travel plan and smoothly processes reservations by the user simply by inputting conditions such as budget, schedule, desired means of transportation, and accommodations.

[1326] This system is configured as follows:

[1327] 1. User Input Phase

[1328] Users first access a website or application and enter details of their travel plans, including budget, travel dates, mode of transportation (e.g., plane, car, train), and accommodation preferences (e.g., with hot springs, near a specific station, etc.). This data is collected by the device and sent to the server.

[1329] 2. Data Collection Phase

[1330] The server receives the travel condition data sent by the user. The server collects the necessary information through external information providers (e.g., transportation information providers, accommodation information providers, etc.). Specifically, it obtains the availability and prices of Shinkansen trains and hotels with hot springs.

[1331] 3. Travel plan generation phase

[1332] The server creates a travel plan based on the collected information. It automatically generates multiple plans and evaluates each plan based on factors such as cost, travel time, and accommodation ratings. In this step, the server compares multiple travel plans that best fit the user's criteria.

[1333] 4. Plan presentation phase

[1334] The server sends the generated itineraries to the terminal, which then visually displays the itineraries to the user. The displayed itineraries are easy for users to compare, and each plan's cost, travel time, and accommodation details are displayed in a list.

[1335] 5. Booking process phase

[1336] Once the user selects the desired travel plan, the terminal transmits the selection to the server, which then calls the external information provider again to confirm the selected Shinkansen and hotel reservations.

[1337] 6. Reservation confirmation and completion phase

[1338] The server retrieves the reservation confirmation and sends it to the terminal, which displays it to the user, informing them that their travel plans have been successfully completed.

[1339] Specific examples

[1340] For example, suppose a user logs into a website and enters the following information:

[1341] Budget: 100,000 yen

[1342] Travel dates: December 1, 2023 to December 5, 2023

[1343] Transportation: Shinkansen

[1344] Accommodation: Hot Spring Hotel

[1345] The device collects this information and sends it to a server, which then collects data such as bullet train schedules and availability of hotels with hot springs, and generates multiple travel plans.

[1346] for example,

[1347] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[1348] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[1349] The server sends these plans to the terminal, which displays them to the user. If the user selects Plan A, the terminal sends the information to the server, which then confirms the Shinkansen and hotel reservations. Finally, the reservation confirmation information is displayed on the terminal, letting the user know that their travel plan has been successfully completed.

[1350] The system configuration described above allows for consistent and efficient travel planning and booking.

[1351] The processing flow will be explained below.

[1352] Step 1:

[1353] A user opens a website or application and accesses a form to enter travel planning details.

[1354] Step 2:

[1355] The user inputs travel conditions such as budget, travel dates, transportation, desired accommodation, etc. The input data is collected by the terminal.

[1356] Step 3:

[1357] The device sends the collected travel condition data to the server in JSON format.

[1358] Step 4:

[1359] The server analyzes the received travel condition data, and based on the analyzed data, prepares to call an external information providing means for collecting information that matches the travel conditions.

[1360] Step 5:

[1361] The server uses external information providers (e.g., transportation information providers, accommodation information providers) to collect necessary travel-related data, such as the availability and prices of Shinkansen trains and hotels with hot springs.

[1362] Step 6:

[1363] The server integrates the collected data and automatically generates travel plans. Multiple travel plans are created and each plan is evaluated based on criteria such as cost, travel time, and accommodation ratings.

[1364] Step 7:

[1365] The server compares the generated travel plans and selects the optimal travel plan based on the comparison results.

[1366] Step 8:

[1367] The server sends the optimal travel plan to the device. The plan information is sent to the device in JSON format.

[1368] Step 9:

[1369] The terminal displays the received travel plans to the user. Details of multiple plans are listed and presented in a format that makes it easy for the user to compare them.

[1370] Step 10:

[1371] The user selects a desired travel plan, and the selected plan information is sent to the server by the terminal.

[1372] Step 11:

[1373] The server starts the reservation procedure for the selected travel plan, and confirms the reservation of the selected Shinkansen and hotel through an external information providing means.

[1374] Step 12:

[1375] The server acquires the reservation confirmation information, which is then sent to the terminal.

[1376] Step 13:

[1377] The terminal displays the reservation confirmation information to the user, who confirms that the travel plan has been successfully completed.

[1378] Through the above processing steps, a series of processes from travel planning to reservation completion can be efficiently executed.

[1379] Example 1

[1380] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1381] In conventional travel planning systems, after users input their travel requirements, they had to manually compare many options to find the best plan, which was time-consuming and labor-intensive. Furthermore, when comparing several plans, the detailed information was not visually centrally managed, which reduced user convenience. To solve these problems, a new system was needed.

[1382] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1383] In this invention, the server includes an interface means for users to input travel conditions, a processing means for automatically generating multiple travel plans based on the travel conditions, a display means for displaying the multiple travel plans, a reservation procedure means for reserving the travel plan selected by the user, a notification means for notifying the results of the reservation procedure, an information collection means for collecting transportation information and accommodation information via an external information providing means, a comparison means for comparing the generated travel plans based on cost and time criteria, and an optimization means for optimizing the multiple travel plans based on an evolutionary algorithm. This allows users to easily input travel conditions and compare, select, and reserve from a wide range of options in a unified manner.

[1384] The "interface means" is a means by which a user inputs travel conditions into the system.

[1385] The "processing means" is a means for automatically generating a plurality of travel plans based on travel conditions input by the user.

[1386] The "display means" is a means for visually displaying the generated travel plans to the user.

[1387] The "reservation procedure means" is a means by which a user confirms the travel plan selected and completes the reservation procedure.

[1388] "Notification means" is a means for notifying the user of the results of the reservation procedure.

[1389] "External information providing means" refers to a means for receiving data such as transportation information and accommodation information from outside.

[1390] "Information gathering means" refers to a means for gathering necessary transportation information and accommodation information via external information providing means.

[1391] The "comparison means" is a means for comparing the generated travel plans based on cost and time.

[1392] The "optimization means" is a means for optimizing multiple travel plans based on an evolutionary algorithm and providing the optimal plan to the user.

[1393] The "graphical user interface means" is a means for visually displaying multiple travel plans in an easy-to-understand manner, allowing users to operate the system intuitively.

[1394] The "update means" is a means for automatically updating the reservation information confirmed by the reservation procedure means.

[1395] The present invention relates to a system that handles everything from travel planning to reservations all at once, and in particular to a system that automatically generates an optimal travel plan and smoothly processes reservations by the user simply by inputting conditions such as budget, schedule, desired means of transportation, and accommodations.

[1396] User Input Phase

[1397] Users first access a website or application and enter details of their travel plans, including budget, travel dates, mode of transportation (e.g., plane, car, train), and accommodation preferences (e.g., with hot springs, near a specific station, etc.). This data is collected by the device and sent to the server.

[1398] Data Collection Phase

[1399] The server receives the travel condition data sent by the user. The server collects the necessary information through external information providers (e.g., transportation information providers, accommodation information providers, etc.). Specifically, it obtains the availability and prices of Shinkansen trains and hotels with hot springs.

[1400] Travel plan generation phase

[1401] The server creates a travel plan based on the collected information. It automatically generates multiple plans and evaluates each plan based on factors such as cost, travel time, and accommodation ratings. In this step, the server uses a generative AI model to compare multiple travel plans that best fit the user's requirements.

[1402] Plan presentation phase

[1403] The server sends the generated itineraries to the terminal, which then visually displays the itineraries to the user. The displayed itineraries are easy for users to compare, and each plan's cost, travel time, and accommodation details are displayed in a list.

[1404] Booking process phase

[1405] Once the user selects the desired travel plan, the terminal transmits the selection to the server, which then calls the external information provider again to confirm the selected Shinkansen and hotel reservations.

[1406] Reservation confirmation and completion phase

[1407] The server retrieves the reservation confirmation and sends it to the terminal, which displays it to the user, informing them that their travel plans have been successfully completed.

[1408] Specific examples

[1409] For example, say a user logs into a website and enters the following information:

[1410] Budget: 100,000 yen

[1411] Travel dates: December 1, 2023 to December 5, 2023

[1412] Transportation: Shinkansen

[1413] Accommodation: Hot Spring Hotel

[1414] The device collects this information and sends it to a server, which then collects data such as bullet train schedules and availability of hotels with hot springs, and generates multiple travel plans.

[1415] for example,

[1416] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[1417] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[1418] The server sends these plans to the terminal, which displays them to the user. If the user selects Plan A, the terminal sends the information to the server, which then confirms the Shinkansen and hotel reservations. Finally, the reservation confirmation information is displayed on the terminal, letting the user know that their travel plan has been successfully completed.

[1419] Example prompts to be input to the generative AI model

[1420] "Generate the best travel plan based on your budget, mode of transportation, and other factors."

[1421] "Please suggest multiple travel plans based on the following criteria: budget \100,000, travel dates December 1, 2023 to December 5, 2023, Shinkansen, hotel with hot spring."

[1422] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1423] Step 1: User Input Phase

[1424] Input: User's travel conditions (budget, travel dates, transportation, accommodation preferences)

[1425] Output: Collected travel condition data

[1426] Specific behavior:

[1427] A user accesses a website or application and enters details of their travel plans. They select their travel dates in a calendar format and enter their budget in text boxes. They select transportation and accommodation options using drop-down lists. After completing all the input fields, the user clicks the "Submit" button. This action causes the device to collect the data and send it to the server.

[1428] Step 2: Data collection phase

[1429] Input: Travel condition data

[1430] Output: Transportation information, accommodation information

[1431] Specific behavior:

[1432] The server receives the travel condition data that has been sent. It then collects the necessary information through external information providers (e.g., transportation information providers, accommodation information providers). The server sends an API request to obtain the availability and prices of Shinkansen trains and hotels with hot springs. The API response is returned to the server, where the collected information is integrated.

[1433] Step 3: Travel plan generation phase

[1434] Input: Collected transportation information, accommodation information, and travel condition data

[1435] Output: Multiple itineraries

[1436] Specific behavior:

[1437] The server uses a generative AI model to generate multiple travel plans based on the collected data and the user's travel conditions. Each plan is evaluated taking into account factors such as cost, travel time, and accommodation ratings. The generated plans are temporarily stored in the server's internal database.

[1438] Step 4: Plan presentation phase

[1439] Input: Multiple itineraries

[1440] Output: Travel plan information for display

[1441] Specific behavior:

[1442] The server sends the generated travel plans to the device, which receives data for visually displaying these plans. The user can compare the details of the plans through the device, and the plan costs, travel times, and accommodation details are displayed in a list.

[1443] Step 5: Booking process phase

[1444] Input: User selected itinerary

[1445] Output: Confirmed reservation information

[1446] Specific behavior:

[1447] Once the user selects their desired travel plan, the device sends the selection information to the server. The server then calls the external information provider again to confirm the selected Shinkansen and hotel reservations. The necessary reservation information is sent and received via the API.

[1448] Step 6: Reservation confirmation and completion phase

[1449] Input: Confirmed reservation information

[1450] Output: Reservation confirmation information, notifications

[1451] Specific behavior:

[1452] The server retrieves the reservation confirmation and sends it to the terminal, which receives it and displays it to the user. The reservation confirmation includes details such as the reservation number, dates, and hotel address, and also displays the message "Your reservation is complete."

[1453] (Application example 1)

[1454] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1455] Conventional food delivery planning systems have the drawback of requiring users to input their preferences through a complex interface, making it difficult to compare multiple options and select the optimal plan. When users have specific requests, there is a need for a system that can generate the optimal plan and quickly complete the ordering process.

[1456] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1457] In this invention, the server includes input means for the user to input conditions, processing means for automatically generating multiple delivery plans based on the conditions, display means for displaying the multiple delivery plans, ordering means for ordering the delivery plan selected by the user, and notification means for notifying the result of the ordering process. This allows the optimal delivery plan based on the conditions desired by the user to be generated quickly and efficiently, and makes the ordering process even easier.

[1458] "Input means" refers to a device or interface for inputting desired conditions or information by a user.

[1459] The "processing means" is a device or software for executing a specific process based on data received from the input means.

[1460] The "display means" is a device or interface for visually presenting the generated plans and information to the user.

[1461] "Order processing means" refers to a device or software that allows a user to confirm the plan selected and execute an order.

[1462] "Notification means" refers to a device or interface for notifying users of the results of the order process and updated information.

[1463] "Information collection means" refers to a device or software for acquiring necessary data from an external information providing service.

[1464] A "comparison tool" is a device or software used to evaluate and compare multiple plans based on specific criteria.

[1465] This invention is a system that automatically generates an optimal delivery plan based on the conditions entered by the user and performs the entire ordering process. This system uses the following hardware and software.

[1466] Hardware and software used:

[1467] Server: AWS EC2 (data processing)

[1468] Database: AWS RDS (data storage)

[1469] API: Google Places API, Yelp API (collecting restaurant information)

[1470] Frontend: React Native (smartphone app)

[1471] Overall system processing overview:

[1472] 1. Data entry method

[1473] Users enter their order requirements through a smartphone app, including budget, desired delivery time, type of food, and specific requests (e.g., vegetarian). This information is sent from the user's device to the server.

[1474] 2. Information gathering methods

[1475] The server uses external information providers (such as Google Places API and Yelp API) to collect information about restaurants and delivery services based on the entered conditions, including restaurant menus, prices, ratings, and available delivery times.

[1476] 3. Plan Generation Method

[1477] The server automatically generates multiple delivery plans based on the collected information, taking into account criteria such as cost, delivery time, and review ratings.

[1478] 4. Plan display method

[1479] The generated multiple delivery plans are sent to the device and displayed on the user's smartphone app, where they are presented in a format that makes it easy to compare plans by cost and delivery time.

[1480] 5. Order Processing Methods

[1481] Once the user selects the desired plan, the selection information is sent to the server, which then confirms the order with the delivery service and completes the order process.

[1482] 6. Means of notification

[1483] The server receives order confirmation information and estimated delivery time and sends it to the user's device. The user can then check the order confirmation and estimated delivery time on their smartphone app.

[1484] Examples:

[1485] For example, suppose a user enters information into a smartphone app under the following conditions:

[1486] Budget: 2,000 yen

[1487] Delivery time: 18:00~19:00

[1488] Cuisine type: Japanese

[1489] Specific requests: Vegetarian

[1490] Based on this information, the server uses the Google Places API and Yelp API to gather relevant restaurant information and generate a delivery plan like this:

[1491] Plan A: Japanese restaurant (¥1,800) + delivery fee (¥200) + delivery time: 18:30

[1492] Plan B: New Japanese restaurant (¥1,500) + Delivery fee (¥300) + Delivery time: 18:45

[1493] These plans are displayed on the user's smartphone app, and once the user selects the desired plan and confirms the order, the results are notified.

[1494] Example prompts to be input to the generative AI model:

[1495] "I'd like Japanese food delivered by 8 PM for under 2,000 yen. I'd like to choose a vegetarian option. Please prioritize restaurants with high reviews."

[1496] This allows users to quickly and efficiently generate the optimal delivery plan based on their specific preferences and easily complete the ordering process.

[1497] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1498] Step 1:

[1499] The user launches the smartphone app and inputs conditions such as budget, delivery time, desired type of food, specific requests, etc. The following information is then sent to the server via the input means:

[1500] Input: Budget (e.g., ¥2,000), Delivery time (e.g., 18:00-19:00), Cuisine type (e.g., Japanese), Specific requests (e.g., Vegetarian)

[1501] Output: The condition is sent to the server

[1502] Step 2:

[1503] The server calls external information providers (e.g., Google Places API, Yelp API) based on the received condition data to obtain related restaurant information and delivery service information. Specifically, it makes an API call and collects data that matches the conditions.

[1504] Input: User condition data

[1505] Output: Restaurant information (e.g., menu, prices, ratings) and delivery service information (e.g., delivery times, delivery fees)

[1506] Step 3:

[1507] The server automatically generates multiple delivery plans based on the collected restaurant and delivery service information. Specifically, it evaluates each combination of restaurant and delivery service based on evaluation criteria and creates plans.

[1508] Input: Collected restaurant and delivery service information

[1509] Output: Multiple delivery plans (e.g., Plan A: Japanese restaurant (¥1,800) + delivery fee (¥200), Plan B: New Japanese restaurant (¥1,500) + delivery fee (¥300))

[1510] Step 4:

[1511] The server sends the generated multiple delivery plans to the user's device and displays them on the user's smartphone app. Specifically, it formats the data to display the plans in a format that makes it easy to visually compare them.

[1512] Input: Multiple delivery plans

[1513] Output: A formatted list of plans

[1514] Step 5:

[1515] The user selects the desired delivery plan on the smartphone app. Once selected, the information is sent to the server. Specifically, data is sent to reflect the user's selection on the server.

[1516] Input: User's selected plan

[1517] Output: Selection information is sent to the server

[1518] Step 6:

[1519] The server confirms the order with the delivery service based on the received selection information. Specifically, it calls the delivery service's API and performs order confirmation processing.

[1520] Input: User selection information

[1521] Output: Order confirmation information

[1522] Step 7:

[1523] The server receives the order confirmation information and estimated delivery time and sends it to the user's device. The user can then confirm the order confirmation and estimated delivery time on the smartphone app. Specifically, the confirmation information is sent via a notification means.

[1524] Input: Order confirmation information and estimated delivery time

[1525] Output: A confirmation message is displayed on the user's terminal.

[1526] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1527] This invention relates to a system that combines a system that handles everything from travel planning to booking with an emotion engine that recognizes the user's emotions, and is able to propose optimal travel plans taking into account the user's emotional state.By analyzing the user's emotional information in real time and adjusting and proposing travel plans based on this, it is possible to provide travel plans that will provide a higher level of satisfaction.

[1528] This system is configured as follows:

[1529] 1. User Input Phase

[1530] A user opens a website or application and enters details of their travel plans, including budget, travel dates, mode of transportation (e.g., plane, car, train), and accommodation preferences (e.g., with hot springs, near a specific station), and this data is collected by the device and sent to the server.

[1531] 2. Data Collection Phase

[1532] The server receives the travel condition data sent by the user. The server collects the necessary travel-related data through external information providers (e.g., transportation information providers, accommodation information providers). For example, it obtains information such as the availability and price of Shinkansen trains and the availability and price of hotels with hot springs.

[1533] 3. Travel plan generation phase

[1534] The server integrates the collected data and automatically generates travel plans. Multiple travel plans are created and each plan is evaluated based on criteria such as cost, travel time, and accommodation ratings.

[1535] 4. Sentiment Analysis Phase

[1536] The emotion engine analyzes the user's input data and emotional information such as the user's facial expressions and tone of voice. This analyzed emotional data is reflected in the travel plan generation, and the plan is adjusted to match the user's emotional state.

[1537] 5. Plan presentation phase

[1538] The server generates multiple travel plans and sends them to the device. The plan information, which also takes into account the results of sentiment analysis, is displayed to the user on the device. The details of the multiple plans are displayed in a list format that makes it easy for the user to compare and consider them.

[1539] 6. Real-time feedback phase

[1540] The emotion engine monitors the user's emotional state in real time and provides immediate feedback when selecting a desired travel plan. For example, if the engine detects anxiety or dissatisfaction with the user's chosen plan, it will present other options.

[1541] 7. Booking Process Phase

[1542] After the user selects the desired travel plan, the selected information is sent from the terminal to the server, which then calls the external information providing means again to confirm the selected Shinkansen and hotel reservations.

[1543] 8. Reservation confirmation and completion phase

[1544] The server retrieves the reservation confirmation information and sends it to the terminal, which displays it to the user, indicating that the travel plan has been successfully completed.

[1545] Specific examples

[1546] For example, suppose a user logs into a website and enters the following information:

[1547] Budget: 100,000 yen

[1548] Travel dates: December 1, 2023 to December 5, 2023

[1549] Transportation: Shinkansen

[1550] Accommodation: Hot Spring Hotel

[1551] The device collects this information and sends it to a server, which then collects data such as bullet train schedules and availability of hotels with hot springs, and generates multiple travel plans.

[1552] for example,

[1553] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[1554] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[1555] At the same time, the emotion engine analyzes the user's emotional state from their facial expressions and tone of voice, and if the user is looking to relax, it makes adjustments such as adding hotels with hot springs to the recommendations.

[1556] The server sends these adjusted plans to the device, which displays them to the user. Real-time feedback suggests alternative plans if the user expresses anxiety or dissatisfaction when selecting Plan A.

[1557] Finally, the user selects the best plan and the reservation is confirmed. The reservation confirmation information is displayed on the terminal, and the user can confirm that the travel plan has been successfully completed.

[1558] The system configuration described above makes it possible to consistently and efficiently carry out travel planning and booking while taking emotional information into consideration.

[1559] The processing flow will be explained below.

[1560] Step 1:

[1561] A user opens a website or application and accesses a form to enter travel planning details.

[1562] Step 2:

[1563] The user inputs travel conditions such as budget, travel schedule, transportation, desired accommodation, etc. The input data is collected by the terminal and sent to the server.

[1564] Step 3:

[1565] The server analyzes the received travel condition data, and based on the analyzed data, prepares to call an external information providing means for collecting information that matches the travel conditions.

[1566] Step 4:

[1567] The server uses external information providers (e.g., transportation information providers, accommodation information providers) to collect necessary travel-related data, such as the availability and prices of Shinkansen trains and hotels with hot springs.

[1568] Step 5:

[1569] The server integrates the collected data and automatically generates travel plans. Multiple travel plans are created and each plan is evaluated based on criteria such as cost, travel time, and accommodation ratings.

[1570] Step 6:

[1571] The emotion engine collects user input data and emotion information such as the user's facial expressions and tone of voice, and analyzes the user's emotional state.

[1572] Step 7:

[1573] The server then adjusts the travel plan based on the results of the sentiment analysis. For example, if the user wants a relaxing trip, it will prioritize accommodations with quiet environments.

[1574] Step 8:

[1575] The server sends the adjusted travel plans to the terminal, which displays them to the user. The plan details are displayed in a list format that makes it easy for the user to compare them.

[1576] Step 9:

[1577] The emotion engine monitors the user's emotional state in real time, analyzing the emotional state and providing immediate feedback when the user selects a desired travel plan from the presented plans.

[1578] Step 10:

[1579] The user selects a desired travel plan, and the selected plan information is sent to the server by the terminal.

[1580] Step 11:

[1581] The server starts the reservation procedure for the selected travel plan, and confirms the reservation of the selected Shinkansen and hotel through an external information providing means.

[1582] Step 12:

[1583] The server receives the reservation confirmation information and sends it to the terminal, which displays it to the user.

[1584] Step 13:

[1585] The user checks the reservation confirmation information and finds out that the travel plan has been successfully completed. Through the above processing steps, a series of processes from travel planning to reservation completion is efficiently executed, taking into account the user's emotional information.

[1586] Example 2

[1587] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1588] Conventional travel planning systems only automatically generate travel plans based on user-entered conditions, but are unable to consider the user's emotional state, making it difficult to propose highly satisfying travel plans. Furthermore, there have been few systems that can analyze the user's emotions in real time and adjust or propose travel plans based on those emotions.

[1589] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1590] In this invention, the server includes an input means for a user to input travel conditions, a generation means for automatically generating multiple travel plans based on the travel conditions, and an emotion analysis means for adjusting the travel plans based on the user's emotional state, thereby making it possible to propose highly satisfying travel plans that reflect the user's emotional state in real time.

[1591] "Input means" refers to a means by which a user inputs travel conditions.

[1592] The "generation means" is a means for automatically generating multiple itineraries based on input travel conditions.

[1593] The "emotion analysis means" is a means for analyzing the user's emotional state and adjusting the travel plan based on that.

[1594] The "display means" is a means for displaying the generated itineraries to the user.

[1595] The "reservation procedure means" refers to the procedure means for reserving the travel plan selected by the user.

[1596] The "notification means" is a means for notifying the user of the results of the reservation procedure.

[1597] "Data collection means" refers to a means for collecting transportation information and accommodation information via external data providing means.

[1598] A "comparison tool" is a tool for comparing multiple travel plans based on cost and time criteria.

[1599] This invention is a system that handles everything from travel planning to booking, and is particularly equipped with an emotion engine that recognizes the user's emotions. This system can propose optimal travel plans taking into account the user's emotional state.

[1600] Specific configuration

[1601] Hardware and Software

[1602] The system uses the following hardware and software:

[1603] Server: Data processing, travel plan generation, and sentiment analysis.

[1604] Terminal: A PC or smartphone that performs user input and displays plans.

[1605] Emotion engine: Analyzes user emotions in real time and adjusts and suggests plans.

[1606] The software used includes:

[1607] Web application: Provides an interface for users to input their travel plans.

[1608] External API: Used to collect data to obtain information on transportation and accommodation.

[1609] Generative AI models: Used for itinerary generation and sentiment analysis.

[1610] How to use

[1611] 1. User Input

[1612] A user opens a website or application and enters the following information:

[1613] budget

[1614] travel itinerary

[1615] Transportation (e.g., bullet train, car, airplane, etc.)

[1616] Accommodation preferences (e.g. with hot springs, specific location, etc.)

[1617] The terminal collects this information and sends it to the server.

[1618] 2. Data Collection

[1619] The server receives the input data and collects information on Shinkansen trains and accommodations through external data providers, specifically information on Shinkansen availability and prices, availability and prices of hotels with hot springs, etc.

[1620] 3. Travel plan generation

[1621] The server generates multiple travel plans based on the collected data, and the plans are scored based on criteria such as cost, travel time, and accommodation ratings.

[1622] 4. Emotion analysis

[1623] The emotion engine analyzes the user's emotional state based on input data, facial expressions, tone of voice, etc. The travel plan is then adjusted based on the analyzed data. For example, if the user is looking for relaxation, hotels with hot springs will be suggested first.

[1624] 5. Plan presentation

[1625] The server sends the generated travel plan to the terminal, which then displays it to the user. The plan information is presented in an easy-to-compare format.

[1626] 6. Real-time feedback

[1627] The emotion engine monitors users' emotions in real time and provides immediate feedback when selecting travel plans, suggesting alternative plans if anxiety or dissatisfaction is detected.

[1628] 7. Reservation Procedure

[1629] After the user selects the desired travel plan, the information is sent to the server, which then calls the external data providing means again to confirm the selected Shinkansen and hotel reservations.

[1630] 8. Reservation Confirmation

[1631] The server receives the reservation confirmation information and sends it to the terminal, which displays the reservation completion information to the user.

[1632] Specific examples

[1633] For example, if a user logs into a website and enters the following information:

[1634] Budget: 100,000 yen

[1635] Travel dates: December 1, 2023 to December 5, 2023

[1636] Transportation: Shinkansen

[1637] Accommodation: Hot Spring Hotel

[1638] The device collects information and sends it to the server. The server collects data on Shinkansen trains and hotels with hot springs, and generates multiple travel plans:

[1639] Plan A: Shinkansen round trip (40,000 yen) + Hotel with hot spring (50,000 yen)

[1640] Plan B: Shinkansen round trip (¥35,000) + Hotel with hot spring (¥55,000)

[1641] The emotion engine analyzes the user's emotions and recommends hotels with hot springs if they are looking for relaxation. The server sends the adjusted plan to the device and displays it to the user. Real-time feedback allows the user to select the best plan.

[1642] Finally, the user selects a plan and the reservation is confirmed. Reservation confirmation information is displayed on the terminal, and the user confirms that the travel plan has been successfully completed.

[1643] Generative AI model prompt example

[1644] "Please tell us about your next trip. Explain your budget, travel dates, transportation, and accommodation preferences. Also, tell us how you want to feel during your trip."

[1645] The system configuration described above makes it possible to consistently and efficiently carry out travel planning and booking while taking emotional information into consideration.

[1646] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1647] Step 1:

[1648] User Input Phase

[1649] A user opens a website or application and enters details of their travel plans (budget, travel dates, transportation options, and preferred accommodations). The data entered by the user is collected by the device and sent to the server. The server then receives the travel conditions data. As a specific example, a user enters information such as a budget of 100,000 yen, travel dates from December 1st to 5th, 2023, travel by Shinkansen, and a hotel with a hot spring.

[1650] Input: Travel conditions entered by the user (budget, travel dates, transportation, accommodation preferences)

[1651] Output: Travel condition data sent to the server

[1652] Step 2:

[1653] Data Collection Phase

[1654] After the server receives the travel condition data, it collects transportation information and accommodation information via external data provision means. As a specific example, the server calls the APIs of a Shinkansen train operation information service and an accommodation information service to obtain the availability and prices of Shinkansen trains and hotels with hot springs.

[1655] Input: Travel condition data

[1656] Output: Transportation and accommodation information

[1657] Step 3:

[1658] Travel plan generation phase

[1659] The server automatically generates multiple travel plans based on the data collected. The server scores the generated plans based on criteria such as cost, travel time, and accommodation ratings. Specifically, the server combines Shinkansen prices and hotel prices to calculate the cost of the travel plan.

[1660] Input: Transportation and accommodation information

[1661] Output: Multiple itineraries

[1662] Step 4:

[1663] Sentiment Analysis Phase

[1664] The emotion engine analyzes the user's input data, facial expressions, and tone of voice to extract the user's emotional state. Based on this emotional data, the server adjusts the travel plan. For example, if the user is looking for relaxation, it will emphasize and suggest hotels with hot springs.

[1665] Input: User's emotional information

[1666] Output: Adjusted itinerary

[1667] Step 5:

[1668] Plan presentation phase

[1669] The server sends the generated travel plan to the terminal, which displays it to the user. The terminal displays a list of details of multiple travel plans so that the user can easily compare them.

[1670] Input: Adjusted travel plans

[1671] Output: The itinerary displayed to the user

[1672] Step 6:

[1673] Real-time feedback phase

[1674] The emotion engine monitors the user's emotional state in real time and provides immediate feedback when selecting a travel plan, for example suggesting other options if the user expresses anxiety or dissatisfaction with the plan they have chosen.

[1675] Input: Real-time user emotion information

[1676] Output: Alternative suggestions

[1677] Step 7:

[1678] Booking process phase

[1679] When the user selects the desired travel plan, the information is sent from the terminal to the server, which then calls the external data providing means again to confirm the reservation of the selected Shinkansen and hotel.

[1680] Input: User selected travel plan

[1681] Output: Confirmed reservation information

[1682] Step 8:

[1683] Reservation confirmation and completion phase

[1684] The server retrieves the reservation confirmation information and sends it to the terminal, which displays the reservation confirmation information to the user, informing them that their travel plans have been successfully completed.

[1685] Input: Confirmed reservation information

[1686] Output: Booking confirmation information displayed to the user

[1687] (Application example 2)

[1688] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1689] In conventional autonomous vehicles, operation plans are determined without taking the driver's emotions into consideration, which increases driver stress and anxiety and reduces driving comfort. Furthermore, there was no system that provided real-time feedback to the driver based on their emotions when proposing a plan or during operation. Therefore, in order to provide a safer and more comfortable driving environment, a system that analyzes the driver's emotions in real time and adjusts operation plans based on that information is needed.

[1690] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes emotion analysis means for analyzing the user's emotion information, plan adjustment means for adjusting the travel plan based on the emotion information, and feedback means for presenting the adjusted plan. This makes it possible to analyze the driver's emotion information in real time and adjust or propose a trip plan based on it.

[1691] "User" refers to the general public who uses the system to input travel conditions.

[1692] "Interface means" refers to an input device or software that allows a user to input travel conditions.

[1693] "Processing means" refers to a device or software that automatically generates multiple travel plans based on input travel conditions.

[1694] "Display means" refers to a device or software that visually presents the generated travel plans to the user.

[1695] "Reservation procedure means" refers to the device or software used to actually reserve the travel plan selected by the user.

[1696] "Notification means" refers to devices or software for notifying users of the results of their reservation procedures.

[1697] "Emotion analysis means" refers to devices or software for analyzing users' emotional information.

[1698] "Plan adjustment means" refers to a device or software for adjusting a travel plan based on the analyzed emotional information.

[1699] "Feedback means" refers to a device or software for presenting the adjusted plan to the user and providing feedback in real time.

[1700] "External information providing means" refers to devices and services for collecting external information such as transportation information and accommodation information.

[1701] This invention combines an emotion engine with a system that handles everything from user travel planning to reservations, and is a specific form for providing safe and comfortable driving plans for drivers of self-driving vehicles. The details are provided below.

[1702] 1. User input phase

[1703] Users use an application installed on their smartphone or head-mounted display to input travel conditions such as destination, travel time, desired route, etc. This allows the user's desired travel plan to be clearly collected.

[1704] 2. Data Collection Phase

[1705] The server collects information on traffic, weather, and congestion at destinations through external information providers, obtaining the latest data in real time via a REST API.

[1706] 3. Operation plan generation phase

[1707] Based on the collected data, the server generates multiple operation plans using Python scripts and machine learning algorithms (e.g., optimization algorithms) and evaluates each one.

[1708] 4. Sentiment Analysis Phase

[1709] Using the camera and microphone installed on the smartphone or head-mounted display, and using face recognition API and voice recognition API, emotional information is analyzed in real time from the user's facial expressions and tone of voice, thereby obtaining emotional information about the user.

[1710] 5. Plan presentation phase

[1711] The server presents the user with multiple operation plans that take emotion information into consideration through a display means, thereby providing the user with an intuitive interface for selecting the optimal plan based on emotion.

[1712] 6. Real-time feedback phase

[1713] Based on the analyzed emotional information, the server adjusts and re-presents the operation plan according to the user's real-time emotions through feedback means. For example, if the server detects anxiety or dissatisfaction with the plan selected by the user, it will present an alternative plan.

[1714] 7. Operational Phase

[1715] After the user selects the optimal plan, the app sends driving instructions to the autonomous vehicle's onboard computer, starting a safe and comfortable journey.

[1716] Specific examples

[1717] User input: Destination "Tokyo", travel time "within 2 hours"

[1718] Prompt (when inputting to a generative AI model):

[1719] Suggest a route that will make the driver feel safe and relaxed. Emotions are a little tense right now. Choose a safe and efficient route within the desired timeframe.

[1720] As described above, this invention is a system that analyzes user emotions in real time and adjusts and proposes operation plans based on that analysis, providing a safe and comfortable operating environment for self-driving vehicles.

[1721] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1722] Step 1:

[1723] The terminal provides an interface where users can input travel conditions (destination, travel time, desired route, etc.) using a smartphone or head-mounted display. The input information is sent from the terminal to the server. The input is the user's desired conditions, and the output is data sent to the server.

[1724] Step 2:

[1725] The server collects traffic information, weather information, and congestion information at the destination via external information providing means based on the travel conditions sent from the terminal. The collected information is stored in the server. The input is the travel conditions and external information, and the output is the storage of the collected data.

[1726] Step 3:

[1727] The server generates a trip plan using the collected data. It uses Python scripts and machine learning algorithms to apply optimization algorithms to create multiple trip plans. The generated trip plans are stored on the server. The input is the collected data and the algorithm, and the output is the trip plan.

[1728] Step 4:

[1729] The device uses the camera and microphone of the smartphone or head-mounted display to analyze the user's face and voice, and analyzes facial expressions and tone of voice in real time using an emotion analysis API. The analysis results are sent from the device to a server. The input is facial expression and voice data, and the output is the emotion analysis results.

[1730] Step 5:

[1731] The server adjusts the previously created operation plan based on the emotion analysis results. Using the plan adjustment means, it selects the operation plan that best suits the user's emotional state and creates an adjusted operation plan. The input is the emotion analysis results and the operation plan, and the output is the adjusted operation plan.

[1732] Step 6:

[1733] The terminal presents the adjusted operation plans from the server to the user. The plans are displayed in a visually easy-to-compare format to make it easier for the user to select a plan. The input is the adjusted operation plan, and the output is the plan presented to the user.

[1734] Step 7:

[1735] The server provides real-time feedback on the trip plan selected by the user, detects the user's anxiety and dissatisfaction, and presents alternative plans as necessary. The input is the user's preferences and emotional state, and the output is the presentation of alternative plans.

[1736] Step 8:

[1737] The terminal confirms the user's final selected trip plan and sends instructions to the autonomous vehicle's onboard computer, which then executes the selected trip plan. The input is the final selected trip plan, and the output is instructions to the onboard computer.

[1738] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1739] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1740] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1741] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1742] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1743] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1744] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1745] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1746] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1747] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1748] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1749] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1750] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1751] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1752] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1753] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1754] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1755] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1756] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1757] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1758] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1759] The following is further disclosed regarding the above embodiment.

[1760] (Claim 1)

[1761] an interface means for a user to input travel conditions;

[1762] a processing means for automatically generating a plurality of travel plans based on the travel conditions;

[1763] a display means for displaying the plurality of travel plans;

[1764] A reservation procedure for booking the travel plan selected by the user;

[1765] Notification means for notifying the result of the reservation procedure

[1766] A system including:

[1767] (Claim 2)

[1768] 2. The system according to claim 1, further comprising information collection means for collecting transportation information and accommodation information via external information providing means according to said travel conditions.

[1769] (Claim 3)

[1770] 10. The system of claim 1, further comprising a comparison means for comparing the plurality of travel plans based on cost and time criteria.

[1771] "Example 1"

[1772] (Claim 1)

[1773] an interface means for a user to input travel conditions;

[1774] a processing means for automatically generating a plurality of travel plans based on the travel conditions;

[1775] a display means for displaying the plurality of travel plans;

[1776] A reservation procedure for booking the travel plan selected by the user;

[1777] notification means for notifying the result of the reservation procedure;

[1778] an information collection means for collecting transportation information and accommodation information via an external information providing means;

[1779] a comparison means for comparing the generated travel plans on cost and time criteria;

[1780] An optimization method for optimizing multiple travel plans based on evolutionary algorithms

[1781] A system including:

[1782] (Claim 2)

[1783] 10. The system of claim 1, further comprising a graphical user interface means for visually displaying said plurality of travel plans.

[1784] (Claim 3)

[1785] 2. The system according to claim 1, further comprising an update means for automatically updating the reservation information confirmed by said reservation procedure means.

[1786] "Application Example 1"

[1787] (Claim 1)

[1788] an input means for a user to input conditions;

[1789] processing means for automatically generating a plurality of delivery plans based on the conditions;

[1790] a display means for displaying the plurality of delivery plans;

[1791] an ordering procedure for ordering the delivery plan selected by the user;

[1792] Notification means for notifying the result of the ordering procedure

[1793] A system including:

[1794] (Claim 2)

[1795] 2. The system according to claim 1, further comprising information collecting means for collecting restaurant information and delivery service information via external information providing means according to said conditions.

[1796] (Claim 3)

[1797] 10. The system of claim 1, further comprising a comparison means for comparing the plurality of delivery plans based on cost and delivery time criteria.

[1798] "Example 2: Combining Emotion Engines"

[1799] (Claim 1)

[1800] an input means for a user to input travel conditions;

[1801] a generation means for automatically generating a plurality of travel plans based on the travel conditions;

[1802] emotion analysis means for adjusting the travel plan based on the user's emotional state;

[1803] a display means for displaying the plurality of travel plans;

[1804] a reservation procedure for booking a user's selected travel plan;

[1805] Notification means for notifying the result of the reservation procedure

[1806] A system including:

[1807] (Claim 2)

[1808] 2. The system according to claim 1, further comprising a data collection means for collecting transportation information and accommodation information via an external data providing means according to the travel conditions.

[1809] (Claim 3)

[1810] 10. The system of claim 1, further comprising a comparison means for comparing said plurality of travel plans on cost and time criteria.

[1811] "Application example 2 when combining emotion engines"

[1812] (Claim 1)

[1813] an interface means for a user to input travel conditions;

[1814] a processing means for automatically generating a plurality of travel plans based on the travel conditions;

[1815] a display means for displaying the plurality of travel plans;

[1816] A reservation procedure for booking the travel plan selected by the user;

[1817] notification means for notifying the result of the reservation procedure;

[1818] emotion analysis means for analyzing emotion information of a user;

[1819] plan adjustment means for adjusting a travel plan based on the emotion information;

[1820] feedback means for presenting said adjusted plan;

[1821] A system including:

[1822] (Claim 2)

[1823] 2. The system according to claim 1, further comprising information collection means for collecting transportation information and accommodation information via external information providing means according to said travel conditions.

[1824] (Claim 3)

[1825] 10. The system of claim 1, further comprising a comparison means for comparing the plurality of travel plans based on cost and time criteria. [Explanation of symbols]

[1826] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. an interface means for a user to input travel conditions; a processing means for automatically generating a plurality of travel plans based on the travel conditions; a display means for displaying the plurality of travel plans; A reservation procedure for booking the travel plan selected by the user; Notification means for notifying the result of the reservation procedure A system including:

2. 2. The system according to claim 1, further comprising information gathering means for gathering transportation information and accommodation information via external information providing means according to said travel conditions.

3. 2. The system of claim 1, further comprising a comparison means for comparing said plurality of travel plans on cost and time criteria.

Images