System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving terms of a travel request from a user; means for generating a plurality of travel plans based on the terms; and means for presenting the generated travel plans to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, users have come to have diverse travel styles and detailed requests. However, manually selecting travel destinations that match these specific requests is often tedious and time-consuming. Furthermore, it is difficult to generate travel plans that take into account specific dietary restrictions, the availability of facilities, and the types of activities, making it difficult for users to plan the trip they desire. Therefore, there is a need for a system that can efficiently generate and propose travel plans based on the user's specific requests. [Means for solving the problem]

[0005] The present invention provides a system that receives travel requirements from a user, generates multiple travel plans based on those requirements using a generative AI model, and presents them to the user. This system includes a means for efficiently generating optimal travel plans based on specific requirements such as landscape imagery, activity types, facility availability, dietary preferences and restrictions, length of stay, and number of people. This allows users to easily select the travel plan that best suits their needs.

[0006] "User" refers to an individual or organization who uses the system by entering desired travel plan conditions.

[0007] "Travel preferences" refers to the specific elements that users look for when deciding on travel plans (image of scenery, type of activity, level of facilities, dietary preferences and restrictions, length of stay, number of people, etc.).

[0008] A "generative AI model" refers to an algorithm or machine learning model that uses artificial intelligence to generate optimal travel plans based on input criteria.

[0009] "Travel Plan" refers to a specific travel proposal that combines information on travel destinations, accommodations, activities, etc. that meet the user's requirements.

[0010] "Scenery image" refers to the specific landscape or natural environment (e.g., mountains, lakes, torii gates, etc.) desired by users at their travel destination.

[0011] "Activity type" refers to the specific type of activity or experience (e.g., hiking, river rafting, hot air ballooning, etc.) that a user wants to engage in at a travel destination.

[0012] "Completeness of facilities" refers to the content and level of facilities and services provided at accommodation and tourist facilities.

[0013] "Dietary preferences or restrictions" refers to any specific dietary needs you may have while traveling (e.g., vegan, halal, allergy-friendly, etc.).

[0014] "Length of stay" refers to the length of time (number of days or dates) that a user stays at a travel destination.

[0015] "Number of people" refers to the total number of users participating in the trip. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention specifically demonstrates a system that, when a user inputs their desired travel conditions, uses a generative AI model to generate an optimal travel plan and propose multiple options to the user.

[0038] Program Overview

[0039] The system has the following main functions:

[0040] 1. Receiving user input

[0041] 2. Analysis of input data

[0042] 3. Generate travel plans that meet the conditions

[0043] 4. Proposing and displaying travel plans

[0044] Detailed program processing

[0045] 1. Receiving user input

[0046] Users input their desired travel conditions into a form displayed on the screen via their device, including information such as landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting, hot air ballooning), level of facilities, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, and number of people.

[0047] The terminal receives the data entered by the user and transmits it to the server.

[0048] 2. Analysis of input data

[0049] The server receives and analyzes user data sent from the device, extracting it into individual parameters and preparing them for passing to the generative AI model.

[0050] 3. Generate travel plans that meet the conditions

[0051] The server passes the conditions based on the extracted parameters to a generative AI model (machine learning algorithm), which then searches the database for suitable travel destination information based on the conditions, selects candidate travel destinations that meet each condition, and collects detailed information for each destination (such as region, accommodation, activity locations, and temperature data).

[0052] The generative AI model generates multiple optimal travel plans from the collected data and returns them to the server.

[0053] 4. Proposing and displaying travel plans

[0054] The server sends the generated itineraries to the device, which then displays them to the user along with detailed information. Each itinerary includes details about the area, accommodation, temperature, and activities. The user can then select the itinerary that best suits their needs.

[0055] Specific examples

[0056] For example, if a user enters the following criteria:

[0057] A place with mountains and lakes

[0058] Where the torii gate is located

[0059] River rafting

[0060] Cottage-style accommodation

[0061] Suitable for those with egg allergies

[0062] Open-air hot spring

[0063] The device sends these conditions to the server, which then detects "Hakone" and "Nikko" as travel destinations that match the conditions. The generative AI model generates a "Hakone Plan" and a "Nikko Plan" and returns them to the server along with detailed information about each plan.

[0064] A detailed example of a Hakone plan includes Lake Ashi, the Hakone Shrine's torii gate, a river cruise, cottage-style accommodation, meals for those with egg allergies, and an open-air hot spring. Similarly, a Nikko plan comprehensively suggests scenery, activities, and meals based on the user's requirements.

[0065] This allows users to compare the details of the Hakone and Nikko plans and choose the travel plan that best suits their needs.

[0066] This system makes it easy for users to plan their trips by efficiently generating travel plans based on the user's detailed wishes and presenting them as multiple options.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] The user enters the desired travel conditions.

[0070] Users enter their desired travel conditions (image of scenery, type of activity, level of facilities, dietary preferences and restrictions, length of stay, number of people, etc.) into a form displayed on the device screen.

[0071] Step 2:

[0072] The terminal receives the user's input and sends it to the server.

[0073] The terminal formats the data entered by the user and sends a request to the server.

[0074] Step 3:

[0075] The server receives and analyzes the data from the user.

[0076] The server analyzes the received data and extracts each condition as a separate parameter.

[0077] Step 4:

[0078] The server prepares to pass the extracted parameters to the generative AI model.

[0079] The server converts the extracted conditions into an appropriate format and performs preprocessing to pass them to the generative AI model.

[0080] Step 5:

[0081] The generative AI model searches the database based on the criteria.

[0082] The generative AI model searches for suitable travel destination information from within the database based on the conditions passed from the server.

[0083] Step 6:

[0084] The generative AI model selects suitable travel destinations and collects detailed information about each one.

[0085] The generative AI model collects detailed information such as scenery, activities, accommodation, food options, and temperature data to select the best candidates.

[0086] Step 7:

[0087] A generative AI model creates multiple itineraries.

[0088] The generative AI model uses the collected information to generate multiple travel plans that meet the user's requirements.

[0089] Step 8:

[0090] The server transmits the generated travel plan to the terminal.

[0091] The server sends multiple travel plans received from the generative AI model to the user's device.

[0092] Step 9:

[0093] The device displays the travel plan to the user.

[0094] The terminal receives the travel plan sent from the server and displays it to the user along with detailed information.

[0095] Step 10:

[0096] The user selects the desired travel plan from the presented plans.

[0097] Users can select the travel plan that best suits their needs from the plans presented.

[0098] Step 11:

[0099] The device will display details of the plan the user has selected and the booking process.

[0100] The device will display detailed information about the plan the user has selected, a link to the reservation page, and guide them on their next steps.

[0101] Example 1

[0102] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0103] Conventional travel plan generation systems have difficulty quickly and automatically generating the optimal travel plan based on a user's desired conditions and providing multiple options. There is also a need for a system that can efficiently analyze and reflect travel plans based on the user's detailed desired conditions. This has led to the problem that it is difficult for users to select the travel plan that best meets their needs.

[0104] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0105] In this invention, the server includes means for receiving desired travel conditions from a user, means for generating multiple travel plans based on the conditions, means for generating an optimal travel plan based on the conditions using a generative AI model, and means for presenting the generated travel plans to the user. This makes it possible to quickly and efficiently generate an optimal travel plan based on the user's detailed desired conditions and provide it to the user as multiple options.

[0106] "User" means an individual or group that requests a travel plan and enters its requirements.

[0107] "Travel preferences" are the specific elements and requirements that users desire for their trip (e.g., landscape image, type of activity, level of facilities, dietary preferences or restrictions, length of stay, number of people, etc.).

[0108] "Terminal" refers to a device used by a user, such as a computer, smartphone, or tablet, which is used to input and receive travel preferences.

[0109] A "server" is a computer system that receives, analyzes, and processes data sent from a terminal.

[0110] A "generative AI model" is an algorithm or software that uses artificial intelligence technology to generate optimal travel plans based on input criteria.

[0111] A "trip plan" is a detailed travel plan based on a user's desired conditions, including destinations, accommodations, activities, temperature data, etc.

[0112] A "database" is an online or offline data storage system for storing information such as travel destination information, accommodation information, activity locations, temperature data, etc.

[0113] An "HTTP request" is part of a communication protocol for sending and receiving data between a terminal and a server, and is used to send the user's conditions to the server.

[0114] "JSON format" is an abbreviation for JavaScript Object Notation, a text-based format used as a data exchange format.

[0115] "Parameters" are the individual elements that make up the desired travel conditions (e.g., mountains, lakes, torii gates, river rafting, cottages, etc.).

[0116] The present invention is a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and then suggests multiple options to the user.

[0117] The system uses the following hardware and software:

[0118] 1. Terminal: A device such as a computer, smartphone, or tablet where users input their desired travel conditions. This terminal provides a user interface using a web browser or dedicated application.

[0119] 2. Server: A computer system for receiving, analyzing, and processing data sent by users. This server is responsible for storing user data, analyzing it, inputting it into the generative AI model, receiving the results, and generating the itinerary.

[0120] 3. Generative AI model: An algorithm or software that uses artificial intelligence technology to generate optimal travel plans based on input criteria. Examples include machine learning algorithms and natural language processing technology.

[0121] Specific processing of the system

[0122] Accepting user input:

[0123] Users input the desired travel conditions into a form displayed on the device screen. These conditions include landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting, hot air ballooning), facility availability, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, number of people, etc. The device stores the input information in its memory and sends this data to the server when the "send" button is pressed.

[0124] Analysis of input data:

[0125] The server receives the HTTP POST request sent from the device and parses the request body in JSON format. The server then breaks down the parsed data into individual parameters (e.g., "mountain," "lake," "torii gate," "river boating," "cottage," etc.) and converts them into a format suitable for input to the generative AI model.

[0126] Generate matching itineraries:

[0127] The server passes the formatted parameters to the generative AI model, which queries the database based on the received parameters to find suitable travel destinations, collects detailed information about the suitable destinations (such as region, accommodation, activity locations, and temperature data), and creates multiple optimal itineraries. This information is returned to the server in JSON format.

[0128] Suggest and display itineraries:

[0129] The server receives and analyzes the JSON data returned by the generative AI model. The server converts the travel plan information into a format that is easy for the user to understand and sends it to the device as an HTTP response. The device displays the received data on the screen and presents the user with multiple travel plans. The user can compare the displayed plans and select the one that best suits their needs.

[0130] Specific examples

[0131] For example, if a user enters the following criteria:

[0132] A place with mountains and lakes

[0133] Where the torii gate is located

[0134] River rafting

[0135] Cottage-style accommodation

[0136] Suitable for those with egg allergies

[0137] Open-air hot spring

[0138] An example prompt is:

[0139] Prompt statement:

[0140] We want the user to suggest itineraries related to "places with mountains and lakes, places with torii gates, river rafting, cottages, egg allergy friendly, and open-air hot springs." Based on this, please generate the optimal itinerary.

[0141] These conditions are sent from the device to a server, which then detects "Location A" and "Location B" as travel destinations that meet the conditions. The generative AI model uses this information to generate a "Location A Plan" and a "Location B Plan," and returns them to the server along with detailed information about each plan (mountains and lakes, torii gates, river rafting spots, cottage-style accommodations, meals for people with egg allergies, open-air hot springs, etc.). This allows users to compare the details of the "Location A Plan" and "Location B Plan" and choose the travel plan that best suits their needs.

[0142] This system allows users to efficiently generate travel plans based on their detailed desired conditions, and easily select the most suitable plan from multiple options.

[0143] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0144] Step 1: Receiving User Input

[0145] Description: The user enters the desired travel conditions into a form displayed on the device screen. The user's input (e.g., a place with mountains and lakes, a place with a torii gate, river rafting, a cottage, suitable for egg allergies, an open-air hot spring) is provided to the device as input data. The device stores this input data in its memory, and when the submit button on the form is pressed, it sends the stored data to the server via an HTTP POST request. Specifically, the device's browser or application sends the user input to the server in JSON format.

[0146] Step 2: Analyzing the input data

[0147] Description: The server receives an HTTP POST request sent from the device and parses the request body in JSON format. The parsed data is provided to the server as input data. The server breaks this data down into individual parameters (e.g., scenery, activity, dietary restrictions, etc.). Specifically, the server-side analysis module parses the JSON data and converts it into a format that can be input into a generative AI model based on the parameters.

[0148] Step 3: Generate itineraries that meet your criteria

[0149] Description: The server passes formatted parameters to the generative AI model. The generative AI model receives these parameters as input data and searches its database for suitable travel destinations based on that data. The search results are provided to the generative AI model as intermediate data. The generative AI model then collects detailed information (e.g., region, accommodation, activity locations, temperature data, etc.) based on the search results and creates multiple optimal travel plans. Specifically, the generative AI model executes database queries, assembles a travel plan from the obtained information, and returns it to the server in JSON format.

[0150] Step 4: Propose and display travel plans

[0151] Description: The server receives the JSON data returned from the generative AI model and provides it as output data. This data is parsed and converted into a format that is easy for the user to understand (e.g., HTML or text format). The server then sends the converted data to the device as an HTTP response. The final output data, a travel plan, is provided to the device. The device receives this data and displays it to the user. The user can compare the displayed multiple travel plans and select the one that best suits their needs. Specifically, the device's display module visually presents the travel plan information, allowing the user to compare them.

[0152] (Application example 1)

[0153] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0154] In recent years, there has been a demand for providing meals that meet individual preferences and health conditions. However, selecting the optimal menu based on detailed conditions regarding dietary restrictions and specific ingredients requires a lot of time and effort. For this reason, there is a need for a system that can efficiently suggest and optimize menus that meet the conditions.

[0155] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0156] In this invention, the server includes a means for receiving meal conditions requested by the user, a means for generating multiple meal menus based on the conditions, and a means for presenting the user with dining facilities that offer the generated meal menus, thereby making it possible to propose the optimal meal menu according to the user's detailed conditions.

[0157] "User" means an individual or group of people who utilizes the System to enter dietary requirements.

[0158] "Dietary requirements" refers to specific conditions such as the user's desired ingredients, dietary restrictions, level of spiciness, and serving amount.

[0159] "Food Menu" refers to a selection of food suggestions generated based on the user's dietary requirements.

[0160] "Food and beverage establishments" are establishments such as restaurants and cafes that offer food menus and allow users to order.

[0161] A "generative AI model" refers to a machine learning algorithm that generates optimal cooking menus based on the user's dietary requirements.

[0162] This invention is a system that uses a generative AI model to propose optimal meal menus based on a user's desired meal conditions. This system receives the user's meal conditions as input, generates multiple meal menus based on them, and then suggests dining establishments that meet the conditions entered by the user.

[0163] First, the user uses the terminal to input their desired dietary requirements. Dietary requirements include dietary restrictions (e.g., vegetarian, gluten-free), spiciness level (e.g., normal, spicy), desired ingredients (e.g., tomato, cheese), and serving size (e.g., two servings). The terminal receives the data entered by the user and sends it to the server.

[0164] The server receives and analyzes the user's meal requirements sent from the device. The analyzed data is extracted as individual parameters and prepared to be passed to the generative AI model. The generative AI model searches the database for food menus that match the user's requirements and collects detailed information about each (such as the name of the dish, dining facility, price, and dish details). The generative AI model generates multiple optimal food menus from the collected data and returns them to the server.

[0165] The server sends the generated menus to the terminal, which displays them with detailed information to the user, allowing the user to select the menu that best suits their needs and order delivery.

[0166] For example, if a user enters the following criteria:

[0167] Dietary: Vegetarian

[0168] Allergies: Gluten

[0169] Spice level: Medium

[0170] Desired ingredients: Tomato, cheese

[0171] Serving size: 2 servings

[0172] Based on this, the server will suggest the following menu options:

[0173] 1. Dish name: Vegetarian pizza with tomato and cheese

[0174] Dining options: Vegetarian Restaurant A

[0175] Price: 1500 yen

[0176] Details: Made with gluten-free pizza dough, topped with tomato and mozzarella cheese

[0177] 2. Dish name: Cheese and tomato pasta

[0178] Dining facilities: Healthy Cafe B

[0179] Price: 1200 yen

[0180] Details: Gluten-free pasta, fresh tomato sauce and Parmesan cheese

[0181] An example prompt might look like this:

[0182] conditions:

[0183] Dietary: Vegetarian

[0184] Allergies: Gluten

[0185] Spice level: Medium

[0186] Desired ingredients: Tomato, cheese

[0187] Serving size: 2 servings

[0188] Based on this, please suggest an appropriate menu.

[0189] In this way, a food menu based on the user's detailed preferences is efficiently generated and presented as multiple options, making meal selections easier for the user.

[0190] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0191] Step 1:

[0192] The user inputs their desired dietary requirements using a terminal. The input information includes dietary restrictions, spiciness level, desired ingredients, serving size, etc. This input information is sent from the terminal to the server.

[0193] Step 2:

[0194] The server receives and analyzes the user's dietary requirements sent from the device. During the analysis process, the input data is extracted as individual parameters, which are then prepared for passing to the generative AI model.

[0195] Step 3:

[0196] The server then passes the analyzed data to a generative AI model, which then searches a database for food menus that match the user's criteria and narrows down the menu based on the criteria.

[0197] Step 4:

[0198] The server uses a generative AI model to generate multiple optimal menus from the collected data, including the name of the dish, the dining establishment, the price, and details of the dish.

[0199] Step 5:

[0200] The server sends the generated menus to the terminal, which displays them to the user along with detailed information. The user compares the details of each menu and selects the desired dish.

[0201] Step 6:

[0202] The user selects the desired menu item and places a delivery order. The terminal sends the selected order information to the server, which then transmits the order information to the restaurant.

[0203] This allows users to easily select the most suitable food menu that meets their specific requirements and use the delivery service.

[0204] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0205] This invention specifically describes a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and suggests multiple options to the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it also suggests travel plans that correspond to the user's emotions.

[0206] Program Overview

[0207] The system has the following main functions:

[0208] 1. Receiving user input

[0209] 2. Analysis of input data

[0210] 3. Generate travel plans that meet the conditions

[0211] 4. Proposing and displaying travel plans

[0212] 5. Emotion engine recognizes and reflects user emotions

[0213] Detailed program processing

[0214] 1. Receiving user input

[0215] Users input their desired travel conditions into a form displayed on the screen via their device, including information such as landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting, hot air ballooning), level of facilities, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, and number of people.

[0216] The terminal receives the data entered by the user and transmits it to the server.

[0217] 2. Analysis of input data

[0218] The server receives and analyzes user data sent from the device, extracting it into individual parameters and preparing them for passing to the generative AI model.

[0219] 3. Generate travel plans that meet the conditions

[0220] The server passes the conditions based on the extracted parameters to a generative AI model (machine learning algorithm), which then searches the database for suitable travel destination information based on the conditions, selects candidate travel destinations that meet each condition, and collects detailed information for each destination (such as region, accommodation, activity locations, and temperature data).

[0221] The generative AI model generates multiple optimal travel plans from the collected data and returns them to the server.

[0222] 4. Proposing and displaying travel plans

[0223] The server sends the generated itineraries to the device, which then displays them to the user along with detailed information. Each itinerary includes details about the area, accommodation, temperature, and activities. The user can then select the itinerary that best suits their needs.

[0224] 5. Emotion engine recognizes and reflects user emotions

[0225] The emotion engine analyzes the user's emotional state in real time as they type, using the device's camera, microphone, and input data to analyze the user's facial expressions, tone of voice, and typing speed.

[0226] The server uses the emotion data obtained from the emotion engine to adjust the conditions based on the user's emotional state. For example, if the user is feeling stressed, the server instructs the generative AI model to prioritize generating relaxing travel plans.

[0227] The emotion engine also stores the user's emotional state as a history and uses it as a reference when generating the next travel plan, allowing it to propose an even more optimal plan based on the user's emotional history.

[0228] Specific examples

[0229] For example, if a user enters the following criteria:

[0230] A place with mountains and lakes

[0231] Where the torii gate is located

[0232] River rafting

[0233] Cottage-style accommodation

[0234] Suitable for those with egg allergies

[0235] Open-air hot spring

[0236] The device sends these conditions to the server, and the emotion engine recognizes that the user has a happy expression when entering the information. The emotion engine then sends this information to the server, which then detects "Hakone" and "Nikko" as travel destinations that match the conditions.

[0237] The generative AI model generates a "Hakone Plan" and a "Nikko Plan," and returns each plan to the server along with detailed information. Examples of detailed Hakone plans include Lake Ashi, the torii gates of Hakone Shrine, a river cruise, cottage-style accommodation, meals suitable for egg allergies, and open-air hot springs. Similarly, Nikko plans comprehensively suggest scenery, activities, and meal options based on the user's requirements.

[0238] This allows users to compare the details of the Hakone and Nikko plans and choose the travel plan that best suits their needs. The emotion engine supports plan generation based on the user's emotional state, resulting in travel suggestions that will satisfy the user even more.

[0239] The processing flow will be explained below.

[0240] Step 1:

[0241] The user enters the desired travel conditions.

[0242] Users enter their desired travel conditions (for example, "a place with mountains and lakes" as a scenic image, "hiking" as the type of activity, "cottage-style accommodation" as the level of facilities, "accommodation for egg allergies" as a dietary preference or restriction, length of stay, number of people, etc.) into a form displayed on the device screen.

[0243] Step 2:

[0244] The terminal receives the user's input and sends it to the server.

[0245] The terminal formats the data entered by the user and sends a request to the server.

[0246] Step 3:

[0247] The emotion engine analyzes user emotions in real time.

[0248] The device's emotion engine uses the camera and microphone to analyze the user's facial expressions and tone of voice to generate emotional data.

[0249] Step 4:

[0250] The device transmits the emotion data to the server.

[0251] The device transmits the analyzed emotion data to the server, which delivers it together with the travel conditions.

[0252] Step 5:

[0253] The server receives and analyzes the data from the user.

[0254] The server analyzes the received user data and emotion data and extracts each condition as an individual parameter.

[0255] Step 6:

[0256] The server prepares to pass the extracted parameters to the generative AI model.

[0257] The server converts the extracted conditions and emotion data into an appropriate format and performs preprocessing to pass it to the generative AI model.

[0258] Step 7:

[0259] The generative AI model searches the database based on the criteria and emotion data.

[0260] The generative AI model searches for suitable travel destination information from within the database based on the conditions and emotional data passed from the server.

[0261] Step 8:

[0262] The generative AI model selects suitable travel destinations and collects detailed information about each one.

[0263] The generative AI model collects detailed information such as scenery, activities, accommodation, food options, and temperature data to select the best candidates.

[0264] Step 9:

[0265] A generative AI model creates a travel plan that reflects emotional data.

[0266] The generative AI model generates the optimal travel plan for the user based on the collected information and the user's emotional state.

[0267] Step 10:

[0268] The server transmits the generated travel plan to the terminal.

[0269] The server sends multiple travel plans received from the generative AI model to the user's device.

[0270] Step 11:

[0271] The device displays the travel plan to the user.

[0272] The terminal receives the travel plan sent from the server and displays it to the user along with detailed information.

[0273] Step 12:

[0274] The user selects the desired travel plan from the presented plans.

[0275] Users can select the travel plan that best suits their needs from the plans presented.

[0276] Step 13:

[0277] The device will display details of the plan the user has selected and the booking process.

[0278] The device will display detailed information about the plan the user has selected, a link to the reservation page, and guide them on their next steps.

[0279] Example 2

[0280] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0281] Conventional travel plan generation systems can propose plans that reflect the user's desired travel conditions, but they cannot adjust the plans to reflect the user's real-time emotional state. This makes it difficult to provide optimal travel plans that reflect the user's emotions. Furthermore, because the system does not take the user's emotions into consideration, the proposed travel plans often do not meet the user's expectations.

[0282] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0283] In this invention, the server includes means for receiving desired travel conditions from a user, means for generating multiple travel plans based on the conditions, means for presenting the generated travel plans to the user, emotion analysis means for analyzing the emotional state of the user, and means for adjusting the travel plans based on the emotional state of the user. This makes it possible to propose travel plans that correspond to the emotional state of the user, and to provide travel plans that will highly satisfy the user.

[0284] "Travel preferences" refer to the specific elements and requests that users have for their travel plans, including landscape image, types of activities, level of facilities, dietary preferences and restrictions, length of stay, number of people, etc.

[0285] "Generative AI model" refers to machine learning algorithms and natural language processing algorithms used to generate optimal travel plans based on a user's desired travel conditions.

[0286] "Emotion analysis means" refers to a function that analyzes a user's emotional state in real time, and includes technology that uses a camera or microphone to analyze a user's facial expressions and tone of voice.

[0287] A "Travel Plan" is a proposal that details the activities and accommodations desired by a User at a particular travel destination.

[0288] "Server" refers to the equipment or system that receives input data from users, analyzes it, works with the generative AI model to generate a travel plan, and finally sends it to the terminal.

[0289] A "terminal" is a device (such as a PC, smartphone, or tablet) through which a user inputs desired travel conditions and receives and displays the generated travel plan.

[0290] This invention is a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and suggests multiple options to the user. Furthermore, by combining it with an emotion analysis means that recognizes the user's emotions, it also suggests travel plans that correspond to the user's emotions.

[0291] System configuration

[0292] The system consists of the following main components:

[0293] 1. Terminal: This is a device (e.g., PC, smartphone, tablet) on which the user enters their travel preferences and displays the generated travel plan.

[0294] 2. Server: This is the device or system that receives input data from users, analyzes it, works with the generative AI model to generate a travel plan, and finally sends it to the device.

[0295] 3. Generative AI model: This uses machine learning and natural language processing algorithms to generate optimal travel plans based on the user's travel requirements.

[0296] 4. Emotion analysis means: This includes technology that uses a camera or microphone to analyze the user's emotional state in real time and transmits the data to a server.

[0297] Detailed program processing

[0298] Receiving user input

[0299] The user enters their desired travel conditions into a form displayed on the screen via their device. For example, they can provide the following information:

[0300] Landscape image (e.g. mountains and lake, torii gate)

[0301] Type of activity (e.g. hiking, river rafting, hot air ballooning)

[0302] Completeness of facilities

[0303] Dietary preferences or restrictions (e.g., vegan, halal, allergy-friendly)

[0304] Length of stay

[0305] Number of people

[0306] When the user enters these conditions and presses the submit button, the device sends the input data to the server. The device application sends the data to the server in JSON format using an HTTP request or AJAX.

[0307] Analyzing input data

[0308] The server receives user data sent from the device, breaks it down into individual parameters, and analyzes them. This process is performed using scripts written in programming languages ​​such as Python or Java. The server then prepares the analyzed parameters to be passed to the generative AI model.

[0309] Generate travel plans that match your criteria

[0310] The server passes data to the generative AI model based on the analyzed parameters. The generative AI model searches the database for suitable travel destination information based on the user's criteria and selects candidate travel destinations that meet each criteria. For example, if "Hakone" and "Nikko" are selected, data on the respective regions, accommodations, activity locations, temperature, etc. is collected. The generative AI model generates multiple optimal travel plans based on this data and returns them to the server.

[0311] Proposing and displaying travel plans

[0312] The server sends multiple travel plans received from the generative AI model to the device, which then displays the plans to the user. For example, the details of the Hakone plan include Lake Ashi, the torii gates of Hakone Shrine, a river rafting spot, cottage-style accommodation, meals for people with egg allergies, and an open-air hot spring. The user can then view the plan that best suits their needs.

[0313] Sentiment analysis and re-suggestion

[0314] The emotion analysis means uses the device's camera and microphone to analyze the user's emotional state in real time. For example, if the user has a happy expression, it sends that emotion to the server. The server then adjusts the travel plan based on the emotional data, prioritizing plans that allow for relaxation. It also saves a history of the user's emotional state and uses it as a reference when generating the next travel plan.

[0315] Specific examples

[0316] For example, if a user enters the following criteria:

[0317] A place with mountains and lakes

[0318] Where the torii gate is located

[0319] River rafting

[0320] Cottage-style accommodation

[0321] Suitable for those with egg allergies

[0322] Open-air hot spring

[0323] The device sends these conditions to the server. The emotion analysis means recognizes that the user has a happy expression and sends this information to the server. The server detects "Hakone" and "Nikko" as travel destinations that match the conditions. The generative AI model generates a "Hakone Plan" and a "Nikko Plan" and returns them to the server along with detailed information about each.

[0324] Example prompt sentence:

[0325] "Please suggest a travel plan that includes mountains, a lake, a torii gate, river rafting, cottage-style accommodations, accommodations that cater to egg allergies, and an open-air hot spring."

[0326] Based on this prompt, the system generates an appropriate travel plan and suggests it to the user. The emotional analysis means analyzes the user's emotional state in real time and provides the optimal plan accordingly.

[0327] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0328] Step 1:

[0329] The user enters the desired travel conditions.

[0330] Users enter their desired travel conditions into an input form via their device. The conditions entered include landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting), facility availability, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, number of people, etc. When the user presses the "Submit" button, the entered data is sent to the server by the device. Input is made using text fields, and the data is sent to the server in JSON format.

[0331] Step 2:

[0332] The server receives and analyzes the data sent by the user.

[0333] The server receives the JSON data sent from the device. It parses the data to extract individual parameters, such as the landscape image and the type of activity. Python scripts are used to parse the data, and the parsed data is temporarily stored in an internal database. The output is a parsed parameter set, which is passed to the next processing step.

[0334] Step 3:

[0335] The server passes the analyzed data to the generative AI model.

[0336] The server sends the extracted parameter set to the API endpoint of the generative AI model. The generative AI model uses a machine learning algorithm to search the database for travel destination information that meets the user's criteria and generate optimal travel plan candidates. The input to this step is the analysis parameters, and the output is a set of generated multiple travel plans.

[0337] Step 4:

[0338] The generative AI model generates an optimal travel plan and returns it to the server.

[0339] Based on the received parameters, the generative AI model performs a database search to select candidate travel destinations that meet each condition. Specifically, it collects data such as region, accommodation, activity locations, and temperature. Once optimal travel plans are generated, these plans are returned to the server. The input is a set of parameters for the generative AI model, and the output is detailed information about the travel plan.

[0340] Step 5:

[0341] The server transmits the generated travel plan to the terminal.

[0342] The server sends the travel plan received from the generative AI model to the device. This transmission uses JSON format as an HTTP response. The device receives this and displays it to the user. The input is detailed information about the travel plan, and the output is data sent to the device.

[0343] Step 6:

[0344] The terminal displays the travel plan to the user.

[0345] The terminal displays the travel plans received from the server in a user-friendly format. Each plan includes details such as area, accommodation, activities, and meal options. For example, the information is presented visually using diagrams and tables on a web page. The input is the travel plan details, and the output is the display to the user.

[0346] Step 7:

[0347] The emotion analysis means analyzes the user's emotional state and transmits it to the server.

[0348] The emotion analysis means analyzes the user's facial expressions and voice in real time using the device's built-in camera and microphone. For example, if the user looks happy, that information is sent to the server. The input is data from the camera and microphone, and the output is analyzed emotional data.

[0349] Step 8:

[0350] The server adjusts the travel plan based on the emotional data and sends it back to the device.

[0351] The server adjusts the travel plan based on the emotional data obtained from the emotion analysis means. If necessary, it may regenerate the plan by issuing new instructions to the generative AI model. The adjusted plan is then sent back to the device. The input is the emotional data and the existing travel plan, and the output is the adjusted travel plan.

[0352] Step 9:

[0353] The device will then redisplay the adjusted itinerary to the user.

[0354] The terminal redisplays the adjusted travel plan sent from the server to the user, providing the optimal travel plan that matches the user's emotions. The input is the adjusted travel plan, and the output is the redisplay to the user.

[0355] (Application example 2)

[0356] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0357] Conventional travel plan generation systems generate plans based on user input, but are unable to reflect the user's real-time emotional state. As a result, proposals are made that are not optimized for the user's emotional state, which can reduce user satisfaction. To solve this problem, a system is needed that can recognize the user's emotions in real time and propose optimal travel plans accordingly.

[0358] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving desired travel conditions from a user, means for generating multiple travel plans based on the conditions, means for presenting the generated travel plans to the user, means for recognizing the user's emotional state using emotion analysis technology, and means for optimizing the travel plans based on the recognized emotional state. This makes it possible to propose an optimal travel plan that is adapted to the user's emotional state.

[0359] "User" means a person who uses the System to input travel preferences.

[0360] "Travel preferences" refers to the user's detailed preferences, such as desired travel destinations, activities, and dietary restrictions.

[0361] "Multiple Travel Plans" are multiple travel options suggested by the generative AI model based on the user's criteria.

[0362] A "generative AI model" is a machine learning algorithm that generates optimal travel plans from collected data.

[0363] "Emotion analysis technology" is a technology that recognizes a user's emotional state by analyzing their facial expressions, tone of voice, etc.

[0364] "Emotional state" refers to the real-time emotions of users as they enter their travel requirements.

[0365] "Presentation means" refers to an interface that displays the generated travel plan in an easy-to-understand manner to the user.

[0366] The "optimization method" is a mechanism that uses emotion analysis technology to adjust and optimize travel plans based on the user's emotional state.

[0367] The "server" is a centralized system that receives, analyzes, and processes user input and emotional states.

[0368] This invention provides a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and suggests multiple options to the user. Furthermore, by combining this with emotion analysis technology that recognizes the user's emotional state, it also suggests travel plans that correspond to the user's emotions.

[0369] Hardware and software used

[0370] Hardware

[0371] Device: Smart glasses, head-mounted display (e.g., Oculus Quest 2)

[0372] Camera: A camera built into smart glasses or HMD

[0373] Microphone: A microphone built into smart glasses or HMD

[0374] software

[0375] Generative AI model: OpenAI GPT-3

[0376] Emotion analysis technology: Affectiva SDK

[0377] Development platform: Unity, Unreal Engine

[0378] Data analysis: Python, TensorFlow

[0379] Server side: Flask (Python web framework)

[0380] Specific processing flow

[0381] 1. Accepting user input:

[0382] Users wear smart glasses or a head-mounted display and input their desired travel conditions into the virtual space. This input is done through a form, and the data is sent from the device to a server.

[0383] 2. Analysis of input data:

[0384] The server analyzes the received condition data and extracts individual parameters, which are then passed to the generative AI model.

[0385] 3. Generate a travel plan:

[0386] The generative AI model (OpenAI GPT-3) generates multiple itineraries based on criteria, and the generation process is done using Python and TensorFlow.

[0387] 4. Present your travel plan:

[0388] The generated travel plan is sent from the server to the device and displayed on the user's smart glasses or HMD, using Unity or Unreal Engine.

[0389] 5. Emotional state recognition and suggestion optimization:

[0390] Using emotion analysis technology (Affectiva SDK), the system analyzes the user's facial expressions and tone of voice in real time. This data is sent to a server, and the travel plan is optimized based on the user's emotional state.

[0391] Specific examples

[0392] Consider the case where the user enters the following criteria:

[0393] Landscape image: Mountains and lake

[0394] Activity:River Rafting

[0395] Accommodation type: Cottage

[0396] Meals: Egg allergy friendly

[0397] Length of stay: 3 days

[0398] Number of people: 2 people

[0399] Based on these input conditions, the server uses a generative AI model (GPT-3) to generate the optimal travel plan. For example, a "Hakone Plan" and a "Nikko Plan" are generated, each containing detailed information.

[0400] Example prompt sentence:

[0401] The user wants to be somewhere with mountains and a lake, so you propose an egg-friendly itinerary that includes cottage-style accommodations and a river cruise. The trip is three days long and for two people. Make sure the user is relaxed and has fun.

[0402] The server sends the generated plan to the device, where the user experiences it in a virtual space through smart glasses or an HMD. Emotion analysis technology recognizes the user's real-time emotional state and prioritizes, for example, relaxing plans that are less stressful. This allows the user to choose the optimal travel plan that adapts to their emotional state.

[0403] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0404] Step 1:

[0405] The user puts on smart glasses or a head-mounted display and enters the desired travel conditions in a virtual space. The input form includes items such as landscape image, type of activity, accommodation type, dietary restrictions, length of stay, and number of people. The entered conditions are sent from the device to the server. Input: User's condition data. Output: Condition data sent to the server.

[0406] Step 2:

[0407] The server receives and analyzes the user's condition data sent from the device. This analysis extracts the condition data as individual parameters (scenery image, type of activity, etc.). These parameters are then formatted to be passed to the generative AI model. Input: User's condition data. Output: A set of parameters to be passed to the generative AI model.

[0408] Step 3:

[0409] The server inputs the analyzed parameter set into the generative AI model (GPT-3) to generate the optimal travel plan. At this time, the generative AI model searches the database for information on suitable travel destinations and collects detailed information on candidate locations that meet the conditions. Input: Parameter set. Output: Generated multiple travel plans.

[0410] Step 4:

[0411] The server sends the generated multiple travel plans to the terminal. The terminal displays the travel plans to the user in a virtual space. The user can view each plan along with detailed information (region, accommodation, activities, meal conditions, etc.). Input: Generated travel plans. Output: Travel plans displayed on the user's terminal.

[0412] Step 5:

[0413] Using emotion analysis technology, the device analyzes the user's facial expressions and tone of voice in real time. This emotion data is sent from the device to the server. Input: User emotion data. Output: Emotion data sent to the server.

[0414] Step 6:

[0415] The server analyzes the user's emotional state based on the emotional data it receives. The server then optimizes the travel plan based on this emotional state. For example, if the user is feeling stressed, it will prioritize relaxing plans. Input: Emotional data. Output: Regenerate the optimized travel plan.

[0416] Step 7:

[0417] The server sends the regenerated optimized travel plan to the terminal and presents it to the user. The user can view and select the new plan optimized based on their emotional state in the virtual space. Input: Optimized travel plan. Output: Display of the optimized travel plan on the user's terminal.

[0418] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0419] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0420] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0421] [Second embodiment]

[0422] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0423] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0424] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0425] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0426] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0427] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0428] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0429] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0430] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0431] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0432] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0433] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0434] This invention specifically demonstrates a system that, when a user inputs their desired travel conditions, uses a generative AI model to generate an optimal travel plan and propose multiple options to the user.

[0435] Program Overview

[0436] The system has the following main functions:

[0437] 1. Receiving user input

[0438] 2. Analysis of input data

[0439] 3. Generate travel plans that meet the conditions

[0440] 4. Proposing and displaying travel plans

[0441] Detailed program processing

[0442] 1. Receiving user input

[0443] Users input their desired travel conditions into a form displayed on the screen via their device, including information such as landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting, hot air ballooning), level of facilities, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, and number of people.

[0444] The terminal receives the data entered by the user and transmits it to the server.

[0445] 2. Analysis of input data

[0446] The server receives and analyzes user data sent from the device, extracting it into individual parameters and preparing them for passing to the generative AI model.

[0447] 3. Generate travel plans that meet the conditions

[0448] The server passes the conditions based on the extracted parameters to a generative AI model (machine learning algorithm), which then searches the database for suitable travel destination information based on the conditions, selects candidate travel destinations that meet each condition, and collects detailed information for each destination (such as region, accommodation, activity locations, and temperature data).

[0449] The generative AI model generates multiple optimal travel plans from the collected data and returns them to the server.

[0450] 4. Proposing and displaying travel plans

[0451] The server sends the generated itineraries to the device, which then displays them to the user along with detailed information. Each itinerary includes details about the area, accommodation, temperature, and activities. The user can then select the itinerary that best suits their needs.

[0452] Specific examples

[0453] For example, if a user enters the following criteria:

[0454] A place with mountains and lakes

[0455] Where the torii gate is located

[0456] River rafting

[0457] Cottage-style accommodation

[0458] Suitable for those with egg allergies

[0459] Open-air hot spring

[0460] The device sends these conditions to the server, which then detects "Hakone" and "Nikko" as travel destinations that match the conditions. The generative AI model generates a "Hakone Plan" and a "Nikko Plan" and returns them to the server along with detailed information about each plan.

[0461] A detailed example of a Hakone plan includes Lake Ashi, the Hakone Shrine's torii gate, a river cruise, cottage-style accommodation, meals for those with egg allergies, and an open-air hot spring. Similarly, a Nikko plan comprehensively suggests scenery, activities, and meals based on the user's requirements.

[0462] This allows users to compare the details of the Hakone and Nikko plans and choose the travel plan that best suits their needs.

[0463] This system makes it easy for users to plan their trips by efficiently generating travel plans based on the user's detailed wishes and presenting them as multiple options.

[0464] The processing flow will be explained below.

[0465] Step 1:

[0466] The user enters the desired travel conditions.

[0467] Users enter their desired travel conditions (image of scenery, type of activity, level of facilities, dietary preferences and restrictions, length of stay, number of people, etc.) into a form displayed on the device screen.

[0468] Step 2:

[0469] The terminal receives the user's input and sends it to the server.

[0470] The terminal formats the data entered by the user and sends a request to the server.

[0471] Step 3:

[0472] The server receives and analyzes the data from the user.

[0473] The server analyzes the received data and extracts each condition as a separate parameter.

[0474] Step 4:

[0475] The server prepares to pass the extracted parameters to the generative AI model.

[0476] The server converts the extracted conditions into an appropriate format and performs preprocessing to pass them to the generative AI model.

[0477] Step 5:

[0478] The generative AI model searches the database based on the criteria.

[0479] The generative AI model searches for suitable travel destination information from within the database based on the conditions passed from the server.

[0480] Step 6:

[0481] The generative AI model selects suitable travel destinations and collects detailed information about each one.

[0482] The generative AI model collects detailed information such as scenery, activities, accommodation, food options, and temperature data to select the best candidates.

[0483] Step 7:

[0484] A generative AI model creates multiple itineraries.

[0485] The generative AI model uses the collected information to generate multiple travel plans that meet the user's requirements.

[0486] Step 8:

[0487] The server transmits the generated travel plan to the terminal.

[0488] The server sends multiple travel plans received from the generative AI model to the user's device.

[0489] Step 9:

[0490] The device displays the travel plan to the user.

[0491] The terminal receives the travel plan sent from the server and displays it to the user along with detailed information.

[0492] Step 10:

[0493] The user selects the desired travel plan from the presented plans.

[0494] Users can select the travel plan that best suits their needs from the plans presented.

[0495] Step 11:

[0496] The device will display details of the plan the user has selected and the booking process.

[0497] The device will display detailed information about the plan the user has selected, a link to the reservation page, and guide them on their next steps.

[0498] Example 1

[0499] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0500] Conventional travel plan generation systems have difficulty quickly and automatically generating the optimal travel plan based on a user's desired conditions and providing multiple options. There is also a need for a system that can efficiently analyze and reflect travel plans based on the user's detailed desired conditions. This has led to the problem that it is difficult for users to select the travel plan that best meets their needs.

[0501] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0502] In this invention, the server includes means for receiving desired travel conditions from a user, means for generating multiple travel plans based on the conditions, means for generating an optimal travel plan based on the conditions using a generative AI model, and means for presenting the generated travel plans to the user. This makes it possible to quickly and efficiently generate an optimal travel plan based on the user's detailed desired conditions and provide it to the user as multiple options.

[0503] "User" means an individual or group that requests a travel plan and enters its requirements.

[0504] "Travel preferences" are the specific elements and requirements that users desire for their trip (e.g., landscape image, type of activity, level of facilities, dietary preferences or restrictions, length of stay, number of people, etc.).

[0505] "Terminal" refers to a device used by a user, such as a computer, smartphone, or tablet, which is used to input and receive travel preferences.

[0506] A "server" is a computer system that receives, analyzes, and processes data sent from a terminal.

[0507] A "generative AI model" is an algorithm or software that uses artificial intelligence technology to generate optimal travel plans based on input criteria.

[0508] A "trip plan" is a detailed travel plan based on a user's desired conditions, including destinations, accommodations, activities, temperature data, etc.

[0509] A "database" is an online or offline data storage system for storing information such as travel destination information, accommodation information, activity locations, temperature data, etc.

[0510] An "HTTP request" is part of a communication protocol for sending and receiving data between a terminal and a server, and is used to send the user's conditions to the server.

[0511] "JSON format" is an abbreviation for JavaScript Object Notation, a text-based format used as a data exchange format.

[0512] "Parameters" are the individual elements that make up the desired travel conditions (e.g., mountains, lakes, torii gates, river rafting, cottages, etc.).

[0513] The present invention is a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and then suggests multiple options to the user.

[0514] The system uses the following hardware and software:

[0515] 1. Terminal: A device such as a computer, smartphone, or tablet where users input their desired travel conditions. This terminal provides a user interface using a web browser or dedicated application.

[0516] 2. Server: A computer system for receiving, analyzing, and processing data sent by users. This server is responsible for storing user data, analyzing it, inputting it into the generative AI model, receiving the results, and generating the itinerary.

[0517] 3. Generative AI model: An algorithm or software that uses artificial intelligence technology to generate optimal travel plans based on input criteria. Examples include machine learning algorithms and natural language processing technology.

[0518] Specific processing of the system

[0519] Accepting user input:

[0520] Users input the desired travel conditions into a form displayed on the device screen. These conditions include landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting, hot air ballooning), facility availability, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, number of people, etc. The device stores the input information in its memory and sends this data to the server when the "send" button is pressed.

[0521] Analysis of input data:

[0522] The server receives the HTTP POST request sent from the device and parses the request body in JSON format. The server then breaks down the parsed data into individual parameters (e.g., "mountain," "lake," "torii gate," "river boating," "cottage," etc.) and converts them into a format suitable for input to the generative AI model.

[0523] Generate matching itineraries:

[0524] The server passes the formatted parameters to the generative AI model, which queries the database based on the received parameters to find suitable travel destinations, collects detailed information about the suitable destinations (such as region, accommodation, activity locations, and temperature data), and creates multiple optimal itineraries. This information is returned to the server in JSON format.

[0525] Suggest and display itineraries:

[0526] The server receives and analyzes the JSON data returned by the generative AI model. The server converts the travel plan information into a format that is easy for the user to understand and sends it to the device as an HTTP response. The device displays the received data on the screen and presents the user with multiple travel plans. The user can compare the displayed plans and select the one that best suits their needs.

[0527] Specific examples

[0528] For example, if a user enters the following criteria:

[0529] A place with mountains and lakes

[0530] Where the torii gate is located

[0531] River rafting

[0532] Cottage-style accommodation

[0533] Suitable for those with egg allergies

[0534] Open-air hot spring

[0535] An example prompt is:

[0536] Prompt statement:

[0537] We want the user to suggest itineraries related to "places with mountains and lakes, places with torii gates, river rafting, cottages, egg allergy friendly, and open-air hot springs." Based on this, please generate the optimal itinerary.

[0538] These conditions are sent from the device to a server, which then detects "Location A" and "Location B" as travel destinations that meet the conditions. The generative AI model uses this information to generate a "Location A Plan" and a "Location B Plan," and returns them to the server along with detailed information about each plan (mountains and lakes, torii gates, river rafting spots, cottage-style accommodations, meals for people with egg allergies, open-air hot springs, etc.). This allows users to compare the details of the "Location A Plan" and "Location B Plan" and choose the travel plan that best suits their needs.

[0539] This system allows users to efficiently generate travel plans based on their detailed desired conditions, and easily select the most suitable plan from multiple options.

[0540] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0541] Step 1: Receiving User Input

[0542] Description: The user enters the desired travel conditions into a form displayed on the device screen. The user's input (e.g., a place with mountains and lakes, a place with a torii gate, river rafting, a cottage, suitable for egg allergies, an open-air hot spring) is provided to the device as input data. The device stores this input data in its memory, and when the submit button on the form is pressed, it sends the stored data to the server via an HTTP POST request. Specifically, the device's browser or application sends the user input to the server in JSON format.

[0543] Step 2: Analyzing the input data

[0544] Description: The server receives an HTTP POST request sent from the device and parses the request body in JSON format. The parsed data is provided to the server as input data. The server breaks this data down into individual parameters (e.g., scenery, activity, dietary restrictions, etc.). Specifically, the server-side analysis module parses the JSON data and converts it into a format that can be input into a generative AI model based on the parameters.

[0545] Step 3: Generate itineraries that meet your criteria

[0546] Description: The server passes formatted parameters to the generative AI model. The generative AI model receives these parameters as input data and searches its database for suitable travel destinations based on that data. The search results are provided to the generative AI model as intermediate data. The generative AI model then collects detailed information (e.g., region, accommodation, activity locations, temperature data, etc.) based on the search results and creates multiple optimal travel plans. Specifically, the generative AI model executes database queries, assembles a travel plan from the obtained information, and returns it to the server in JSON format.

[0547] Step 4: Propose and display travel plans

[0548] Description: The server receives the JSON data returned from the generative AI model and provides it as output data. This data is parsed and converted into a format that is easy for the user to understand (e.g., HTML or text format). The server then sends the converted data to the device as an HTTP response. The final output data, a travel plan, is provided to the device. The device receives this data and displays it to the user. The user can compare the displayed multiple travel plans and select the one that best suits their needs. Specifically, the device's display module visually presents the travel plan information, allowing the user to compare them.

[0549] (Application example 1)

[0550] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0551] In recent years, there has been a demand for providing meals that meet individual preferences and health conditions. However, selecting the optimal menu based on detailed conditions regarding dietary restrictions and specific ingredients requires a lot of time and effort. For this reason, there is a need for a system that can efficiently suggest and optimize menus that meet the conditions.

[0552] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0553] In this invention, the server includes a means for receiving meal conditions requested by the user, a means for generating multiple meal menus based on the conditions, and a means for presenting the user with dining facilities that offer the generated meal menus, thereby making it possible to propose the optimal meal menu according to the user's detailed conditions.

[0554] "User" means an individual or group of people who utilizes the System to enter dietary requirements.

[0555] "Dietary requirements" refers to specific conditions such as the user's desired ingredients, dietary restrictions, level of spiciness, and serving amount.

[0556] "Food Menu" refers to a selection of food suggestions generated based on the user's dietary requirements.

[0557] "Food and beverage establishments" are establishments such as restaurants and cafes that offer food menus and allow users to order.

[0558] A "generative AI model" refers to a machine learning algorithm that generates optimal cooking menus based on the user's dietary requirements.

[0559] This invention is a system that uses a generative AI model to propose optimal meal menus based on a user's desired meal conditions. This system receives the user's meal conditions as input, generates multiple meal menus based on them, and then suggests dining establishments that meet the conditions entered by the user.

[0560] First, the user uses the terminal to input their desired dietary requirements. Dietary requirements include dietary restrictions (e.g., vegetarian, gluten-free), spiciness level (e.g., normal, spicy), desired ingredients (e.g., tomato, cheese), and serving size (e.g., two servings). The terminal receives the data entered by the user and sends it to the server.

[0561] The server receives and analyzes the user's meal requirements sent from the device. The analyzed data is extracted as individual parameters and prepared to be passed to the generative AI model. The generative AI model searches the database for food menus that match the user's requirements and collects detailed information about each (such as the name of the dish, dining facility, price, and dish details). The generative AI model generates multiple optimal food menus from the collected data and returns them to the server.

[0562] The server sends the generated menus to the terminal, which displays them with detailed information to the user, allowing the user to select the menu that best suits their needs and order delivery.

[0563] For example, if a user enters the following criteria:

[0564] Dietary: Vegetarian

[0565] Allergies: Gluten

[0566] Spice level: Medium

[0567] Desired ingredients: Tomato, cheese

[0568] Serving size: 2 servings

[0569] Based on this, the server will suggest the following menu options:

[0570] 1. Dish name: Vegetarian pizza with tomato and cheese

[0571] Dining options: Vegetarian Restaurant A

[0572] Price: 1500 yen

[0573] Details: Made with gluten-free pizza dough, topped with tomato and mozzarella cheese

[0574] 2. Dish name: Cheese and tomato pasta

[0575] Dining facilities: Healthy Cafe B

[0576] Price: 1200 yen

[0577] Details: Gluten-free pasta, fresh tomato sauce and Parmesan cheese

[0578] An example prompt might look like this:

[0579] conditions:

[0580] Dietary: Vegetarian

[0581] Allergies: Gluten

[0582] Spice level: Medium

[0583] Desired ingredients: Tomato, cheese

[0584] Serving size: 2 servings

[0585] Based on this, please suggest an appropriate menu.

[0586] In this way, a food menu based on the user's detailed preferences is efficiently generated and presented as multiple options, making meal selections easier for the user.

[0587] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0588] Step 1:

[0589] The user inputs their desired dietary requirements using a terminal. The input information includes dietary restrictions, spiciness level, desired ingredients, serving size, etc. This input information is sent from the terminal to the server.

[0590] Step 2:

[0591] The server receives and analyzes the user's dietary requirements sent from the device. During the analysis process, the input data is extracted as individual parameters, which are then prepared for passing to the generative AI model.

[0592] Step 3:

[0593] The server then passes the analyzed data to a generative AI model, which then searches a database for food menus that match the user's criteria and narrows down the menu based on the criteria.

[0594] Step 4:

[0595] The server uses a generative AI model to generate multiple optimal menus from the collected data, including the name of the dish, the dining establishment, the price, and details of the dish.

[0596] Step 5:

[0597] The server sends the generated menus to the terminal, which displays them to the user along with detailed information. The user compares the details of each menu and selects the desired dish.

[0598] Step 6:

[0599] The user selects the desired menu item and places a delivery order. The terminal sends the selected order information to the server, which then transmits the order information to the restaurant.

[0600] This allows users to easily select the most suitable food menu that meets their specific requirements and use the delivery service.

[0601] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0602] This invention specifically describes a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and suggests multiple options to the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it also suggests travel plans that correspond to the user's emotions.

[0603] Program Overview

[0604] The system has the following main functions:

[0605] 1. Receiving user input

[0606] 2. Analysis of input data

[0607] 3. Generate travel plans that meet the conditions

[0608] 4. Proposing and displaying travel plans

[0609] 5. Emotion engine recognizes and reflects user emotions

[0610] Detailed program processing

[0611] 1. Receiving user input

[0612] Users input their desired travel conditions into a form displayed on the screen via their device, including information such as landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting, hot air ballooning), level of facilities, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, and number of people.

[0613] The terminal receives the data entered by the user and transmits it to the server.

[0614] 2. Analysis of input data

[0615] The server receives and analyzes user data sent from the device, extracting it into individual parameters and preparing them for passing to the generative AI model.

[0616] 3. Generate travel plans that meet the conditions

[0617] The server passes the conditions based on the extracted parameters to a generative AI model (machine learning algorithm), which then searches the database for suitable travel destination information based on the conditions, selects candidate travel destinations that meet each condition, and collects detailed information for each destination (such as region, accommodation, activity locations, and temperature data).

[0618] The generative AI model generates multiple optimal travel plans from the collected data and returns them to the server.

[0619] 4. Proposing and displaying travel plans

[0620] The server sends the generated itineraries to the device, which then displays them to the user along with detailed information. Each itinerary includes details about the area, accommodation, temperature, and activities. The user can then select the itinerary that best suits their needs.

[0621] 5. Emotion engine recognizes and reflects user emotions

[0622] The emotion engine analyzes the user's emotional state in real time as they type, using the device's camera, microphone, and input data to analyze the user's facial expressions, tone of voice, and typing speed.

[0623] The server uses the emotion data obtained from the emotion engine to adjust the conditions based on the user's emotional state. For example, if the user is feeling stressed, the server instructs the generative AI model to prioritize generating relaxing travel plans.

[0624] The emotion engine also stores the user's emotional state as a history and uses it as a reference when generating the next travel plan, allowing it to propose an even more optimal plan based on the user's emotional history.

[0625] Specific examples

[0626] For example, if a user enters the following criteria:

[0627] A place with mountains and lakes

[0628] Where the torii gate is located

[0629] River rafting

[0630] Cottage-style accommodation

[0631] Suitable for those with egg allergies

[0632] Open-air hot spring

[0633] The device sends these conditions to the server, and the emotion engine recognizes that the user has a happy expression when entering the information. The emotion engine then sends this information to the server, which then detects "Hakone" and "Nikko" as travel destinations that match the conditions.

[0634] The generative AI model generates a "Hakone Plan" and a "Nikko Plan," and returns each plan to the server along with detailed information. Examples of detailed Hakone plans include Lake Ashi, the torii gates of Hakone Shrine, a river cruise, cottage-style accommodation, meals suitable for egg allergies, and open-air hot springs. Similarly, Nikko plans comprehensively suggest scenery, activities, and meal options based on the user's requirements.

[0635] This allows users to compare the details of the Hakone and Nikko plans and choose the travel plan that best suits their needs. The emotion engine supports plan generation based on the user's emotional state, resulting in travel suggestions that will satisfy the user even more.

[0636] The processing flow will be explained below.

[0637] Step 1:

[0638] The user enters the desired travel conditions.

[0639] Users enter their desired travel conditions (for example, "a place with mountains and lakes" as a scenic image, "hiking" as the type of activity, "cottage-style accommodation" as the level of facilities, "accommodation for egg allergies" as a dietary preference or restriction, length of stay, number of people, etc.) into a form displayed on the device screen.

[0640] Step 2:

[0641] The terminal receives the user's input and sends it to the server.

[0642] The terminal formats the data entered by the user and sends a request to the server.

[0643] Step 3:

[0644] The emotion engine analyzes user emotions in real time.

[0645] The device's emotion engine uses the camera and microphone to analyze the user's facial expressions and tone of voice to generate emotional data.

[0646] Step 4:

[0647] The device transmits the emotion data to the server.

[0648] The device transmits the analyzed emotion data to the server, which delivers it together with the travel conditions.

[0649] Step 5:

[0650] The server receives and analyzes the data from the user.

[0651] The server analyzes the received user data and emotion data and extracts each condition as an individual parameter.

[0652] Step 6:

[0653] The server prepares to pass the extracted parameters to the generative AI model.

[0654] The server converts the extracted conditions and emotion data into an appropriate format and performs preprocessing to pass it to the generative AI model.

[0655] Step 7:

[0656] The generative AI model searches the database based on the criteria and emotion data.

[0657] The generative AI model searches for suitable travel destination information from within the database based on the conditions and emotional data passed from the server.

[0658] Step 8:

[0659] The generative AI model selects suitable travel destinations and collects detailed information about each one.

[0660] The generative AI model collects detailed information such as scenery, activities, accommodation, food options, and temperature data to select the best candidates.

[0661] Step 9:

[0662] A generative AI model creates a travel plan that reflects emotional data.

[0663] The generative AI model generates the optimal travel plan for the user based on the collected information and the user's emotional state.

[0664] Step 10:

[0665] The server transmits the generated travel plan to the terminal.

[0666] The server sends multiple travel plans received from the generative AI model to the user's device.

[0667] Step 11:

[0668] The device displays the travel plan to the user.

[0669] The terminal receives the travel plan sent from the server and displays it to the user along with detailed information.

[0670] Step 12:

[0671] The user selects the desired travel plan from the presented plans.

[0672] Users can select the travel plan that best suits their needs from the plans presented.

[0673] Step 13:

[0674] The device will display details of the plan the user has selected and the booking process.

[0675] The device will display detailed information about the plan the user has selected, a link to the reservation page, and guide them on their next steps.

[0676] Example 2

[0677] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0678] Conventional travel plan generation systems can propose plans that reflect the user's desired travel conditions, but they cannot adjust the plans to reflect the user's real-time emotional state. This makes it difficult to provide optimal travel plans that reflect the user's emotions. Furthermore, because the system does not take the user's emotions into consideration, the proposed travel plans often do not meet the user's expectations.

[0679] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0680] In this invention, the server includes means for receiving desired travel conditions from a user, means for generating multiple travel plans based on the conditions, means for presenting the generated travel plans to the user, emotion analysis means for analyzing the emotional state of the user, and means for adjusting the travel plans based on the emotional state of the user. This makes it possible to propose travel plans that correspond to the emotional state of the user, and to provide travel plans that will highly satisfy the user.

[0681] "Travel preferences" refer to the specific elements and requests that users have for their travel plans, including landscape image, types of activities, level of facilities, dietary preferences and restrictions, length of stay, number of people, etc.

[0682] "Generative AI model" refers to machine learning algorithms and natural language processing algorithms used to generate optimal travel plans based on a user's desired travel conditions.

[0683] "Emotion analysis means" refers to a function that analyzes a user's emotional state in real time, and includes technology that uses a camera or microphone to analyze a user's facial expressions and tone of voice.

[0684] A "Travel Plan" is a proposal that details the activities and accommodations desired by a User at a particular travel destination.

[0685] "Server" refers to the equipment or system that receives input data from users, analyzes it, works with the generative AI model to generate a travel plan, and finally sends it to the terminal.

[0686] A "terminal" is a device (such as a PC, smartphone, or tablet) through which a user inputs desired travel conditions and receives and displays the generated travel plan.

[0687] This invention is a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and suggests multiple options to the user. Furthermore, by combining it with an emotion analysis means that recognizes the user's emotions, it also suggests travel plans that correspond to the user's emotions.

[0688] System configuration

[0689] The system consists of the following main components:

[0690] 1. Terminal: This is a device (e.g., PC, smartphone, tablet) on which the user enters their travel preferences and displays the generated travel plan.

[0691] 2. Server: This is the device or system that receives input data from users, analyzes it, works with the generative AI model to generate a travel plan, and finally sends it to the device.

[0692] 3. Generative AI model: This uses machine learning and natural language processing algorithms to generate optimal travel plans based on the user's travel requirements.

[0693] 4. Emotion analysis means: This includes technology that uses a camera or microphone to analyze the user's emotional state in real time and transmits the data to a server.

[0694] Detailed program processing

[0695] Receiving user input

[0696] The user enters their desired travel conditions into a form displayed on the screen via their device. For example, they can provide the following information:

[0697] Landscape image (e.g. mountains and lake, torii gate)

[0698] Type of activity (e.g. hiking, river rafting, hot air ballooning)

[0699] Completeness of facilities

[0700] Dietary preferences or restrictions (e.g., vegan, halal, allergy-friendly)

[0701] Length of stay

[0702] Number of people

[0703] When the user enters these conditions and presses the submit button, the device sends the input data to the server. The device application sends the data to the server in JSON format using an HTTP request or AJAX.

[0704] Analyzing input data

[0705] The server receives user data sent from the device, breaks it down into individual parameters, and analyzes them. This process is performed using scripts written in programming languages ​​such as Python or Java. The server then prepares the analyzed parameters to be passed to the generative AI model.

[0706] Generate travel plans that match your criteria

[0707] The server passes data to the generative AI model based on the analyzed parameters. The generative AI model searches the database for suitable travel destination information based on the user's criteria and selects candidate travel destinations that meet each criteria. For example, if "Hakone" and "Nikko" are selected, data on the respective regions, accommodations, activity locations, temperature, etc. is collected. The generative AI model generates multiple optimal travel plans based on this data and returns them to the server.

[0708] Proposing and displaying travel plans

[0709] The server sends multiple travel plans received from the generative AI model to the device, which then displays the plans to the user. For example, the details of the Hakone plan include Lake Ashi, the torii gates of Hakone Shrine, a river rafting spot, cottage-style accommodation, meals for people with egg allergies, and an open-air hot spring. The user can then view the plan that best suits their needs.

[0710] Sentiment analysis and re-suggestion

[0711] The emotion analysis means uses the device's camera and microphone to analyze the user's emotional state in real time. For example, if the user has a happy expression, it sends that emotion to the server. The server then adjusts the travel plan based on the emotional data, prioritizing plans that allow for relaxation. It also saves a history of the user's emotional state and uses it as a reference when generating the next travel plan.

[0712] Specific examples

[0713] For example, if a user enters the following criteria:

[0714] A place with mountains and lakes

[0715] Where the torii gate is located

[0716] River rafting

[0717] Cottage-style accommodation

[0718] Suitable for those with egg allergies

[0719] Open-air hot spring

[0720] The device sends these conditions to the server. The emotion analysis means recognizes that the user has a happy expression and sends this information to the server. The server detects "Hakone" and "Nikko" as travel destinations that match the conditions. The generative AI model generates a "Hakone Plan" and a "Nikko Plan" and returns them to the server along with detailed information about each.

[0721] Example prompt sentence:

[0722] "Please suggest a travel plan that includes mountains, a lake, a torii gate, river rafting, cottage-style accommodations, accommodations that cater to egg allergies, and an open-air hot spring."

[0723] Based on this prompt, the system generates an appropriate travel plan and suggests it to the user. The emotional analysis means analyzes the user's emotional state in real time and provides the optimal plan accordingly.

[0724] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0725] Step 1:

[0726] The user enters the desired travel conditions.

[0727] Users enter their desired travel conditions into an input form via their device. The conditions entered include landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting), facility availability, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, number of people, etc. When the user presses the "Submit" button, the entered data is sent to the server by the device. Input is made using text fields, and the data is sent to the server in JSON format.

[0728] Step 2:

[0729] The server receives and analyzes the data sent by the user.

[0730] The server receives the JSON data sent from the device. It parses the data to extract individual parameters, such as the landscape image and the type of activity. Python scripts are used to parse the data, and the parsed data is temporarily stored in an internal database. The output is a parsed parameter set, which is passed to the next processing step.

[0731] Step 3:

[0732] The server passes the analyzed data to the generative AI model.

[0733] The server sends the extracted parameter set to the API endpoint of the generative AI model. The generative AI model uses a machine learning algorithm to search the database for travel destination information that meets the user's criteria and generate optimal travel plan candidates. The input to this step is the analysis parameters, and the output is a set of generated multiple travel plans.

[0734] Step 4:

[0735] The generative AI model generates an optimal travel plan and returns it to the server.

[0736] Based on the received parameters, the generative AI model performs a database search to select candidate travel destinations that meet each condition. Specifically, it collects data such as region, accommodation, activity locations, and temperature. Once optimal travel plans are generated, these plans are returned to the server. The input is a set of parameters for the generative AI model, and the output is detailed information about the travel plan.

[0737] Step 5:

[0738] The server transmits the generated travel plan to the terminal.

[0739] The server sends the travel plan received from the generative AI model to the device. This transmission uses JSON format as an HTTP response. The device receives this and displays it to the user. The input is detailed information about the travel plan, and the output is data sent to the device.

[0740] Step 6:

[0741] The terminal displays the travel plan to the user.

[0742] The terminal displays the travel plans received from the server in a user-friendly format. Each plan includes details such as area, accommodation, activities, and meal options. For example, the information is presented visually using diagrams and tables on a web page. The input is the travel plan details, and the output is the display to the user.

[0743] Step 7:

[0744] The emotion analysis means analyzes the user's emotional state and transmits it to the server.

[0745] The emotion analysis means analyzes the user's facial expressions and voice in real time using the device's built-in camera and microphone. For example, if the user looks happy, that information is sent to the server. The input is data from the camera and microphone, and the output is analyzed emotional data.

[0746] Step 8:

[0747] The server adjusts the travel plan based on the emotional data and sends it back to the device.

[0748] The server adjusts the travel plan based on the emotional data obtained from the emotion analysis means. If necessary, it may regenerate the plan by issuing new instructions to the generative AI model. The adjusted plan is then sent back to the device. The input is the emotional data and the existing travel plan, and the output is the adjusted travel plan.

[0749] Step 9:

[0750] The device will then redisplay the adjusted itinerary to the user.

[0751] The terminal redisplays the adjusted travel plan sent from the server to the user, providing the optimal travel plan that matches the user's emotions. The input is the adjusted travel plan, and the output is the redisplay to the user.

[0752] (Application example 2)

[0753] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0754] Conventional travel plan generation systems generate plans based on user input, but are unable to reflect the user's real-time emotional state. As a result, proposals are made that are not optimized for the user's emotional state, which can reduce user satisfaction. To solve this problem, a system is needed that can recognize the user's emotions in real time and propose optimal travel plans accordingly.

[0755] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving desired travel conditions from a user, means for generating multiple travel plans based on the conditions, means for presenting the generated travel plans to the user, means for recognizing the user's emotional state using emotion analysis technology, and means for optimizing the travel plans based on the recognized emotional state. This makes it possible to propose an optimal travel plan that is adapted to the user's emotional state.

[0756] "User" means a person who uses the System to input travel preferences.

[0757] "Travel preferences" refers to the user's detailed preferences, such as desired travel destinations, activities, and dietary restrictions.

[0758] "Multiple Travel Plans" are multiple travel options suggested by the generative AI model based on the user's criteria.

[0759] A "generative AI model" is a machine learning algorithm that generates optimal travel plans from collected data.

[0760] "Emotion analysis technology" is a technology that recognizes a user's emotional state by analyzing their facial expressions, tone of voice, etc.

[0761] "Emotional state" refers to the real-time emotions of users as they enter their travel requirements.

[0762] "Presentation means" refers to an interface that displays the generated travel plan in an easy-to-understand manner to the user.

[0763] The "optimization method" is a mechanism that uses emotion analysis technology to adjust and optimize travel plans based on the user's emotional state.

[0764] The "server" is a centralized system that receives, analyzes, and processes user input and emotional states.

[0765] This invention provides a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and suggests multiple options to the user. Furthermore, by combining this with emotion analysis technology that recognizes the user's emotional state, it also suggests travel plans that correspond to the user's emotions.

[0766] Hardware and software used

[0767] Hardware

[0768] Device: Smart glasses, head-mounted display (e.g., Oculus Quest 2)

[0769] Camera: A camera built into smart glasses or HMD

[0770] Microphone: A microphone built into smart glasses or HMD

[0771] software

[0772] Generative AI model: OpenAI GPT-3

[0773] Emotion analysis technology: Affectiva SDK

[0774] Development platform: Unity, Unreal Engine

[0775] Data analysis: Python, TensorFlow

[0776] Server side: Flask (Python web framework)

[0777] Specific processing flow

[0778] 1. Accepting user input:

[0779] Users wear smart glasses or a head-mounted display and input their desired travel conditions into the virtual space. This input is done through a form, and the data is sent from the device to a server.

[0780] 2. Analysis of input data:

[0781] The server analyzes the received condition data and extracts individual parameters, which are then passed to the generative AI model.

[0782] 3. Generate a travel plan:

[0783] The generative AI model (OpenAI GPT-3) generates multiple itineraries based on criteria, and the generation process is done using Python and TensorFlow.

[0784] 4. Present your travel plan:

[0785] The generated travel plan is sent from the server to the device and displayed on the user's smart glasses or HMD, using Unity or Unreal Engine.

[0786] 5. Emotional state recognition and suggestion optimization:

[0787] Using emotion analysis technology (Affectiva SDK), the system analyzes the user's facial expressions and tone of voice in real time. This data is sent to a server, and the travel plan is optimized based on the user's emotional state.

[0788] Specific examples

[0789] Consider the case where the user enters the following criteria:

[0790] Landscape image: Mountains and lake

[0791] Activity:River Rafting

[0792] Accommodation type: Cottage

[0793] Meals: Egg allergy friendly

[0794] Length of stay: 3 days

[0795] Number of people: 2 people

[0796] Based on these input conditions, the server uses a generative AI model (GPT-3) to generate the optimal travel plan. For example, a "Hakone Plan" and a "Nikko Plan" are generated, each containing detailed information.

[0797] Example prompt sentence:

[0798] The user wants to be somewhere with mountains and a lake, so you propose an egg-friendly itinerary that includes cottage-style accommodations and a river cruise. The trip is three days long and for two people. Make sure the user is relaxed and has fun.

[0799] The server sends the generated plan to the device, where the user experiences it in a virtual space through smart glasses or an HMD. Emotion analysis technology recognizes the user's real-time emotional state and prioritizes, for example, relaxing plans that are less stressful. This allows the user to choose the optimal travel plan that adapts to their emotional state.

[0800] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0801] Step 1:

[0802] The user puts on smart glasses or a head-mounted display and enters the desired travel conditions in a virtual space. The input form includes items such as landscape image, type of activity, accommodation type, dietary restrictions, length of stay, and number of people. The entered conditions are sent from the device to the server. Input: User's condition data. Output: Condition data sent to the server.

[0803] Step 2:

[0804] The server receives and analyzes the user's condition data sent from the device. This analysis extracts the condition data as individual parameters (scenery image, type of activity, etc.). These parameters are then formatted to be passed to the generative AI model. Input: User's condition data. Output: A set of parameters to be passed to the generative AI model.

[0805] Step 3:

[0806] The server inputs the analyzed parameter set into the generative AI model (GPT-3) to generate the optimal travel plan. At this time, the generative AI model searches the database for information on suitable travel destinations and collects detailed information on candidate locations that meet the conditions. Input: Parameter set. Output: Generated multiple travel plans.

[0807] Step 4:

[0808] The server sends the generated multiple travel plans to the terminal. The terminal displays the travel plans to the user in a virtual space. The user can view each plan along with detailed information (region, accommodation, activities, meal conditions, etc.). Input: Generated travel plans. Output: Travel plans displayed on the user's terminal.

[0809] Step 5:

[0810] Using emotion analysis technology, the device analyzes the user's facial expressions and tone of voice in real time. This emotion data is sent from the device to the server. Input: User emotion data. Output: Emotion data sent to the server.

[0811] Step 6:

[0812] The server analyzes the user's emotional state based on the emotional data it receives. The server then optimizes the travel plan based on this emotional state. For example, if the user is feeling stressed, it will prioritize relaxing plans. Input: Emotional data. Output: Regenerate the optimized travel plan.

[0813] Step 7:

[0814] The server sends the regenerated optimized travel plan to the terminal and presents it to the user. The user can view and select the new plan optimized based on their emotional state in the virtual space. Input: Optimized travel plan. Output: Display of the optimized travel plan on the user's terminal.

[0815] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0816] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0817] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0818] [Third embodiment]

[0819] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0820] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0821] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0822] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0823] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0824] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0825] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0826] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0827] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0828] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0829] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0830] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0831] This invention specifically demonstrates a system that, when a user inputs their desired travel conditions, uses a generative AI model to generate an optimal travel plan and propose multiple options to the user.

[0832] Program Overview

[0833] The system has the following main functions:

[0834] 1. Receiving user input

[0835] 2. Analysis of input data

[0836] 3. Generate travel plans that meet the conditions

[0837] 4. Proposing and displaying travel plans

[0838] Detailed program processing

[0839] 1. Receiving user input

[0840] Users input their desired travel conditions into a form displayed on the screen via their device, including information such as landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting, hot air ballooning), level of facilities, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, and number of people.

[0841] The terminal receives the data entered by the user and transmits it to the server.

[0842] 2. Analysis of input data

[0843] The server receives and analyzes user data sent from the device, extracting it into individual parameters and preparing them for passing to the generative AI model.

[0844] 3. Generate travel plans that meet the conditions

[0845] The server passes the conditions based on the extracted parameters to a generative AI model (machine learning algorithm), which then searches the database for suitable travel destination information based on the conditions, selects candidate travel destinations that meet each condition, and collects detailed information for each destination (such as region, accommodation, activity locations, and temperature data).

[0846] The generative AI model generates multiple optimal travel plans from the collected data and returns them to the server.

[0847] 4. Proposing and displaying travel plans

[0848] The server sends the generated itineraries to the device, which then displays them to the user along with detailed information. Each itinerary includes details about the area, accommodation, temperature, and activities. The user can then select the itinerary that best suits their needs.

[0849] Specific examples

[0850] For example, if a user enters the following criteria:

[0851] A place with mountains and lakes

[0852] Where the torii gate is located

[0853] River rafting

[0854] Cottage-style accommodation

[0855] Suitable for those with egg allergies

[0856] Open-air hot spring

[0857] The device sends these conditions to the server, which then detects "Hakone" and "Nikko" as travel destinations that match the conditions. The generative AI model generates a "Hakone Plan" and a "Nikko Plan" and returns them to the server along with detailed information about each plan.

[0858] A detailed example of a Hakone plan includes Lake Ashi, the Hakone Shrine's torii gate, a river cruise, cottage-style accommodation, meals for those with egg allergies, and an open-air hot spring. Similarly, a Nikko plan comprehensively suggests scenery, activities, and meals based on the user's requirements.

[0859] This allows users to compare the details of the Hakone and Nikko plans and choose the travel plan that best suits their needs.

[0860] This system makes it easy for users to plan their trips by efficiently generating travel plans based on the user's detailed wishes and presenting them as multiple options.

[0861] The processing flow will be explained below.

[0862] Step 1:

[0863] The user enters the desired travel conditions.

[0864] Users enter their desired travel conditions (image of scenery, type of activity, level of facilities, dietary preferences and restrictions, length of stay, number of people, etc.) into a form displayed on the device screen.

[0865] Step 2:

[0866] The terminal receives the user's input and sends it to the server.

[0867] The terminal formats the data entered by the user and sends a request to the server.

[0868] Step 3:

[0869] The server receives and analyzes the data from the user.

[0870] The server analyzes the received data and extracts each condition as a separate parameter.

[0871] Step 4:

[0872] The server prepares to pass the extracted parameters to the generative AI model.

[0873] The server converts the extracted conditions into an appropriate format and performs preprocessing to pass them to the generative AI model.

[0874] Step 5:

[0875] The generative AI model searches the database based on the criteria.

[0876] The generative AI model searches for suitable travel destination information from within the database based on the conditions passed from the server.

[0877] Step 6:

[0878] The generative AI model selects suitable travel destinations and collects detailed information about each one.

[0879] The generative AI model collects detailed information such as scenery, activities, accommodation, food options, and temperature data to select the best candidates.

[0880] Step 7:

[0881] A generative AI model creates multiple itineraries.

[0882] The generative AI model uses the collected information to generate multiple travel plans that meet the user's requirements.

[0883] Step 8:

[0884] The server transmits the generated travel plan to the terminal.

[0885] The server sends multiple travel plans received from the generative AI model to the user's device.

[0886] Step 9:

[0887] The device displays the travel plan to the user.

[0888] The terminal receives the travel plan sent from the server and displays it to the user along with detailed information.

[0889] Step 10:

[0890] The user selects the desired travel plan from the presented plans.

[0891] Users can select the travel plan that best suits their needs from the plans presented.

[0892] Step 11:

[0893] The device will display details of the plan the user has selected and the booking process.

[0894] The device will display detailed information about the plan the user has selected, a link to the reservation page, and guide them on their next steps.

[0895] Example 1

[0896] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0897] Conventional travel plan generation systems have difficulty quickly and automatically generating the optimal travel plan based on a user's desired conditions and providing multiple options. There is also a need for a system that can efficiently analyze and reflect travel plans based on the user's detailed desired conditions. This has led to the problem that it is difficult for users to select the travel plan that best meets their needs.

[0898] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0899] In this invention, the server includes means for receiving desired travel conditions from a user, means for generating multiple travel plans based on the conditions, means for generating an optimal travel plan based on the conditions using a generative AI model, and means for presenting the generated travel plans to the user. This makes it possible to quickly and efficiently generate an optimal travel plan based on the user's detailed desired conditions and provide it to the user as multiple options.

[0900] "User" means an individual or group that requests a travel plan and enters its requirements.

[0901] "Travel preferences" are the specific elements and requirements that users desire for their trip (e.g., landscape image, type of activity, level of facilities, dietary preferences or restrictions, length of stay, number of people, etc.).

[0902] "Terminal" refers to a device used by a user, such as a computer, smartphone, or tablet, which is used to input and receive travel preferences.

[0903] A "server" is a computer system that receives, analyzes, and processes data sent from a terminal.

[0904] A "generative AI model" is an algorithm or software that uses artificial intelligence technology to generate optimal travel plans based on input criteria.

[0905] A "trip plan" is a detailed travel plan based on a user's desired conditions, including destinations, accommodations, activities, temperature data, etc.

[0906] A "database" is an online or offline data storage system for storing information such as travel destination information, accommodation information, activity locations, temperature data, etc.

[0907] An "HTTP request" is part of a communication protocol for sending and receiving data between a terminal and a server, and is used to send the user's conditions to the server.

[0908] "JSON format" is an abbreviation for JavaScript Object Notation, a text-based format used as a data exchange format.

[0909] "Parameters" are the individual elements that make up the desired travel conditions (e.g., mountains, lakes, torii gates, river rafting, cottages, etc.).

[0910] The present invention is a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and then suggests multiple options to the user.

[0911] The system uses the following hardware and software:

[0912] 1. Terminal: A device such as a computer, smartphone, or tablet where users input their desired travel conditions. This terminal provides a user interface using a web browser or dedicated application.

[0913] 2. Server: A computer system for receiving, analyzing, and processing data sent by users. This server is responsible for storing user data, analyzing it, inputting it into the generative AI model, receiving the results, and generating the itinerary.

[0914] 3. Generative AI model: An algorithm or software that uses artificial intelligence technology to generate optimal travel plans based on input criteria. Examples include machine learning algorithms and natural language processing technology.

[0915] Specific processing of the system

[0916] Accepting user input:

[0917] Users input the desired travel conditions into a form displayed on the device screen. These conditions include landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting, hot air ballooning), facility availability, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, number of people, etc. The device stores the input information in its memory and sends this data to the server when the "send" button is pressed.

[0918] Analysis of input data:

[0919] The server receives the HTTP POST request sent from the device and parses the request body in JSON format. The server then breaks down the parsed data into individual parameters (e.g., "mountain," "lake," "torii gate," "river boating," "cottage," etc.) and converts them into a format suitable for input to the generative AI model.

[0920] Generate matching itineraries:

[0921] The server passes the formatted parameters to the generative AI model, which queries the database based on the received parameters to find suitable travel destinations, collects detailed information about the suitable destinations (such as region, accommodation, activity locations, and temperature data), and creates multiple optimal itineraries. This information is returned to the server in JSON format.

[0922] Suggest and display itineraries:

[0923] The server receives and analyzes the JSON data returned by the generative AI model. The server converts the travel plan information into a format that is easy for the user to understand and sends it to the device as an HTTP response. The device displays the received data on the screen and presents the user with multiple travel plans. The user can compare the displayed plans and select the one that best suits their needs.

[0924] Specific examples

[0925] For example, if a user enters the following criteria:

[0926] A place with mountains and lakes

[0927] Where the torii gate is located

[0928] River rafting

[0929] Cottage-style accommodation

[0930] Suitable for those with egg allergies

[0931] Open-air hot spring

[0932] An example prompt is:

[0933] Prompt statement:

[0934] We want the user to suggest itineraries related to "places with mountains and lakes, places with torii gates, river rafting, cottages, egg allergy friendly, and open-air hot springs." Based on this, please generate the optimal itinerary.

[0935] These conditions are sent from the device to a server, which then detects "Location A" and "Location B" as travel destinations that meet the conditions. The generative AI model uses this information to generate a "Location A Plan" and a "Location B Plan," and returns them to the server along with detailed information about each plan (mountains and lakes, torii gates, river rafting spots, cottage-style accommodations, meals for people with egg allergies, open-air hot springs, etc.). This allows users to compare the details of the "Location A Plan" and "Location B Plan" and choose the travel plan that best suits their needs.

[0936] This system allows users to efficiently generate travel plans based on their detailed desired conditions, and easily select the most suitable plan from multiple options.

[0937] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0938] Step 1: Receiving User Input

[0939] Description: The user enters the desired travel conditions into a form displayed on the device screen. The user's input (e.g., a place with mountains and lakes, a place with a torii gate, river rafting, a cottage, suitable for egg allergies, an open-air hot spring) is provided to the device as input data. The device stores this input data in its memory, and when the submit button on the form is pressed, it sends the stored data to the server via an HTTP POST request. Specifically, the device's browser or application sends the user input to the server in JSON format.

[0940] Step 2: Analyzing the input data

[0941] Description: The server receives an HTTP POST request sent from the device and parses the request body in JSON format. The parsed data is provided to the server as input data. The server breaks this data down into individual parameters (e.g., scenery, activity, dietary restrictions, etc.). Specifically, the server-side analysis module parses the JSON data and converts it into a format that can be input into a generative AI model based on the parameters.

[0942] Step 3: Generate itineraries that meet your criteria

[0943] Description: The server passes formatted parameters to the generative AI model. The generative AI model receives these parameters as input data and searches its database for suitable travel destinations based on that data. The search results are provided to the generative AI model as intermediate data. The generative AI model then collects detailed information (e.g., region, accommodation, activity locations, temperature data, etc.) based on the search results and creates multiple optimal travel plans. Specifically, the generative AI model executes database queries, assembles a travel plan from the obtained information, and returns it to the server in JSON format.

[0944] Step 4: Propose and display travel plans

[0945] Description: The server receives the JSON data returned from the generative AI model and provides it as output data. This data is parsed and converted into a format that is easy for the user to understand (e.g., HTML or text format). The server then sends the converted data to the device as an HTTP response. The final output data, a travel plan, is provided to the device. The device receives this data and displays it to the user. The user can compare the displayed multiple travel plans and select the one that best suits their needs. Specifically, the device's display module visually presents the travel plan information, allowing the user to compare them.

[0946] (Application example 1)

[0947] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0948] In recent years, there has been a demand for providing meals that meet individual preferences and health conditions. However, selecting the optimal menu based on detailed conditions regarding dietary restrictions and specific ingredients requires a lot of time and effort. For this reason, there is a need for a system that can efficiently suggest and optimize menus that meet the conditions.

[0949] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0950] In this invention, the server includes a means for receiving meal conditions requested by the user, a means for generating multiple meal menus based on the conditions, and a means for presenting the user with dining facilities that offer the generated meal menus, thereby making it possible to propose the optimal meal menu according to the user's detailed conditions.

[0951] "User" means an individual or group of people who utilizes the System to enter dietary requirements.

[0952] "Dietary requirements" refers to specific conditions such as the user's desired ingredients, dietary restrictions, level of spiciness, and serving amount.

[0953] "Food Menu" refers to a selection of food suggestions generated based on the user's dietary requirements.

[0954] "Food and beverage establishments" are establishments such as restaurants and cafes that offer food menus and allow users to order.

[0955] A "generative AI model" refers to a machine learning algorithm that generates optimal cooking menus based on the user's dietary requirements.

[0956] This invention is a system that uses a generative AI model to propose optimal meal menus based on a user's desired meal conditions. This system receives the user's meal conditions as input, generates multiple meal menus based on them, and then suggests dining establishments that meet the conditions entered by the user.

[0957] First, the user uses the terminal to input their desired dietary requirements. Dietary requirements include dietary restrictions (e.g., vegetarian, gluten-free), spiciness level (e.g., normal, spicy), desired ingredients (e.g., tomato, cheese), and serving size (e.g., two servings). The terminal receives the data entered by the user and sends it to the server.

[0958] The server receives and analyzes the user's meal requirements sent from the device. The analyzed data is extracted as individual parameters and prepared to be passed to the generative AI model. The generative AI model searches the database for food menus that match the user's requirements and collects detailed information about each (such as the name of the dish, dining facility, price, and dish details). The generative AI model generates multiple optimal food menus from the collected data and returns them to the server.

[0959] The server sends the generated menus to the terminal, which displays them with detailed information to the user, allowing the user to select the menu that best suits their needs and order delivery.

[0960] For example, if a user enters the following criteria:

[0961] Dietary: Vegetarian

[0962] Allergies: Gluten

[0963] Spice level: Medium

[0964] Desired ingredients: Tomato, cheese

[0965] Serving size: 2 servings

[0966] Based on this, the server will suggest the following menu options:

[0967] 1. Dish name: Vegetarian pizza with tomato and cheese

[0968] Dining options: Vegetarian Restaurant A

[0969] Price: 1500 yen

[0970] Details: Made with gluten-free pizza dough, topped with tomato and mozzarella cheese

[0971] 2. Dish name: Cheese and tomato pasta

[0972] Dining facilities: Healthy Cafe B

[0973] Price: 1200 yen

[0974] Details: Gluten-free pasta, fresh tomato sauce and Parmesan cheese

[0975] An example prompt might look like this:

[0976] conditions:

[0977] Dietary: Vegetarian

[0978] Allergies: Gluten

[0979] Spice level: Medium

[0980] Desired ingredients: Tomato, cheese

[0981] Serving size: 2 servings

[0982] Based on this, please suggest an appropriate menu.

[0983] In this way, a food menu based on the user's detailed preferences is efficiently generated and presented as multiple options, making meal selections easier for the user.

[0984] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0985] Step 1:

[0986] The user inputs their desired dietary requirements using a terminal. The input information includes dietary restrictions, spiciness level, desired ingredients, serving size, etc. This input information is sent from the terminal to the server.

[0987] Step 2:

[0988] The server receives and analyzes the user's dietary requirements sent from the device. During the analysis process, the input data is extracted as individual parameters, which are then prepared for passing to the generative AI model.

[0989] Step 3:

[0990] The server then passes the analyzed data to a generative AI model, which then searches a database for food menus that match the user's criteria and narrows down the menu based on the criteria.

[0991] Step 4:

[0992] The server uses a generative AI model to generate multiple optimal menus from the collected data, including the name of the dish, the dining establishment, the price, and details of the dish.

[0993] Step 5:

[0994] The server sends the generated menus to the terminal, which displays them to the user along with detailed information. The user compares the details of each menu and selects the desired dish.

[0995] Step 6:

[0996] The user selects the desired menu item and places a delivery order. The terminal sends the selected order information to the server, which then transmits the order information to the restaurant.

[0997] This allows users to easily select the most suitable food menu that meets their specific requirements and use the delivery service.

[0998] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0999] This invention specifically describes a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and suggests multiple options to the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it also suggests travel plans that correspond to the user's emotions.

[1000] Program Overview

[1001] The system has the following main functions:

[1002] 1. Receiving user input

[1003] 2. Analysis of input data

[1004] 3. Generate travel plans that meet the conditions

[1005] 4. Proposing and displaying travel plans

[1006] 5. Emotion engine recognizes and reflects user emotions

[1007] Detailed program processing

[1008] 1. Receiving user input

[1009] Users input their desired travel conditions into a form displayed on the screen via their device, including information such as landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting, hot air ballooning), level of facilities, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, and number of people.

[1010] The terminal receives the data entered by the user and transmits it to the server.

[1011] 2. Analysis of input data

[1012] The server receives and analyzes user data sent from the device, extracting it into individual parameters and preparing them for passing to the generative AI model.

[1013] 3. Generate travel plans that meet the conditions

[1014] The server passes the conditions based on the extracted parameters to a generative AI model (machine learning algorithm), which then searches the database for suitable travel destination information based on the conditions, selects candidate travel destinations that meet each condition, and collects detailed information for each destination (such as region, accommodation, activity locations, and temperature data).

[1015] The generative AI model generates multiple optimal travel plans from the collected data and returns them to the server.

[1016] 4. Proposing and displaying travel plans

[1017] The server sends the generated itineraries to the device, which then displays them to the user along with detailed information. Each itinerary includes details about the area, accommodation, temperature, and activities. The user can then select the itinerary that best suits their needs.

[1018] 5. Emotion engine recognizes and reflects user emotions

[1019] The emotion engine analyzes the user's emotional state in real time as they type, using the device's camera, microphone, and input data to analyze the user's facial expressions, tone of voice, and typing speed.

[1020] The server uses the emotion data obtained from the emotion engine to adjust the conditions based on the user's emotional state. For example, if the user is feeling stressed, the server instructs the generative AI model to prioritize generating relaxing travel plans.

[1021] The emotion engine also stores the user's emotional state as a history and uses it as a reference when generating the next travel plan, allowing it to propose an even more optimal plan based on the user's emotional history.

[1022] Specific examples

[1023] For example, if a user enters the following criteria:

[1024] A place with mountains and lakes

[1025] Where the torii gate is located

[1026] River rafting

[1027] Cottage-style accommodation

[1028] Suitable for those with egg allergies

[1029] Open-air hot spring

[1030] The device sends these conditions to the server, and the emotion engine recognizes that the user has a happy expression when entering the information. The emotion engine then sends this information to the server, which then detects "Hakone" and "Nikko" as travel destinations that match the conditions.

[1031] The generative AI model generates a "Hakone Plan" and a "Nikko Plan," and returns each plan to the server along with detailed information. Examples of detailed Hakone plans include Lake Ashi, the torii gates of Hakone Shrine, a river cruise, cottage-style accommodation, meals suitable for egg allergies, and open-air hot springs. Similarly, Nikko plans comprehensively suggest scenery, activities, and meal options based on the user's requirements.

[1032] This allows users to compare the details of the Hakone and Nikko plans and choose the travel plan that best suits their needs. The emotion engine supports plan generation based on the user's emotional state, resulting in travel suggestions that will satisfy the user even more.

[1033] The processing flow will be explained below.

[1034] Step 1:

[1035] The user enters the desired travel conditions.

[1036] Users enter their desired travel conditions (for example, "a place with mountains and lakes" as a scenic image, "hiking" as the type of activity, "cottage-style accommodation" as the level of facilities, "accommodation for egg allergies" as a dietary preference or restriction, length of stay, number of people, etc.) into a form displayed on the device screen.

[1037] Step 2:

[1038] The terminal receives the user's input and sends it to the server.

[1039] The terminal formats the data entered by the user and sends a request to the server.

[1040] Step 3:

[1041] The emotion engine analyzes user emotions in real time.

[1042] The device's emotion engine uses the camera and microphone to analyze the user's facial expressions and tone of voice to generate emotional data.

[1043] Step 4:

[1044] The device transmits the emotion data to the server.

[1045] The device transmits the analyzed emotion data to the server, which delivers it together with the travel conditions.

[1046] Step 5:

[1047] The server receives and analyzes the data from the user.

[1048] The server analyzes the received user data and emotion data and extracts each condition as an individual parameter.

[1049] Step 6:

[1050] The server prepares to pass the extracted parameters to the generative AI model.

[1051] The server converts the extracted conditions and emotion data into an appropriate format and performs preprocessing to pass it to the generative AI model.

[1052] Step 7:

[1053] The generative AI model searches the database based on the criteria and emotion data.

[1054] The generative AI model searches for suitable travel destination information from within the database based on the conditions and emotional data passed from the server.

[1055] Step 8:

[1056] The generative AI model selects suitable travel destinations and collects detailed information about each one.

[1057] The generative AI model collects detailed information such as scenery, activities, accommodation, food options, and temperature data to select the best candidates.

[1058] Step 9:

[1059] A generative AI model creates a travel plan that reflects emotional data.

[1060] The generative AI model generates the optimal travel plan for the user based on the collected information and the user's emotional state.

[1061] Step 10:

[1062] The server transmits the generated travel plan to the terminal.

[1063] The server sends multiple travel plans received from the generative AI model to the user's device.

[1064] Step 11:

[1065] The device displays the travel plan to the user.

[1066] The terminal receives the travel plan sent from the server and displays it to the user along with detailed information.

[1067] Step 12:

[1068] The user selects the desired travel plan from the presented plans.

[1069] Users can select the travel plan that best suits their needs from the plans presented.

[1070] Step 13:

[1071] The device will display details of the plan the user has selected and the booking process.

[1072] The device will display detailed information about the plan the user has selected, a link to the reservation page, and guide them on their next steps.

[1073] Example 2

[1074] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1075] Conventional travel plan generation systems can propose plans that reflect the user's desired travel conditions, but they cannot adjust the plans to reflect the user's real-time emotional state. This makes it difficult to provide optimal travel plans that reflect the user's emotions. Furthermore, because the system does not take the user's emotions into consideration, the proposed travel plans often do not meet the user's expectations.

[1076] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1077] In this invention, the server includes means for receiving desired travel conditions from a user, means for generating multiple travel plans based on the conditions, means for presenting the generated travel plans to the user, emotion analysis means for analyzing the emotional state of the user, and means for adjusting the travel plans based on the emotional state of the user. This makes it possible to propose travel plans that correspond to the emotional state of the user, and to provide travel plans that will highly satisfy the user.

[1078] "Travel preferences" refer to the specific elements and requests that users have for their travel plans, including landscape image, types of activities, level of facilities, dietary preferences and restrictions, length of stay, number of people, etc.

[1079] "Generative AI model" refers to machine learning algorithms and natural language processing algorithms used to generate optimal travel plans based on a user's desired travel conditions.

[1080] "Emotion analysis means" refers to a function that analyzes a user's emotional state in real time, and includes technology that uses a camera or microphone to analyze a user's facial expressions and tone of voice.

[1081] A "Travel Plan" is a proposal that details the activities and accommodations desired by a User at a particular travel destination.

[1082] "Server" refers to the equipment or system that receives input data from users, analyzes it, works with the generative AI model to generate a travel plan, and finally sends it to the terminal.

[1083] A "terminal" is a device (such as a PC, smartphone, or tablet) through which a user inputs desired travel conditions and receives and displays the generated travel plan.

[1084] This invention is a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and suggests multiple options to the user. Furthermore, by combining it with an emotion analysis means that recognizes the user's emotions, it also suggests travel plans that correspond to the user's emotions.

[1085] System configuration

[1086] The system consists of the following main components:

[1087] 1. Terminal: This is a device (e.g., PC, smartphone, tablet) on which the user enters their travel preferences and displays the generated travel plan.

[1088] 2. Server: This is the device or system that receives input data from users, analyzes it, works with the generative AI model to generate a travel plan, and finally sends it to the device.

[1089] 3. Generative AI model: This uses machine learning and natural language processing algorithms to generate optimal travel plans based on the user's travel requirements.

[1090] 4. Emotion analysis means: This includes technology that uses a camera or microphone to analyze the user's emotional state in real time and transmits the data to a server.

[1091] Detailed program processing

[1092] Receiving user input

[1093] The user enters their desired travel conditions into a form displayed on the screen via their device. For example, they can provide the following information:

[1094] Landscape image (e.g. mountains and lake, torii gate)

[1095] Type of activity (e.g. hiking, river rafting, hot air ballooning)

[1096] Completeness of facilities

[1097] Dietary preferences or restrictions (e.g., vegan, halal, allergy-friendly)

[1098] Length of stay

[1099] Number of people

[1100] When the user enters these conditions and presses the submit button, the device sends the input data to the server. The device application sends the data to the server in JSON format using an HTTP request or AJAX.

[1101] Analyzing input data

[1102] The server receives user data sent from the device, breaks it down into individual parameters, and analyzes them. This process is performed using scripts written in programming languages ​​such as Python or Java. The server then prepares the analyzed parameters to be passed to the generative AI model.

[1103] Generate travel plans that match your criteria

[1104] The server passes data to the generative AI model based on the analyzed parameters. The generative AI model searches the database for suitable travel destination information based on the user's criteria and selects candidate travel destinations that meet each criteria. For example, if "Hakone" and "Nikko" are selected, data on the respective regions, accommodations, activity locations, temperature, etc. is collected. The generative AI model generates multiple optimal travel plans based on this data and returns them to the server.

[1105] Proposing and displaying travel plans

[1106] The server sends multiple travel plans received from the generative AI model to the device, which then displays the plans to the user. For example, the details of the Hakone plan include Lake Ashi, the torii gates of Hakone Shrine, a river rafting spot, cottage-style accommodation, meals for people with egg allergies, and an open-air hot spring. The user can then view the plan that best suits their needs.

[1107] Sentiment analysis and re-suggestion

[1108] The emotion analysis means uses the device's camera and microphone to analyze the user's emotional state in real time. For example, if the user has a happy expression, it sends that emotion to the server. The server then adjusts the travel plan based on the emotional data, prioritizing plans that allow for relaxation. It also saves a history of the user's emotional state and uses it as a reference when generating the next travel plan.

[1109] Specific examples

[1110] For example, if a user enters the following criteria:

[1111] A place with mountains and lakes

[1112] Where the torii gate is located

[1113] River rafting

[1114] Cottage-style accommodation

[1115] Suitable for those with egg allergies

[1116] Open-air hot spring

[1117] The device sends these conditions to the server. The emotion analysis means recognizes that the user has a happy expression and sends this information to the server. The server detects "Hakone" and "Nikko" as travel destinations that match the conditions. The generative AI model generates a "Hakone Plan" and a "Nikko Plan" and returns them to the server along with detailed information about each.

[1118] Example prompt sentence:

[1119] "Please suggest a travel plan that includes mountains, a lake, a torii gate, river rafting, cottage-style accommodations, accommodations that cater to egg allergies, and an open-air hot spring."

[1120] Based on this prompt, the system generates an appropriate travel plan and suggests it to the user. The emotional analysis means analyzes the user's emotional state in real time and provides the optimal plan accordingly.

[1121] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1122] Step 1:

[1123] The user enters the desired travel conditions.

[1124] Users enter their desired travel conditions into an input form via their device. The conditions entered include landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting), facility availability, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, number of people, etc. When the user presses the "Submit" button, the entered data is sent to the server by the device. Input is made using text fields, and the data is sent to the server in JSON format.

[1125] Step 2:

[1126] The server receives and analyzes the data sent by the user.

[1127] The server receives the JSON data sent from the device. It parses the data to extract individual parameters, such as the landscape image and the type of activity. Python scripts are used to parse the data, and the parsed data is temporarily stored in an internal database. The output is a parsed parameter set, which is passed to the next processing step.

[1128] Step 3:

[1129] The server passes the analyzed data to the generative AI model.

[1130] The server sends the extracted parameter set to the API endpoint of the generative AI model. The generative AI model uses a machine learning algorithm to search the database for travel destination information that meets the user's criteria and generate optimal travel plan candidates. The input to this step is the analysis parameters, and the output is a set of generated multiple travel plans.

[1131] Step 4:

[1132] The generative AI model generates an optimal travel plan and returns it to the server.

[1133] Based on the received parameters, the generative AI model performs a database search to select candidate travel destinations that meet each condition. Specifically, it collects data such as region, accommodation, activity locations, and temperature. Once optimal travel plans are generated, these plans are returned to the server. The input is a set of parameters for the generative AI model, and the output is detailed information about the travel plan.

[1134] Step 5:

[1135] The server transmits the generated travel plan to the terminal.

[1136] The server sends the travel plan received from the generative AI model to the device. This transmission uses JSON format as an HTTP response. The device receives this and displays it to the user. The input is detailed information about the travel plan, and the output is data sent to the device.

[1137] Step 6:

[1138] The terminal displays the travel plan to the user.

[1139] The terminal displays the travel plans received from the server in a user-friendly format. Each plan includes details such as area, accommodation, activities, and meal options. For example, the information is presented visually using diagrams and tables on a web page. The input is the travel plan details, and the output is the display to the user.

[1140] Step 7:

[1141] The emotion analysis means analyzes the user's emotional state and transmits it to the server.

[1142] The emotion analysis means analyzes the user's facial expressions and voice in real time using the device's built-in camera and microphone. For example, if the user looks happy, that information is sent to the server. The input is data from the camera and microphone, and the output is analyzed emotional data.

[1143] Step 8:

[1144] The server adjusts the travel plan based on the emotional data and sends it back to the device.

[1145] The server adjusts the travel plan based on the emotional data obtained from the emotion analysis means. If necessary, it may regenerate the plan by issuing new instructions to the generative AI model. The adjusted plan is then sent back to the device. The input is the emotional data and the existing travel plan, and the output is the adjusted travel plan.

[1146] Step 9:

[1147] The device will then redisplay the adjusted itinerary to the user.

[1148] The terminal redisplays the adjusted travel plan sent from the server to the user, providing the optimal travel plan that matches the user's emotions. The input is the adjusted travel plan, and the output is the redisplay to the user.

[1149] (Application example 2)

[1150] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1151] Conventional travel plan generation systems generate plans based on user input, but are unable to reflect the user's real-time emotional state. As a result, proposals are made that are not optimized for the user's emotional state, which can reduce user satisfaction. To solve this problem, a system is needed that can recognize the user's emotions in real time and propose optimal travel plans accordingly.

[1152] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving desired travel conditions from a user, means for generating multiple travel plans based on the conditions, means for presenting the generated travel plans to the user, means for recognizing the user's emotional state using emotion analysis technology, and means for optimizing the travel plans based on the recognized emotional state. This makes it possible to propose an optimal travel plan that is adapted to the user's emotional state.

[1153] "User" means a person who uses the System to input travel preferences.

[1154] "Travel preferences" refers to the user's detailed preferences, such as desired travel destinations, activities, and dietary restrictions.

[1155] "Multiple Travel Plans" are multiple travel options suggested by the generative AI model based on the user's criteria.

[1156] A "generative AI model" is a machine learning algorithm that generates optimal travel plans from collected data.

[1157] "Emotion analysis technology" is a technology that recognizes a user's emotional state by analyzing their facial expressions, tone of voice, etc.

[1158] "Emotional state" refers to the real-time emotions of users as they enter their travel requirements.

[1159] "Presentation means" refers to an interface that displays the generated travel plan in an easy-to-understand manner to the user.

[1160] The "optimization method" is a mechanism that uses emotion analysis technology to adjust and optimize travel plans based on the user's emotional state.

[1161] The "server" is a centralized system that receives, analyzes, and processes user input and emotional states.

[1162] This invention provides a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and suggests multiple options to the user. Furthermore, by combining this with emotion analysis technology that recognizes the user's emotional state, it also suggests travel plans that correspond to the user's emotions.

[1163] Hardware and software used

[1164] Hardware

[1165] Device: Smart glasses, head-mounted display (e.g., Oculus Quest 2)

[1166] Camera: A camera built into smart glasses or HMD

[1167] Microphone: A microphone built into smart glasses or HMD

[1168] software

[1169] Generative AI model: OpenAI GPT-3

[1170] Emotion analysis technology: Affectiva SDK

[1171] Development platform: Unity, Unreal Engine

[1172] Data analysis: Python, TensorFlow

[1173] Server side: Flask (Python web framework)

[1174] Specific processing flow

[1175] 1. Accepting user input:

[1176] Users wear smart glasses or a head-mounted display and input their desired travel conditions into the virtual space. This input is done through a form, and the data is sent from the device to a server.

[1177] 2. Analysis of input data:

[1178] The server analyzes the received condition data and extracts individual parameters, which are then passed to the generative AI model.

[1179] 3. Generate a travel plan:

[1180] The generative AI model (OpenAI GPT-3) generates multiple itineraries based on criteria, and the generation process is done using Python and TensorFlow.

[1181] 4. Present your travel plan:

[1182] The generated travel plan is sent from the server to the device and displayed on the user's smart glasses or HMD, using Unity or Unreal Engine.

[1183] 5. Emotional state recognition and suggestion optimization:

[1184] Using emotion analysis technology (Affectiva SDK), the system analyzes the user's facial expressions and tone of voice in real time. This data is sent to a server, and the travel plan is optimized based on the user's emotional state.

[1185] Specific examples

[1186] Consider the case where the user enters the following criteria:

[1187] Landscape image: Mountains and lake

[1188] Activity:River Rafting

[1189] Accommodation type: Cottage

[1190] Meals: Egg allergy friendly

[1191] Length of stay: 3 days

[1192] Number of people: 2 people

[1193] Based on these input conditions, the server uses a generative AI model (GPT-3) to generate the optimal travel plan. For example, a "Hakone Plan" and a "Nikko Plan" are generated, each containing detailed information.

[1194] Example prompt sentence:

[1195] The user wants to be somewhere with mountains and a lake, so you propose an egg-friendly itinerary that includes cottage-style accommodations and a river cruise. The trip is three days long and for two people. Make sure the user is relaxed and has fun.

[1196] The server sends the generated plan to the device, where the user experiences it in a virtual space through smart glasses or an HMD. Emotion analysis technology recognizes the user's real-time emotional state and prioritizes, for example, relaxing plans that are less stressful. This allows the user to choose the optimal travel plan that adapts to their emotional state.

[1197] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1198] Step 1:

[1199] The user puts on smart glasses or a head-mounted display and enters the desired travel conditions in a virtual space. The input form includes items such as landscape image, type of activity, accommodation type, dietary restrictions, length of stay, and number of people. The entered conditions are sent from the device to the server. Input: User's condition data. Output: Condition data sent to the server.

[1200] Step 2:

[1201] The server receives and analyzes the user's condition data sent from the device. This analysis extracts the condition data as individual parameters (scenery image, type of activity, etc.). These parameters are then formatted to be passed to the generative AI model. Input: User's condition data. Output: A set of parameters to be passed to the generative AI model.

[1202] Step 3:

[1203] The server inputs the analyzed parameter set into the generative AI model (GPT-3) to generate the optimal travel plan. At this time, the generative AI model searches the database for information on suitable travel destinations and collects detailed information on candidate locations that meet the conditions. Input: Parameter set. Output: Generated multiple travel plans.

[1204] Step 4:

[1205] The server sends the generated multiple travel plans to the terminal. The terminal displays the travel plans to the user in a virtual space. The user can view each plan along with detailed information (region, accommodation, activities, meal conditions, etc.). Input: Generated travel plans. Output: Travel plans displayed on the user's terminal.

[1206] Step 5:

[1207] Using emotion analysis technology, the device analyzes the user's facial expressions and tone of voice in real time. This emotion data is sent from the device to the server. Input: User emotion data. Output: Emotion data sent to the server.

[1208] Step 6:

[1209] The server analyzes the user's emotional state based on the emotional data it receives. The server then optimizes the travel plan based on this emotional state. For example, if the user is feeling stressed, it will prioritize relaxing plans. Input: Emotional data. Output: Regenerate the optimized travel plan.

[1210] Step 7:

[1211] The server sends the regenerated optimized travel plan to the terminal and presents it to the user. The user can view and select the new plan optimized based on their emotional state in the virtual space. Input: Optimized travel plan. Output: Display of the optimized travel plan on the user's terminal.

[1212] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1213] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1214] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1215] [Fourth embodiment]

[1216] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1217] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1218] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1219] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1220] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1221] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1222] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1223] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1224] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1225] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1226] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1227] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1228] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1229] This invention specifically demonstrates a system that, when a user inputs their desired travel conditions, uses a generative AI model to generate an optimal travel plan and propose multiple options to the user.

[1230] Program Overview

[1231] The system has the following main functions:

[1232] 1. Receiving user input

[1233] 2. Analysis of input data

[1234] 3. Generate travel plans that meet the conditions

[1235] 4. Proposing and displaying travel plans

[1236] Detailed program processing

[1237] 1. Receiving user input

[1238] Users input their desired travel conditions into a form displayed on the screen via their device, including information such as landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting, hot air ballooning), level of facilities, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, and number of people.

[1239] The terminal receives the data entered by the user and transmits it to the server.

[1240] 2. Analysis of input data

[1241] The server receives and analyzes user data sent from the device, extracting it into individual parameters and preparing them for passing to the generative AI model.

[1242] 3. Generate travel plans that meet the conditions

[1243] The server passes the conditions based on the extracted parameters to a generative AI model (machine learning algorithm), which then searches the database for suitable travel destination information based on the conditions, selects candidate travel destinations that meet each condition, and collects detailed information for each destination (such as region, accommodation, activity locations, and temperature data).

[1244] The generative AI model generates multiple optimal travel plans from the collected data and returns them to the server.

[1245] 4. Proposing and displaying travel plans

[1246] The server sends the generated itineraries to the device, which then displays them to the user along with detailed information. Each itinerary includes details about the area, accommodation, temperature, and activities. The user can then select the itinerary that best suits their needs.

[1247] Specific examples

[1248] For example, if a user enters the following criteria:

[1249] A place with mountains and lakes

[1250] Where the torii gate is located

[1251] River rafting

[1252] Cottage-style accommodation

[1253] Suitable for those with egg allergies

[1254] Open-air hot spring

[1255] The device sends these conditions to the server, which then detects "Hakone" and "Nikko" as travel destinations that match the conditions. The generative AI model generates a "Hakone Plan" and a "Nikko Plan" and returns them to the server along with detailed information about each plan.

[1256] A detailed example of a Hakone plan includes Lake Ashi, the Hakone Shrine's torii gate, a river cruise, cottage-style accommodation, meals for those with egg allergies, and an open-air hot spring. Similarly, a Nikko plan comprehensively suggests scenery, activities, and meals based on the user's requirements.

[1257] This allows users to compare the details of the Hakone and Nikko plans and choose the travel plan that best suits their needs.

[1258] This system makes it easy for users to plan their trips by efficiently generating travel plans based on the user's detailed wishes and presenting them as multiple options.

[1259] The processing flow will be explained below.

[1260] Step 1:

[1261] The user enters the desired travel conditions.

[1262] Users enter their desired travel conditions (image of scenery, type of activity, level of facilities, dietary preferences and restrictions, length of stay, number of people, etc.) into a form displayed on the device screen.

[1263] Step 2:

[1264] The terminal receives the user's input and sends it to the server.

[1265] The terminal formats the data entered by the user and sends a request to the server.

[1266] Step 3:

[1267] The server receives and analyzes the data from the user.

[1268] The server analyzes the received data and extracts each condition as a separate parameter.

[1269] Step 4:

[1270] The server prepares to pass the extracted parameters to the generative AI model.

[1271] The server converts the extracted conditions into an appropriate format and performs preprocessing to pass them to the generative AI model.

[1272] Step 5:

[1273] The generative AI model searches the database based on the criteria.

[1274] The generative AI model searches for suitable travel destination information from within the database based on the conditions passed from the server.

[1275] Step 6:

[1276] The generative AI model selects suitable travel destinations and collects detailed information about each one.

[1277] The generative AI model collects detailed information such as scenery, activities, accommodation, food options, and temperature data to select the best candidates.

[1278] Step 7:

[1279] A generative AI model creates multiple itineraries.

[1280] The generative AI model uses the collected information to generate multiple travel plans that meet the user's requirements.

[1281] Step 8:

[1282] The server transmits the generated travel plan to the terminal.

[1283] The server sends multiple travel plans received from the generative AI model to the user's device.

[1284] Step 9:

[1285] The device displays the travel plan to the user.

[1286] The terminal receives the travel plan sent from the server and displays it to the user along with detailed information.

[1287] Step 10:

[1288] The user selects the desired travel plan from the presented plans.

[1289] Users can select the travel plan that best suits their needs from the plans presented.

[1290] Step 11:

[1291] The device will display details of the plan the user has selected and the booking process.

[1292] The device will display detailed information about the plan the user has selected, a link to the reservation page, and guide them on their next steps.

[1293] Example 1

[1294] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1295] Conventional travel plan generation systems have difficulty quickly and automatically generating the optimal travel plan based on a user's desired conditions and providing multiple options. There is also a need for a system that can efficiently analyze and reflect travel plans based on the user's detailed desired conditions. This has led to the problem that it is difficult for users to select the travel plan that best meets their needs.

[1296] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1297] In this invention, the server includes means for receiving desired travel conditions from a user, means for generating multiple travel plans based on the conditions, means for generating an optimal travel plan based on the conditions using a generative AI model, and means for presenting the generated travel plans to the user. This makes it possible to quickly and efficiently generate an optimal travel plan based on the user's detailed desired conditions and provide it to the user as multiple options.

[1298] "User" means an individual or group that requests a travel plan and enters its requirements.

[1299] "Travel preferences" are the specific elements and requirements that users desire for their trip (e.g., landscape image, type of activity, level of facilities, dietary preferences or restrictions, length of stay, number of people, etc.).

[1300] "Terminal" refers to a device used by a user, such as a computer, smartphone, or tablet, which is used to input and receive travel preferences.

[1301] A "server" is a computer system that receives, analyzes, and processes data sent from a terminal.

[1302] A "generative AI model" is an algorithm or software that uses artificial intelligence technology to generate optimal travel plans based on input criteria.

[1303] A "trip plan" is a detailed travel plan based on a user's desired conditions, including destinations, accommodations, activities, temperature data, etc.

[1304] A "database" is an online or offline data storage system for storing information such as travel destination information, accommodation information, activity locations, temperature data, etc.

[1305] An "HTTP request" is part of a communication protocol for sending and receiving data between a terminal and a server, and is used to send the user's conditions to the server.

[1306] "JSON format" is an abbreviation for JavaScript Object Notation, a text-based format used as a data exchange format.

[1307] "Parameters" are the individual elements that make up the desired travel conditions (e.g., mountains, lakes, torii gates, river rafting, cottages, etc.).

[1308] The present invention is a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and then suggests multiple options to the user.

[1309] The system uses the following hardware and software:

[1310] 1. Terminal: A device such as a computer, smartphone, or tablet where users input their desired travel conditions. This terminal provides a user interface using a web browser or dedicated application.

[1311] 2. Server: A computer system for receiving, analyzing, and processing data sent by users. This server is responsible for storing user data, analyzing it, inputting it into the generative AI model, receiving the results, and generating the itinerary.

[1312] 3. Generative AI model: An algorithm or software that uses artificial intelligence technology to generate optimal travel plans based on input criteria. Examples include machine learning algorithms and natural language processing technology.

[1313] Specific processing of the system

[1314] Accepting user input:

[1315] Users input the desired travel conditions into a form displayed on the device screen. These conditions include landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting, hot air ballooning), facility availability, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, number of people, etc. The device stores the input information in its memory and sends this data to the server when the "send" button is pressed.

[1316] Analysis of input data:

[1317] The server receives the HTTP POST request sent from the device and parses the request body in JSON format. The server then breaks down the parsed data into individual parameters (e.g., "mountain," "lake," "torii gate," "river boating," "cottage," etc.) and converts them into a format suitable for input to the generative AI model.

[1318] Generate matching itineraries:

[1319] The server passes the formatted parameters to the generative AI model, which queries the database based on the received parameters to find suitable travel destinations, collects detailed information about the suitable destinations (such as region, accommodation, activity locations, and temperature data), and creates multiple optimal itineraries. This information is returned to the server in JSON format.

[1320] Suggest and display itineraries:

[1321] The server receives and analyzes the JSON data returned by the generative AI model. The server converts the travel plan information into a format that is easy for the user to understand and sends it to the device as an HTTP response. The device displays the received data on the screen and presents the user with multiple travel plans. The user can compare the displayed plans and select the one that best suits their needs.

[1322] Specific examples

[1323] For example, if a user enters the following criteria:

[1324] A place with mountains and lakes

[1325] Where the torii gate is located

[1326] River rafting

[1327] Cottage-style accommodation

[1328] Suitable for those with egg allergies

[1329] Open-air hot spring

[1330] An example prompt is:

[1331] Prompt statement:

[1332] We want the user to suggest itineraries related to "places with mountains and lakes, places with torii gates, river rafting, cottages, egg allergy friendly, and open-air hot springs." Based on this, please generate the optimal itinerary.

[1333] These conditions are sent from the device to a server, which then detects "Location A" and "Location B" as travel destinations that meet the conditions. The generative AI model uses this information to generate a "Location A Plan" and a "Location B Plan," and returns them to the server along with detailed information about each plan (mountains and lakes, torii gates, river rafting spots, cottage-style accommodations, meals for people with egg allergies, open-air hot springs, etc.). This allows users to compare the details of the "Location A Plan" and "Location B Plan" and choose the travel plan that best suits their needs.

[1334] This system allows users to efficiently generate travel plans based on their detailed desired conditions, and easily select the most suitable plan from multiple options.

[1335] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1336] Step 1: Receiving User Input

[1337] Description: The user enters the desired travel conditions into a form displayed on the device screen. The user's input (e.g., a place with mountains and lakes, a place with a torii gate, river rafting, a cottage, suitable for egg allergies, an open-air hot spring) is provided to the device as input data. The device stores this input data in its memory, and when the submit button on the form is pressed, it sends the stored data to the server via an HTTP POST request. Specifically, the device's browser or application sends the user input to the server in JSON format.

[1338] Step 2: Analyzing the input data

[1339] Description: The server receives an HTTP POST request sent from the device and parses the request body in JSON format. The parsed data is provided to the server as input data. The server breaks this data down into individual parameters (e.g., scenery, activity, dietary restrictions, etc.). Specifically, the server-side analysis module parses the JSON data and converts it into a format that can be input into a generative AI model based on the parameters.

[1340] Step 3: Generate itineraries that meet your criteria

[1341] Description: The server passes formatted parameters to the generative AI model. The generative AI model receives these parameters as input data and searches its database for suitable travel destinations based on that data. The search results are provided to the generative AI model as intermediate data. The generative AI model then collects detailed information (e.g., region, accommodation, activity locations, temperature data, etc.) based on the search results and creates multiple optimal travel plans. Specifically, the generative AI model executes database queries, assembles a travel plan from the obtained information, and returns it to the server in JSON format.

[1342] Step 4: Propose and display travel plans

[1343] Description: The server receives the JSON data returned from the generative AI model and provides it as output data. This data is parsed and converted into a format that is easy for the user to understand (e.g., HTML or text format). The server then sends the converted data to the device as an HTTP response. The final output data, a travel plan, is provided to the device. The device receives this data and displays it to the user. The user can compare the displayed multiple travel plans and select the one that best suits their needs. Specifically, the device's display module visually presents the travel plan information, allowing the user to compare them.

[1344] (Application example 1)

[1345] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1346] In recent years, there has been a demand for providing meals that meet individual preferences and health conditions. However, selecting the optimal menu based on detailed conditions regarding dietary restrictions and specific ingredients requires a lot of time and effort. For this reason, there is a need for a system that can efficiently suggest and optimize menus that meet the conditions.

[1347] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1348] In this invention, the server includes a means for receiving meal conditions requested by the user, a means for generating multiple meal menus based on the conditions, and a means for presenting the user with dining facilities that offer the generated meal menus, thereby making it possible to propose the optimal meal menu according to the user's detailed conditions.

[1349] "User" means an individual or group of people who utilizes the System to enter dietary requirements.

[1350] "Dietary requirements" refers to specific conditions such as the user's desired ingredients, dietary restrictions, level of spiciness, and serving amount.

[1351] "Food Menu" refers to a selection of food suggestions generated based on the user's dietary requirements.

[1352] "Food and beverage establishments" are establishments such as restaurants and cafes that offer food menus and allow users to order.

[1353] A "generative AI model" refers to a machine learning algorithm that generates optimal cooking menus based on the user's dietary requirements.

[1354] This invention is a system that uses a generative AI model to propose optimal meal menus based on a user's desired meal conditions. This system receives the user's meal conditions as input, generates multiple meal menus based on them, and then suggests dining establishments that meet the conditions entered by the user.

[1355] First, the user uses the terminal to input their desired dietary requirements. Dietary requirements include dietary restrictions (e.g., vegetarian, gluten-free), spiciness level (e.g., normal, spicy), desired ingredients (e.g., tomato, cheese), and serving size (e.g., two servings). The terminal receives the data entered by the user and sends it to the server.

[1356] The server receives and analyzes the user's meal requirements sent from the device. The analyzed data is extracted as individual parameters and prepared to be passed to the generative AI model. The generative AI model searches the database for food menus that match the user's requirements and collects detailed information about each (such as the name of the dish, dining facility, price, and dish details). The generative AI model generates multiple optimal food menus from the collected data and returns them to the server.

[1357] The server sends the generated menus to the terminal, which displays them with detailed information to the user, allowing the user to select the menu that best suits their needs and order delivery.

[1358] For example, if a user enters the following criteria:

[1359] Dietary: Vegetarian

[1360] Allergies: Gluten

[1361] Spice level: Medium

[1362] Desired ingredients: Tomato, cheese

[1363] Serving size: 2 servings

[1364] Based on this, the server will suggest the following menu options:

[1365] 1. Dish name: Vegetarian pizza with tomato and cheese

[1366] Dining options: Vegetarian Restaurant A

[1367] Price: 1500 yen

[1368] Details: Made with gluten-free pizza dough, topped with tomato and mozzarella cheese

[1369] 2. Dish name: Cheese and tomato pasta

[1370] Dining facilities: Healthy Cafe B

[1371] Price: 1200 yen

[1372] Details: Gluten-free pasta, fresh tomato sauce and Parmesan cheese

[1373] An example prompt might look like this:

[1374] conditions:

[1375] Dietary: Vegetarian

[1376] Allergies: Gluten

[1377] Spice level: Medium

[1378] Desired ingredients: Tomato, cheese

[1379] Serving size: 2 servings

[1380] Based on this, please suggest an appropriate menu.

[1381] In this way, a food menu based on the user's detailed preferences is efficiently generated and presented as multiple options, making meal selections easier for the user.

[1382] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1383] Step 1:

[1384] The user inputs their desired dietary requirements using a terminal. The input information includes dietary restrictions, spiciness level, desired ingredients, serving size, etc. This input information is sent from the terminal to the server.

[1385] Step 2:

[1386] The server receives and analyzes the user's dietary requirements sent from the device. During the analysis process, the input data is extracted as individual parameters, which are then prepared for passing to the generative AI model.

[1387] Step 3:

[1388] The server then passes the analyzed data to a generative AI model, which then searches a database for food menus that match the user's criteria and narrows down the menu based on the criteria.

[1389] Step 4:

[1390] The server uses a generative AI model to generate multiple optimal menus from the collected data, including the name of the dish, the dining establishment, the price, and details of the dish.

[1391] Step 5:

[1392] The server sends the generated menus to the terminal, which displays them to the user along with detailed information. The user compares the details of each menu and selects the desired dish.

[1393] Step 6:

[1394] The user selects the desired menu item and places a delivery order. The terminal sends the selected order information to the server, which then transmits the order information to the restaurant.

[1395] This allows users to easily select the most suitable food menu that meets their specific requirements and use the delivery service.

[1396] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1397] This invention specifically describes a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and suggests multiple options to the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it also suggests travel plans that correspond to the user's emotions.

[1398] Program Overview

[1399] The system has the following main functions:

[1400] 1. Receiving user input

[1401] 2. Analysis of input data

[1402] 3. Generate travel plans that meet the conditions

[1403] 4. Proposing and displaying travel plans

[1404] 5. Emotion engine recognizes and reflects user emotions

[1405] Detailed program processing

[1406] 1. Receiving user input

[1407] Users input their desired travel conditions into a form displayed on the screen via their device, including information such as landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting, hot air ballooning), level of facilities, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, and number of people.

[1408] The terminal receives the data entered by the user and transmits it to the server.

[1409] 2. Analysis of input data

[1410] The server receives and analyzes user data sent from the device, extracting it into individual parameters and preparing them for passing to the generative AI model.

[1411] 3. Generate travel plans that meet the conditions

[1412] The server passes the conditions based on the extracted parameters to a generative AI model (machine learning algorithm), which then searches the database for suitable travel destination information based on the conditions, selects candidate travel destinations that meet each condition, and collects detailed information for each destination (such as region, accommodation, activity locations, and temperature data).

[1413] The generative AI model generates multiple optimal travel plans from the collected data and returns them to the server.

[1414] 4. Proposing and displaying travel plans

[1415] The server sends the generated itineraries to the device, which then displays them to the user along with detailed information. Each itinerary includes details about the area, accommodation, temperature, and activities. The user can then select the itinerary that best suits their needs.

[1416] 5. Emotion engine recognizes and reflects user emotions

[1417] The emotion engine analyzes the user's emotional state in real time as they type, using the device's camera, microphone, and input data to analyze the user's facial expressions, tone of voice, and typing speed.

[1418] The server uses the emotion data obtained from the emotion engine to adjust the conditions based on the user's emotional state. For example, if the user is feeling stressed, the server instructs the generative AI model to prioritize generating relaxing travel plans.

[1419] The emotion engine also stores the user's emotional state as a history and uses it as a reference when generating the next travel plan, allowing it to propose an even more optimal plan based on the user's emotional history.

[1420] Specific examples

[1421] For example, if a user enters the following criteria:

[1422] A place with mountains and lakes

[1423] Where the torii gate is located

[1424] River rafting

[1425] Cottage-style accommodation

[1426] Suitable for those with egg allergies

[1427] Open-air hot spring

[1428] The device sends these conditions to the server, and the emotion engine recognizes that the user has a happy expression when entering the information. The emotion engine then sends this information to the server, which then detects "Hakone" and "Nikko" as travel destinations that match the conditions.

[1429] The generative AI model generates a "Hakone Plan" and a "Nikko Plan," and returns each plan to the server along with detailed information. Examples of detailed Hakone plans include Lake Ashi, the torii gates of Hakone Shrine, a river cruise, cottage-style accommodation, meals suitable for egg allergies, and open-air hot springs. Similarly, Nikko plans comprehensively suggest scenery, activities, and meal options based on the user's requirements.

[1430] This allows users to compare the details of the Hakone and Nikko plans and choose the travel plan that best suits their needs. The emotion engine supports plan generation based on the user's emotional state, resulting in travel suggestions that will satisfy the user even more.

[1431] The processing flow will be explained below.

[1432] Step 1:

[1433] The user enters the desired travel conditions.

[1434] Users enter their desired travel conditions (for example, "a place with mountains and lakes" as a scenic image, "hiking" as the type of activity, "cottage-style accommodation" as the level of facilities, "accommodation for egg allergies" as a dietary preference or restriction, length of stay, number of people, etc.) into a form displayed on the device screen.

[1435] Step 2:

[1436] The terminal receives the user's input and sends it to the server.

[1437] The terminal formats the data entered by the user and sends a request to the server.

[1438] Step 3:

[1439] The emotion engine analyzes user emotions in real time.

[1440] The device's emotion engine uses the camera and microphone to analyze the user's facial expressions and tone of voice to generate emotional data.

[1441] Step 4:

[1442] The device transmits the emotion data to the server.

[1443] The device transmits the analyzed emotion data to the server, which delivers it together with the travel conditions.

[1444] Step 5:

[1445] The server receives and analyzes the data from the user.

[1446] The server analyzes the received user data and emotion data and extracts each condition as an individual parameter.

[1447] Step 6:

[1448] The server prepares to pass the extracted parameters to the generative AI model.

[1449] The server converts the extracted conditions and emotion data into an appropriate format and performs preprocessing to pass it to the generative AI model.

[1450] Step 7:

[1451] The generative AI model searches the database based on the criteria and emotion data.

[1452] The generative AI model searches for suitable travel destination information from within the database based on the conditions and emotional data passed from the server.

[1453] Step 8:

[1454] The generative AI model selects suitable travel destinations and collects detailed information about each one.

[1455] The generative AI model collects detailed information such as scenery, activities, accommodation, food options, and temperature data to select the best candidates.

[1456] Step 9:

[1457] A generative AI model creates a travel plan that reflects emotional data.

[1458] The generative AI model generates the optimal travel plan for the user based on the collected information and the user's emotional state.

[1459] Step 10:

[1460] The server transmits the generated travel plan to the terminal.

[1461] The server sends multiple travel plans received from the generative AI model to the user's device.

[1462] Step 11:

[1463] The device displays the travel plan to the user.

[1464] The terminal receives the travel plan sent from the server and displays it to the user along with detailed information.

[1465] Step 12:

[1466] The user selects the desired travel plan from the presented plans.

[1467] Users can select the travel plan that best suits their needs from the plans presented.

[1468] Step 13:

[1469] The device will display details of the plan the user has selected and the booking process.

[1470] The device will display detailed information about the plan the user has selected, a link to the reservation page, and guide them on their next steps.

[1471] Example 2

[1472] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1473] Conventional travel plan generation systems can propose plans that reflect the user's desired travel conditions, but they cannot adjust the plans to reflect the user's real-time emotional state. This makes it difficult to provide optimal travel plans that reflect the user's emotions. Furthermore, because the system does not take the user's emotions into consideration, the proposed travel plans often do not meet the user's expectations.

[1474] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1475] In this invention, the server includes means for receiving desired travel conditions from a user, means for generating multiple travel plans based on the conditions, means for presenting the generated travel plans to the user, emotion analysis means for analyzing the emotional state of the user, and means for adjusting the travel plans based on the emotional state of the user. This makes it possible to propose travel plans that correspond to the emotional state of the user, and to provide travel plans that will highly satisfy the user.

[1476] "Travel preferences" refer to the specific elements and requests that users have for their travel plans, including landscape image, types of activities, level of facilities, dietary preferences and restrictions, length of stay, number of people, etc.

[1477] "Generative AI model" refers to machine learning algorithms and natural language processing algorithms used to generate optimal travel plans based on a user's desired travel conditions.

[1478] "Emotion analysis means" refers to a function that analyzes a user's emotional state in real time, and includes technology that uses a camera or microphone to analyze a user's facial expressions and tone of voice.

[1479] A "Travel Plan" is a proposal that details the activities and accommodations desired by a User at a particular travel destination.

[1480] "Server" refers to the equipment or system that receives input data from users, analyzes it, works with the generative AI model to generate a travel plan, and finally sends it to the terminal.

[1481] A "terminal" is a device (such as a PC, smartphone, or tablet) through which a user inputs desired travel conditions and receives and displays the generated travel plan.

[1482] This invention is a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and suggests multiple options to the user. Furthermore, by combining it with an emotion analysis means that recognizes the user's emotions, it also suggests travel plans that correspond to the user's emotions.

[1483] System configuration

[1484] The system consists of the following main components:

[1485] 1. Terminal: This is a device (e.g., PC, smartphone, tablet) on which the user enters their travel preferences and displays the generated travel plan.

[1486] 2. Server: This is the device or system that receives input data from users, analyzes it, works with the generative AI model to generate a travel plan, and finally sends it to the device.

[1487] 3. Generative AI model: This uses machine learning and natural language processing algorithms to generate optimal travel plans based on the user's travel requirements.

[1488] 4. Emotion analysis means: This includes technology that uses a camera or microphone to analyze the user's emotional state in real time and transmits the data to a server.

[1489] Detailed program processing

[1490] Receiving user input

[1491] The user enters their desired travel conditions into a form displayed on the screen via their device. For example, they can provide the following information:

[1492] Landscape image (e.g. mountains and lake, torii gate)

[1493] Type of activity (e.g. hiking, river rafting, hot air ballooning)

[1494] Completeness of facilities

[1495] Dietary preferences or restrictions (e.g., vegan, halal, allergy-friendly)

[1496] Length of stay

[1497] Number of people

[1498] When the user enters these conditions and presses the submit button, the device sends the input data to the server. The device application sends the data to the server in JSON format using an HTTP request or AJAX.

[1499] Analyzing input data

[1500] The server receives user data sent from the device, breaks it down into individual parameters, and analyzes them. This process is performed using scripts written in programming languages ​​such as Python or Java. The server then prepares the analyzed parameters to be passed to the generative AI model.

[1501] Generate travel plans that match your criteria

[1502] The server passes data to the generative AI model based on the analyzed parameters. The generative AI model searches the database for suitable travel destination information based on the user's criteria and selects candidate travel destinations that meet each criteria. For example, if "Hakone" and "Nikko" are selected, data on the respective regions, accommodations, activity locations, temperature, etc. is collected. The generative AI model generates multiple optimal travel plans based on this data and returns them to the server.

[1503] Proposing and displaying travel plans

[1504] The server sends multiple travel plans received from the generative AI model to the device, which then displays the plans to the user. For example, the details of the Hakone plan include Lake Ashi, the torii gates of Hakone Shrine, a river rafting spot, cottage-style accommodation, meals for people with egg allergies, and an open-air hot spring. The user can then view the plan that best suits their needs.

[1505] Sentiment analysis and re-suggestion

[1506] The emotion analysis means uses the device's camera and microphone to analyze the user's emotional state in real time. For example, if the user has a happy expression, it sends that emotion to the server. The server then adjusts the travel plan based on the emotional data, prioritizing plans that allow for relaxation. It also saves a history of the user's emotional state and uses it as a reference when generating the next travel plan.

[1507] Specific examples

[1508] For example, if a user enters the following criteria:

[1509] A place with mountains and lakes

[1510] Where the torii gate is located

[1511] River rafting

[1512] Cottage-style accommodation

[1513] Suitable for those with egg allergies

[1514] Open-air hot spring

[1515] The device sends these conditions to the server. The emotion analysis means recognizes that the user has a happy expression and sends this information to the server. The server detects "Hakone" and "Nikko" as travel destinations that match the conditions. The generative AI model generates a "Hakone Plan" and a "Nikko Plan" and returns them to the server along with detailed information about each.

[1516] Example prompt sentence:

[1517] "Please suggest a travel plan that includes mountains, a lake, a torii gate, river rafting, cottage-style accommodations, accommodations that cater to egg allergies, and an open-air hot spring."

[1518] Based on this prompt, the system generates an appropriate travel plan and suggests it to the user. The emotional analysis means analyzes the user's emotional state in real time and provides the optimal plan accordingly.

[1519] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1520] Step 1:

[1521] The user enters the desired travel conditions.

[1522] Users enter their desired travel conditions into an input form via their device. The conditions entered include landscape image (e.g., mountains and lakes, torii gates), type of activity (e.g., hiking, river rafting), facility availability, dietary preferences and restrictions (e.g., vegan, halal, allergy-friendly), length of stay, number of people, etc. When the user presses the "Submit" button, the entered data is sent to the server by the device. Input is made using text fields, and the data is sent to the server in JSON format.

[1523] Step 2:

[1524] The server receives and analyzes the data sent by the user.

[1525] The server receives the JSON data sent from the device. It parses the data to extract individual parameters, such as the landscape image and the type of activity. Python scripts are used to parse the data, and the parsed data is temporarily stored in an internal database. The output is a parsed parameter set, which is passed to the next processing step.

[1526] Step 3:

[1527] The server passes the analyzed data to the generative AI model.

[1528] The server sends the extracted parameter set to the API endpoint of the generative AI model. The generative AI model uses a machine learning algorithm to search the database for travel destination information that meets the user's criteria and generate optimal travel plan candidates. The input to this step is the analysis parameters, and the output is a set of generated multiple travel plans.

[1529] Step 4:

[1530] The generative AI model generates an optimal travel plan and returns it to the server.

[1531] Based on the received parameters, the generative AI model performs a database search to select candidate travel destinations that meet each condition. Specifically, it collects data such as region, accommodation, activity locations, and temperature. Once optimal travel plans are generated, these plans are returned to the server. The input is a set of parameters for the generative AI model, and the output is detailed information about the travel plan.

[1532] Step 5:

[1533] The server transmits the generated travel plan to the terminal.

[1534] The server sends the travel plan received from the generative AI model to the device. This transmission uses JSON format as an HTTP response. The device receives this and displays it to the user. The input is detailed information about the travel plan, and the output is data sent to the device.

[1535] Step 6:

[1536] The terminal displays the travel plan to the user.

[1537] The terminal displays the travel plans received from the server in a user-friendly format. Each plan includes details such as area, accommodation, activities, and meal options. For example, the information is presented visually using diagrams and tables on a web page. The input is the travel plan details, and the output is the display to the user.

[1538] Step 7:

[1539] The emotion analysis means analyzes the user's emotional state and transmits it to the server.

[1540] The emotion analysis means analyzes the user's facial expressions and voice in real time using the device's built-in camera and microphone. For example, if the user looks happy, that information is sent to the server. The input is data from the camera and microphone, and the output is analyzed emotional data.

[1541] Step 8:

[1542] The server adjusts the travel plan based on the emotional data and sends it back to the device.

[1543] The server adjusts the travel plan based on the emotional data obtained from the emotion analysis means. If necessary, it may regenerate the plan by issuing new instructions to the generative AI model. The adjusted plan is then sent back to the device. The input is the emotional data and the existing travel plan, and the output is the adjusted travel plan.

[1544] Step 9:

[1545] The device will then redisplay the adjusted itinerary to the user.

[1546] The terminal redisplays the adjusted travel plan sent from the server to the user, providing the optimal travel plan that matches the user's emotions. The input is the adjusted travel plan, and the output is the redisplay to the user.

[1547] (Application example 2)

[1548] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1549] Conventional travel plan generation systems generate plans based on user input, but are unable to reflect the user's real-time emotional state. As a result, proposals are made that are not optimized for the user's emotional state, which can reduce user satisfaction. To solve this problem, a system is needed that can recognize the user's emotions in real time and propose optimal travel plans accordingly.

[1550] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving desired travel conditions from a user, means for generating multiple travel plans based on the conditions, means for presenting the generated travel plans to the user, means for recognizing the user's emotional state using emotion analysis technology, and means for optimizing the travel plans based on the recognized emotional state. This makes it possible to propose an optimal travel plan that is adapted to the user's emotional state.

[1551] "User" means a person who uses the System to input travel preferences.

[1552] "Travel preferences" refers to the user's detailed preferences, such as desired travel destinations, activities, and dietary restrictions.

[1553] "Multiple Travel Plans" are multiple travel options suggested by the generative AI model based on the user's criteria.

[1554] A "generative AI model" is a machine learning algorithm that generates optimal travel plans from collected data.

[1555] "Emotion analysis technology" is a technology that recognizes a user's emotional state by analyzing their facial expressions, tone of voice, etc.

[1556] "Emotional state" refers to the real-time emotions of users as they enter their travel requirements.

[1557] "Presentation means" refers to an interface that displays the generated travel plan in an easy-to-understand manner to the user.

[1558] The "optimization method" is a mechanism that uses emotion analysis technology to adjust and optimize travel plans based on the user's emotional state.

[1559] The "server" is a centralized system that receives, analyzes, and processes user input and emotional states.

[1560] This invention provides a system that uses a generative AI model to generate an optimal travel plan when a user inputs their desired travel conditions, and suggests multiple options to the user. Furthermore, by combining this with emotion analysis technology that recognizes the user's emotional state, it also suggests travel plans that correspond to the user's emotions.

[1561] Hardware and software used

[1562] Hardware

[1563] Device: Smart glasses, head-mounted display (e.g., Oculus Quest 2)

[1564] Camera: A camera built into smart glasses or HMD

[1565] Microphone: A microphone built into smart glasses or HMD

[1566] software

[1567] Generative AI model: OpenAI GPT-3

[1568] Emotion analysis technology: Affectiva SDK

[1569] Development platform: Unity, Unreal Engine

[1570] Data analysis: Python, TensorFlow

[1571] Server side: Flask (Python web framework)

[1572] Specific processing flow

[1573] 1. Accepting user input:

[1574] Users wear smart glasses or a head-mounted display and input their desired travel conditions into the virtual space. This input is done through a form, and the data is sent from the device to a server.

[1575] 2. Analysis of input data:

[1576] The server analyzes the received condition data and extracts individual parameters, which are then passed to the generative AI model.

[1577] 3. Generate a travel plan:

[1578] The generative AI model (OpenAI GPT-3) generates multiple itineraries based on criteria, and the generation process is done using Python and TensorFlow.

[1579] 4. Present your travel plan:

[1580] The generated travel plan is sent from the server to the device and displayed on the user's smart glasses or HMD, using Unity or Unreal Engine.

[1581] 5. Emotional state recognition and suggestion optimization:

[1582] Using emotion analysis technology (Affectiva SDK), the system analyzes the user's facial expressions and tone of voice in real time. This data is sent to a server, and the travel plan is optimized based on the user's emotional state.

[1583] Specific examples

[1584] Consider the case where the user enters the following criteria:

[1585] Landscape image: Mountains and lake

[1586] Activity:River Rafting

[1587] Accommodation type: Cottage

[1588] Meals: Egg allergy friendly

[1589] Length of stay: 3 days

[1590] Number of people: 2 people

[1591] Based on these input conditions, the server uses a generative AI model (GPT-3) to generate the optimal travel plan. For example, a "Hakone Plan" and a "Nikko Plan" are generated, each containing detailed information.

[1592] Example prompt sentence:

[1593] The user wants to be somewhere with mountains and a lake, so you propose an egg-friendly itinerary that includes cottage-style accommodations and a river cruise. The trip is three days long and for two people. Make sure the user is relaxed and has fun.

[1594] The server sends the generated plan to the device, where the user experiences it in a virtual space through smart glasses or an HMD. Emotion analysis technology recognizes the user's real-time emotional state and prioritizes, for example, relaxing plans that are less stressful. This allows the user to choose the optimal travel plan that adapts to their emotional state.

[1595] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1596] Step 1:

[1597] The user puts on smart glasses or a head-mounted display and enters the desired travel conditions in a virtual space. The input form includes items such as landscape image, type of activity, accommodation type, dietary restrictions, length of stay, and number of people. The entered conditions are sent from the device to the server. Input: User's condition data. Output: Condition data sent to the server.

[1598] Step 2:

[1599] The server receives and analyzes the user's condition data sent from the device. This analysis extracts the condition data as individual parameters (scenery image, type of activity, etc.). These parameters are then formatted to be passed to the generative AI model. Input: User's condition data. Output: A set of parameters to be passed to the generative AI model.

[1600] Step 3:

[1601] The server inputs the analyzed parameter set into the generative AI model (GPT-3) to generate the optimal travel plan. At this time, the generative AI model searches the database for information on suitable travel destinations and collects detailed information on candidate locations that meet the conditions. Input: Parameter set. Output: Generated multiple travel plans.

[1602] Step 4:

[1603] The server sends the generated multiple travel plans to the terminal. The terminal displays the travel plans to the user in a virtual space. The user can view each plan along with detailed information (region, accommodation, activities, meal conditions, etc.). Input: Generated travel plans. Output: Travel plans displayed on the user's terminal.

[1604] Step 5:

[1605] Using emotion analysis technology, the device analyzes the user's facial expressions and tone of voice in real time. This emotion data is sent from the device to the server. Input: User emotion data. Output: Emotion data sent to the server.

[1606] Step 6:

[1607] The server analyzes the user's emotional state based on the emotional data it receives. The server then optimizes the travel plan based on this emotional state. For example, if the user is feeling stressed, it will prioritize relaxing plans. Input: Emotional data. Output: Regenerate the optimized travel plan.

[1608] Step 7:

[1609] The server sends the regenerated optimized travel plan to the terminal and presents it to the user. The user can view and select the new plan optimized based on their emotional state in the virtual space. Input: Optimized travel plan. Output: Display of the optimized travel plan on the user's terminal.

[1610] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1611] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1612] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1613] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1614] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1615] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1616] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1617] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1618] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1619] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1620] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1621] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1622] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1623] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1624] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1625] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1626] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1627] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1628] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1629] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1630] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1631] The following is further disclosed regarding the above embodiment.

[1632] (Claim 1)

[1633] A means for receiving travel preferences from users;

[1634] means for generating a plurality of travel plans based on the conditions;

[1635] means for presenting the generated travel plan to a user;

[1636] A system including:

[1637] (Claim 2)

[1638] 2. The system according to claim 1, wherein the desired travel conditions include a landscape image, a type of activity, a level of facilities, dietary preferences or restrictions, a length of stay, and a number of people.

[1639] (Claim 3)

[1640] 10. The system of claim 1, further comprising means for generating an optimal travel plan based on said conditions using a generative AI model.

[1641] "Example 1"

[1642] (Claim 1)

[1643] A means for receiving travel preferences from users;

[1644] means for generating a plurality of travel plans based on the conditions;

[1645] A means for generating an optimal travel plan based on the conditions using a generative AI model;

[1646] means for presenting the generated travel plan to a user;

[1647] A system including:

[1648] (Claim 2)

[1649] 2. The system according to claim 1, wherein the desired travel conditions include a landscape image, a type of activity, a level of facilities, dietary preferences or restrictions, a length of stay, and a number of people.

[1650] (Claim 3)

[1651] means for analyzing user data received from the device and extracting individual parameters;

[1652] means for converting the extracted parameters into a format for input into a generative AI model;

[1653] A means for the generative AI model to search a database based on criteria and collect information on suitable travel destinations;

[1654] means for generating a plurality of optimal travel plans using the collected information;

[1655] 2. The system of claim 1, comprising:

[1656] "Application Example 1"

[1657] (Claim 1)

[1658] a means for receiving dietary requirements from a user;

[1659] means for generating a plurality of dish menus based on the conditions;

[1660] a means for presenting to a user dining establishments that offer the generated food menu;

[1661] A system including:

[1662] (Claim 2)

[1663] The system of claim 1, wherein the dietary requirements include dietary restrictions, spiciness level, desired ingredients, and serving size.

[1664] (Claim 3)

[1665] 2. The system of claim 1, further comprising means for generating an optimal cooking menu based on said conditions using a generative AI model.

[1666] "Example 2: Combining Emotion Engines"

[1667] (Claim 1)

[1668] A means for receiving travel preferences from users;

[1669] means for generating a plurality of travel plans based on the conditions;

[1670] means for presenting the generated travel plan to a user;

[1671] an emotion analysis means for analyzing the emotional state of a user;

[1672] means for adjusting a travel plan based on the emotional state of the user;

[1673] A system including:

[1674] (Claim 2)

[1675] 2. The system according to claim 1, wherein the desired travel conditions include a landscape image, a type of activity, a level of facilities, dietary preferences or restrictions, a length of stay, and a number of people.

[1676] (Claim 3)

[1677] 10. The system of claim 1, further comprising means for generating an optimal travel plan based on said conditions using a generative AI model.

[1678] "Application example 2 when combining emotion engines"

[1679] (Claim 1)

[1680] A means for receiving travel preferences from users;

[1681] means for generating a plurality of travel plans based on the conditions;

[1682] means for presenting the generated travel plan to a user;

[1683] means for recognizing a user's emotional state using emotion analysis techniques;

[1684] means for optimizing a travel plan based on the recognized emotional state;

[1685] A system including:

[1686] (Claim 2)

[1687] 2. The system according to claim 1, wherein the desired travel conditions include a landscape image, a type of activity, a level of facilities, dietary preferences or restrictions, a length of stay, and a number of people.

[1688] (Claim 3)

[1689] 10. The system of claim 1, further comprising means for generating an optimal travel plan based on said conditions using a generative AI model. [Explanation of symbols]

[1690] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for receiving travel preferences from users; means for generating a plurality of travel plans based on the conditions; means for presenting the generated travel plan to a user; A system including:

2. 2. The system according to claim 1, wherein the desired travel conditions include a landscape image, a type of activity, a level of facilities, dietary preferences or restrictions, a length of stay, and a number of people.

3. 10. The system of claim 1, further comprising means for generating an optimal travel plan based on said conditions using a generative AI model.

Images