System

Abstract

A system is provided.SOLUTION: The system includes a means for receiving initial information on a travel plan from a user, a means for generating an optimum travel plan on the basis of the initial information, a means for executing a reservation procedure in cooperation with various reservation services on the basis of the travel plan, and a means for generating a promotion video on the basis of the travel plan.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] There are challenges such as the high cost of tours offered by travel agencies and the difficulty of creating a travel plan that perfectly suits your preferences. Furthermore, arranging everything yourself requires a lot of effort, so many people find planning a trip a hassle. In this situation, there is a demand for tools that allow users to create optimal travel plans smoothly and enjoyably. [Means for solving the problem]

[0005] To solve the above-mentioned problems, the present invention provides the following means. First, a means is provided for receiving initial information about travel plans from a user. Then, a system is provided that includes a means for generating an optimal travel plan based on this initial information. Then, a means is provided for linking with various reservation services based on this travel plan and executing reservation procedures for airline tickets, accommodations, restaurants, and activities. Furthermore, by introducing a means for generating a promotional video based on the travel plan, it is possible to increase anticipation before the trip. In this way, the present invention realizes a system that enables users to plan their travels smoothly and enjoyably.

[0006] "User" means a person who uses the system to make travel plans.

[0007] "Travel Plan" means the schedule and contents of a trip, including details such as destination, dates, number of people, budget, activity and dining preferences.

[0008] "Initial information" refers to the basic information required for travel planning that the user provides to the system.

[0009] The "optimal travel plan" is a travel schedule and reservation details that are automatically generated based on the user's initial information.

[0010] "Means of generation" refers to the process of creating a travel plan based on the user's initial information using algorithms or AI within the system.

[0011] "Reservation Service" means an online platform for making reservations such as flights, accommodation, restaurants, and activities.

[0012] "Reservation Process" means the process of making reservations for flights, accommodations, restaurants, activities, etc. through the Reservation Service.

[0013] A "promotional video" is a video created based on a user's travel plan, intended to build anticipation before the trip.

[0014] "Multimodal AI" is an artificial intelligence technology that processes and integrates multiple data modalities such as text, images, and voice. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. It also generates promotional videos to build excitement before the trip.

[0037] 1. Initial settings and information entry

[0038] User Action

[0039] Users launch the travel planning app on their smartphone or PC and enter initial information about their trip plans in the chat box displayed on the interface screen, including the destination, dates, number of people, budget, desired activities, and dining style.

[0040] Specific examples

[0041] A user types, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[0042] 2. Generate a travel plan

[0043] Server Processing

[0044] The server analyzes the initial information received from the user to identify their needs. Then, using algorithms and AI within the system, it generates an optimal travel plan based on the user's requests. The travel plan includes transportation (airplane, bullet train, etc.), accommodation, restaurants, and activities.

[0045] Specific examples

[0046] The server generates an itinerary like this:

[0047] Dates: August 1st to August 4th

[0048] Accommodation: Japanese-style inn in Kyoto city

[0049] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[0050] Restaurant: Japanese restaurant

[0051] Transportation: Shinkansen round-trip ticket

[0052] 3. Making various reservations

[0053] Server Processing

[0054] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking flights, accommodations, restaurants, and activities all at once.

[0055] Specific examples

[0056] The server makes the following reservation through the "Reservation Service API":

[0057] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[0058] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[0059] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[0060] Transportation: Shinkansen round-trip ticket

[0061] 4. Generate promotional videos

[0062] Server Processing

[0063] The server generates a promotional video based on the travel plan. By using multimodal AI, the details of the travel plan are presented in video format and provided to the user.

[0064] Specific examples

[0065] The promotional video, generated by the server using multi-mode AI, includes the following highlights:

[0066] A morning visit scene at the beautiful Kinkakuji Temple

[0067] Enjoyable matcha experience

[0068] Relaxing time at a Japanese-style inn

[0069] Delicious meal scene at a Japanese restaurant

[0070] 5. User plan confirmation and travel preparation

[0071] User Action

[0072] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[0073] Specific examples

[0074] Users click on the provided link to watch the promotional video, which builds excitement before the trip. The user feels, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[0075] As described above, the system of the present invention allows users to smoothly create travel plans that suit their preferences and easily complete the reservation procedure. In addition, the system also provides promotional videos that build anticipation before the trip, making the entire travel planning process enjoyable.

[0076] The processing flow will be explained below.

[0077] Step 1:

[0078] A user launches a travel planning app on their smartphone or PC. The device displays the interface screen and a chat box.

[0079] Step 2:

[0080] The user enters initial information about their travel plans into the chat box, such as destination, dates, number of people, budget, desired activities and dining style, etc. The device then sends this information to the server.

[0081] Step 3:

[0082] The server analyzes the initial information received and identifies the user's needs. Specifically, it breaks down the input content into categories and prepares to retrieve the necessary information from the database.

[0083] Step 4:

[0084] The server uses algorithms and AI within the system to generate an optimal travel plan based on the user's requests, including details on transportation, accommodation, restaurants, and activities.

[0085] Step 5:

[0086] The server accesses various reservation services (airline tickets, hotels, restaurants, activities, etc.) to collect availability and price information. For example, it queries each reservation site through a "reservation service API."

[0087] Step 6:

[0088] Based on the information collected, the server selects the reservation options that best fit the user's requirements, and then finalizes the travel plan based on this selection.

[0089] Step 7:

[0090] Based on the details of the reservation options selected by the server, the reservation process is carried out in bulk through various reservation APIs, specifically for air tickets, accommodations, restaurants, and activities.

[0091] Step 8:

[0092] The server collects the reservation confirmation information and sends it to the terminal, which displays a reservation confirmation notice to the user.

[0093] Step 9:

[0094] The server collects the details of the itinerary and uses multimodal AI to generate a promotional video, which includes the trip's highlights and recommended spots.

[0095] Step 10:

[0096] The server saves the generated promotional video in cloud storage and provides the link to the user, who then receives the link to watch the video on the device.

[0097] Step 11:

[0098] Users can watch the promotional video from the provided link and get a sense of their trip, which helps them prepare and build anticipation before their trip.

[0099] This is the specific process flow of the travel planning system. By using this system, users can easily create a travel plan that suits their preferences and smoothly proceed with reservations and preparations.

[0100] Example 1

[0101] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0102] Traditional travel planning requires users to go through multiple booking sites, which requires a lot of time and effort. In addition, there is a lack of information to build anticipation before a trip, which can make pre-trip preparations cumbersome.

[0103] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0104] In this invention, the server includes means for receiving initial information about a travel plan from a user, means for generating an optimal travel plan based on the initial information, means for executing reservation procedures in cooperation with various reservation services based on the travel plan, and means for providing the promotional video to the user to heighten pre-travel excitement. This allows users to efficiently carry out everything from planning a travel plan to making reservations and gathering pre-travel information using a single system.

[0105] "User" refers to an individual or organization making travel plans.

[0106] "Travel Itinerary" includes information and plans for traveling to a particular location.

[0107] "Initial Information" refers to basic information provided by the User, such as the travel destination, itinerary, number of people, budget, and desired activities and dining style.

[0108] "Travel Plan" refers to a plan generated based on the User's initial information, including details of transportation, accommodation, dining options, activities, etc.

[0109] "Reservation Process" refers to the process of making reservations for transportation, accommodation, dining, activities, etc.

[0110] A "promotional video" refers to a video generated based on a travel plan that outlines the attractions of a travel destination and activities.

[0111] "Multimodal AI" refers to artificial intelligence technology that integrates multiple data formats, such as text, images, and audio, and uses them to analyze and generate information.

[0112] "Server" refers to a computer system that receives and processes information based on user operations.

[0113] MODE FOR CARRYING OUT THE INVENTION

[0114] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. It also generates promotional videos to build excitement before the trip.

[0115] Initial setup and information entry

[0116] A user launches a travel planning app on a smartphone or PC. They enter initial information about their travel plans into a chat box displayed on the app's interface screen. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. This information is sent to the server using a data structure such as JSON.

[0117] Specific examples

[0118] A user types, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[0119] Generate a travel plan

[0120] The server uses a text analysis engine to analyze the initial information it receives. As a result of this analysis, it identifies the user's needs and generates an optimal travel plan using the system's algorithms and generative AI model. The generative AI model learns from past travel data and the user's preferences and proposes the optimal plan.

[0121] The travel plan includes transportation (airplane tickets, bullet trains, etc.), accommodations, restaurants, and activities. The generated plan is provided to the user through an interface for confirmation.

[0122] Specific examples

[0123] The server generates an itinerary like this:

[0124] Dates: August 1st to August 4th

[0125] Accommodation: Japanese-style inn in Kyoto city

[0126] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[0127] Restaurant: Japanese restaurant

[0128] Transportation: Shinkansen round-trip ticket

[0129] Making various reservations

[0130] The server then coordinates with various reservation services to complete the reservation process based on the generated travel plan. Using the reservation API, reservations for flights, accommodations, restaurants, and activities are made all at once. Once the reservation is complete, the information is sent to the user.

[0131] Specific examples

[0132] The server makes the following reservation through the "Reservation Service API":

[0133] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[0134] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[0135] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[0136] Transportation: Shinkansen round-trip ticket

[0137] Promotional video generation

[0138] The server uses multimodal AI to generate a promotional video based on the travel plan. The multimodal AI integrates multiple data formats, such as text, images, and audio, analyzes the information, and presents the details of the travel plan in video format. The generated promotional video is provided to the user.

[0139] Specific examples

[0140] The promotional video generated by Sarva using multimodal AI includes the following highlights:

[0141] A morning visit scene at the beautiful Kinkakuji Temple

[0142] Enjoyable matcha experience

[0143] Relaxing time at a Japanese-style inn

[0144] Delicious meal scene at a Japanese restaurant

[0145] Check user plans and prepare for trip

[0146] The user can review the provided travel plan and reservation confirmation information, and click on the provided link to watch a promotional video to further enhance their travel plans. At this stage, they can begin making any necessary preparations.

[0147] Specific examples

[0148] Users click on the provided link to watch the promotional video, which builds excitement before the trip. The user feels, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[0149] The system allows users to smoothly create a travel plan that suits their preferences, simplify the booking process, and even provides promotional videos to build excitement before the trip, making the whole travel planning process fun.

[0150] Prompt Sentence Examples

[0151] "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is under 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[0152] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0153] Step 1:

[0154] A user launches a travel planning app on their smartphone or PC. They enter initial information about their travel plans into a chat box displayed on the app's interface screen. Specifically, they enter the travel destination, itinerary, number of people, budget, desired activities, and dining style. This input information is converted into JSON-formatted data by the application and sent to the server.

[0155] Input: Initial information about the travel plan entered by the user

[0156] Output: Initial information sent to the server in JSON format.

[0157] Specific actions: The user enters, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[0158] Step 2:

[0159] The server launches a text analysis engine to analyze the initial information received. This engine uses natural language processing technology to extract information such as destination, itinerary, number of people, budget, activities, and dining style, and identifies each element. Based on this analysis, the system uses its algorithms and generative AI models to generate an optimal travel plan.

[0160] Input: Initial information in JSON format

[0161] Output: Analyzed travel plan elements and a travel plan based on them

[0162] Specific operation: The server extracts information such as "Kyoto," "August 1st to August 4th," "2 people," "200,000 yen," "matcha experience," "temple tour," and "Japanese cuisine" to create the optimal plan.

[0163] Step 3:

[0164] The server connects with various reservation services based on the generated travel plan. Specifically, it uses a reservation API to make reservations for flights, accommodations, restaurants, and activities all at once. Once the reservation is complete, the server notifies the user of the reservation confirmation information.

[0165] Input: Travel Plan

[0166] Output: Reservation confirmation information

[0167] Specific operation: The server executes the following reservation through the "Reservation Service API":

[0168] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[0169] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[0170] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[0171] Transportation: Shinkansen round-trip ticket

[0172] Step 4:

[0173] The server generates a promotional video based on the travel plan, activates multimodal AI, and integrates multiple data formats such as text, images, and audio to present the details of the travel plan in video format, and provides the generated promotional video to the user.

[0174] Input: Travel plan details

[0175] Output: Promotional video

[0176] What happens: The server generates a promotional video that includes the following:

[0177] A morning visit scene at the beautiful Kinkakuji Temple

[0178] Enjoyable matcha experience

[0179] Relaxing time at a Japanese-style inn

[0180] Delicious meal scene at a Japanese restaurant

[0181] Step 5:

[0182] The user confirms the provided travel plan and reservation confirmation information, and then clicks on the provided link to watch a promotional video to further enhance their travel plans and make the necessary preparations.

[0183] Input: Promotional video link and booking confirmation information

[0184] Output: Specific actions that users can take to prepare and expectations before the trip

[0185] Specific behavior: The user clicks on the provided link, watches the promotional video, and feels, "After watching the video, I'm really excited. I'll start preparing for my trip."

[0186] (Application example 1)

[0187] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0188] Conventional travel planning and reservation systems require users to individually visit various reservation sites to make reservations, which takes a lot of time and effort. Furthermore, the system lacks sufficient promotional content to build excitement during the travel planning process, resulting in low user satisfaction. Furthermore, local guide information and gourmet spot information are scattered across multiple platforms, making it difficult for users to efficiently obtain the information they need.

[0189] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0190] In this invention, the server includes means for receiving initial information about a travel plan from a user, means for generating an optimal travel plan based on the initial information, means for executing a reservation procedure in cooperation with various reservation services based on the travel plan, means for generating and distributing promotional content based on the travel plan, and means for including guide content for each travel destination and content introducing gourmet spots in the promotional content. This allows users to easily and efficiently plan a travel plan and complete the reservation procedure, and further enables them to obtain promotional content and local information that will increase their excitement for the trip.

[0191] "User" means any person or entity that uses the System to make travel plans.

[0192] A "trip plan" is the overall schedule and content of a trip generated based on the destination, dates, number of people, budget, and desired activities and dining style.

[0193] "Initial information" refers to information such as destination, dates, number of people, budget, desired activities and dining style that a user inputs into the system when making travel plans.

[0194] A "travel plan" is a detailed travel schedule including transportation, accommodation, restaurants, activities, etc., generated based on initial information.

[0195] A "reservation service" is a service that provides reservations for transportation, accommodation, restaurants, activities, etc. via the Internet.

[0196] "Promotional content" refers to content that includes visual and audio information created to attractively introduce the contents of a travel plan.

[0197] "Guide content" refers to explanatory and guidance information about tourist attractions, culture, and activities related to a travel destination.

[0198] "Gourmet spots" refers to information about restaurants and types of cuisine that are accessible at a travel destination.

[0199] "Generative AI model" means an artificial intelligence model used to automatically generate promotional content based on initial information and travel plans.

[0200] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations in one go. It also provides promotional content to build excitement before a trip.

[0201] 1. Initial settings and information entry

[0202] User operations

[0203] Users launch the travel planning app on their smartphone or PC and enter initial information about their trip plans in the chat box displayed on the interface screen, including the destination, dates, number of people, budget, desired activities, and dining style.

[0204] Specific examples

[0205] For example, a user might enter, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[0206] 2. Generate a travel plan

[0207] Server Processing

[0208] The server analyzes the initial information received from the user to identify their needs, then uses algorithms and generative AI models within the system to generate an optimal travel plan based on the user's requests, including transportation, accommodation, restaurants, and activities.

[0209] Specific examples

[0210] The server generates an itinerary as follows:

[0211] Dates: August 1st to August 4th

[0212] Accommodation: Japanese-style inn in Kyoto city

[0213] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[0214] Restaurant: Japanese restaurant

[0215] Transportation: Shinkansen round-trip ticket

[0216] 3. Making various reservations

[0217] Server Processing

[0218] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking transportation, accommodation, restaurants, and activities all at once.

[0219] Specific examples

[0220] The server makes the following reservation through the "Reservation Service API":

[0221] Accommodation: Japanese-style inn in Kyoto city (August 1st to August 4th)

[0222] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[0223] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[0224] Transportation: Shinkansen round-trip ticket

[0225] 4. Generate promotional content

[0226] Server Processing

[0227] The server generates and delivers promotional content based on the travel plan. This content presents the travel plan details to the user using visual and audio information. Using a generative AI model, engaging and personalized multimedia content is generated.

[0228] Specific examples

[0229] For example, a server-generated promotional video includes the following highlights:

[0230] A morning visit scene at the beautiful Kinkakuji Temple

[0231] Enjoyable matcha experience

[0232] Relaxing time at a Japanese-style inn

[0233] Delicious meal scene at a Japanese restaurant

[0234] 5. User plan confirmation and travel preparation

[0235] User operations

[0236] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[0237] Specific examples

[0238] Users click on the provided link to watch the promotional video, which builds excitement before the trip. They then feel, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[0239] Hardware and software used

[0240] The following hardware and software are used to realize this system.

[0241] Hardware: Smartphones, PCs, servers

[0242] Software: Python, REST API, generative AI models

[0243] Prompt Sentence Examples

[0244] Destination: Kyoto

[0245] Date: "2023-08-01" to "2023-08-04"

[0246] Number of people: 2 people

[0247] Budget: 200,000 yen

[0248] Activities: Matcha experience, temple tours, Japanese cuisine

[0249] As described above, the system of the present invention allows users to smoothly create travel plans that suit their preferences and easily complete the reservation procedure. Furthermore, it also provides promotional content that builds anticipation before the trip, making the entire travel planning process enjoyable.

[0250] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0251] Step 1:

[0252] A user launches a travel planning app on their smartphone or PC and enters initial information about their trip plan in a chat box displayed on the interface screen, including the destination, dates, number of people, budget, desired activities, and dining style.

[0253] Input: Initial information entered by the user (e.g., destination, dates, number of people, budget, activities)

[0254] Output: Initial information sent to the server

[0255] Specific actions: The user types in the chat box, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is under 200,000 yen, and we would like to experience matcha tea, visit temples, and enjoy Japanese cuisine."

[0256] Step 2:

[0257] The server analyzes the initial information received to identify the user's needs, then uses the system's algorithms and generative AI models to generate an optimal travel plan based on the user's requests.

[0258] Input: Initial information (destination, dates, number of people, budget, activities)

[0259] Output: Optimal travel plan (transportation, accommodation, restaurants, activities)

[0260] Specific operation: The server analyzes "Destination: Kyoto, Dates: August 1st to August 4th, Number of people: 2, Budget: 200,000 yen, Activities: Matcha experience, Temple tour, Japanese cuisine" and generates the optimal travel plan.

[0261] Step 3:

[0262] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, including reservations for transportation, accommodation, restaurants, and activities.

[0263] Input: Generated itinerary

[0264] Output: Confirmed reservation information

[0265] Specific operation: The server makes a reservation through the "Reservation Service API" for "a Japanese-style inn in Kyoto City (August 1st to August 4th), a matcha tea experience (August 2nd from 14:00), a visit to Kinkakuji Temple (August 3rd from 10:00), and a round-trip Shinkansen ticket."

[0266] Step 4:

[0267] The server generates and distributes promotional content based on the travel plan, which presents the details of the travel plan to the user using visual and audio information.

[0268] Input: Generated itinerary

[0269] Output: Promotional content (videos, guide content, gourmet spot introductions)

[0270] Specific operation: The server uses the generative AI model to generate a promotional video including scenes of the beautiful Kinkaku-ji Temple, a matcha tea experience, relaxing at a Japanese-style inn, and dining at a Japanese restaurant, and notifies the user.

[0271] Step 5:

[0272] The user checks the provided travel plan and reservation confirmation information, views the promotional content, and makes the necessary travel preparations based on the content they have viewed.

[0273] Input: Travel itinerary, booking confirmation information, promotional content

[0274] Output: User's travel preparations

[0275] What happens: The user clicks on the provided link, watches the promotional video, and feels, "The video made me really excited. I'm going to start preparing for my trip."

[0276] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0277] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it becomes possible to propose more personalized travel plans. The system also generates promotional videos to build anticipation before the trip.

[0278] 1. Initial settings and information entry

[0279] User Action

[0280] Users launch the travel planning app on their smartphone or PC and enter initial information about their travel plans in the chat box displayed on the interface screen. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. The emotion engine also recognizes emotions from the user's facial expressions and voice, and acquires emotional information.

[0281] Specific examples

[0282] A user inputs, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine." The emotion engine analyzes the user's facial expressions and voice when they input information, and interprets the emotions they are looking forward to.

[0283] 2. Generate a travel plan

[0284] Server Processing

[0285] The server analyzes the initial information and emotional information received from the user to identify their needs. Then, using algorithms and AI within the system, it generates an optimal travel plan based on the user's requests and emotions. The travel plan includes transportation (airplane tickets, bullet trains, etc.), accommodation, restaurants, and activities.

[0286] Specific examples

[0287] The server generates an itinerary like this:

[0288] Dates: August 1st to August 4th

[0289] Accommodation: Japanese-style inn in Kyoto city

[0290] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[0291] Restaurant: Japanese restaurant

[0292] Transportation: Shinkansen round-trip ticket

[0293] Based on emotional information, activities that the user is particularly looking forward to are prioritized.

[0294] 3. Making various reservations

[0295] Server Processing

[0296] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking flights, accommodations, restaurants, and activities all at once.

[0297] Specific examples

[0298] The server makes the following reservation through the "Reservation Service API":

[0299] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[0300] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[0301] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[0302] Transportation: Shinkansen round-trip ticket

[0303] 4. Generate promotional videos

[0304] Server Processing

[0305] The server collects travel plan details and sentiment information and uses multimodal AI to generate a promotional video, highlighting activities and attractions that the user has expressed particular interest in, further building anticipation.

[0306] Specific examples

[0307] The promotional video generated by Sarva using multimodal AI includes the following highlights:

[0308] A morning visit scene at the beautiful Kinkakuji Temple

[0309] Enjoyable matcha experience

[0310] Relaxing time at a Japanese-style inn

[0311] Delicious meal scene at a Japanese restaurant

[0312] Based on emotional information, scenes of activities that the user is particularly looking forward to are included with priority.

[0313] 5. User plan confirmation and travel preparation

[0314] User Action

[0315] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[0316] Specific examples

[0317] Users click on the provided link to watch the promotional video, which builds excitement before the trip. The user feels, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[0318] As described above, the system of the present invention allows users to smoothly create travel plans that suit their preferences and easily complete the reservation procedure. Furthermore, by using the emotion engine, it is possible to provide more personalized plans and further increase anticipation before the trip.

[0319] The processing flow will be explained below.

[0320] Step 1:

[0321] A user launches a travel planning app on their smartphone or PC. The device displays the interface screen and a chat box.

[0322] Step 2:

[0323] The user enters initial information about their travel plans (destination, dates, number of people, budget, desired activities and dining style) into the chat box, and the device sends this information to the server.

[0324] Step 3:

[0325] The device activates the emotion engine and acquires emotion information by analyzing the user's facial expressions and voice. The emotion engine then transmits the acquired emotion information to the server.

[0326] Step 4:

[0327] The server analyzes the initial information and emotional information received to identify the user's needs. Specifically, it centrally manages input content and emotional information and estimates what the user is looking forward to.

[0328] Step 5:

[0329] The server uses the system's algorithms and AI to generate an optimal travel plan based on the user's requests and feelings. The plan includes transportation (airplane, bullet train, etc.), accommodation, restaurants, and activities.

[0330] Step 6:

[0331] The server accesses various reservation services (airline tickets, hotels, restaurants, activities, etc.) to collect availability and price information. For example, it queries each reservation site through a "reservation service API."

[0332] Step 7:

[0333] Based on the information collected, the server selects the reservation options that best fit the user's requirements, and then finalizes the travel plan based on this selection.

[0334] Step 8:

[0335] Based on the details of the reservation options selected by the server, the reservation process is carried out in bulk through various reservation APIs, specifically for air tickets, accommodations, restaurants, and activities.

[0336] Step 9:

[0337] The server collects the reservation confirmation information and sends it to the terminal, which displays a reservation confirmation notice to the user.

[0338] Step 10:

[0339] The server collects itinerary details and sentiment information and uses multimodal AI to generate a promotional video that highlights activities and attractions that the user has expressed particular interest in.

[0340] Step 11:

[0341] The server saves the generated promotional video in cloud storage and provides the link to the user, who then receives the link to watch the video on the device.

[0342] Step 12:

[0343] Users can watch the promotional video from the provided link and get a sense of their trip, which helps them prepare and build anticipation before their trip.

[0344] Example 2

[0345] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0346] Conventional travel planning systems make it difficult for users to create travel plans based on their own emotions and preferences. Furthermore, few systems offer a single solution for everything from travel planning to various reservations, and there are few ways to build anticipation before a trip. This results in users spending a lot of time and effort on planning and booking trips, preventing smooth travel preparations.

[0347] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0348] In this invention, the server includes means for receiving initial information about a travel plan from a user, means for acquiring emotional information about the user, means for generating an optimal travel plan based on the initial information and emotional information, means for executing a reservation procedure in cooperation with various reservation services based on the travel plan, and means for generating a promotional video based on the travel plan. This makes it possible to create a personalized travel plan that matches the user's preferences and emotions, realize a smooth reservation procedure, and increase anticipation before the trip.

[0349] "User" means an individual or group that uses the System to make travel plans.

[0350] "Initial information" refers to basic information about travel plans such as destination, dates, number of people, budget, and desired activities and dining options.

[0351] "Emotional information" refers to data related to emotions obtained from the user's facial expressions, voice, etc.

[0352] "Travel Itinerary" refers to a plan of a trip including details of transportation, accommodation, dining and activities generated based on the initial information and sentiment information.

[0353] "Reservation Service" means a service that handles reservations for transportation, accommodation, dining, and activities.

[0354] "Promotional video" refers to video content that is generated based on travel plans and is produced to attract the user's interest while taking into account the user's emotional information.

[0355] A "generative AI model" refers to an artificial intelligence model that can process data in a variety of formats and generate new information and content.

[0356] The following is a detailed description of an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it becomes possible to propose more personalized travel plans. The system also generates promotional videos to build anticipation before the trip.

[0357] Initial setup and information entry

[0358] User Action

[0359] Users launch the travel planning app on their smartphone or PC and enter initial information about their travel plans in the chat box displayed on the interface screen. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. The emotion engine also recognizes emotions from the user's facial expressions and voice, and acquires emotional information.

[0360] Specific examples

[0361] A user inputs, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine." The emotion engine analyzes the user's facial expressions and voice when they input information, and interprets the emotions they are looking forward to.

[0362] Generate a travel plan

[0363] Server Processing

[0364] The server analyzes the initial information and emotional information received from the user to identify their needs. Then, using generative AI models and other algorithms within the system, it generates an optimal travel plan based on the user's needs and emotions. The travel plan includes transportation (airplane, bullet train, etc.), accommodation, dining, and activities.

[0365] Specific examples

[0366] The server generates an itinerary like this:

[0367] Dates: August 1st to August 4th

[0368] Accommodation: Japanese-style inn in Kyoto city

[0369] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[0370] Restaurant: Japanese restaurant

[0371] Transportation: Shinkansen round-trip ticket

[0372] Based on emotional information, activities that the user is particularly looking forward to are prioritized.

[0373] Making various reservations

[0374] Server Processing

[0375] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking transportation, accommodation, dining, and activities all at once.

[0376] Specific examples

[0377] The server makes the following reservation through the Reservation Service API:

[0378] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[0379] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[0380] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[0381] Transportation: Shinkansen round-trip ticket

[0382] Promotional video generation

[0383] Server Processing

[0384] The server collects travel plan details and emotional information and uses multimodal AI to generate promotional videos, highlighting activities and tourist attractions that the user has shown particular interest in, further building anticipation.

[0385] Specific examples

[0386] The promotional video, generated by Sarva using multimodal AI, includes the following highlights:

[0387] A morning visit scene at the beautiful Kinkakuji Temple

[0388] Enjoyable matcha experience

[0389] Relaxing time at a Japanese-style inn

[0390] Delicious meal scene at a Japanese restaurant

[0391] Based on emotional information, scenes of activities that the user is particularly looking forward to are included with priority.

[0392] Check user plans and prepare for trip

[0393] User Action

[0394] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[0395] Specific examples

[0396] The user clicks on the provided link, watches the promotional video, and feels, "After watching the video, I'm really excited. I'll start preparing for my trip."

[0397] (Example of a prompt)

[0398] "I'd like to plan a trip for two people to Kyoto for four days and three nights, with a budget of 200,000 yen or less. I'd like to experience matcha tea, visit temples, and enjoy Japanese cuisine."

[0399] As described above, the system utilizes an emotion engine and generative AI models to provide users with highly personalized travel plans and a smooth booking process, allowing them to prepare for their trip efficiently and enjoyably.

[0400] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0401] The specific flow of the program processing of this system

[0402] Step 1:

[0403] Entering initial user information

[0404] A user launches a travel planning app on a smartphone or PC. They enter initial information about their travel plans into a chat box displayed on the interface. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. Based on the input data, the emotion engine extracts emotional information from the user's facial expressions and voice.

[0405] input

[0406] Destination, dates, number of people, budget, desired activities and dining style

[0407] User's facial expressions and voice

[0408] output

[0409] Initial Information Data

[0410] Emotional information data

[0411] Specific actions

[0412] A user inputs, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha green tea experience, visiting temples, and Japanese cuisine." The system saves the input data, and at the same time, the emotion engine analyzes the user's facial expressions and voice to obtain emotional information.

[0413] Step 2:

[0414] Analysis of initial information and emotional information

[0415] The server analyzes the initial information and emotional information received from the user. The system identifies the user's needs and performs appropriate data processing and calculations accordingly.

[0416] input

[0417] Initial Information Data

[0418] Emotional information data

[0419] output

[0420] User needs data

[0421] Specific actions

[0422] The server analyzes the initial information and the emotional information to identify the user's desired activities, budget, and emotional characteristics. The analyzed information is used in the next step.

[0423] Step 3:

[0424] Generate optimal travel plans

[0425] The server uses generative AI models and other algorithms within the system to generate optimal travel plans based on the user's requests and emotions.

[0426] input

[0427] User needs data

[0428] output

[0429] Generated travel plan data

[0430] Specific actions

[0431] The server uses the generative AI model to generate a travel plan like this:

[0432] Dates: August 1st to August 4th

[0433] Accommodation: Japanese-style inn in Kyoto city

[0434] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[0435] Restaurant: Japanese restaurant

[0436] Transportation: Shinkansen round-trip ticket

[0437] Based on emotional information, activities that the user is particularly looking forward to are prioritized in the plan.

[0438] Step 4:

[0439] Making various reservations

[0440] Based on the generated travel plan, the server connects with various reservation services and executes the reservation process, specifically booking transportation, accommodation, dining, and activities all at once.

[0441] input

[0442] Generated travel plan data

[0443] output

[0444] Various reservation confirmation data

[0445] Specific actions

[0446] The server connects to the "Reservation Service API" and performs the following reservation:

[0447] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[0448] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[0449] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[0450] Transportation: Shinkansen round-trip ticket

[0451] Step 5:

[0452] Promotional video generation

[0453] The server collects itinerary details and sentiment information and uses multimodal AI to generate promotional videos that highlight activities and attractions that the user has expressed particular interest in.

[0454] input

[0455] Travel Plan Data

[0456] Emotional information data

[0457] output

[0458] Generated promotional video

[0459] Specific actions

[0460] The server selects footage and audio based on the details of the itinerary and runs a video editing algorithm. The multimodal AI-generated footage includes highlights of:

[0461] A morning visit scene at the beautiful Kinkakuji Temple

[0462] Enjoyable matcha experience

[0463] Relaxing time at a Japanese-style inn

[0464] Delicious Japanese food scene

[0465] Based on emotional information, scenes of activities that the user is particularly looking forward to are included with priority.

[0466] Step 6:

[0467] Check user plans and prepare for trip

[0468] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[0469] input

[0470] Generated itinerary and booking confirmation data

[0471] Generated promotional video

[0472] output

[0473] User Readiness

[0474] Specific actions

[0475] The user clicks on the provided link, watches the promotional video, and feels, "After watching the video, I'm really excited. I'll start preparing for my trip." Based on this, the user makes specific preparations for the trip.

[0476] Through these steps, the system supports users through a series of steps, from entering initial information to creating a travel plan, various reservations, providing promotional videos, and final confirmation and preparation, providing a personalized travel experience.

[0477] (Application example 2)

[0478] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0479] In recent years, as demand for food delivery has increased, there is a demand for personalized meal suggestions tailored to user needs and more efficient ordering procedures. It is also important to provide an experience that allows users to enjoy ordering, rather than simply ordering a meal. Current food delivery systems do not consider users' emotional information when making ordering suggestions, nor do they foster anticipation through promotional media, making it difficult to improve the user experience.

[0480] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving initial information regarding a meal order from a user, means for generating an optimal food plan based on the initial information, means for executing a reservation procedure in cooperation with various reservation services based on the food plan, means for generating promotional media based on the food plan, means for acquiring emotional information of the user using an emotion engine and proposing a personalized plan, and means for providing the promotional media generated based on the emotional information. This makes it possible to propose personalized food delivery plans that take the user's emotions into consideration, enabling the user to have an enjoyable and improved ordering experience.

[0481] "User" refers to the end user who uses the system.

[0482] "Initial information" refers to basic information provided by the user, such as the destination, itinerary, number of people, budget, desired activities, and dining style.

[0483] A "travel plan" refers to a travel schedule or itinerary that is generated based on the above initial information and emotional information and is optimized to meet the user's needs.

[0484] "Reservation process" refers to the process of making reservations for accommodation, transportation, restaurants, activities, etc. in one go, in collaboration with various reservation services based on your travel plans.

[0485] "Promotional media" refers to video or video content created with the purpose of increasing user interest or emotion.

[0486] An "emotion engine" refers to an algorithm or program that recognizes emotions from a user's facial expressions and voice and acquires emotional information.

[0487] A "personalized plan" refers to an itinerary or proposal that is individually optimized to reflect the user's emotional information.

[0488] "Multimodal artificial intelligence technology" refers to artificial intelligence technology that integrates and processes multiple types of data (text, images, audio, etc.).

[0489] A "server" refers to a computer system that processes and stores data on the cloud and performs various system functions.

[0490] The following describes an embodiment of the present invention. The system generates a personalized food plan when a user places a meal order, enabling efficient ordering. Furthermore, by combining an emotion engine, the system generates suggestions and promotional media based on the user's emotions, improving the ordering experience.

[0491] 1. Input of initial information and emotional information

[0492] User Action

[0493] Users launch the food delivery app on their smartphone and enter initial information about their meal in the chat box displayed on the interface screen. This information includes the desired food category, budget, and number of people. The emotion engine then recognizes the user's emotions from their facial expressions and voice, and acquires their emotional information.

[0494] Specific examples

[0495] A user inputs, "I want to eat pizza and pasta for three people within a budget of 5,000 yen." The emotion engine analyzes the user's facial expressions and voice when they input, and interprets the user's emotions.

[0496] 2. Generate a food plan

[0497] Server Processing

[0498] The server analyzes the initial information and emotional information received from the user to identify their needs. It then uses a generative AI model (e.g., GPT-4) within the system to generate an optimal meal plan based on the user's requests and emotions. The meal plan includes suggested restaurants and specific menu items, with priority given to suggestions that enhance the user's emotions.

[0499] Specific examples

[0500] The server generates a food plan like this:

[0501] Restaurant: Popular Italian restaurant

[0502] Menu: Margherita pizza, carbonara pasta

[0503] 3. Making reservations and ordering

[0504] Server Processing

[0505] Based on the generated food plan, the server connects with various services to carry out the ordering process, including sending the order details to the restaurant and handling the payment process all at once.

[0506] Specific examples

[0507] The server executes the following procedure through the "Order Service API":

[0508] Send order to restaurant: Margherita pizza (3 slices), Carbonara pasta (3 servings)

[0509] Payment process

[0510] 4. Promotional Media Creation

[0511] Server Processing

[0512] The server collects details of the food plan and emotional information, and generates promotional media using multimodal artificial intelligence technology (e.g., DALL-E). This media introduces the menu and restaurant atmosphere that the user is particularly interested in, building anticipation.

[0513] Specific examples

[0514] Promotional media generated by the server using multimodal artificial intelligence technology includes the following highlights:

[0515] Beautiful pizza being baked

[0516] Delicious looking pasta being served

[0517] 5. User plan confirmation and order confirmation

[0518] User Action

[0519] The user checks the provided food plan and reservation confirmation information, and views the promotional media to confirm and confirm the order details.

[0520] Specific examples

[0521] The user clicks on the provided link, watches the promotional media, and decides to "confirm my order from this menu."

[0522] As described above, the system of the present invention allows users to smoothly create a meal plan that suits their preferences and easily complete the ordering process. Furthermore, by utilizing the emotion engine, it is possible to provide more personalized suggestions and further increase anticipation before ordering.

[0523] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0524] Step 1:

[0525] The user launches the food delivery app on their smartphone and enters initial information about the meal in the chat box displayed on the interface screen. This information includes the desired food category, budget, and number of people. The emotion engine recognizes emotions from the user's facial expressions and voice and acquires emotional information. The entered information is sent to the server.

[0526] input:

[0527] Desired food category

[0528] budget

[0529] Number of people

[0530] Facial expression data

[0531] Audio data

[0532] output:

[0533] Initial information

[0534] emotional information

[0535] Specific behavior:

[0536] The user enters, "I want to eat pizza and pasta for three people within a budget of 5,000 yen."

[0537] The emotion engine recognizes facial expressions and voices through the smartphone's camera and microphone.

[0538] The server receives the initial information and emotion information.

[0539] Step 2:

[0540] The server analyzes the initial information and emotional information received from the user. Then, using a generative AI model (e.g., GPT-4) within the system, it generates an optimal meal plan based on the user's requests and emotions. Based on the analyzed data, it suggests the most suitable restaurant and specific menu.

[0541] input:

[0542] Initial information

[0543] emotional information

[0544] output:

[0545] Food Plan

[0546] Specific behavior:

[0547] The server analyzes the initial information and emotional information to identify the user's needs.

[0548] A generative AI model was used to generate a food plan that included Margherita pizza and carbonara pasta.

[0549] Step 3:

[0550] Based on the generated food plan, the server connects with various services to carry out the ordering process, including sending the order details to the restaurant and handling the payment process all at once.

[0551] input:

[0552] Food Plan

[0553] output:

[0554] Order Confirmation Information

[0555] Payment completion information

[0556] Specific behavior:

[0557] The server sends the order details to the restaurant via the "Order Service API."

[0558] Process payments and generate order confirmation and payment completion information.

[0559] Step 4:

[0560] The server collects details of the food plan and emotional information, and generates promotional media using multimodal artificial intelligence technology (e.g., DALL-E). This media introduces the menu and restaurant atmosphere that the user is particularly interested in, building anticipation.

[0561] input:

[0562] Food Plan

[0563] emotional information

[0564] output:

[0565] Promotional Media

[0566] Specific behavior:

[0567] The server uses multimodal artificial intelligence technology to generate compelling images of pizza and pasta.

[0568] Create links to provide promotional media to users.

[0569] Step 5:

[0570] The user checks the provided food plan and reservation confirmation information, watches the promotional media, and confirms and confirms the order details. If satisfied, the user finally confirms the order.

[0571] input:

[0572] Promotional Media

[0573] Order Confirmation Information

[0574] output:

[0575] Final order confirmation information

[0576] Specific behavior:

[0577] User clicks on the provided link to view the promotional media.

[0578] After watching, confirm your order by clicking "Confirm your order from this menu."

[0579] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0580] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0581] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0582] [Second embodiment]

[0583] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0584] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0585] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0586] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0587] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0588] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0589] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0590] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0591] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0592] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0593] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0594] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0595] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. It also generates promotional videos to build excitement before the trip.

[0596] 1. Initial settings and information entry

[0597] User Action

[0598] Users launch the travel planning app on their smartphone or PC and enter initial information about their trip plans in the chat box displayed on the interface screen, including the destination, dates, number of people, budget, desired activities, and dining style.

[0599] Specific examples

[0600] A user types, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[0601] 2. Generate a travel plan

[0602] Server Processing

[0603] The server analyzes the initial information received from the user to identify their needs. Then, using algorithms and AI within the system, it generates an optimal travel plan based on the user's requests. The travel plan includes transportation (airplane, bullet train, etc.), accommodation, restaurants, and activities.

[0604] Specific examples

[0605] The server generates an itinerary like this:

[0606] Dates: August 1st to August 4th

[0607] Accommodation: Japanese-style inn in Kyoto city

[0608] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[0609] Restaurant: Japanese restaurant

[0610] Transportation: Shinkansen round-trip ticket

[0611] 3. Making various reservations

[0612] Server Processing

[0613] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking flights, accommodations, restaurants, and activities all at once.

[0614] Specific examples

[0615] The server makes the following reservation through the "Reservation Service API":

[0616] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[0617] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[0618] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[0619] Transportation: Shinkansen round-trip ticket

[0620] 4. Generate promotional videos

[0621] Server Processing

[0622] The server generates a promotional video based on the travel plan. By using multimodal AI, the details of the travel plan are presented in video format and provided to the user.

[0623] Specific examples

[0624] The promotional video, generated by the server using multi-mode AI, includes the following highlights:

[0625] A morning visit scene at the beautiful Kinkakuji Temple

[0626] Enjoyable matcha experience

[0627] Relaxing time at a Japanese-style inn

[0628] Delicious meal scene at a Japanese restaurant

[0629] 5. User plan confirmation and travel preparation

[0630] User Action

[0631] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[0632] Specific examples

[0633] Users click on the provided link to watch the promotional video, which builds excitement before the trip. The user feels, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[0634] As described above, the system of the present invention allows users to smoothly create travel plans that suit their preferences and easily complete the reservation procedure. In addition, the system also provides promotional videos that build anticipation before the trip, making the entire travel planning process enjoyable.

[0635] The processing flow will be explained below.

[0636] Step 1:

[0637] A user launches a travel planning app on their smartphone or PC. The device displays the interface screen and a chat box.

[0638] Step 2:

[0639] The user enters initial information about their travel plans into the chat box, such as destination, dates, number of people, budget, desired activities and dining style, etc. The device then sends this information to the server.

[0640] Step 3:

[0641] The server analyzes the initial information received and identifies the user's needs. Specifically, it breaks down the input content into categories and prepares to retrieve the necessary information from the database.

[0642] Step 4:

[0643] The server uses algorithms and AI within the system to generate an optimal travel plan based on the user's requests, including details on transportation, accommodation, restaurants, and activities.

[0644] Step 5:

[0645] The server accesses various reservation services (airline tickets, hotels, restaurants, activities, etc.) to collect availability and price information. For example, it queries each reservation site through a "reservation service API."

[0646] Step 6:

[0647] Based on the information collected, the server selects the reservation options that best fit the user's requirements, and then finalizes the travel plan based on this selection.

[0648] Step 7:

[0649] Based on the details of the reservation options selected by the server, the reservation process is carried out in bulk through various reservation APIs, specifically for air tickets, accommodations, restaurants, and activities.

[0650] Step 8:

[0651] The server collects the reservation confirmation information and sends it to the terminal, which displays a reservation confirmation notice to the user.

[0652] Step 9:

[0653] The server collects the details of the itinerary and uses multimodal AI to generate a promotional video, which includes the trip's highlights and recommended spots.

[0654] Step 10:

[0655] The server saves the generated promotional video in cloud storage and provides the link to the user, who then receives the link to watch the video on the device.

[0656] Step 11:

[0657] Users can watch the promotional video from the provided link and get a sense of their trip, which helps them prepare for and build anticipation before their trip.

[0658] This is the specific process flow of the travel planning system. By using this system, users can easily create a travel plan that suits their preferences and smoothly proceed with reservations and preparations.

[0659] Example 1

[0660] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0661] Traditional travel planning requires users to go through multiple booking sites, which requires a lot of time and effort. In addition, there is a lack of information to build anticipation before a trip, which can make pre-trip preparations cumbersome.

[0662] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0663] In this invention, the server includes means for receiving initial information about a travel plan from a user, means for generating an optimal travel plan based on the initial information, means for executing reservation procedures in cooperation with various reservation services based on the travel plan, and means for providing the promotional video to the user to heighten pre-travel excitement. This allows users to efficiently carry out everything from planning a travel plan to making reservations and gathering pre-travel information using a single system.

[0664] "User" refers to an individual or organization making travel plans.

[0665] "Travel Itinerary" includes information and plans for traveling to a particular location.

[0666] "Initial Information" refers to basic information provided by the User, such as the travel destination, itinerary, number of people, budget, and desired activities and dining style.

[0667] "Travel Plan" refers to a plan generated based on the User's initial information, including details of transportation, accommodation, dining options, activities, etc.

[0668] "Reservation Process" refers to the process of making reservations for transportation, accommodation, dining, activities, etc.

[0669] A "promotional video" refers to a video generated based on a travel plan that outlines the attractions of a travel destination and activities.

[0670] "Multimodal AI" refers to artificial intelligence technology that integrates multiple data formats, such as text, images, and audio, and uses them to analyze and generate information.

[0671] "Server" refers to a computer system that receives and processes information based on user operations.

[0672] MODE FOR CARRYING OUT THE INVENTION

[0673] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. It also generates promotional videos to build excitement before the trip.

[0674] Initial setup and information entry

[0675] A user launches a travel planning app on a smartphone or PC. They enter initial information about their travel plans into a chat box displayed on the app's interface screen. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. This information is sent to the server using a data structure such as JSON.

[0676] Specific examples

[0677] A user types, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[0678] Generate a travel plan

[0679] The server uses a text analysis engine to analyze the initial information it receives. As a result of this analysis, it identifies the user's needs and generates an optimal travel plan using the system's algorithms and generative AI model. The generative AI model learns from past travel data and the user's preferences and proposes the optimal plan.

[0680] The travel plan includes transportation (airplane tickets, bullet trains, etc.), accommodations, restaurants, and activities. The generated plan is provided to the user through an interface for confirmation.

[0681] Specific examples

[0682] The server generates an itinerary like this:

[0683] Dates: August 1st to August 4th

[0684] Accommodation: Japanese-style inn in Kyoto city

[0685] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[0686] Restaurant: Japanese restaurant

[0687] Transportation: Shinkansen round-trip ticket

[0688] Making various reservations

[0689] The server then coordinates with various reservation services to complete the reservation process based on the generated travel plan. Using the reservation API, reservations for flights, accommodations, restaurants, and activities are made all at once. Once the reservation is complete, the information is sent to the user.

[0690] Specific examples

[0691] The server makes the following reservation through the "Reservation Service API":

[0692] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[0693] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[0694] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[0695] Transportation: Shinkansen round-trip ticket

[0696] Promotional video generation

[0697] The server uses multimodal AI to generate a promotional video based on the travel plan. The multimodal AI integrates multiple data formats, such as text, images, and audio, analyzes the information, and presents the details of the travel plan in video format. The generated promotional video is provided to the user.

[0698] Specific examples

[0699] The promotional video generated by Sarva using multimodal AI includes the following highlights:

[0700] A morning visit scene at the beautiful Kinkakuji Temple

[0701] Enjoyable matcha experience

[0702] Relaxing time at a Japanese-style inn

[0703] Delicious meal scene at a Japanese restaurant

[0704] Check user plans and prepare for trip

[0705] The user can review the provided travel plan and reservation confirmation information, and click on the provided link to watch a promotional video to further enhance their travel plans. At this stage, they can begin making any necessary preparations.

[0706] Specific examples

[0707] Users click on the provided link to watch the promotional video, which builds excitement before the trip. The user feels, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[0708] The system allows users to smoothly create a travel plan that suits their preferences, simplify the booking process, and even provides promotional videos to build excitement before the trip, making the whole travel planning process fun.

[0709] Prompt Sentence Examples

[0710] "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is under 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[0711] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0712] Step 1:

[0713] A user launches a travel planning app on their smartphone or PC. They enter initial information about their travel plans into a chat box displayed on the app's interface screen. Specifically, they enter the travel destination, itinerary, number of people, budget, desired activities, and dining style. This input information is converted into JSON-formatted data by the application and sent to the server.

[0714] Input: Initial information about the travel plan entered by the user

[0715] Output: Initial information sent to the server in JSON format.

[0716] Specific actions: The user enters, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[0717] Step 2:

[0718] The server launches a text analysis engine to analyze the initial information received. This engine uses natural language processing technology to extract information such as destination, itinerary, number of people, budget, activities, and dining style, and identifies each element. Based on this analysis, the system uses its algorithms and generative AI models to generate an optimal travel plan.

[0719] Input: Initial information in JSON format

[0720] Output: Analyzed travel plan elements and a travel plan based on them

[0721] Specific operation: The server extracts information such as "Kyoto," "August 1st to August 4th," "2 people," "200,000 yen," "matcha experience," "temple tour," and "Japanese cuisine" to create the optimal plan.

[0722] Step 3:

[0723] The server connects with various reservation services based on the generated travel plan. Specifically, it uses a reservation API to make reservations for flights, accommodations, restaurants, and activities all at once. Once the reservation is complete, the server notifies the user of the reservation confirmation information.

[0724] Input: Travel Plan

[0725] Output: Reservation confirmation information

[0726] Specific operation: The server executes the following reservation through the "Reservation Service API":

[0727] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[0728] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[0729] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[0730] Transportation: Shinkansen round-trip ticket

[0731] Step 4:

[0732] The server generates a promotional video based on the travel plan, activates multimodal AI, and integrates multiple data formats such as text, images, and audio to present the details of the travel plan in video format, and provides the generated promotional video to the user.

[0733] Input: Travel plan details

[0734] Output: Promotional video

[0735] What happens: The server generates a promotional video that includes the following:

[0736] A morning visit scene at the beautiful Kinkakuji Temple

[0737] Enjoyable matcha experience

[0738] Relaxing time at a Japanese-style inn

[0739] Delicious meal scene at a Japanese restaurant

[0740] Step 5:

[0741] The user confirms the provided travel plan and reservation confirmation information, and then clicks on the provided link to watch a promotional video to further enhance their travel plans and make the necessary preparations.

[0742] Input: Promotional video link and booking confirmation information

[0743] Output: Specific actions that users can take to prepare and expectations before the trip

[0744] Specific behavior: The user clicks on the provided link, watches the promotional video, and feels, "After watching the video, I'm really excited. I'll start preparing for my trip."

[0745] (Application example 1)

[0746] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0747] Conventional travel planning and reservation systems require users to individually visit various reservation sites to make reservations, which takes a lot of time and effort. Furthermore, the system lacks sufficient promotional content to build excitement during the travel planning process, resulting in low user satisfaction. Furthermore, local guide information and gourmet spot information are scattered across multiple platforms, making it difficult for users to efficiently obtain the information they need.

[0748] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0749] In this invention, the server includes means for receiving initial information about a travel plan from a user, means for generating an optimal travel plan based on the initial information, means for executing a reservation procedure in cooperation with various reservation services based on the travel plan, means for generating and distributing promotional content based on the travel plan, and means for including guide content for each travel destination and content introducing gourmet spots in the promotional content. This allows users to easily and efficiently plan a travel plan and complete the reservation procedure, and further enables them to obtain promotional content and local information that will increase their excitement for the trip.

[0750] "User" means any person or entity that uses the System to make travel plans.

[0751] A "trip plan" is the overall schedule and content of a trip generated based on the destination, dates, number of people, budget, and desired activities and dining style.

[0752] "Initial information" refers to information such as destination, dates, number of people, budget, desired activities and dining style that a user inputs into the system when making travel plans.

[0753] A "travel plan" is a detailed travel schedule including transportation, accommodation, restaurants, activities, etc., generated based on initial information.

[0754] A "reservation service" is a service that provides reservations for transportation, accommodation, restaurants, activities, etc. via the Internet.

[0755] "Promotional content" refers to content that includes visual and audio information created to attractively introduce the contents of a travel plan.

[0756] "Guide content" refers to explanatory and guidance information about tourist attractions, culture, and activities related to a travel destination.

[0757] "Gourmet spots" refers to information about restaurants and types of cuisine that are accessible at a travel destination.

[0758] "Generative AI model" means an artificial intelligence model used to automatically generate promotional content based on initial information and travel plans.

[0759] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations in one go. It also provides promotional content to build excitement before a trip.

[0760] 1. Initial settings and information entry

[0761] User operations

[0762] Users launch the travel planning app on their smartphone or PC and enter initial information about their trip plans in the chat box displayed on the interface screen, including the destination, dates, number of people, budget, desired activities, and dining style.

[0763] Specific examples

[0764] For example, a user might enter, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[0765] 2. Generate a travel plan

[0766] Server Processing

[0767] The server analyzes the initial information received from the user to identify their needs, then uses algorithms and generative AI models within the system to generate an optimal travel plan based on the user's requests, including transportation, accommodation, restaurants, and activities.

[0768] Specific examples

[0769] The server generates an itinerary as follows:

[0770] Dates: August 1st to August 4th

[0771] Accommodation: Japanese-style inn in Kyoto city

[0772] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[0773] Restaurant: Japanese restaurant

[0774] Transportation: Shinkansen round-trip ticket

[0775] 3. Making various reservations

[0776] Server Processing

[0777] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking transportation, accommodation, restaurants, and activities all at once.

[0778] Specific examples

[0779] The server makes the following reservation through the "Reservation Service API":

[0780] Accommodation: Japanese-style inn in Kyoto city (August 1st to August 4th)

[0781] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[0782] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[0783] Transportation: Shinkansen round-trip ticket

[0784] 4. Generate promotional content

[0785] Server Processing

[0786] The server generates and delivers promotional content based on the travel plan. This content presents the travel plan details to the user using visual and audio information. Using a generative AI model, engaging and personalized multimedia content is generated.

[0787] Specific examples

[0788] For example, a server-generated promotional video includes the following highlights:

[0789] A morning visit scene at the beautiful Kinkakuji Temple

[0790] Enjoyable matcha experience

[0791] Relaxing time at a Japanese-style inn

[0792] Delicious meal scene at a Japanese restaurant

[0793] 5. User plan confirmation and travel preparation

[0794] User operations

[0795] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[0796] Specific examples

[0797] Users click on the provided link to watch the promotional video, which builds excitement before the trip. They then feel, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[0798] Hardware and software used

[0799] The following hardware and software are used to realize this system.

[0800] Hardware: Smartphones, PCs, servers

[0801] Software: Python, REST API, generative AI models

[0802] Prompt Sentence Examples

[0803] Destination: Kyoto

[0804] Date: "2023-08-01" to "2023-08-04"

[0805] Number of people: 2 people

[0806] Budget: 200,000 yen

[0807] Activities: Matcha experience, temple tours, Japanese cuisine

[0808] As described above, the system of the present invention allows users to smoothly create travel plans that suit their preferences and easily complete the reservation procedure. Furthermore, it also provides promotional content that builds anticipation before the trip, making the entire travel planning process enjoyable.

[0809] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0810] Step 1:

[0811] A user launches a travel planning app on their smartphone or PC and enters initial information about their trip plan in a chat box displayed on the interface screen, including the destination, dates, number of people, budget, desired activities, and dining style.

[0812] Input: Initial information entered by the user (e.g., destination, dates, number of people, budget, activities)

[0813] Output: Initial information sent to the server

[0814] Specific actions: The user types in the chat box, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is under 200,000 yen, and we would like to experience matcha tea, visit temples, and enjoy Japanese cuisine."

[0815] Step 2:

[0816] The server analyzes the initial information received to identify the user's needs, then uses the system's algorithms and generative AI models to generate an optimal travel plan based on the user's requests.

[0817] Input: Initial information (destination, dates, number of people, budget, activities)

[0818] Output: Optimal travel plan (transportation, accommodation, restaurants, activities)

[0819] Specific operation: The server analyzes "Destination: Kyoto, Dates: August 1st to August 4th, Number of people: 2, Budget: 200,000 yen, Activities: Matcha experience, Temple tour, Japanese cuisine" and generates the optimal travel plan.

[0820] Step 3:

[0821] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, including reservations for transportation, accommodation, restaurants, and activities.

[0822] Input: Generated itinerary

[0823] Output: Confirmed reservation information

[0824] Specific operation: The server makes a reservation through the "Reservation Service API" for "a Japanese-style inn in Kyoto City (August 1st to August 4th), a matcha tea experience (August 2nd from 14:00), a visit to Kinkakuji Temple (August 3rd from 10:00), and a round-trip Shinkansen ticket."

[0825] Step 4:

[0826] The server generates and distributes promotional content based on the travel plan, which presents the details of the travel plan to the user using visual and audio information.

[0827] Input: Generated itinerary

[0828] Output: Promotional content (videos, guide content, gourmet spot introductions)

[0829] Specific operation: The server uses the generative AI model to generate a promotional video including scenes of the beautiful Kinkaku-ji Temple, a matcha tea experience, relaxing at a Japanese-style inn, and dining at a Japanese restaurant, and notifies the user.

[0830] Step 5:

[0831] The user checks the provided travel plan and reservation confirmation information, views the promotional content, and makes the necessary travel preparations based on the content they have viewed.

[0832] Input: Travel itinerary, booking confirmation information, promotional content

[0833] Output: User's travel preparations

[0834] What happens: The user clicks on the provided link, watches the promotional video, and feels, "The video made me really excited. I'm going to start preparing for my trip."

[0835] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0836] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it becomes possible to propose more personalized travel plans. The system also generates promotional videos to build anticipation before the trip.

[0837] 1. Initial settings and information entry

[0838] User Action

[0839] Users launch the travel planning app on their smartphone or PC and enter initial information about their travel plans in the chat box displayed on the interface screen. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. The emotion engine also recognizes emotions from the user's facial expressions and voice, and acquires emotional information.

[0840] Specific examples

[0841] A user inputs, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine." The emotion engine analyzes the user's facial expressions and voice when they input information, and interprets the emotions they are looking forward to.

[0842] 2. Generate a travel plan

[0843] Server Processing

[0844] The server analyzes the initial information and emotional information received from the user to identify their needs. Then, using algorithms and AI within the system, it generates an optimal travel plan based on the user's requests and emotions. The travel plan includes transportation (airplane tickets, bullet trains, etc.), accommodation, restaurants, and activities.

[0845] Specific examples

[0846] The server generates an itinerary like this:

[0847] Dates: August 1st to August 4th

[0848] Accommodation: Japanese-style inn in Kyoto city

[0849] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[0850] Restaurant: Japanese restaurant

[0851] Transportation: Shinkansen round-trip ticket

[0852] Based on emotional information, activities that the user is particularly looking forward to are prioritized.

[0853] 3. Making various reservations

[0854] Server Processing

[0855] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking flights, accommodations, restaurants, and activities all at once.

[0856] Specific examples

[0857] The server makes the following reservation through the "Reservation Service API":

[0858] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[0859] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[0860] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[0861] Transportation: Shinkansen round-trip ticket

[0862] 4. Generate promotional videos

[0863] Server Processing

[0864] The server collects travel plan details and sentiment information and uses multimodal AI to generate a promotional video, highlighting activities and attractions that the user has expressed particular interest in, further building anticipation.

[0865] Specific examples

[0866] The promotional video generated by Sarva using multimodal AI includes the following highlights:

[0867] A morning visit scene at the beautiful Kinkakuji Temple

[0868] Enjoyable matcha experience

[0869] Relaxing time at a Japanese-style inn

[0870] Delicious meal scene at a Japanese restaurant

[0871] Based on emotional information, scenes of activities that the user is particularly looking forward to are included with priority.

[0872] 5. User plan confirmation and travel preparation

[0873] User Action

[0874] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[0875] Specific examples

[0876] Users click on the provided link to watch the promotional video, which builds excitement before the trip. The user feels, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[0877] As described above, the system of the present invention allows users to smoothly create travel plans that suit their preferences and easily complete the reservation procedure. Furthermore, by using the emotion engine, it is possible to provide more personalized plans and further increase anticipation before the trip.

[0878] The processing flow will be explained below.

[0879] Step 1:

[0880] A user launches a travel planning app on their smartphone or PC. The device displays the interface screen and a chat box.

[0881] Step 2:

[0882] The user enters initial information about their travel plans (destination, dates, number of people, budget, desired activities and dining style) into the chat box, and the device sends this information to the server.

[0883] Step 3:

[0884] The device activates the emotion engine and acquires emotion information by analyzing the user's facial expressions and voice. The emotion engine then transmits the acquired emotion information to the server.

[0885] Step 4:

[0886] The server analyzes the initial information and emotional information received to identify the user's needs. Specifically, it centrally manages input content and emotional information and estimates what the user is looking forward to.

[0887] Step 5:

[0888] The server uses the system's algorithms and AI to generate an optimal travel plan based on the user's requests and feelings. The plan includes transportation (airplane, bullet train, etc.), accommodation, restaurants, and activities.

[0889] Step 6:

[0890] The server accesses various reservation services (airline tickets, hotels, restaurants, activities, etc.) to collect availability and price information. For example, it queries each reservation site through a "reservation service API."

[0891] Step 7:

[0892] Based on the information collected, the server selects the reservation options that best fit the user's requirements, and then finalizes the travel plan based on this selection.

[0893] Step 8:

[0894] Based on the details of the reservation options selected by the server, the reservation process is carried out in bulk through various reservation APIs, specifically for air tickets, accommodations, restaurants, and activities.

[0895] Step 9:

[0896] The server collects the reservation confirmation information and sends it to the terminal, which displays a reservation confirmation notice to the user.

[0897] Step 10:

[0898] The server collects itinerary details and sentiment information and uses multimodal AI to generate a promotional video that highlights activities and attractions that the user has expressed particular interest in.

[0899] Step 11:

[0900] The server saves the generated promotional video in cloud storage and provides the link to the user, who then receives the link to watch the video on the device.

[0901] Step 12:

[0902] Users can watch the promotional video from the provided link and get a sense of their trip, which helps them prepare for and build anticipation before their trip.

[0903] Example 2

[0904] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0905] Conventional travel planning systems make it difficult for users to create travel plans based on their own emotions and preferences. Furthermore, few systems offer a single solution for everything from travel planning to various reservations, and there are few ways to build anticipation before a trip. This results in users spending a lot of time and effort on planning and booking trips, preventing smooth travel preparations.

[0906] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0907] In this invention, the server includes means for receiving initial information about a travel plan from a user, means for acquiring emotional information about the user, means for generating an optimal travel plan based on the initial information and emotional information, means for executing a reservation procedure in cooperation with various reservation services based on the travel plan, and means for generating a promotional video based on the travel plan. This makes it possible to create a personalized travel plan that matches the user's preferences and emotions, realize a smooth reservation procedure, and increase anticipation before the trip.

[0908] "User" means an individual or group that uses the System to make travel plans.

[0909] "Initial information" refers to basic information about travel plans such as destination, dates, number of people, budget, and desired activities and dining options.

[0910] "Emotional information" refers to data related to emotions obtained from the user's facial expressions, voice, etc.

[0911] "Travel Itinerary" refers to a plan of a trip including details of transportation, accommodation, dining and activities generated based on the initial information and sentiment information.

[0912] "Reservation Service" means a service that handles reservations for transportation, accommodation, dining, and activities.

[0913] "Promotional video" refers to video content that is generated based on travel plans and is produced to attract the user's interest while taking into account the user's emotional information.

[0914] A "generative AI model" refers to an artificial intelligence model that can process data in a variety of formats and generate new information and content.

[0915] The following is a detailed description of an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it becomes possible to propose more personalized travel plans. The system also generates promotional videos to build anticipation before the trip.

[0916] Initial setup and information entry

[0917] User Action

[0918] Users launch the travel planning app on their smartphone or PC and enter initial information about their travel plans in the chat box displayed on the interface screen. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. The emotion engine also recognizes emotions from the user's facial expressions and voice, and acquires emotional information.

[0919] Specific examples

[0920] A user inputs, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine." The emotion engine analyzes the user's facial expressions and voice when they input information, and interprets the emotions they are looking forward to.

[0921] Generate a travel plan

[0922] Server Processing

[0923] The server analyzes the initial information and emotional information received from the user to identify their needs. Then, using generative AI models and other algorithms within the system, it generates an optimal travel plan based on the user's needs and emotions. The travel plan includes transportation (airplane, bullet train, etc.), accommodation, dining, and activities.

[0924] Specific examples

[0925] The server generates an itinerary like this:

[0926] Dates: August 1st to August 4th

[0927] Accommodation: Japanese-style inn in Kyoto city

[0928] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[0929] Restaurant: Japanese restaurant

[0930] Transportation: Shinkansen round-trip ticket

[0931] Based on emotional information, activities that the user is particularly looking forward to are prioritized.

[0932] Making various reservations

[0933] Server Processing

[0934] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking transportation, accommodation, dining, and activities all at once.

[0935] Specific examples

[0936] The server makes the following reservation through the Reservation Service API:

[0937] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[0938] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[0939] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[0940] Transportation: Shinkansen round-trip ticket

[0941] Promotional video generation

[0942] Server Processing

[0943] The server collects travel plan details and emotional information and uses multimodal AI to generate promotional videos, highlighting activities and tourist attractions that the user has shown particular interest in, further building anticipation.

[0944] Specific examples

[0945] The promotional video, generated by Sarva using multimodal AI, includes the following highlights:

[0946] A morning visit scene at the beautiful Kinkakuji Temple

[0947] Enjoyable matcha experience

[0948] Relaxing time at a Japanese-style inn

[0949] Delicious meal scene at a Japanese restaurant

[0950] Based on emotional information, scenes of activities that the user is particularly looking forward to are included with priority.

[0951] Check user plans and prepare for trip

[0952] User Action

[0953] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[0954] Specific examples

[0955] The user clicks on the provided link, watches the promotional video, and feels, "After watching the video, I'm really excited. I'll start preparing for my trip."

[0956] (Example of a prompt)

[0957] "I'd like to plan a trip for two people to Kyoto for four days and three nights, with a budget of 200,000 yen or less. I'd like to experience matcha tea, visit temples, and enjoy Japanese cuisine."

[0958] As described above, the system utilizes an emotion engine and generative AI models to provide users with highly personalized travel plans and a smooth booking process, allowing them to prepare for their trip efficiently and enjoyably.

[0959] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0960] The specific flow of the program processing of this system

[0961] Step 1:

[0962] Entering initial user information

[0963] A user launches a travel planning app on a smartphone or PC. They enter initial information about their travel plans into a chat box displayed on the interface. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. Based on the input data, the emotion engine extracts emotional information from the user's facial expressions and voice.

[0964] input

[0965] Destination, dates, number of people, budget, desired activities and dining style

[0966] User's facial expressions and voice

[0967] output

[0968] Initial Information Data

[0969] Emotional information data

[0970] Specific actions

[0971] A user inputs, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha green tea experience, visiting temples, and Japanese cuisine." The system saves the input data, and at the same time, the emotion engine analyzes the user's facial expressions and voice to obtain emotional information.

[0972] Step 2:

[0973] Analysis of initial information and emotional information

[0974] The server analyzes the initial information and emotional information received from the user. The system identifies the user's needs and performs appropriate data processing and calculations accordingly.

[0975] input

[0976] Initial Information Data

[0977] Emotional information data

[0978] output

[0979] User needs data

[0980] Specific actions

[0981] The server analyzes the initial information and the emotional information to identify the user's desired activities, budget, and emotional characteristics. The analyzed information is used in the next step.

[0982] Step 3:

[0983] Generate optimal travel plans

[0984] The server uses generative AI models and other algorithms within the system to generate optimal travel plans based on the user's requests and emotions.

[0985] input

[0986] User needs data

[0987] output

[0988] Generated travel plan data

[0989] Specific actions

[0990] The server uses the generative AI model to generate a travel plan like this:

[0991] Dates: August 1st to August 4th

[0992] Accommodation: Japanese-style inn in Kyoto city

[0993] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[0994] Restaurant: Japanese restaurant

[0995] Transportation: Shinkansen round-trip ticket

[0996] Based on emotional information, activities that the user is particularly looking forward to are prioritized in the plan.

[0997] Step 4:

[0998] Making various reservations

[0999] Based on the generated travel plan, the server connects with various reservation services and executes the reservation process, specifically booking transportation, accommodation, dining, and activities all at once.

[1000] input

[1001] Generated travel plan data

[1002] output

[1003] Various reservation confirmation data

[1004] Specific actions

[1005] The server connects to the "Reservation Service API" and performs the following reservation:

[1006] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[1007] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[1008] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[1009] Transportation: Shinkansen round-trip ticket

[1010] Step 5:

[1011] Promotional video generation

[1012] The server collects itinerary details and sentiment information and uses multimodal AI to generate promotional videos that highlight activities and attractions that the user has expressed particular interest in.

[1013] input

[1014] Travel Plan Data

[1015] Emotional information data

[1016] output

[1017] Generated promotional video

[1018] Specific actions

[1019] The server selects footage and audio based on the details of the itinerary and runs a video editing algorithm. The multimodal AI-generated footage includes highlights of:

[1020] A morning visit scene at the beautiful Kinkakuji Temple

[1021] Enjoyable matcha experience

[1022] Relaxing time at a Japanese-style inn

[1023] Delicious Japanese food scene

[1024] Based on emotional information, scenes of activities that the user is particularly looking forward to are included with priority.

[1025] Step 6:

[1026] Check user plans and prepare for trip

[1027] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[1028] input

[1029] Generated itinerary and booking confirmation data

[1030] Generated promotional video

[1031] output

[1032] User Readiness

[1033] Specific actions

[1034] The user clicks on the provided link, watches the promotional video, and feels, "After watching the video, I'm really excited. I'll start preparing for my trip." Based on this, the user makes specific preparations for the trip.

[1035] Through these steps, the system supports users through a series of steps, from entering initial information to creating a travel plan, various reservations, providing promotional videos, and final confirmation and preparation, providing a personalized travel experience.

[1036] (Application example 2)

[1037] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1038] In recent years, as demand for food delivery has increased, there is a demand for personalized meal suggestions tailored to user needs and more efficient ordering procedures. It is also important to provide an experience that allows users to enjoy ordering, rather than simply ordering a meal. Current food delivery systems do not consider users' emotional information when making ordering suggestions, nor do they foster anticipation through promotional media, making it difficult to improve the user experience.

[1039] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving initial information regarding a meal order from a user, means for generating an optimal food plan based on the initial information, means for executing a reservation procedure in cooperation with various reservation services based on the food plan, means for generating promotional media based on the food plan, means for acquiring emotional information of the user using an emotion engine and proposing a personalized plan, and means for providing the promotional media generated based on the emotional information. This makes it possible to propose personalized food delivery plans that take the user's emotions into consideration, enabling the user to have an enjoyable and improved ordering experience.

[1040] "User" refers to the end user who uses the system.

[1041] "Initial information" refers to basic information provided by the user, such as the destination, itinerary, number of people, budget, desired activities, and dining style.

[1042] A "travel plan" refers to a travel schedule or itinerary that is generated based on the above initial information and emotional information and is optimized to meet the user's needs.

[1043] "Reservation process" refers to the process of making reservations for accommodation, transportation, restaurants, activities, etc. in one go, in collaboration with various reservation services based on your travel plans.

[1044] "Promotional media" refers to video or video content created with the purpose of increasing user interest or emotion.

[1045] An "emotion engine" refers to an algorithm or program that recognizes emotions from a user's facial expressions and voice and acquires emotional information.

[1046] A "personalized plan" refers to an itinerary or proposal that is individually optimized to reflect the user's emotional information.

[1047] "Multimodal artificial intelligence technology" refers to artificial intelligence technology that integrates and processes multiple types of data (text, images, audio, etc.).

[1048] A "server" refers to a computer system that processes and stores data on the cloud and performs various system functions.

[1049] The following describes an embodiment of the present invention. The system generates a personalized food plan when a user places a meal order, enabling efficient ordering. Furthermore, by combining an emotion engine, the system generates suggestions and promotional media based on the user's emotions, improving the ordering experience.

[1050] 1. Input of initial information and emotional information

[1051] User Action

[1052] Users launch the food delivery app on their smartphone and enter initial information about their meal in the chat box displayed on the interface screen. This information includes the desired food category, budget, and number of people. The emotion engine then recognizes the user's emotions from their facial expressions and voice, and acquires their emotional information.

[1053] Specific examples

[1054] A user inputs, "I want to eat pizza and pasta for three people within a budget of 5,000 yen." The emotion engine analyzes the user's facial expressions and voice when they input, and interprets the user's emotions.

[1055] 2. Generate a food plan

[1056] Server Processing

[1057] The server analyzes the initial information and emotional information received from the user to identify their needs. It then uses a generative AI model (e.g., GPT-4) within the system to generate an optimal meal plan based on the user's requests and emotions. The meal plan includes suggested restaurants and specific menu items, with priority given to suggestions that enhance the user's emotions.

[1058] Specific examples

[1059] The server generates a food plan like this:

[1060] Restaurant: Popular Italian restaurant

[1061] Menu: Margherita pizza, carbonara pasta

[1062] 3. Making reservations and ordering

[1063] Server Processing

[1064] Based on the generated food plan, the server connects with various services to carry out the ordering process, including sending the order details to the restaurant and handling the payment process all at once.

[1065] Specific examples

[1066] The server executes the following procedure through the "Order Service API":

[1067] Send order to restaurant: Margherita pizza (3 slices), Carbonara pasta (3 servings)

[1068] Payment process

[1069] 4. Promotional Media Creation

[1070] Server Processing

[1071] The server collects details of the food plan and emotional information, and generates promotional media using multimodal artificial intelligence technology (e.g., DALL-E). This media introduces the menu and restaurant atmosphere that the user is particularly interested in, building anticipation.

[1072] Specific examples

[1073] Promotional media generated by the server using multimodal artificial intelligence technology includes the following highlights:

[1074] Beautiful pizza being baked

[1075] Delicious looking pasta being served

[1076] 5. User plan confirmation and order confirmation

[1077] User Action

[1078] The user checks the provided food plan and reservation confirmation information, and views the promotional media to confirm and confirm the order details.

[1079] Specific examples

[1080] The user clicks on the provided link, watches the promotional media, and decides to "confirm my order from this menu."

[1081] As described above, the system of the present invention allows users to smoothly create a meal plan that suits their preferences and easily complete the ordering process. Furthermore, by utilizing the emotion engine, it is possible to provide more personalized suggestions and further increase anticipation before ordering.

[1082] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1083] Step 1:

[1084] The user launches the food delivery app on their smartphone and enters initial information about the meal in the chat box displayed on the interface screen. This information includes the desired food category, budget, and number of people. The emotion engine recognizes emotions from the user's facial expressions and voice and acquires emotional information. The entered information is sent to the server.

[1085] input:

[1086] Desired food category

[1087] budget

[1088] Number of people

[1089] Facial expression data

[1090] Audio data

[1091] output:

[1092] Initial information

[1093] emotional information

[1094] Specific behavior:

[1095] The user enters, "I want to eat pizza and pasta for three people within a budget of 5,000 yen."

[1096] The emotion engine recognizes facial expressions and voices through the smartphone's camera and microphone.

[1097] The server receives the initial information and emotion information.

[1098] Step 2:

[1099] The server analyzes the initial information and emotional information received from the user. Then, using a generative AI model (e.g., GPT-4) within the system, it generates an optimal meal plan based on the user's requests and emotions. Based on the analyzed data, it suggests the most suitable restaurant and specific menu.

[1100] input:

[1101] Initial information

[1102] emotional information

[1103] output:

[1104] Food Plan

[1105] Specific behavior:

[1106] The server analyzes the initial information and emotional information to identify the user's needs.

[1107] A generative AI model was used to generate a food plan that included Margherita pizza and carbonara pasta.

[1108] Step 3:

[1109] Based on the generated food plan, the server connects with various services to carry out the ordering process, including sending the order details to the restaurant and handling the payment process all at once.

[1110] input:

[1111] Food Plan

[1112] output:

[1113] Order Confirmation Information

[1114] Payment completion information

[1115] Specific behavior:

[1116] The server sends the order details to the restaurant via the "Order Service API."

[1117] Process payments and generate order confirmation and payment completion information.

[1118] Step 4:

[1119] The server collects details of the food plan and emotional information, and generates promotional media using multimodal artificial intelligence technology (e.g., DALL-E). This media introduces the menu and restaurant atmosphere that the user is particularly interested in, building anticipation.

[1120] input:

[1121] Food Plan

[1122] emotional information

[1123] output:

[1124] Promotional Media

[1125] Specific behavior:

[1126] The server uses multimodal artificial intelligence technology to generate compelling images of pizza and pasta.

[1127] Create links to provide promotional media to users.

[1128] Step 5:

[1129] The user checks the provided food plan and reservation confirmation information, watches the promotional media, and confirms and confirms the order details. If satisfied, the user finally confirms the order.

[1130] input:

[1131] Promotional Media

[1132] Order Confirmation Information

[1133] output:

[1134] Final order confirmation information

[1135] Specific behavior:

[1136] User clicks on the provided link to view the promotional media.

[1137] After watching, confirm your order by clicking "Confirm your order from this menu."

[1138] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1139] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1140] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1141] [Third embodiment]

[1142] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1143] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1144] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1145] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1146] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1147] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1148] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1149] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1150] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1151] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1152] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1153] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1154] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. It also generates promotional videos to build excitement before the trip.

[1155] 1. Initial settings and information entry

[1156] User Action

[1157] Users launch the travel planning app on their smartphone or PC and enter initial information about their trip plans in the chat box displayed on the interface screen, including the destination, dates, number of people, budget, desired activities, and dining style.

[1158] Specific examples

[1159] A user types, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[1160] 2. Generate a travel plan

[1161] Server Processing

[1162] The server analyzes the initial information received from the user to identify their needs. Then, using algorithms and AI within the system, it generates an optimal travel plan based on the user's requests. The travel plan includes transportation (airplane, bullet train, etc.), accommodation, restaurants, and activities.

[1163] Specific examples

[1164] The server generates an itinerary like this:

[1165] Dates: August 1st to August 4th

[1166] Accommodation: Japanese-style inn in Kyoto city

[1167] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[1168] Restaurant: Japanese restaurant

[1169] Transportation: Shinkansen round-trip ticket

[1170] 3. Making various reservations

[1171] Server Processing

[1172] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking flights, accommodations, restaurants, and activities all at once.

[1173] Specific examples

[1174] The server makes the following reservation through the "Reservation Service API":

[1175] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[1176] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[1177] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[1178] Transportation: Shinkansen round-trip ticket

[1179] 4. Generate promotional videos

[1180] Server Processing

[1181] The server generates a promotional video based on the travel plan. By using multimodal AI, the details of the travel plan are presented in video format and provided to the user.

[1182] Specific examples

[1183] The promotional video, generated by the server using multi-mode AI, includes the following highlights:

[1184] A morning visit scene at the beautiful Kinkakuji Temple

[1185] Enjoyable matcha experience

[1186] Relaxing time at a Japanese-style inn

[1187] Delicious meal scene at a Japanese restaurant

[1188] 5. User plan confirmation and travel preparation

[1189] User Action

[1190] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[1191] Specific examples

[1192] Users click on the provided link to watch the promotional video, which builds excitement before the trip. The user feels, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[1193] As described above, the system of the present invention allows users to smoothly create travel plans that suit their preferences and easily complete the reservation procedure. In addition, the system also provides promotional videos that build anticipation before the trip, making the entire travel planning process enjoyable.

[1194] The processing flow will be explained below.

[1195] Step 1:

[1196] A user launches a travel planning app on their smartphone or PC. The device displays the interface screen and a chat box.

[1197] Step 2:

[1198] The user enters initial information about their travel plans into the chat box, such as destination, dates, number of people, budget, desired activities and dining style, etc. The device then sends this information to the server.

[1199] Step 3:

[1200] The server analyzes the initial information received and identifies the user's needs. Specifically, it breaks down the input content into categories and prepares to retrieve the necessary information from the database.

[1201] Step 4:

[1202] The server uses algorithms and AI within the system to generate an optimal travel plan based on the user's requests, including details on transportation, accommodation, restaurants, and activities.

[1203] Step 5:

[1204] The server accesses various reservation services (airline tickets, hotels, restaurants, activities, etc.) to collect availability and price information. For example, it queries each reservation site through a "reservation service API."

[1205] Step 6:

[1206] Based on the information collected, the server selects the reservation options that best fit the user's requirements, and then finalizes the travel plan based on this selection.

[1207] Step 7:

[1208] Based on the details of the reservation options selected by the server, the reservation process is carried out in bulk through various reservation APIs, specifically for air tickets, accommodations, restaurants, and activities.

[1209] Step 8:

[1210] The server collects the reservation confirmation information and sends it to the terminal, which displays a reservation confirmation notice to the user.

[1211] Step 9:

[1212] The server collects the details of the itinerary and uses multimodal AI to generate a promotional video, which includes the trip's highlights and recommended spots.

[1213] Step 10:

[1214] The server saves the generated promotional video in cloud storage and provides the link to the user, who then receives the link to watch the video on the device.

[1215] Step 11:

[1216] Users can watch the promotional video from the provided link and get a sense of their trip, which helps them prepare for and build anticipation before their trip.

[1217] This is the specific process flow of the travel planning system. By using this system, users can easily create a travel plan that suits their preferences and smoothly proceed with reservations and preparations.

[1218] Example 1

[1219] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1220] Traditional travel planning requires users to go through multiple booking sites, which requires a lot of time and effort. In addition, there is a lack of information to build anticipation before a trip, which can make pre-trip preparations cumbersome.

[1221] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1222] In this invention, the server includes means for receiving initial information about a travel plan from a user, means for generating an optimal travel plan based on the initial information, means for executing reservation procedures in cooperation with various reservation services based on the travel plan, and means for providing the promotional video to the user to heighten pre-travel excitement. This allows users to efficiently carry out everything from planning a travel plan to making reservations and gathering pre-travel information using a single system.

[1223] "User" refers to an individual or organization making travel plans.

[1224] "Travel Itinerary" includes information and plans for traveling to a particular location.

[1225] "Initial Information" refers to basic information provided by the User, such as the travel destination, itinerary, number of people, budget, and desired activities and dining style.

[1226] "Travel Plan" refers to a plan generated based on the User's initial information, including details of transportation, accommodation, dining options, activities, etc.

[1227] "Reservation Process" refers to the process of making reservations for transportation, accommodation, dining, activities, etc.

[1228] A "promotional video" refers to a video generated based on a travel plan that outlines the attractions of a travel destination and activities.

[1229] "Multimodal AI" refers to artificial intelligence technology that integrates multiple data formats, such as text, images, and audio, and uses them to analyze and generate information.

[1230] "Server" refers to a computer system that receives and processes information based on user operations.

[1231] MODE FOR CARRYING OUT THE INVENTION

[1232] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. It also generates promotional videos to build excitement before the trip.

[1233] Initial setup and information entry

[1234] A user launches a travel planning app on a smartphone or PC. They enter initial information about their travel plans into a chat box displayed on the app's interface screen. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. This information is sent to the server using a data structure such as JSON.

[1235] Specific examples

[1236] A user types, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[1237] Generate a travel plan

[1238] The server uses a text analysis engine to analyze the initial information it receives. As a result of this analysis, it identifies the user's needs and generates an optimal travel plan using the system's algorithms and generative AI model. The generative AI model learns from past travel data and the user's preferences and proposes the optimal plan.

[1239] The travel plan includes transportation (airplane tickets, bullet trains, etc.), accommodations, restaurants, and activities. The generated plan is provided to the user through an interface for confirmation.

[1240] Specific examples

[1241] The server generates an itinerary like this:

[1242] Dates: August 1st to August 4th

[1243] Accommodation: Japanese-style inn in Kyoto city

[1244] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[1245] Restaurant: Japanese restaurant

[1246] Transportation: Shinkansen round-trip ticket

[1247] Making various reservations

[1248] The server then coordinates with various reservation services to complete the reservation process based on the generated travel plan. Using the reservation API, reservations for flights, accommodations, restaurants, and activities are made all at once. Once the reservation is complete, the information is sent to the user.

[1249] Specific examples

[1250] The server makes the following reservation through the "Reservation Service API":

[1251] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[1252] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[1253] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[1254] Transportation: Shinkansen round-trip ticket

[1255] Promotional video generation

[1256] The server uses multimodal AI to generate a promotional video based on the travel plan. The multimodal AI integrates multiple data formats, such as text, images, and audio, analyzes the information, and presents the details of the travel plan in video format. The generated promotional video is provided to the user.

[1257] Specific examples

[1258] The promotional video generated by Sarva using multimodal AI includes the following highlights:

[1259] A morning visit scene at the beautiful Kinkakuji Temple

[1260] Enjoyable matcha experience

[1261] Relaxing time at a Japanese-style inn

[1262] Delicious meal scene at a Japanese restaurant

[1263] Check user plans and prepare for trip

[1264] The user can review the provided travel plan and reservation confirmation information, and click on the provided link to watch a promotional video to further enhance their travel plans. At this stage, they can begin making any necessary preparations.

[1265] Specific examples

[1266] Users click on the provided link to watch the promotional video, which builds excitement before the trip. The user feels, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[1267] The system allows users to smoothly create a travel plan that suits their preferences, simplify the booking process, and even provides promotional videos to build excitement before the trip, making the whole travel planning process fun.

[1268] Prompt Sentence Examples

[1269] "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is under 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[1270] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1271] Step 1:

[1272] A user launches a travel planning app on their smartphone or PC. They enter initial information about their travel plans into a chat box displayed on the app's interface screen. Specifically, they enter the travel destination, itinerary, number of people, budget, desired activities, and dining style. This input information is converted into JSON-formatted data by the application and sent to the server.

[1273] Input: Initial information about the travel plan entered by the user

[1274] Output: Initial information sent to the server in JSON format.

[1275] Specific actions: The user enters, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[1276] Step 2:

[1277] The server launches a text analysis engine to analyze the initial information received. This engine uses natural language processing technology to extract information such as destination, itinerary, number of people, budget, activities, and dining style, and identifies each element. Based on this analysis, the system uses its algorithms and generative AI models to generate an optimal travel plan.

[1278] Input: Initial information in JSON format

[1279] Output: Analyzed travel plan elements and a travel plan based on them

[1280] Specific operation: The server extracts information such as "Kyoto," "August 1st to August 4th," "2 people," "200,000 yen," "matcha experience," "temple tour," and "Japanese cuisine" to create the optimal plan.

[1281] Step 3:

[1282] The server connects with various reservation services based on the generated travel plan. Specifically, it uses a reservation API to make reservations for flights, accommodations, restaurants, and activities all at once. Once the reservation is complete, the server notifies the user of the reservation confirmation information.

[1283] Input: Travel Plan

[1284] Output: Reservation confirmation information

[1285] Specific operation: The server executes the following reservation through the "Reservation Service API":

[1286] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[1287] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[1288] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[1289] Transportation: Shinkansen round-trip ticket

[1290] Step 4:

[1291] The server generates a promotional video based on the travel plan, activates multimodal AI, and integrates multiple data formats such as text, images, and audio to present the details of the travel plan in video format, and provides the generated promotional video to the user.

[1292] Input: Travel plan details

[1293] Output: Promotional video

[1294] What happens: The server generates a promotional video that includes the following:

[1295] A morning visit scene at the beautiful Kinkakuji Temple

[1296] Enjoyable matcha experience

[1297] Relaxing time at a Japanese-style inn

[1298] Delicious meal scene at a Japanese restaurant

[1299] Step 5:

[1300] The user confirms the provided travel plan and reservation confirmation information, and then clicks on the provided link to watch a promotional video to further enhance their travel plans and make the necessary preparations.

[1301] Input: Promotional video link and booking confirmation information

[1302] Output: Specific actions that users can take to prepare and expectations before the trip

[1303] Specific behavior: The user clicks on the provided link, watches the promotional video, and feels, "After watching the video, I'm really excited. I'll start preparing for my trip."

[1304] (Application example 1)

[1305] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1306] Conventional travel planning and reservation systems require users to individually visit various reservation sites to make reservations, which takes a lot of time and effort. Furthermore, the system lacks sufficient promotional content to build excitement during the travel planning process, resulting in low user satisfaction. Furthermore, local guide information and gourmet spot information are scattered across multiple platforms, making it difficult for users to efficiently obtain the information they need.

[1307] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1308] In this invention, the server includes means for receiving initial information about a travel plan from a user, means for generating an optimal travel plan based on the initial information, means for executing a reservation procedure in cooperation with various reservation services based on the travel plan, means for generating and distributing promotional content based on the travel plan, and means for including guide content for each travel destination and content introducing gourmet spots in the promotional content. This allows users to easily and efficiently plan a travel plan and complete the reservation procedure, and further enables them to obtain promotional content and local information that will increase their excitement for the trip.

[1309] "User" means any person or entity that uses the System to make travel plans.

[1310] A "trip plan" is the overall schedule and content of a trip generated based on the destination, dates, number of people, budget, and desired activities and dining style.

[1311] "Initial information" refers to information such as destination, dates, number of people, budget, desired activities and dining style that a user inputs into the system when making travel plans.

[1312] A "travel plan" is a detailed travel schedule including transportation, accommodation, restaurants, activities, etc., generated based on initial information.

[1313] A "reservation service" is a service that provides reservations for transportation, accommodation, restaurants, activities, etc. via the Internet.

[1314] "Promotional content" refers to content that includes visual and audio information created to attractively introduce the contents of a travel plan.

[1315] "Guide content" refers to explanatory and guidance information about tourist attractions, culture, and activities related to a travel destination.

[1316] "Gourmet spots" refers to information about restaurants and types of cuisine that are accessible at a travel destination.

[1317] "Generative AI model" means an artificial intelligence model used to automatically generate promotional content based on initial information and travel plans.

[1318] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations in one go. It also provides promotional content to build excitement before a trip.

[1319] 1. Initial settings and information entry

[1320] User operations

[1321] Users launch the travel planning app on their smartphone or PC and enter initial information about their trip plans in the chat box displayed on the interface screen, including the destination, dates, number of people, budget, desired activities, and dining style.

[1322] Specific examples

[1323] For example, a user might enter, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[1324] 2. Generate a travel plan

[1325] Server Processing

[1326] The server analyzes the initial information received from the user to identify their needs, then uses algorithms and generative AI models within the system to generate an optimal travel plan based on the user's requests, including transportation, accommodation, restaurants, and activities.

[1327] Specific examples

[1328] The server generates an itinerary as follows:

[1329] Dates: August 1st to August 4th

[1330] Accommodation: Japanese-style inn in Kyoto city

[1331] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[1332] Restaurant: Japanese restaurant

[1333] Transportation: Shinkansen round-trip ticket

[1334] 3. Making various reservations

[1335] Server Processing

[1336] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking transportation, accommodation, restaurants, and activities all at once.

[1337] Specific examples

[1338] The server makes the following reservation through the "Reservation Service API":

[1339] Accommodation: Japanese-style inn in Kyoto city (August 1st to August 4th)

[1340] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[1341] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[1342] Transportation: Shinkansen round-trip ticket

[1343] 4. Generate promotional content

[1344] Server Processing

[1345] The server generates and delivers promotional content based on the travel plan. This content presents the travel plan details to the user using visual and audio information. Using a generative AI model, engaging and personalized multimedia content is generated.

[1346] Specific examples

[1347] For example, a server-generated promotional video includes the following highlights:

[1348] A morning visit scene at the beautiful Kinkakuji Temple

[1349] Enjoyable matcha experience

[1350] Relaxing time at a Japanese-style inn

[1351] Delicious meal scene at a Japanese restaurant

[1352] 5. User plan confirmation and travel preparation

[1353] User operations

[1354] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[1355] Specific examples

[1356] Users click on the provided link to watch the promotional video, which builds excitement before the trip. They then feel, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[1357] Hardware and software used

[1358] The following hardware and software are used to realize this system.

[1359] Hardware: Smartphones, PCs, servers

[1360] Software: Python, REST API, generative AI models

[1361] Prompt Sentence Examples

[1362] Destination: Kyoto

[1363] Date: "2023-08-01" to "2023-08-04"

[1364] Number of people: 2 people

[1365] Budget: 200,000 yen

[1366] Activities: Matcha experience, temple tours, Japanese cuisine

[1367] As described above, the system of the present invention allows users to smoothly create travel plans that suit their preferences and easily complete the reservation procedure. Furthermore, it also provides promotional content that builds anticipation before the trip, making the entire travel planning process enjoyable.

[1368] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1369] Step 1:

[1370] A user launches a travel planning app on their smartphone or PC and enters initial information about their trip plan in a chat box displayed on the interface screen, including the destination, dates, number of people, budget, desired activities, and dining style.

[1371] Input: Initial information entered by the user (e.g., destination, dates, number of people, budget, activities)

[1372] Output: Initial information sent to the server

[1373] Specific actions: The user types in the chat box, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is under 200,000 yen, and we would like to experience matcha tea, visit temples, and enjoy Japanese cuisine."

[1374] Step 2:

[1375] The server analyzes the initial information received to identify the user's needs, then uses the system's algorithms and generative AI models to generate an optimal travel plan based on the user's requests.

[1376] Input: Initial information (destination, dates, number of people, budget, activities)

[1377] Output: Optimal travel plan (transportation, accommodation, restaurants, activities)

[1378] Specific operation: The server analyzes "Destination: Kyoto, Dates: August 1st to August 4th, Number of people: 2, Budget: 200,000 yen, Activities: Matcha experience, Temple tour, Japanese cuisine" and generates the optimal travel plan.

[1379] Step 3:

[1380] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, including reservations for transportation, accommodation, restaurants, and activities.

[1381] Input: Generated itinerary

[1382] Output: Confirmed reservation information

[1383] Specific operation: The server makes a reservation through the "Reservation Service API" for "a Japanese-style inn in Kyoto City (August 1st to August 4th), a matcha tea experience (August 2nd from 14:00), a visit to Kinkakuji Temple (August 3rd from 10:00), and a round-trip Shinkansen ticket."

[1384] Step 4:

[1385] The server generates and distributes promotional content based on the travel plan, which presents the details of the travel plan to the user using visual and audio information.

[1386] Input: Generated itinerary

[1387] Output: Promotional content (videos, guide content, gourmet spot introductions)

[1388] Specific operation: The server uses the generative AI model to generate a promotional video including scenes of the beautiful Kinkaku-ji Temple, a matcha tea experience, relaxing at a Japanese-style inn, and dining at a Japanese restaurant, and notifies the user.

[1389] Step 5:

[1390] The user checks the provided travel plan and reservation confirmation information, views the promotional content, and makes the necessary travel preparations based on the content they have viewed.

[1391] Input: Travel itinerary, booking confirmation information, promotional content

[1392] Output: User's travel preparations

[1393] What happens: The user clicks on the provided link, watches the promotional video, and feels, "The video made me really excited. I'm going to start preparing for my trip."

[1394] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1395] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it becomes possible to propose more personalized travel plans. The system also generates promotional videos to build anticipation before the trip.

[1396] 1. Initial settings and information entry

[1397] User Action

[1398] Users launch the travel planning app on their smartphone or PC and enter initial information about their travel plans in the chat box displayed on the interface screen. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. The emotion engine also recognizes emotions from the user's facial expressions and voice, and acquires emotional information.

[1399] Specific examples

[1400] A user inputs, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine." The emotion engine analyzes the user's facial expressions and voice when they input information, and interprets the emotions they are looking forward to.

[1401] 2. Generate a travel plan

[1402] Server Processing

[1403] The server analyzes the initial information and emotional information received from the user to identify their needs. Then, using algorithms and AI within the system, it generates an optimal travel plan based on the user's requests and emotions. The travel plan includes transportation (airplane tickets, bullet trains, etc.), accommodation, restaurants, and activities.

[1404] Specific examples

[1405] The server generates an itinerary like this:

[1406] Dates: August 1st to August 4th

[1407] Accommodation: Japanese-style inn in Kyoto city

[1408] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[1409] Restaurant: Japanese restaurant

[1410] Transportation: Shinkansen round-trip ticket

[1411] Based on emotional information, activities that the user is particularly looking forward to are prioritized.

[1412] 3. Making various reservations

[1413] Server Processing

[1414] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking flights, accommodations, restaurants, and activities all at once.

[1415] Specific examples

[1416] The server makes the following reservation through the "Reservation Service API":

[1417] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[1418] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[1419] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[1420] Transportation: Shinkansen round-trip ticket

[1421] 4. Generate promotional videos

[1422] Server Processing

[1423] The server collects travel plan details and sentiment information and uses multimodal AI to generate a promotional video, highlighting activities and attractions that the user has expressed particular interest in, further building anticipation.

[1424] Specific examples

[1425] The promotional video generated by Sarva using multimodal AI includes the following highlights:

[1426] A morning visit scene at the beautiful Kinkakuji Temple

[1427] Enjoyable matcha experience

[1428] Relaxing time at a Japanese-style inn

[1429] Delicious meal scene at a Japanese restaurant

[1430] Based on emotional information, scenes of activities that the user is particularly looking forward to are included with priority.

[1431] 5. User plan confirmation and travel preparation

[1432] User Action

[1433] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[1434] Specific examples

[1435] Users click on the provided link to watch the promotional video, which builds excitement before the trip. The user feels, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[1436] As described above, the system of the present invention allows users to smoothly create travel plans that suit their preferences and easily complete the reservation procedure. Furthermore, by using the emotion engine, it is possible to provide more personalized plans and further increase anticipation before the trip.

[1437] The processing flow will be explained below.

[1438] Step 1:

[1439] A user launches a travel planning app on their smartphone or PC. The device displays the interface screen and a chat box.

[1440] Step 2:

[1441] The user enters initial information about their travel plans (destination, dates, number of people, budget, desired activities and dining style) into the chat box, and the device sends this information to the server.

[1442] Step 3:

[1443] The device activates the emotion engine and acquires emotion information by analyzing the user's facial expressions and voice. The emotion engine then transmits the acquired emotion information to the server.

[1444] Step 4:

[1445] The server analyzes the initial information and emotional information received to identify the user's needs. Specifically, it centrally manages input content and emotional information and estimates what the user is looking forward to.

[1446] Step 5:

[1447] The server uses the system's algorithms and AI to generate an optimal travel plan based on the user's requests and feelings. The plan includes transportation (airplane, bullet train, etc.), accommodation, restaurants, and activities.

[1448] Step 6:

[1449] The server accesses various reservation services (airline tickets, hotels, restaurants, activities, etc.) to collect availability and price information. For example, it queries each reservation site through a "reservation service API."

[1450] Step 7:

[1451] Based on the information collected, the server selects the reservation options that best fit the user's requirements, and then finalizes the travel plan based on this selection.

[1452] Step 8:

[1453] Based on the details of the reservation options selected by the server, the reservation process is carried out in bulk through various reservation APIs, specifically for air tickets, accommodations, restaurants, and activities.

[1454] Step 9:

[1455] The server collects the reservation confirmation information and sends it to the terminal, which displays a reservation confirmation notice to the user.

[1456] Step 10:

[1457] The server collects itinerary details and sentiment information and uses multimodal AI to generate a promotional video that highlights activities and attractions that the user has expressed particular interest in.

[1458] Step 11:

[1459] The server saves the generated promotional video in cloud storage and provides the link to the user, who then receives the link to watch the video on the device.

[1460] Step 12:

[1461] Users can watch the promotional video from the provided link and get a sense of their trip, which helps them prepare for and build anticipation before their trip.

[1462] Example 2

[1463] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1464] Conventional travel planning systems make it difficult for users to create travel plans based on their own emotions and preferences. Furthermore, few systems offer a single solution for everything from travel planning to various reservations, and there are few ways to build anticipation before a trip. This results in users spending a lot of time and effort on planning and booking trips, preventing smooth travel preparations.

[1465] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1466] In this invention, the server includes means for receiving initial information about a travel plan from a user, means for acquiring emotional information about the user, means for generating an optimal travel plan based on the initial information and emotional information, means for executing a reservation procedure in cooperation with various reservation services based on the travel plan, and means for generating a promotional video based on the travel plan. This makes it possible to create a personalized travel plan that matches the user's preferences and emotions, realize a smooth reservation procedure, and increase anticipation before the trip.

[1467] "User" means an individual or group that uses the System to make travel plans.

[1468] "Initial information" refers to basic information about travel plans such as destination, dates, number of people, budget, and desired activities and dining options.

[1469] "Emotional information" refers to data related to emotions obtained from the user's facial expressions, voice, etc.

[1470] "Travel Itinerary" refers to a plan of a trip including details of transportation, accommodation, dining and activities generated based on the initial information and sentiment information.

[1471] "Reservation Service" means a service that handles reservations for transportation, accommodation, dining, and activities.

[1472] "Promotional video" refers to video content that is generated based on travel plans and is produced to attract the user's interest while taking into account the user's emotional information.

[1473] A "generative AI model" refers to an artificial intelligence model that can process data in a variety of formats and generate new information and content.

[1474] The following is a detailed description of an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it becomes possible to propose more personalized travel plans. The system also generates promotional videos to build anticipation before the trip.

[1475] Initial setup and information entry

[1476] User Action

[1477] Users launch the travel planning app on their smartphone or PC and enter initial information about their travel plans in the chat box displayed on the interface screen. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. The emotion engine also recognizes emotions from the user's facial expressions and voice, and acquires emotional information.

[1478] Specific examples

[1479] A user inputs, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine." The emotion engine analyzes the user's facial expressions and voice when they input information, and interprets the emotions they are looking forward to.

[1480] Generate a travel plan

[1481] Server Processing

[1482] The server analyzes the initial information and emotional information received from the user to identify their needs. Then, using generative AI models and other algorithms within the system, it generates an optimal travel plan based on the user's needs and emotions. The travel plan includes transportation (airplane, bullet train, etc.), accommodation, dining, and activities.

[1483] Specific examples

[1484] The server generates an itinerary like this:

[1485] Dates: August 1st to August 4th

[1486] Accommodation: Japanese-style inn in Kyoto city

[1487] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[1488] Restaurant: Japanese restaurant

[1489] Transportation: Shinkansen round-trip ticket

[1490] Based on emotional information, activities that the user is particularly looking forward to are prioritized.

[1491] Making various reservations

[1492] Server Processing

[1493] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking transportation, accommodation, dining, and activities all at once.

[1494] Specific examples

[1495] The server makes the following reservation through the Reservation Service API:

[1496] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[1497] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[1498] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[1499] Transportation: Shinkansen round-trip ticket

[1500] Promotional video generation

[1501] Server Processing

[1502] The server collects travel plan details and emotional information and uses multimodal AI to generate promotional videos, highlighting activities and tourist attractions that the user has shown particular interest in, further building anticipation.

[1503] Specific examples

[1504] The promotional video, generated by Sarva using multimodal AI, includes the following highlights:

[1505] A morning visit scene at the beautiful Kinkakuji Temple

[1506] Enjoyable matcha experience

[1507] Relaxing time at a Japanese-style inn

[1508] Delicious meal scene at a Japanese restaurant

[1509] Based on emotional information, scenes of activities that the user is particularly looking forward to are included with priority.

[1510] Check user plans and prepare for trip

[1511] User Action

[1512] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[1513] Specific examples

[1514] The user clicks on the provided link, watches the promotional video, and feels, "After watching the video, I'm really excited. I'll start preparing for my trip."

[1515] (Example of a prompt)

[1516] "I'd like to plan a trip for two people to Kyoto for four days and three nights, with a budget of 200,000 yen or less. I'd like to experience matcha tea, visit temples, and enjoy Japanese cuisine."

[1517] As described above, the system utilizes an emotion engine and generative AI models to provide users with highly personalized travel plans and a smooth booking process, allowing them to prepare for their trip efficiently and enjoyably.

[1518] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1519] The specific flow of the program processing of this system

[1520] Step 1:

[1521] Entering initial user information

[1522] A user launches a travel planning app on a smartphone or PC. They enter initial information about their travel plans into a chat box displayed on the interface. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. Based on the input data, the emotion engine extracts emotional information from the user's facial expressions and voice.

[1523] input

[1524] Destination, dates, number of people, budget, desired activities and dining style

[1525] User's facial expressions and voice

[1526] output

[1527] Initial Information Data

[1528] Emotional information data

[1529] Specific actions

[1530] A user inputs, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha green tea experience, visiting temples, and Japanese cuisine." The system saves the input data, and at the same time, the emotion engine analyzes the user's facial expressions and voice to obtain emotional information.

[1531] Step 2:

[1532] Analysis of initial information and emotional information

[1533] The server analyzes the initial information and emotional information received from the user. The system identifies the user's needs and performs appropriate data processing and calculations accordingly.

[1534] input

[1535] Initial Information Data

[1536] Emotional information data

[1537] output

[1538] User needs data

[1539] Specific actions

[1540] The server analyzes the initial information and the emotional information to identify the user's desired activities, budget, and emotional characteristics. The analyzed information is used in the next step.

[1541] Step 3:

[1542] Generate optimal travel plans

[1543] The server uses generative AI models and other algorithms within the system to generate optimal travel plans based on the user's requests and emotions.

[1544] input

[1545] User needs data

[1546] output

[1547] Generated travel plan data

[1548] Specific actions

[1549] The server uses the generative AI model to generate a travel plan like this:

[1550] Dates: August 1st to August 4th

[1551] Accommodation: Japanese-style inn in Kyoto city

[1552] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[1553] Restaurant: Japanese restaurant

[1554] Transportation: Shinkansen round-trip ticket

[1555] Based on emotional information, activities that the user is particularly looking forward to are prioritized in the plan.

[1556] Step 4:

[1557] Making various reservations

[1558] Based on the generated travel plan, the server connects with various reservation services and executes the reservation process, specifically booking transportation, accommodation, dining, and activities all at once.

[1559] input

[1560] Generated travel plan data

[1561] output

[1562] Various reservation confirmation data

[1563] Specific actions

[1564] The server connects to the "Reservation Service API" and performs the following reservation:

[1565] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[1566] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[1567] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[1568] Transportation: Shinkansen round-trip ticket

[1569] Step 5:

[1570] Promotional video generation

[1571] The server collects itinerary details and sentiment information and uses multimodal AI to generate promotional videos that highlight activities and attractions that the user has expressed particular interest in.

[1572] input

[1573] Travel Plan Data

[1574] Emotional information data

[1575] output

[1576] Generated promotional video

[1577] Specific actions

[1578] The server selects footage and audio based on the details of the itinerary and runs a video editing algorithm. The multimodal AI-generated footage includes highlights of:

[1579] A morning visit scene at the beautiful Kinkakuji Temple

[1580] Enjoyable matcha experience

[1581] Relaxing time at a Japanese-style inn

[1582] Delicious Japanese food scene

[1583] Based on emotional information, scenes of activities that the user is particularly looking forward to are included with priority.

[1584] Step 6:

[1585] Check user plans and prepare for trip

[1586] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[1587] input

[1588] Generated itinerary and booking confirmation data

[1589] Generated promotional video

[1590] output

[1591] User Readiness

[1592] Specific actions

[1593] The user clicks on the provided link, watches the promotional video, and feels, "After watching the video, I'm really excited. I'll start preparing for my trip." Based on this, the user makes specific preparations for the trip.

[1594] Through these steps, the system supports users through a series of steps, from entering initial information to creating a travel plan, various reservations, providing promotional videos, and final confirmation and preparation, providing a personalized travel experience.

[1595] (Application example 2)

[1596] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1597] In recent years, as demand for food delivery has increased, there is a demand for personalized meal suggestions tailored to user needs and more efficient ordering procedures. It is also important to provide an experience that allows users to enjoy ordering, rather than simply ordering a meal. Current food delivery systems do not consider users' emotional information when making ordering suggestions, nor do they foster anticipation through promotional media, making it difficult to improve the user experience.

[1598] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving initial information regarding a meal order from a user, means for generating an optimal food plan based on the initial information, means for executing a reservation procedure in cooperation with various reservation services based on the food plan, means for generating promotional media based on the food plan, means for acquiring emotional information of the user using an emotion engine and proposing a personalized plan, and means for providing the promotional media generated based on the emotional information. This makes it possible to propose personalized food delivery plans that take the user's emotions into consideration, enabling the user to have an enjoyable and improved ordering experience.

[1599] "User" refers to the end user who uses the system.

[1600] "Initial information" refers to basic information provided by the user, such as the destination, itinerary, number of people, budget, desired activities, and dining style.

[1601] A "travel plan" refers to a travel schedule or itinerary that is generated based on the above initial information and emotional information and is optimized to meet the user's needs.

[1602] "Reservation process" refers to the process of making reservations for accommodation, transportation, restaurants, activities, etc. in one go, in collaboration with various reservation services based on your travel plans.

[1603] "Promotional media" refers to video or video content created with the purpose of increasing user interest or emotion.

[1604] An "emotion engine" refers to an algorithm or program that recognizes emotions from a user's facial expressions and voice and acquires emotional information.

[1605] A "personalized plan" refers to an itinerary or proposal that is individually optimized to reflect the user's emotional information.

[1606] "Multimodal artificial intelligence technology" refers to artificial intelligence technology that integrates and processes multiple types of data (text, images, audio, etc.).

[1607] A "server" refers to a computer system that processes and stores data on the cloud and performs various system functions.

[1608] The following describes an embodiment of the present invention. The system generates a personalized food plan when a user places a meal order, enabling efficient ordering. Furthermore, by combining an emotion engine, the system generates suggestions and promotional media based on the user's emotions, improving the ordering experience.

[1609] 1. Input of initial information and emotional information

[1610] User Action

[1611] Users launch the food delivery app on their smartphone and enter initial information about their meal in the chat box displayed on the interface screen. This information includes the desired food category, budget, and number of people. The emotion engine then recognizes the user's emotions from their facial expressions and voice, and acquires their emotional information.

[1612] Specific examples

[1613] A user inputs, "I want to eat pizza and pasta for three people within a budget of 5,000 yen." The emotion engine analyzes the user's facial expressions and voice when they input, and interprets the user's emotions.

[1614] 2. Generate a food plan

[1615] Server Processing

[1616] The server analyzes the initial information and emotional information received from the user to identify their needs. It then uses a generative AI model (e.g., GPT-4) within the system to generate an optimal meal plan based on the user's requests and emotions. The meal plan includes suggested restaurants and specific menu items, with priority given to suggestions that enhance the user's emotions.

[1617] Specific examples

[1618] The server generates a food plan like this:

[1619] Restaurant: Popular Italian restaurant

[1620] Menu: Margherita pizza, carbonara pasta

[1621] 3. Making reservations and ordering

[1622] Server Processing

[1623] Based on the generated food plan, the server connects with various services to carry out the ordering process, including sending the order details to the restaurant and handling the payment process all at once.

[1624] Specific examples

[1625] The server executes the following procedure through the "Order Service API":

[1626] Send order to restaurant: Margherita pizza (3 slices), Carbonara pasta (3 servings)

[1627] Payment process

[1628] 4. Promotional Media Creation

[1629] Server Processing

[1630] The server collects details of the food plan and emotional information, and generates promotional media using multimodal artificial intelligence technology (e.g., DALL-E). This media introduces the menu and restaurant atmosphere that the user is particularly interested in, building anticipation.

[1631] Specific examples

[1632] Promotional media generated by the server using multimodal artificial intelligence technology includes the following highlights:

[1633] Beautiful pizza being baked

[1634] Delicious looking pasta being served

[1635] 5. User plan confirmation and order confirmation

[1636] User Action

[1637] The user checks the provided food plan and reservation confirmation information, and views the promotional media to confirm and confirm the order details.

[1638] Specific examples

[1639] The user clicks on the provided link, watches the promotional media, and decides to "confirm my order from this menu."

[1640] As described above, the system of the present invention allows users to smoothly create a meal plan that suits their preferences and easily complete the ordering process. Furthermore, by utilizing the emotion engine, it is possible to provide more personalized suggestions and further increase anticipation before ordering.

[1641] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1642] Step 1:

[1643] The user launches the food delivery app on their smartphone and enters initial information about the meal in the chat box displayed on the interface screen. This information includes the desired food category, budget, and number of people. The emotion engine recognizes emotions from the user's facial expressions and voice and acquires emotional information. The entered information is sent to the server.

[1644] input:

[1645] Desired food category

[1646] budget

[1647] Number of people

[1648] Facial expression data

[1649] Audio data

[1650] output:

[1651] Initial information

[1652] emotional information

[1653] Specific behavior:

[1654] The user enters, "I want to eat pizza and pasta for three people within a budget of 5,000 yen."

[1655] The emotion engine recognizes facial expressions and voices through the smartphone's camera and microphone.

[1656] The server receives the initial information and emotion information.

[1657] Step 2:

[1658] The server analyzes the initial information and emotional information received from the user. Then, using a generative AI model (e.g., GPT-4) within the system, it generates an optimal meal plan based on the user's requests and emotions. Based on the analyzed data, it suggests the most suitable restaurant and specific menu.

[1659] input:

[1660] Initial information

[1661] emotional information

[1662] output:

[1663] Food Plan

[1664] Specific behavior:

[1665] The server analyzes the initial information and emotional information to identify the user's needs.

[1666] A generative AI model was used to generate a food plan that included Margherita pizza and carbonara pasta.

[1667] Step 3:

[1668] Based on the generated food plan, the server connects with various services to carry out the ordering process, including sending the order details to the restaurant and handling the payment process all at once.

[1669] input:

[1670] Food Plan

[1671] output:

[1672] Order Confirmation Information

[1673] Payment completion information

[1674] Specific behavior:

[1675] The server sends the order details to the restaurant via the "Order Service API."

[1676] Process payments and generate order confirmation and payment completion information.

[1677] Step 4:

[1678] The server collects details of the food plan and emotional information, and generates promotional media using multimodal artificial intelligence technology (e.g., DALL-E). This media introduces the menu and restaurant atmosphere that the user is particularly interested in, building anticipation.

[1679] input:

[1680] Food Plan

[1681] emotional information

[1682] output:

[1683] Promotional Media

[1684] Specific behavior:

[1685] The server uses multimodal artificial intelligence technology to generate compelling images of pizza and pasta.

[1686] Create links to provide promotional media to users.

[1687] Step 5:

[1688] The user checks the provided food plan and reservation confirmation information, watches the promotional media, and confirms and confirms the order details. If satisfied, the user finally confirms the order.

[1689] input:

[1690] Promotional Media

[1691] Order Confirmation Information

[1692] output:

[1693] Final order confirmation information

[1694] Specific behavior:

[1695] User clicks on the provided link to view the promotional media.

[1696] After watching, confirm your order by clicking "Confirm your order from this menu."

[1697] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1698] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1699] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1700] [Fourth embodiment]

[1701] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1702] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1703] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1704] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1705] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1706] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1707] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1708] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1709] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1710] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1711] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1712] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1713] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1714] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. It also generates promotional videos to build excitement before the trip.

[1715] 1. Initial settings and information entry

[1716] User Action

[1717] Users launch the travel planning app on their smartphone or PC and enter initial information about their trip plans in the chat box displayed on the interface screen, including the destination, dates, number of people, budget, desired activities, and dining style.

[1718] Specific examples

[1719] A user types, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[1720] 2. Generate a travel plan

[1721] Server Processing

[1722] The server analyzes the initial information received from the user to identify their needs. Then, using algorithms and AI within the system, it generates an optimal travel plan based on the user's requests. The travel plan includes transportation (airplane, bullet train, etc.), accommodation, restaurants, and activities.

[1723] Specific examples

[1724] The server generates an itinerary like this:

[1725] Dates: August 1st to August 4th

[1726] Accommodation: Japanese-style inn in Kyoto city

[1727] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[1728] Restaurant: Japanese restaurant

[1729] Transportation: Shinkansen round-trip ticket

[1730] 3. Making various reservations

[1731] Server Processing

[1732] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking flights, accommodations, restaurants, and activities all at once.

[1733] Specific examples

[1734] The server makes the following reservation through the "Reservation Service API":

[1735] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[1736] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[1737] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[1738] Transportation: Shinkansen round-trip ticket

[1739] 4. Generate promotional videos

[1740] Server Processing

[1741] The server generates a promotional video based on the travel plan. By using multimodal AI, the details of the travel plan are presented in video format and provided to the user.

[1742] Specific examples

[1743] The promotional video, generated by the server using multi-mode AI, includes the following highlights:

[1744] A morning visit scene at the beautiful Kinkakuji Temple

[1745] Enjoyable matcha experience

[1746] Relaxing time at a Japanese-style inn

[1747] Delicious meal scene at a Japanese restaurant

[1748] 5. User plan confirmation and travel preparation

[1749] User Action

[1750] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[1751] Specific examples

[1752] Users click on the provided link to watch the promotional video, which builds excitement before the trip. The user feels, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[1753] As described above, the system of the present invention allows users to smoothly create travel plans that suit their preferences and easily complete the reservation procedure. In addition, the system also provides promotional videos that build anticipation before the trip, making the entire travel planning process enjoyable.

[1754] The processing flow will be explained below.

[1755] Step 1:

[1756] A user launches a travel planning app on their smartphone or PC. The device displays the interface screen and a chat box.

[1757] Step 2:

[1758] The user enters initial information about their travel plans into the chat box, such as destination, dates, number of people, budget, desired activities and dining style, etc. The device then sends this information to the server.

[1759] Step 3:

[1760] The server analyzes the initial information received and identifies the user's needs. Specifically, it breaks down the input content into categories and prepares to retrieve the necessary information from the database.

[1761] Step 4:

[1762] The server uses algorithms and AI within the system to generate an optimal travel plan based on the user's requests, including details on transportation, accommodation, restaurants, and activities.

[1763] Step 5:

[1764] The server accesses various reservation services (airline tickets, hotels, restaurants, activities, etc.) to collect availability and price information. For example, it queries each reservation site through a "reservation service API."

[1765] Step 6:

[1766] Based on the information collected, the server selects the reservation options that best fit the user's requirements, and then finalizes the travel plan based on this selection.

[1767] Step 7:

[1768] Based on the details of the reservation options selected by the server, the reservation process is carried out in bulk through various reservation APIs, specifically for air tickets, accommodations, restaurants, and activities.

[1769] Step 8:

[1770] The server collects the reservation confirmation information and sends it to the terminal, which displays a reservation confirmation notice to the user.

[1771] Step 9:

[1772] The server collects the details of the itinerary and uses multimodal AI to generate a promotional video, which includes the trip's highlights and recommended spots.

[1773] Step 10:

[1774] The server saves the generated promotional video in cloud storage and provides the link to the user, who then receives the link to watch the video on the device.

[1775] Step 11:

[1776] Users can watch the promotional video from the provided link and get a sense of their trip, which helps them prepare for and build anticipation before their trip.

[1777] This is the specific process flow of the travel planning system. By using this system, users can easily create a travel plan that suits their preferences and smoothly proceed with reservations and preparations.

[1778] Example 1

[1779] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1780] Traditional travel planning requires users to go through multiple booking sites, which requires a lot of time and effort. In addition, there is a lack of information to build anticipation before a trip, which can make pre-trip preparations cumbersome.

[1781] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1782] In this invention, the server includes means for receiving initial information about a travel plan from a user, means for generating an optimal travel plan based on the initial information, means for executing reservation procedures in cooperation with various reservation services based on the travel plan, and means for providing the promotional video to the user to heighten pre-travel excitement. This allows users to efficiently carry out everything from planning a travel plan to making reservations and gathering pre-travel information using a single system.

[1783] "User" refers to an individual or organization making travel plans.

[1784] "Travel Itinerary" includes information and plans for traveling to a particular location.

[1785] "Initial Information" refers to basic information provided by the User, such as the travel destination, itinerary, number of people, budget, and desired activities and dining style.

[1786] "Travel Plan" refers to a plan generated based on the User's initial information, including details of transportation, accommodation, dining options, activities, etc.

[1787] "Reservation Process" refers to the process of making reservations for transportation, accommodation, dining, activities, etc.

[1788] A "promotional video" refers to a video generated based on a travel plan that outlines the attractions of a travel destination and activities.

[1789] "Multimodal AI" refers to artificial intelligence technology that integrates multiple data formats, such as text, images, and audio, and uses them to analyze and generate information.

[1790] "Server" refers to a computer system that receives and processes information based on user operations.

[1791] MODE FOR CARRYING OUT THE INVENTION

[1792] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. It also generates promotional videos to build excitement before the trip.

[1793] Initial setup and information entry

[1794] A user launches a travel planning app on a smartphone or PC. They enter initial information about their travel plans into a chat box displayed on the app's interface screen. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. This information is sent to the server using a data structure such as JSON.

[1795] Specific examples

[1796] A user types, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[1797] Generate a travel plan

[1798] The server uses a text analysis engine to analyze the initial information it receives. As a result of this analysis, it identifies the user's needs and generates an optimal travel plan using the system's algorithms and generative AI model. The generative AI model learns from past travel data and the user's preferences and proposes the optimal plan.

[1799] The travel plan includes transportation (airplane tickets, bullet trains, etc.), accommodations, restaurants, and activities. The generated plan is provided to the user through an interface for confirmation.

[1800] Specific examples

[1801] The server generates an itinerary like this:

[1802] Dates: August 1st to August 4th

[1803] Accommodation: Japanese-style inn in Kyoto city

[1804] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[1805] Restaurant: Japanese restaurant

[1806] Transportation: Shinkansen round-trip ticket

[1807] Making various reservations

[1808] The server then coordinates with various reservation services to complete the reservation process based on the generated travel plan. Using the reservation API, reservations for flights, accommodations, restaurants, and activities are made all at once. Once the reservation is complete, the information is sent to the user.

[1809] Specific examples

[1810] The server makes the following reservation through the "Reservation Service API":

[1811] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[1812] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[1813] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[1814] Transportation: Shinkansen round-trip ticket

[1815] Promotional video generation

[1816] The server uses multimodal AI to generate a promotional video based on the travel plan. The multimodal AI integrates multiple data formats, such as text, images, and audio, analyzes the information, and presents the details of the travel plan in video format. The generated promotional video is provided to the user.

[1817] Specific examples

[1818] The promotional video generated by Sarva using multimodal AI includes the following highlights:

[1819] A morning visit scene at the beautiful Kinkakuji Temple

[1820] Enjoyable matcha experience

[1821] Relaxing time at a Japanese-style inn

[1822] Delicious meal scene at a Japanese restaurant

[1823] Check user plans and prepare for trip

[1824] The user can review the provided travel plan and reservation confirmation information, and click on the provided link to watch a promotional video to further enhance their travel plans. At this stage, they can begin making any necessary preparations.

[1825] Specific examples

[1826] Users click on the provided link to watch the promotional video, which builds excitement before the trip. The user feels, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[1827] The system allows users to smoothly create a travel plan that suits their preferences, simplify the booking process, and even provides promotional videos to build excitement before the trip, making the whole travel planning process fun.

[1828] Prompt Sentence Examples

[1829] "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is under 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[1830] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1831] Step 1:

[1832] A user launches a travel planning app on their smartphone or PC. They enter initial information about their travel plans into a chat box displayed on the app's interface screen. Specifically, they enter the travel destination, itinerary, number of people, budget, desired activities, and dining style. This input information is converted into JSON-formatted data by the application and sent to the server.

[1833] Input: Initial information about the travel plan entered by the user

[1834] Output: Initial information sent to the server in JSON format.

[1835] Specific actions: The user enters, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[1836] Step 2:

[1837] The server launches a text analysis engine to analyze the initial information received. This engine uses natural language processing technology to extract information such as destination, itinerary, number of people, budget, activities, and dining style, and identifies each element. Based on this analysis, the system uses its algorithms and generative AI models to generate an optimal travel plan.

[1838] Input: Initial information in JSON format

[1839] Output: Analyzed travel plan elements and a travel plan based on them

[1840] Specific operation: The server extracts information such as "Kyoto," "August 1st to August 4th," "2 people," "200,000 yen," "matcha experience," "temple tour," and "Japanese cuisine" to create the optimal plan.

[1841] Step 3:

[1842] The server connects with various reservation services based on the generated travel plan. Specifically, it uses a reservation API to make reservations for flights, accommodations, restaurants, and activities all at once. Once the reservation is complete, the server notifies the user of the reservation confirmation information.

[1843] Input: Travel Plan

[1844] Output: Reservation confirmation information

[1845] Specific operation: The server executes the following reservation through the "Reservation Service API":

[1846] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[1847] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[1848] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[1849] Transportation: Shinkansen round-trip ticket

[1850] Step 4:

[1851] The server generates a promotional video based on the travel plan, activates multimodal AI, and integrates multiple data formats such as text, images, and audio to present the details of the travel plan in video format, and provides the generated promotional video to the user.

[1852] Input: Travel plan details

[1853] Output: Promotional video

[1854] What happens: The server generates a promotional video that includes the following:

[1855] A morning visit scene at the beautiful Kinkakuji Temple

[1856] Enjoyable matcha experience

[1857] Relaxing time at a Japanese-style inn

[1858] Delicious meal scene at a Japanese restaurant

[1859] Step 5:

[1860] The user confirms the provided travel plan and reservation confirmation information, and then clicks on the provided link to watch a promotional video to further enhance their travel plans and make the necessary preparations.

[1861] Input: Promotional video link and booking confirmation information

[1862] Output: Specific actions that users can take to prepare and expectations before the trip

[1863] Specific behavior: The user clicks on the provided link, watches the promotional video, and feels, "After watching the video, I'm really excited. I'll start preparing for my trip."

[1864] (Application example 1)

[1865] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1866] Conventional travel planning and reservation systems require users to individually visit various reservation sites to make reservations, which takes a lot of time and effort. Furthermore, the system lacks sufficient promotional content to build excitement during the travel planning process, resulting in low user satisfaction. Furthermore, local guide information and gourmet spot information are scattered across multiple platforms, making it difficult for users to efficiently obtain the information they need.

[1867] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1868] In this invention, the server includes means for receiving initial information about a travel plan from a user, means for generating an optimal travel plan based on the initial information, means for executing a reservation procedure in cooperation with various reservation services based on the travel plan, means for generating and distributing promotional content based on the travel plan, and means for including guide content for each travel destination and content introducing gourmet spots in the promotional content. This allows users to easily and efficiently plan a travel plan and complete the reservation procedure, and further enables them to obtain promotional content and local information that will increase their excitement for the trip.

[1869] "User" means any person or entity that uses the System to make travel plans.

[1870] A "trip plan" is the overall schedule and content of a trip generated based on the destination, dates, number of people, budget, and desired activities and dining style.

[1871] "Initial information" refers to information such as destination, dates, number of people, budget, desired activities and dining style that a user inputs into the system when making travel plans.

[1872] A "travel plan" is a detailed travel schedule including transportation, accommodation, restaurants, activities, etc., generated based on initial information.

[1873] A "reservation service" is a service that provides reservations for transportation, accommodation, restaurants, activities, etc. via the Internet.

[1874] "Promotional content" refers to content that includes visual and audio information created to attractively introduce the contents of a travel plan.

[1875] "Guide content" refers to explanatory and guidance information about tourist attractions, culture, and activities related to a travel destination.

[1876] "Gourmet spots" refers to information about restaurants and types of cuisine that are accessible at a travel destination.

[1877] "Generative AI model" means an artificial intelligence model used to automatically generate promotional content based on initial information and travel plans.

[1878] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations in one go. It also provides promotional content to build excitement before a trip.

[1879] 1. Initial settings and information entry

[1880] User operations

[1881] Users launch the travel planning app on their smartphone or PC and enter initial information about their trip plans in the chat box displayed on the interface screen, including the destination, dates, number of people, budget, desired activities, and dining style.

[1882] Specific examples

[1883] For example, a user might enter, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine."

[1884] 2. Generate a travel plan

[1885] Server Processing

[1886] The server analyzes the initial information received from the user to identify their needs, then uses algorithms and generative AI models within the system to generate an optimal travel plan based on the user's requests, including transportation, accommodation, restaurants, and activities.

[1887] Specific examples

[1888] The server generates an itinerary as follows:

[1889] Dates: August 1st to August 4th

[1890] Accommodation: Japanese-style inn in Kyoto city

[1891] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[1892] Restaurant: Japanese restaurant

[1893] Transportation: Shinkansen round-trip ticket

[1894] 3. Making various reservations

[1895] Server Processing

[1896] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking transportation, accommodation, restaurants, and activities all at once.

[1897] Specific examples

[1898] The server makes the following reservation through the "Reservation Service API":

[1899] Accommodation: Japanese-style inn in Kyoto city (August 1st to August 4th)

[1900] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[1901] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[1902] Transportation: Shinkansen round-trip ticket

[1903] 4. Generate promotional content

[1904] Server Processing

[1905] The server generates and delivers promotional content based on the travel plan. This content presents the travel plan details to the user using visual and audio information. Using a generative AI model, engaging and personalized multimedia content is generated.

[1906] Specific examples

[1907] For example, a server-generated promotional video includes the following highlights:

[1908] A morning visit scene at the beautiful Kinkakuji Temple

[1909] Enjoyable matcha experience

[1910] Relaxing time at a Japanese-style inn

[1911] Delicious meal scene at a Japanese restaurant

[1912] 5. User plan confirmation and travel preparation

[1913] User operations

[1914] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[1915] Specific examples

[1916] Users click on the provided link to watch the promotional video, which builds excitement before the trip. They then feel, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[1917] Hardware and software used

[1918] The following hardware and software are used to realize this system.

[1919] Hardware: Smartphones, PCs, servers

[1920] Software: Python, REST API, generative AI models

[1921] Prompt Sentence Examples

[1922] Destination: Kyoto

[1923] Date: "2023-08-01" to "2023-08-04"

[1924] Number of people: 2 people

[1925] Budget: 200,000 yen

[1926] Activities: Matcha experience, temple tours, Japanese cuisine

[1927] As described above, the system of the present invention allows users to smoothly create travel plans that suit their preferences and easily complete the reservation procedure. Furthermore, it also provides promotional content that builds anticipation before the trip, making the entire travel planning process enjoyable.

[1928] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1929] Step 1:

[1930] A user launches a travel planning app on their smartphone or PC and enters initial information about their trip plan in a chat box displayed on the interface screen, including the destination, dates, number of people, budget, desired activities, and dining style.

[1931] Input: Initial information entered by the user (e.g., destination, dates, number of people, budget, activities)

[1932] Output: Initial information sent to the server

[1933] Specific actions: The user types in the chat box, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is under 200,000 yen, and we would like to experience matcha tea, visit temples, and enjoy Japanese cuisine."

[1934] Step 2:

[1935] The server analyzes the initial information received to identify the user's needs, then uses the system's algorithms and generative AI models to generate an optimal travel plan based on the user's requests.

[1936] Input: Initial information (destination, dates, number of people, budget, activities)

[1937] Output: Optimal travel plan (transportation, accommodation, restaurants, activities)

[1938] Specific operation: The server analyzes "Destination: Kyoto, Dates: August 1st to August 4th, Number of people: 2, Budget: 200,000 yen, Activities: Matcha experience, Temple tour, Japanese cuisine" and generates the optimal travel plan.

[1939] Step 3:

[1940] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, including reservations for transportation, accommodation, restaurants, and activities.

[1941] Input: Generated itinerary

[1942] Output: Confirmed reservation information

[1943] Specific operation: The server makes a reservation through the "Reservation Service API" for "a Japanese-style inn in Kyoto City (August 1st to August 4th), a matcha tea experience (August 2nd from 14:00), a visit to Kinkakuji Temple (August 3rd from 10:00), and a round-trip Shinkansen ticket."

[1944] Step 4:

[1945] The server generates and distributes promotional content based on the travel plan, which presents the details of the travel plan to the user using visual and audio information.

[1946] Input: Generated itinerary

[1947] Output: Promotional content (videos, guide content, gourmet spot introductions)

[1948] Specific operation: The server uses the generative AI model to generate a promotional video including scenes of the beautiful Kinkaku-ji Temple, a matcha tea experience, relaxing at a Japanese-style inn, and dining at a Japanese restaurant, and notifies the user.

[1949] Step 5:

[1950] The user checks the provided travel plan and reservation confirmation information, views the promotional content, and makes the necessary travel preparations based on the content they have viewed.

[1951] Input: Travel itinerary, booking confirmation information, promotional content

[1952] Output: User's travel preparations

[1953] What happens: The user clicks on the provided link, watches the promotional video, and feels, "The video made me really excited. I'm going to start preparing for my trip."

[1954] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1955] The following describes an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it becomes possible to propose more personalized travel plans. The system also generates promotional videos to build anticipation before the trip.

[1956] 1. Initial settings and information entry

[1957] User Action

[1958] Users launch the travel planning app on their smartphone or PC and enter initial information about their travel plans in the chat box displayed on the interface screen. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. The emotion engine also recognizes emotions from the user's facial expressions and voice, and acquires emotional information.

[1959] Specific examples

[1960] A user inputs, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine." The emotion engine analyzes the user's facial expressions and voice when they input information, and interprets the emotions they are looking forward to.

[1961] 2. Generate a travel plan

[1962] Server Processing

[1963] The server analyzes the initial information and emotional information received from the user to identify their needs. Then, using algorithms and AI within the system, it generates an optimal travel plan based on the user's requests and emotions. The travel plan includes transportation (airplane tickets, bullet trains, etc.), accommodation, restaurants, and activities.

[1964] Specific examples

[1965] The server generates an itinerary like this:

[1966] Dates: August 1st to August 4th

[1967] Accommodation: Japanese-style inn in Kyoto city

[1968] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[1969] Restaurant: Japanese restaurant

[1970] Transportation: Shinkansen round-trip ticket

[1971] Based on emotional information, activities that the user is particularly looking forward to are prioritized.

[1972] 3. Making various reservations

[1973] Server Processing

[1974] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking flights, accommodations, restaurants, and activities all at once.

[1975] Specific examples

[1976] The server makes the following reservation through the "Reservation Service API":

[1977] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[1978] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[1979] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[1980] Transportation: Shinkansen round-trip ticket

[1981] 4. Generate promotional videos

[1982] Server Processing

[1983] The server collects travel plan details and sentiment information and uses multimodal AI to generate a promotional video, highlighting activities and attractions that the user has expressed particular interest in, further building anticipation.

[1984] Specific examples

[1985] The promotional video generated by Sarva using multimodal AI includes the following highlights:

[1986] A morning visit scene at the beautiful Kinkakuji Temple

[1987] Enjoyable matcha experience

[1988] Relaxing time at a Japanese-style inn

[1989] Delicious meal scene at a Japanese restaurant

[1990] Based on emotional information, scenes of activities that the user is particularly looking forward to are included with priority.

[1991] 5. User plan confirmation and travel preparation

[1992] User Action

[1993] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[1994] Specific examples

[1995] Users click on the provided link to watch the promotional video, which builds excitement before the trip. The user feels, "After watching the video, I'm really looking forward to it. I'll start preparing for my trip."

[1996] As described above, the system of the present invention allows users to smoothly create travel plans that suit their preferences and easily complete the reservation procedure. Furthermore, by using the emotion engine, it is possible to provide more personalized plans and further increase anticipation before the trip.

[1997] The processing flow will be explained below.

[1998] Step 1:

[1999] A user launches a travel planning app on their smartphone or PC. The device displays the interface screen and a chat box.

[2000] Step 2:

[2001] The user enters initial information about their travel plans (destination, dates, number of people, budget, desired activities and dining style) into the chat box, and the device sends this information to the server.

[2002] Step 3:

[2003] The device activates the emotion engine and acquires emotion information by analyzing the user's facial expressions and voice. The emotion engine then transmits the acquired emotion information to the server.

[2004] Step 4:

[2005] The server analyzes the initial information and emotional information received to identify the user's needs. Specifically, it centrally manages input content and emotional information and estimates what the user is looking forward to.

[2006] Step 5:

[2007] The server uses the system's algorithms and AI to generate an optimal travel plan based on the user's requests and feelings. The plan includes transportation (airplane, bullet train, etc.), accommodation, restaurants, and activities.

[2008] Step 6:

[2009] The server accesses various reservation services (airline tickets, hotels, restaurants, activities, etc.) to collect availability and price information. For example, it queries each reservation site through a "reservation service API."

[2010] Step 7:

[2011] Based on the information collected, the server selects the reservation options that best fit the user's requirements, and then finalizes the travel plan based on this selection.

[2012] Step 8:

[2013] Based on the details of the reservation options selected by the server, the reservation process is carried out in bulk through various reservation APIs, specifically for air tickets, accommodations, restaurants, and activities.

[2014] Step 9:

[2015] The server collects the reservation confirmation information and sends it to the terminal, which displays a reservation confirmation notice to the user.

[2016] Step 10:

[2017] The server collects itinerary details and sentiment information and uses multimodal AI to generate a promotional video that highlights activities and attractions that the user has expressed particular interest in.

[2018] Step 11:

[2019] The server saves the generated promotional video in cloud storage and provides the link to the user, who then receives the link to watch the video on the device.

[2020] Step 12:

[2021] Users can watch the promotional video from the provided link and get a sense of their trip, which helps them prepare and build anticipation before their trip.

[2022] Example 2

[2023] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2024] Conventional travel planning systems make it difficult for users to create travel plans based on their own emotions and preferences. Furthermore, few systems offer a single solution for everything from travel planning to various reservations, and there are few ways to build anticipation before a trip. This results in users spending a lot of time and effort on planning and booking trips, preventing smooth travel preparations.

[2025] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2026] In this invention, the server includes means for receiving initial information about a travel plan from a user, means for acquiring emotional information about the user, means for generating an optimal travel plan based on the initial information and emotional information, means for executing a reservation procedure in cooperation with various reservation services based on the travel plan, and means for generating a promotional video based on the travel plan. This makes it possible to create a personalized travel plan that matches the user's preferences and emotions, realize a smooth reservation procedure, and increase anticipation before the trip.

[2027] "User" means an individual or group that uses the System to make travel plans.

[2028] "Initial information" refers to basic information about travel plans such as destination, dates, number of people, budget, and desired activities and dining options.

[2029] "Emotional information" refers to data related to emotions obtained from the user's facial expressions, voice, etc.

[2030] "Travel Itinerary" refers to a plan of a trip including details of transportation, accommodation, dining and activities generated based on the initial information and sentiment information.

[2031] "Reservation Service" means a service that handles reservations for transportation, accommodation, dining, and activities.

[2032] "Promotional video" refers to video content that is generated based on travel plans and is produced to attract the user's interest while taking into account the user's emotional information.

[2033] A "generative AI model" refers to an artificial intelligence model that can process data in a variety of formats and generate new information and content.

[2034] The following is a detailed description of an embodiment of the present invention. This system allows users to smoothly and enjoyably create travel plans and make reservations all at once when planning a trip. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it becomes possible to propose more personalized travel plans. The system also generates promotional videos to build anticipation before the trip.

[2035] Initial setup and information entry

[2036] User Action

[2037] Users launch the travel planning app on their smartphone or PC and enter initial information about their travel plans in the chat box displayed on the interface screen. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. The emotion engine also recognizes emotions from the user's facial expressions and voice, and acquires emotional information.

[2038] Specific examples

[2039] A user inputs, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha experience, visiting temples, and Japanese cuisine." The emotion engine analyzes the user's facial expressions and voice when they input information, and interprets the emotions they are looking forward to.

[2040] Generate a travel plan

[2041] Server Processing

[2042] The server analyzes the initial information and emotional information received from the user to identify their needs. Then, using generative AI models and other algorithms within the system, it generates an optimal travel plan based on the user's needs and emotions. The travel plan includes transportation (airplane, bullet train, etc.), accommodation, dining, and activities.

[2043] Specific examples

[2044] The server generates an itinerary like this:

[2045] Dates: August 1st to August 4th

[2046] Accommodation: Japanese-style inn in Kyoto city

[2047] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[2048] Restaurant: Japanese restaurant

[2049] Transportation: Shinkansen round-trip ticket

[2050] Based on emotional information, activities that the user is particularly looking forward to are prioritized.

[2051] Making various reservations

[2052] Server Processing

[2053] Based on the generated travel plan, the server connects with various reservation services to carry out the reservation process, specifically booking transportation, accommodation, dining, and activities all at once.

[2054] Specific examples

[2055] The server makes the following reservation through the Reservation Service API:

[2056] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[2057] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[2058] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[2059] Transportation: Shinkansen round-trip ticket

[2060] Promotional video generation

[2061] Server Processing

[2062] The server collects travel plan details and emotional information and uses multimodal AI to generate promotional videos, highlighting activities and tourist attractions that the user has shown particular interest in, further building anticipation.

[2063] Specific examples

[2064] The promotional video, generated by Sarva using multimodal AI, includes the following highlights:

[2065] A morning visit scene at the beautiful Kinkakuji Temple

[2066] Enjoyable matcha experience

[2067] Relaxing time at a Japanese-style inn

[2068] Delicious meal scene at a Japanese restaurant

[2069] Based on emotional information, scenes of activities that the user is particularly looking forward to are included with priority.

[2070] Check user plans and prepare for trip

[2071] User Action

[2072] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[2073] Specific examples

[2074] The user clicks on the provided link, watches the promotional video, and feels, "After watching the video, I'm really excited. I'll start preparing for my trip."

[2075] (Example of a prompt)

[2076] "I'd like to plan a trip for two people to Kyoto for four days and three nights, with a budget of 200,000 yen or less. I'd like to experience matcha tea, visit temples, and enjoy Japanese cuisine."

[2077] As described above, the system utilizes an emotion engine and generative AI models to provide users with highly personalized travel plans and a smooth booking process, allowing them to prepare for their trip efficiently and enjoyably.

[2078] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2079] The specific flow of the program processing of this system

[2080] Step 1:

[2081] Entering initial user information

[2082] A user launches a travel planning app on a smartphone or PC. They enter initial information about their travel plans into a chat box displayed on the interface. This information includes the destination, itinerary, number of people, budget, desired activities, and dining style. Based on the input data, the emotion engine extracts emotional information from the user's facial expressions and voice.

[2083] input

[2084] Destination, dates, number of people, budget, desired activities and dining style

[2085] User's facial expressions and voice

[2086] output

[2087] Initial Information Data

[2088] Emotional information data

[2089] Specific actions

[2090] A user inputs, "We are planning a four-day, three-night trip to Kyoto for two people. Our budget is within 200,000 yen, and we would like to enjoy a matcha green tea experience, visiting temples, and Japanese cuisine." The system saves the input data, and at the same time, the emotion engine analyzes the user's facial expressions and voice to obtain emotional information.

[2091] Step 2:

[2092] Analysis of initial information and emotional information

[2093] The server analyzes the initial information and emotional information received from the user. The system identifies the user's needs and performs appropriate data processing and calculations accordingly.

[2094] input

[2095] Initial Information Data

[2096] Emotional information data

[2097] output

[2098] User needs data

[2099] Specific actions

[2100] The server analyzes the initial information and the emotional information to identify the user's desired activities, budget, and emotional characteristics. The analyzed information is used in the next step.

[2101] Step 3:

[2102] Generate optimal travel plans

[2103] The server uses generative AI models and other algorithms within the system to generate optimal travel plans based on the user's requests and emotions.

[2104] input

[2105] User needs data

[2106] output

[2107] Generated travel plan data

[2108] Specific actions

[2109] The server uses the generative AI model to generate a travel plan like this:

[2110] Dates: August 1st to August 4th

[2111] Accommodation: Japanese-style inn in Kyoto city

[2112] Activities: Matcha experience, visit to Kinkakuji Temple, tour of Fushimi Inari Shrine

[2113] Restaurant: Japanese restaurant

[2114] Transportation: Shinkansen round-trip ticket

[2115] Based on emotional information, activities that the user is particularly looking forward to are prioritized in the plan.

[2116] Step 4:

[2117] Making various reservations

[2118] Based on the generated travel plan, the server connects with various reservation services and executes the reservation process, specifically booking transportation, accommodation, dining, and activities all at once.

[2119] input

[2120] Generated travel plan data

[2121] output

[2122] Various reservation confirmation data

[2123] Specific actions

[2124] The server connects to the "Reservation Service API" and performs the following reservation:

[2125] Accommodation: Japanese-style inn in Kyoto City (August 1st to August 4th)

[2126] Activities: Matcha experience (August 2nd, 14:00~), Kinkakuji Temple visit (August 3rd, 10:00~)

[2127] Restaurant: Japanese restaurant (August 2nd, 7:00 PM)

[2128] Transportation: Shinkansen round-trip ticket

[2129] Step 5:

[2130] Promotional video generation

[2131] The server collects itinerary details and sentiment information and uses multimodal AI to generate promotional videos that highlight activities and attractions that the user has expressed particular interest in.

[2132] input

[2133] Travel Plan Data

[2134] Emotional information data

[2135] output

[2136] Generated promotional video

[2137] Specific actions

[2138] The server selects footage and audio based on the details of the itinerary and runs a video editing algorithm. The multimodal AI-generated footage includes highlights of:

[2139] A morning visit scene at the beautiful Kinkakuji Temple

[2140] Enjoyable matcha experience

[2141] Relaxing time at a Japanese-style inn

[2142] Delicious Japanese food scene

[2143] Based on emotional information, scenes of activities that the user is particularly looking forward to are included with priority.

[2144] Step 6:

[2145] Check user plans and prepare for trip

[2146] The user checks the provided travel plan and reservation confirmation information, watches the promotional video to expand their travel imagination, and proceeds with the necessary preparations.

[2147] input

[2148] Generated itinerary and booking confirmation data

[2149] Generated promotional video

[2150] output

[2151] User Readiness

[2152] Specific actions

[2153] The user clicks on the provided link, watches the promotional video, and feels, "After watching the video, I'm really excited. I'll start preparing for my trip." Based on this, the user makes specific preparations for the trip.

[2154] Through these steps, the system supports users through a series of steps, from entering initial information to creating a travel plan, various reservations, providing promotional videos, and final confirmation and preparation, providing a personalized travel experience.

[2155] (Application example 2)

[2156] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2157] In recent years, as demand for food delivery has increased, there is a demand for personalized meal suggestions tailored to user needs and more efficient ordering procedures. It is also important to provide an experience that allows users to enjoy ordering, rather than simply ordering a meal. Current food delivery systems do not consider users' emotional information when making ordering suggestions, nor do they foster anticipation through promotional media, making it difficult to improve the user experience.

[2158] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving initial information regarding a meal order from a user, means for generating an optimal food plan based on the initial information, means for executing a reservation procedure in cooperation with various reservation services based on the food plan, means for generating promotional media based on the food plan, means for acquiring emotional information of the user using an emotion engine and proposing a personalized plan, and means for providing the promotional media generated based on the emotional information. This makes it possible to propose personalized food delivery plans that take the user's emotions into consideration, enabling the user to have an enjoyable and improved ordering experience.

[2159] "User" refers to the end user who uses the system.

[2160] "Initial information" refers to basic information provided by the user, such as the destination, itinerary, number of people, budget, desired activities, and dining style.

[2161] A "travel plan" refers to a travel schedule or itinerary that is generated based on the above initial information and emotional information and is optimized to meet the user's needs.

[2162] "Reservation process" refers to the process of making reservations for accommodation, transportation, restaurants, activities, etc. in one go, in collaboration with various reservation services based on your travel plans.

[2163] "Promotional media" refers to video or video content created with the purpose of increasing user interest or emotion.

[2164] An "emotion engine" refers to an algorithm or program that recognizes emotions from a user's facial expressions and voice and acquires emotional information.

[2165] A "personalized plan" refers to an itinerary or proposal that is individually optimized to reflect the user's emotional information.

[2166] "Multimodal artificial intelligence technology" refers to artificial intelligence technology that integrates and processes multiple types of data (text, images, audio, etc.).

[2167] A "server" refers to a computer system that processes and stores data on the cloud and performs various system functions.

[2168] The following describes an embodiment of the present invention. The system generates a personalized food plan when a user places a meal order, enabling efficient ordering. Furthermore, by combining an emotion engine, the system generates suggestions and promotional media based on the user's emotions, improving the ordering experience.

[2169] 1. Input of initial information and emotional information

[2170] User Action

[2171] Users launch the food delivery app on their smartphone and enter initial information about their meal in the chat box displayed on the interface screen. This information includes the desired food category, budget, and number of people. The emotion engine then recognizes the user's emotions from their facial expressions and voice, and acquires their emotional information.

[2172] Specific examples

[2173] A user inputs, "I want to eat pizza and pasta for three people within a budget of 5,000 yen." The emotion engine analyzes the user's facial expressions and voice when they input, and interprets the user's emotions.

[2174] 2. Generate a food plan

[2175] Server Processing

[2176] The server analyzes the initial information and emotional information received from the user to identify their needs. It then uses a generative AI model (e.g., GPT-4) within the system to generate an optimal meal plan based on the user's requests and emotions. The meal plan includes suggested restaurants and specific menu items, with priority given to suggestions that enhance the user's emotions.

[2177] Specific examples

[2178] The server generates a food plan like this:

[2179] Restaurant: Popular Italian restaurant

[2180] Menu: Margherita pizza, carbonara pasta

[2181] 3. Making reservations and ordering

[2182] Server Processing

[2183] Based on the generated food plan, the server connects with various services to carry out the ordering process, including sending the order details to the restaurant and handling the payment process all at once.

[2184] Specific examples

[2185] The server executes the following procedure through the "Order Service API":

[2186] Send order to restaurant: Margherita pizza (3 slices), Carbonara pasta (3 servings)

[2187] Payment process

[2188] 4. Promotional Media Creation

[2189] Server Processing

[2190] The server collects details of the food plan and emotional information, and generates promotional media using multimodal artificial intelligence technology (e.g., DALL-E). This media introduces the menu and restaurant atmosphere that the user is particularly interested in, building anticipation.

[2191] Specific examples

[2192] Promotional media generated by the server using multimodal artificial intelligence technology includes the following highlights:

[2193] Beautiful pizza being baked

[2194] Delicious looking pasta being served

[2195] 5. User plan confirmation and order confirmation

[2196] User Action

[2197] The user checks the provided food plan and reservation confirmation information, and views the promotional media to confirm and confirm the order details.

[2198] Specific examples

[2199] The user clicks on the provided link, watches the promotional media, and decides to "confirm my order from this menu."

[2200] As described above, the system of the present invention allows users to smoothly create a meal plan that suits their preferences and easily complete the ordering process. Furthermore, by utilizing the emotion engine, it is possible to provide more personalized suggestions and further increase anticipation before ordering.

[2201] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2202] Step 1:

[2203] The user launches the food delivery app on their smartphone and enters initial information about the meal in the chat box displayed on the interface screen. This information includes the desired food category, budget, and number of people. The emotion engine recognizes emotions from the user's facial expressions and voice and acquires emotional information. The entered information is sent to the server.

[2204] input:

[2205] Desired food category

[2206] budget

[2207] Number of people

[2208] Facial expression data

[2209] Audio data

[2210] output:

[2211] Initial information

[2212] emotional information

[2213] Specific behavior:

[2214] The user enters, "I want to eat pizza and pasta for three people within a budget of 5,000 yen."

[2215] The emotion engine recognizes facial expressions and voices through the smartphone's camera and microphone.

[2216] The server receives the initial information and emotion information.

[2217] Step 2:

[2218] The server analyzes the initial information and emotional information received from the user. Then, using a generative AI model (e.g., GPT-4) within the system, it generates an optimal meal plan based on the user's requests and emotions. Based on the analyzed data, it suggests the most suitable restaurant and specific menu.

[2219] input:

[2220] Initial information

[2221] emotional information

[2222] output:

[2223] Food Plan

[2224] Specific behavior:

[2225] The server analyzes the initial information and emotional information to identify the user's needs.

[2226] A generative AI model was used to generate a food plan that included Margherita pizza and carbonara pasta.

[2227] Step 3:

[2228] Based on the generated food plan, the server connects with various services to carry out the ordering process, including sending the order details to the restaurant and handling the payment process all at once.

[2229] input:

[2230] Food Plan

[2231] output:

[2232] Order Confirmation Information

[2233] Payment completion information

[2234] Specific behavior:

[2235] The server sends the order details to the restaurant via the "Order Service API."

[2236] Process payments and generate order confirmation and payment completion information.

[2237] Step 4:

[2238] The server collects details of the food plan and emotional information, and generates promotional media using multimodal artificial intelligence technology (e.g., DALL-E). This media introduces the menu and restaurant atmosphere that the user is particularly interested in, building anticipation.

[2239] input:

[2240] Food Plan

[2241] emotional information

[2242] output:

[2243] Promotional Media

[2244] Specific behavior:

[2245] The server uses multimodal artificial intelligence technology to generate compelling images of pizza and pasta.

[2246] Create links to provide promotional media to users.

[2247] Step 5:

[2248] The user checks the provided food plan and reservation confirmation information, watches the promotional media, and confirms and confirms the order details. If satisfied, the user finally confirms the order.

[2249] input:

[2250] Promotional Media

[2251] Order Confirmation Information

[2252] output:

[2253] Final order confirmation information

[2254] Specific behavior:

[2255] User clicks on the provided link to view the promotional media.

[2256] After watching, confirm your order by clicking "Confirm your order from this menu."

[2257] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2258] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2259] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2260] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2261] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2262] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2263] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2264] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2265] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2266] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2267] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2268] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2269] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2270] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2271] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2272] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2273] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2274] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2275] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2276] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2277] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2278] The following is further disclosed regarding the above embodiment.

[2279] (Claim 1)

[2280] means for receiving initial travel plan information from a user;

[2281] means for generating an optimal travel plan based on the initial information;

[2282] A means for executing a reservation procedure in cooperation with various reservation services based on the travel plan;

[2283] The system includes means for generating a promotional video based on the travel plan.

[2284] (Claim 2)

[2285] A means for receiving the initial information, such as destination, itinerary, number of people, budget, and desired activities and dining style;

[2286] 2. The system of claim 1, wherein the means for executing the reservation process includes reserving air tickets, accommodations, restaurants, and activities.

[2287] (Claim 3)

[2288] 10. The system of claim 1, utilizing multimodal AI in generating the promotional video.

[2289] "Example 1"

[2290] (Claim 1)

[2291] means for receiving initial travel plan information from a user;

[2292] means for generating an optimal travel plan based on the initial information;

[2293] A means for executing a reservation procedure in cooperation with various reservation services based on the travel plan;

[2294] means for generating a promotional video based on the travel plan;

[2295] The system includes a means for providing the promotional video to users to increase their anticipation before the trip.

[2296] (Claim 2)

[2297] A means for receiving the initial information, such as a destination, a schedule, a number of people, a budget, and a desired activity and dining style;

[2298] 2. The system of claim 1, wherein the means for executing the reservation procedure includes reservations for transportation, accommodation, restaurants, and activities.

[2299] (Claim 3)

[2300] 10. The system of claim 1, utilizing multimodal AI in generating the promotional video.

[2301] "Application Example 1"

[2302] (Claim 1)

[2303] means for receiving initial travel plan information from a user;

[2304] means for generating an optimal travel plan based on the initial information;

[2305] A means for executing a reservation procedure in cooperation with various reservation services based on the travel plan;

[2306] means for generating and distributing promotional content based on the travel plan;

[2307] The system includes a means for including guide content for each travel destination and content introducing gourmet spots in the promotional content.

[2308] (Claim 2)

[2309] A means for receiving the initial information, such as destination, itinerary, number of people, budget, and desired activities and dining style;

[2310] 2. The system of claim 1, wherein the means for executing the reservation procedure includes reservations for transportation, accommodation, restaurants, and activities.

[2311] (Claim 3)

[2312] 10. The system of claim 1, wherein the generation and delivery of the promotional content utilizes a generative AI model.

[2313] "Example 2: Combining Emotion Engines"

[2314] (Claim 1)

[2315] means for receiving initial travel plan information from a user;

[2316] A means for acquiring emotion information of a user;

[2317] means for generating an optimal travel plan based on the initial information and emotion information;

[2318] a means for executing a reservation procedure in cooperation with various reservation services based on the travel plan;

[2319] The system includes means for generating a promotional video based on the travel plan.

[2320] (Claim 2)

[2321] A means for receiving the initial information, such as destination, itinerary, number of people, budget, and desired activities and meal types;

[2322] means for recognizing emotions from facial expressions and voices of a user as the emotion information;

[2323] 10. The system of claim 1, wherein the means for executing the reservation process includes reservations for transportation, accommodations, dining, and activities.

[2324] (Claim 3)

[2325] The system of claim 1, wherein a generative AI model is utilized in generating the promotional video.

[2326] "Application example 2 when combining emotion engines"

[2327] (Claim 1)

[2328] means for receiving initial travel plan information from a user;

[2329] means for generating an optimal travel plan based on the initial information;

[2330] A means for executing a reservation procedure in cooperation with various reservation services based on the travel plan;

[2331] means for generating promotional media based on the travel plan;

[2332] A means for acquiring user emotion information using an emotion engine and proposing a personalized plan;

[2333] means for providing promotional media generated based on said emotion information;

[2334] A system including:

[2335] (Claim 2)

[2336] A means for receiving the initial information, such as the destination, schedule, number of people, budget, desired activities and dining style;

[2337] 2. The system of claim 1, wherein the means for executing the reservation process includes reservations for transportation, accommodations, dining, and activities.

[2338] (Claim 3)

[2339] The system of claim 1, wherein the generation of the promotional media utilizes multimodal artificial intelligence techniques. [Explanation of symbols]

[2340] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving initial travel plan information from a user; means for generating an optimal travel plan based on the initial information; A means for executing a reservation procedure in cooperation with various reservation services based on the travel plan; The system includes means for generating a promotional video based on the travel plan.

2. A means for receiving the initial information, such as destination, itinerary, number of people, budget, and desired activities and dining style; 2. The system of claim 1, wherein the means for executing the reservation process includes reserving airline tickets, accommodations, restaurants, and activities.

3. The system of claim 1 , wherein the generation of the promotional video utilizes multimodal AI.

Images