System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting user information; means for sending the user information to a server; means for the server learning the user information; means for the server generating a travel plan based on the user information; means for the server sending the travel plan generated to a user terminal; means for the user terminal displaying the travel plan; means for sending feedback from the user to the server; and means for providing support in real-time.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When planning a trip, many people have difficulty selecting a destination and gathering information. This difficulty is particularly exacerbated when they don't have time to visit a travel agency or when they want to create a travel plan tailored to their individual needs. Furthermore, they require real-time support during their trip due to issues or language barriers. Current travel planning services do not adequately address these issues, and a new system is needed to provide users with the ideal travel experience. [Means for solving the problem]

[0005] The present invention provides a system for proposing personalized travel plans. Specifically, the system includes a means for inputting user information and a means for transmitting the information to a server, where the server learns the information. The system also includes a means for the server to generate a travel plan based on the information and transmit the plan to the user's terminal. The user terminal further includes a means for displaying the generated travel plan and transmitting feedback from the user to the server. The system also includes a means for providing real-time support and a means for linking with translation services and other external services for support. In this way, it is possible to provide travel plans tailored to the user's individual needs and comprehensive support during the trip.

[0006] "User Information" includes profile information and preference information provided by users, such as name, age, gender, activities of interest, preferred meals, and budget.

[0007] "Server" refers to a central processing unit for receiving and analyzing user information, and generating and providing travel plans.

[0008] "Travel Plan" means a detailed plan including suggested destinations, itineraries, and activities generated based on a User's specific preferences and criteria.

[0009] "User terminal" means a device used by a user, such as a smartphone, tablet, or PC, that communicates with the server and includes the means to display travel plans and send feedback.

[0010] A "chatbot" is a software program that provides automated responses in real time to user inquiries.

[0011] "Translation Services" means third-party services that translate text and / or audio between different languages ​​to assist Users in communicating in their local language.

[0012] "Taxi Arrangement Service" means an external service that allows a User to call a taxi locally and supports the process of booking transportation.

[0013] "Database" means a collection of data in which the server stores user information, travel plans, and other related data for access and analysis as needed.

[0014] "Machine learning algorithm" refers to a computational technique used by the server to learn user information and generate a travel plan tailored to the user based on that information.

[0015] "Feedback" refers to information provided by a user regarding a proposed travel plan, including any evaluation or requests for revision. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is an AI concierge system that proposes personalized travel plans based on the user's preferences. The program processing of this system is explained in detail below in natural language.

[0038] 1. Collection of User Information

[0039] User

[0040] A user downloads the app and launches it for the first time.

[0041] Enter your profile information (name, age, gender) and travel preferences (favorite meals, activities you're interested in, budget).

[0042] Terminal

[0043] The input information is sent to the server.

[0044] server

[0045] The received user information is stored in a database and trained by a machine learning algorithm, which then creates a dataset to suggest optimal travel plans for the user.

[0046] 2. Generate a travel plan

[0047] User

[0048] Enter the conditions of your desired trip (departure date, destination, travel method, etc.).

[0049] Terminal

[0050] The entered conditions are sent to the server.

[0051] server

[0052] The system analyzes the received information and uses machine learning algorithms to generate a travel plan based on the user's preferences, taking into account weather information, local event information, budget, and other factors to create the optimal plan.

[0053] Terminal

[0054] The travel plan sent from the server is displayed to the user.

[0055] User

[0056] Review the proposed itinerary and provide corrections and feedback as needed.

[0057] 3. In-depth support and customization

[0058] User

[0059] Request specific requests or changes (e.g., a specific restaurant reservation, change of transportation).

[0060] Terminal

[0061] Sends the request to the server.

[0062] server

[0063] We analyze your request, suggest possible customizations, and make any necessary reservations and arrangements.

[0064] Terminal

[0065] Display the final customized plan to the user.

[0066] User

[0067] Review and finalize the plan.

[0068] 4. Real-time support during your trip

[0069] User

[0070] If you get lost or have any other inquiries while you're there, you can contact them through the chatbot.

[0071] Terminal

[0072] The query is sent to the server.

[0073] server

[0074] The chatbot analyzes the inquiry and generates the most appropriate answer, linking with translation services and taxi booking apps as needed.

[0075] Terminal

[0076] Display answers and support information to users.

[0077] User

[0078] Accept the support offered and contact us again if necessary.

[0079] Specific examples

[0080] For new users

[0081] 1. A user launches the app for the first time and enters their profile information and travel preferences.

[0082] 2. The device sends this information to the server.

[0083] 3. The server stores the information in a database and uses machine learning algorithms to learn from it.

[0084] 4. The user enters the desired travel conditions, and the terminal sends them to the server.

[0085] 5. The server generates an optimal travel plan and sends it to the device.

[0086] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[0087] 7. The server receives the feedback, modifies the plan as needed, and provides the final plan to the user.

[0088] Real-time consultation during your trip

[0089] 1. A user gets lost in the area and makes an inquiry through a chatbot.

[0090] 2. The device sends the query to the server.

[0091] 3. The server generates the best answer and, if necessary, connects with a translation service or taxi booking app.

[0092] 4. The device displays answers and support information to the user.

[0093] 5. The user accepts the support offered and contacts us again if necessary.

[0094] In this way, the system of the present invention proposes travel plans based on the user's preferences and provides comprehensive support during the trip, allowing the user to easily realize the ideal travel experience.

[0095] The processing flow will be explained below.

[0096] Step 1:

[0097] A user downloads the app and launches it for the first time.

[0098] Step 2:

[0099] The device displays an interface for entering the user's profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget).

[0100] Step 3:

[0101] The user enters the required information and presses the send button.

[0102] Step 4:

[0103] The terminal transmits the input information to the server.

[0104] Step 5:

[0105] The server stores the received user information in a database.

[0106] Step 6:

[0107] The server runs machine learning algorithms based on the stored user information to learn the user's preferences.

[0108] Step 7:

[0109] The user inputs the conditions of the desired trip (departure date, destination, travel mode, etc.).

[0110] Step 8:

[0111] The terminal transmits the input travel conditions to the server.

[0112] Step 9:

[0113] The server analyzes the travel conditions received and compares them with the user's preference data.

[0114] Step 10:

[0115] The server uses deep learning algorithms to generate the optimal travel plan (destinations, activities, accommodation, etc.).

[0116] Step 11:

[0117] The server transmits the generated travel plan to the terminal.

[0118] Step 12:

[0119] The terminal displays the travel plan sent from the server to the user.

[0120] Step 13:

[0121] The user reviews the proposed itinerary and enters corrections or feedback as necessary.

[0122] Step 14:

[0123] The device sends the user's feedback to the server.

[0124] Step 15:

[0125] The server analyzes the feedback and modifies the travel plan.

[0126] Step 16:

[0127] The server sends the final revised itinerary to the terminal.

[0128] Step 17:

[0129] The device displays the final revised plan to the user.

[0130] Step 18:

[0131] The user reviews and confirms the final plan.

[0132] Step 19:

[0133] If a user gets lost or needs information while in the area, they can make inquiries through the chatbot function.

[0134] Step 20:

[0135] The terminal sends the inquiry to the server.

[0136] Step 21:

[0137] The server uses a chatbot to analyze the inquiry and generate the most appropriate answer.

[0138] Step 22:

[0139] The server will connect with translation services and taxi booking apps as needed.

[0140] Step 23:

[0141] The server generates responses and support information and sends them to the device.

[0142] Step 24:

[0143] The device displays answers and support information to the user.

[0144] Step 25:

[0145] The user accepts the support offered and contacts the company again if necessary.

[0146] Through this series of steps, users can receive suggestions for optimal travel plans and receive support based on their requests while traveling with peace of mind.

[0147] Example 1

[0148] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0149] Conventional travel plan suggestion systems have difficulty in providing advanced customization based on users' preferences and individual needs, and do not provide sufficient real-time support during the trip. Furthermore, it is difficult to provide support in conjunction with external translation services or taxi booking services. This makes it difficult for users to achieve their ideal travel experience.

[0150] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0151] In this invention, the server includes a means for storing user information in a database and learning using a machine learning algorithm, a means for generating a travel plan based on the user's preferences, and a means for analyzing travel conditions and generating an optimal travel plan. This allows for personalized travel plans based on the user's preferences and needs. It can also accommodate customization requests and provide real-time support on-site, allowing users to enjoy an ideal travel experience.

[0152] "User Information" means information including a User's profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget).

[0153] "Server" means a computer system that stores and analyzes information submitted by users and generates travel plans.

[0154] A "database" is a structured collection of data for storing user information and travel plans.

[0155] A "machine learning algorithm" is a computational method for analyzing data based on user preferences and building predictive models.

[0156] A "travel plan" is a plan that includes travel schedules and suggestions generated based on the user's preferences and conditions.

[0157] "Travel conditions" refers to information such as the departure date, destination, and travel mode desired by the user when traveling.

[0158] A "customization request" is a request by a user to communicate specific wishes or modifications to the system.

[0159] "Real-time support" is a service that provides immediate support and assistance to problems and inquiries users may encounter while traveling.

[0160] A "chatbot" is a program that uses natural language processing technology to automatically respond to user questions and requests.

[0161] "External linkage services" are support functions provided by the system in collaboration with external services, such as translation services and taxi booking services.

[0162] This invention is an AI concierge system that proposes personalized travel plans based on the user's preferences. The program processing of this system is described in detail below.

[0163] Collection of User Information

[0164] When a user launches the app on their smartphone or tablet for the first time, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget). The device sends this information to the server. The server stores the received user information in a database and uses a machine learning algorithm (such as Scikit-Learn or TENSORFLOW (registered trademark)) to train the database. This information is used to create a dataset that can be used to suggest optimal travel plans to the user.

[0165] Generate a travel plan

[0166] The user enters the conditions of their desired trip (departure date, destination, travel mode, etc.) into the app. The device sends these conditions to the server. The server analyzes the received conditions and generates an optimal travel plan based on the user's preference data and travel conditions. In doing so, it uses weather information APIs and local event information APIs and also takes budget into consideration. The server sends the generated travel plan to the user's device, which then displays it to the user. The user can review the displayed travel plan and provide feedback if necessary.

[0167] In-depth support and customization

[0168] Users can request specific requests or changes through the app. For example, they can input requests to make a specific restaurant reservation or change their transportation method. The device sends these requests to the server. The server analyzes the request, suggests possible customizations, and makes the necessary reservations and arrangements. This allows for customization that meets the user's individual needs. The final customized plan is generated and displayed on the device. The user reviews and confirms the final plan.

[0169] Real-time support during your trip

[0170] If a user gets lost or needs other support while traveling, they can make an inquiry using the chatbot function within the app. The device sends the inquiry to the server. The server uses the chatbot to analyze the inquiry and generate the most appropriate answer. If necessary, it also connects with translation service APIs and taxi dispatch app APIs. This allows the user to receive the necessary support in real time. The generated answer and support information are displayed on the device, and the user can check and respond.

[0171] Specific examples

[0172] For new users

[0173] 1. A user launches the app for the first time and enters their profile information and travel preferences.

[0174] 2. The device sends this information to the server.

[0175] 3. The server stores the information in a database and uses machine learning algorithms to learn from it.

[0176] 4. The user enters the desired travel conditions, and the terminal sends them to the server.

[0177] 5. The server generates an optimal travel plan and sends it to the device.

[0178] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[0179] 7. The server receives the feedback, modifies the plan as needed, and provides the final plan to the user.

[0180] Real-time consultation during your trip

[0181] 1. A user gets lost in the area and makes an inquiry through a chatbot.

[0182] 2. The device sends the query to the server.

[0183] 3. The server generates the best answer and, if necessary, connects with a translation service or taxi booking app.

[0184] 4. The device displays answers and support information to the user.

[0185] 5. The user accepts the support offered and contacts us again if necessary.

[0186] The system allows users to create optimal travel plans tailored to their preferences and receive comprehensive support during their trip.

[0187] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0188] Program processing flow

[0189] Step 1: Enter your user information

[0190] input:

[0191] When a user launches the app for the first time, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget).

[0192] explanation:

[0193] A user downloads and launches the app on their smartphone or tablet.

[0194] Enter information such as your name, age, gender, and travel preferences into the in-app form.

[0195] The user presses the "Submit" button to confirm the entered information.

[0196] output:

[0197] A dataset of user profile information and preference information

[0198] Step 2: Submit your information

[0199] input:

[0200] User information entered in step 1

[0201] explanation:

[0202] The device sends the user's input information to the backend server via an HTTP POST request.

[0203] Information is transmitted using SSL / TLS encrypted communications.

[0204] output:

[0205] The server receives the user information.

[0206] Step 3: Database storage and machine learning

[0207] input:

[0208] The dataset of user information received in step 2

[0209] explanation:

[0210] The server stores the received user information in a database.

[0211] The database used is MySQL (registered trademark).

[0212] The server uses Python to analyze the received data using machine learning algorithms such as Scikit-Learn and TensorFlow, and performs learning.

[0213] A user preference profile is generated from the learning results.

[0214] output:

[0215] User preference profile

[0216] Step 4: Enter your travel requirements

[0217] input:

[0218] User's desired travel conditions (departure date, destination, travel type, etc.)

[0219] explanation:

[0220] The user enters their travel requirements into a dedicated form within the app.

[0221] Also consider options such as those for families with children or for active outdoorsy people.

[0222] After the user has entered all the required information, he clicks "Submit."

[0223] output:

[0224] Dataset of user desired travel conditions

[0225] Step 5: Sending conditions

[0226] input:

[0227] The data set of travel conditions entered in step 4

[0228] explanation:

[0229] The terminal sends the travel conditions entered by the user to the server via an HTTP POST request.

[0230] Communications are encrypted using SSL / TLS.

[0231] output:

[0232] Travel conditions received

[0233] Step 6: Create a travel plan

[0234] input:

[0235] User information and travel conditions collected in Steps 2 and 4

[0236] explanation:

[0237] The server generates the optimal travel plan based on the user's preference profile and travel conditions using weather information APIs and local event information APIs.

[0238] Travel plans are automatically generated using machine learning algorithms (such as TensorFlow).

[0239] Adjust your plan taking into account your budget.

[0240] output:

[0241] The perfect travel plan for you

[0242] Step 7: Submit and view your plan

[0243] input:

[0244] The itinerary generated in step 6

[0245] explanation:

[0246] The server sends the generated travel plan to the terminal as an HTTP response.

[0247] The device displays the received travel plan to the user.

[0248] output:

[0249] The itinerary displayed to the user

[0250] Step 8: Provide feedback

[0251] input:

[0252] User feedback information

[0253] explanation:

[0254] The user checks the proposed travel plan and, if necessary, inputs changes or corrections as feedback.

[0255] The terminal transmits the feedback information to the server.

[0256] output:

[0257] Feedback information dataset

[0258] Step 9: Submit a customization request

[0259] input:

[0260] Customization Request Information

[0261] explanation:

[0262] The user inputs a specific request (e.g., a specific restaurant reservation, a change of transportation method).

[0263] The terminal sends these requests to the server.

[0264] output:

[0265] Customization request information dataset

[0266] Step 10: Customization and Final Plan Generation

[0267] input:

[0268] Customization request information received in step 9

[0269] explanation:

[0270] The server analyzes the received request and adds any necessary customizations to the itinerary.

[0271] Make any necessary reservations and arrangements and generate your final itinerary.

[0272] output:

[0273] A customized ultimate itinerary

[0274] Step 11: Submit and review your final plan

[0275] input:

[0276] The final itinerary generated in step 10

[0277] explanation:

[0278] The server sends the final customized itinerary to the terminal as an HTTP response.

[0279] The device displays the final plan received to the user.

[0280] The user reviews and confirms the final plan.

[0281] output:

[0282] Final confirmed travel plans

[0283] Step 12: Real-time support during your trip

[0284] input:

[0285] Real-time support inquiries from users

[0286] explanation:

[0287] Users can use chatbots to make inquiries while traveling.

[0288] The terminal sends the inquiry to the server.

[0289] The server analyzes the data using a chatbot and generates the most appropriate answer.

[0290] If necessary, it will also link with translation service APIs and taxi booking app APIs.

[0291] The device displays generated answers and supporting information to the user.

[0292] output:

[0293] Responses to inquiries and support information

[0294] In this way, users can create the perfect travel plan tailored to their preferences and receive comprehensive support during their trip.

[0295] (Application example 1)

[0296] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0297] While conventional travel plan suggestion systems can provide personalized plans based on user preferences, they lack a way for users to visually visualize the atmosphere and locations of their travel destinations. Furthermore, providing real-time support during a trip requires users to use multiple apps simultaneously, which is inconvenient for users. There is a need to solve these problems and realize a more comprehensive travel experience.

[0298] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0299] In this invention, the server includes means for inputting user information, means for transmitting the user information to the server, means for the server to learn the user information, means for the server to generate a travel plan based on the user information, means for the server to transmit the generated travel plan to a user terminal, means for the user terminal to display the travel plan, means for transmitting feedback from the user to the server, means for providing real-time support, and means for providing a virtual reality tour based on the travel plan, thereby enabling the user to visually experience the suggested travel destinations in virtual reality and receive comprehensive real-time support, thereby enabling the user to plan and execute a more satisfying trip.

[0300] "User Information" means data relating to a user's profile information, travel preferences and travel requirements.

[0301] The "server" is a computer system that receives user information, learns and analyzes it, and generates and transmits the optimal travel plan.

[0302] "User terminal" means an electronic device, typically a smartphone or tablet, used to input user information and receive and display travel plans and support information.

[0303] "Virtual reality tour" is a feature that allows users to visually experience the atmosphere and locations of their travel destinations through virtual reality technology.

[0304] A "trip plan" is a plan that includes a suggested travel itinerary and destination details generated based on a user's preferences and conditions.

[0305] "Real-time support" is a support service that provides immediate response and assistance to users' inquiries and problems while traveling.

[0306] A "machine learning algorithm" is an algorithm that learns patterns based on collected user information and generates travel plans based on the results of that learning.

[0307] In order to put the present invention into practice, it is necessary to build a system in which multiple pieces of hardware and software work together. A specific embodiment of this system is shown below.

[0308] 1. Collection and storage of user information

[0309] When a user downloads and launches the application for the first time, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget). This information is sent from the device (e.g., smartphone) used to the server. The server stores the received user information in a database and uses machine learning algorithms to learn from it. This step forms the base dataset for providing personalized travel plans based on the user information.

[0310] 2. Generate a travel plan

[0311] When a user inputs the conditions for their desired travel plan (departure date, destination, travel mode, etc.), these conditions are also sent to the server via their device. The server analyzes the received conditions and uses a machine learning algorithm to generate a travel plan based on the user's preferences. This also takes into account factors such as weather information, local event information, and budget. The generated travel plan is sent from the server to the user's device and displayed to the user.

[0312] 3. Providing VR tours

[0313] After users confirm the proposed itinerary, they can experience a virtual reality (VR) tour of specific tourist spots or points. This function is available by installing an application on a VR headset or smartphone. For example, if a user selects a plan with "Kyoto" as the destination, they can virtually visit Kyoto's famous tourist spots.

[0314] 4. Real-time support during your trip

[0315] If a user gets lost or needs emergency assistance while traveling, they can receive real-time support using the chatbot within the application. When the user enters their inquiry, it is sent from the device to the server, which uses machine learning algorithms to generate the best answer. If necessary, it can also be linked to a translation service or a taxi booking app.

[0316] 5. Specific Examples

[0317] For new users, the flow is as follows:

[0318] 1. A user launches the app for the first time and enters their profile information and travel preferences.

[0319] 2. The device sends this information to the server.

[0320] 3. The server stores the information in a database and uses machine learning algorithms to learn from it.

[0321] 4. The user enters the desired travel conditions, and the device sends them to the server.

[0322] 5. The server generates an optimal travel plan and sends it to the device.

[0323] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[0324] 7. The server receives the feedback, modifies the plan as needed, and provides the final plan to the user.

[0325] As an example of real-time support during travel, here is a specific example of a user getting lost and making an inquiry through a chatbot:

[0326] 1. A user gets lost in the area and makes an inquiry through a chatbot.

[0327] 2. The device sends the query to the server.

[0328] 3. The server generates the best answer and, if necessary, connects with a translation service or taxi booking app.

[0329] 4. The device displays answers and support information to the user.

[0330] 5. The user accepts the support provided and contacts us again if necessary.

[0331] Examples of prompts include:

[0332] "After users download the app and launch it for the first time, they enter the following information:

[0333] Name: User A

[0334] Age: 30

[0335] Gender: Male

[0336] Favorite food: Japanese food

[0337] Interested in: Hiking

[0338] Budget: 50,000 yen

[0339] Next, enter your travel plan requirements:

[0340] Departure date: July 1, 2024

[0341] Destination: Kyoto

[0342] Travel type: Solo travel

[0343] By following these prompts, each function of the system will function properly, providing users with a personalized travel experience.

[0344] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0345] Step 1:

[0346] Collection of User Information

[0347] When a user launches the application for the first time, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget). The device sends this information to the server, which stores the received user information in a database and begins learning using machine learning algorithms. This process forms the base data set for suggesting personalized travel plans to the user.

[0348] Input: User profile information and travel preference data

[0349] Output: User information stored in the database

[0350] Step 2:

[0351] Enter travel plan conditions

[0352] The user inputs the travel plan conditions (departure date, destination, travel mode, etc.). This information is sent to the server via the device. The server analyzes the received conditions and uses machine learning algorithms to generate a travel plan based on the user's preferences. This also takes into account factors such as weather information, local event information, and budget.

[0353] Input: User's travel plan conditions

[0354] Output: Parsed itinerary data

[0355] Step 3:

[0356] Generate and display travel plans

[0357] The server sends the generated travel plan to the user's device, which then displays the plan to the user. The user reviews the proposed plan and provides feedback if necessary. This feedback is sent from the device to the server, which then revises the plan based on that feedback. The final plan is then provided to the user.

[0358] Input: Server-generated travel plan data, user feedback

[0359] Output: Finalized itinerary

[0360] Step 4:

[0361] Providing virtual reality tours

[0362] After reviewing the proposed itinerary, users can experience a virtual reality (VR) tour of specific tourist spots or points. VR content is sent to the device, and users can visually experience the content with a VR headset or smartphone. For example, if a user selects "Kyoto" as their destination, they can virtually visit Kyoto's famous tourist spots.

[0363] Input: Proposed itinerary and related VR content

[0364] Output: The virtual reality tour the user experiences

[0365] Step 5:

[0366] Real-time support during your trip

[0367] If a user gets lost or needs emergency assistance while traveling, they can receive real-time support from a chatbot within the application. When the user enters their inquiry, it is sent from the device to the server. The server uses a machine learning algorithm to generate the optimal answer and sends it to the device. If necessary, it can also be linked to a translation service or a taxi booking app.

[0368] Input: User's support request

[0369] Output: Answers and support information from the chatbot

[0370] The above steps will create a system that allows users to create personalized travel plans, experience virtual reality tours, and receive real-time support throughout their trip.

[0371] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0372] This invention is an AI concierge system that proposes personalized travel plans based on the user's preferences and emotions and provides real-time support. The system incorporates an emotion engine and has the function of adjusting plans and support content according to the user's emotions.

[0373] User information collection and emotion recognition

[0374] User

[0375] Download the app and launch it for the first time.

[0376] Enter your profile information (name, age, gender) and travel preferences (favorite meals, activities you're interested in, budget). The app is equipped with an emotion engine that recognizes the user's emotions through facial expressions and voice.

[0377] Terminal

[0378] The input information and emotion data are sent to the server.

[0379] server

[0380] The received user information and emotion data are stored in a database, which allows the system to learn the relationship between user preferences and emotions.

[0381] Generate a travel plan

[0382] User

[0383] Enter the conditions of your desired trip (departure date, destination, travel method, etc.).

[0384] Terminal

[0385] The entered travel conditions are sent to the server.

[0386] server

[0387] The system analyzes the user's preference and emotional data along with travel conditions to generate an optimal travel plan. Based on the emotional data, it selects activities that will relax the user and events that will excite them.

[0388] Terminal

[0389] The travel plan sent from the server is displayed to the user.

[0390] User

[0391] Review the proposed itinerary and provide corrections and feedback as needed.

[0392] Customization and Real-Time Support

[0393] User

[0394] Request specific requests or changes (e.g., a specific restaurant reservation, change of transportation).

[0395] Terminal

[0396] Send the user's request and emotion data to the server.

[0397] server

[0398] Analyzes the request along with emotional data and proposes optimal customization. Taking emotional data into consideration, adjusts plans to ensure user satisfaction.

[0399] Terminal

[0400] Display the final customized plan to the user.

[0401] User

[0402] Review and confirm the final plan.

[0403] Real-time support during your trip

[0404] User

[0405] If you get lost or have an emergency, you can contact us through the chatbot.

[0406] Terminal

[0407] The inquiry content and emotion data are sent to the server.

[0408] server

[0409] The chatbot analyzes the inquiry content and emotional data to generate the most appropriate response, and if necessary, connects with translation services or taxi booking apps to provide support that takes the user's emotions into consideration.

[0410] Terminal

[0411] Display answers and support information to users.

[0412] User

[0413] Accept the support offered and follow up if necessary.

[0414] Specific examples

[0415] For new users

[0416] 1. A user launches the app for the first time and enters their profile information and travel preferences. The emotion engine recognizes the user's facial expressions and voice to collect emotional data.

[0417] 2. The device sends information and emotion data to the server.

[0418] 3. The server stores the information and emotion data in a database and performs learning.

[0419] 4. The user enters the desired travel conditions, and the terminal sends them to the server.

[0420] 5. The server generates an optimal travel plan based on the conditions and emotion data.

[0421] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[0422] 7. The server analyzes the feedback and sentiment data, modifies the plan as needed, and provides the final plan.

[0423] Real-time consultation during your trip

[0424] 1. When a user gets lost in the area and makes an inquiry through the chatbot, the emotion engine detects the user's stress level.

[0425] 2. The device sends the inquiry and emotion data to the server.

[0426] 3. The server analyzes the inquiry and emotional data to generate the optimal response. If necessary, it connects with translation services or taxi booking apps to provide support that reduces the user's stress.

[0427] 4. The device displays answers and support information to the user.

[0428] 5. The user accepts the support offered and contacts us again if necessary.

[0429] In this way, the system of the present invention incorporating an emotion engine can provide personalized travel plans that take into account not only the user's preferences but also their emotions, providing comprehensive support during the trip.

[0430] The processing flow will be explained below.

[0431] Step 1:

[0432] A user downloads the app and launches it for the first time.

[0433] Step 2:

[0434] The device asks the user to enter their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget).

[0435] Step 3:

[0436] The user enters their profile information and travel preferences and hits the submit button.

[0437] Step 4:

[0438] The terminal transmits the input information to the server.

[0439] Step 5:

[0440] The server stores the received user information in a database.

[0441] Step 6:

[0442] The device uses an emotion engine to collect emotional data from the user's facial expressions and voice.

[0443] Step 7:

[0444] The device transmits the emotion data to the server.

[0445] Step 8:

[0446] The server runs a machine learning algorithm based on user information and emotional data to learn the relationship between user preferences and emotions.

[0447] Step 9:

[0448] The user inputs the conditions of the desired trip (departure date, destination, travel mode, etc.).

[0449] Step 10:

[0450] The terminal transmits the input travel conditions to the server.

[0451] Step 11:

[0452] The server analyzes the travel conditions received and generates an optimal travel plan based on them. Based on the emotional data, it selects activities that will relax or excite the user.

[0453] Step 12:

[0454] The server sends the generated travel plan to the user's terminal.

[0455] Step 13:

[0456] The device displays the travel plan to the user.

[0457] Step 14:

[0458] The user reviews the proposed itinerary and enters corrections or feedback as necessary.

[0459] Step 15:

[0460] The device sends the user's feedback to the server.

[0461] Step 16:

[0462] The server analyzes the feedback and modifies the travel plan, taking into account the emotional data.

[0463] Step 17:

[0464] The server sends the final revised itinerary to the terminal.

[0465] Step 18:

[0466] The device displays the final revised plan to the user.

[0467] Step 19:

[0468] The user reviews and confirms the final plan.

[0469] Step 20:

[0470] If a user gets lost or stressed while in the area, they can contact the chatbot to inquire.

[0471] Step 20A:

[0472] The device monitors the user's emotions in real time and transmits the emotion data along with the query to the server.

[0473] Step 21:

[0474] The server uses a chatbot to analyze the inquiry content and emotional data and generate the most appropriate answer.

[0475] Step 22:

[0476] The server will connect with translation services and taxi booking apps as needed to provide support that takes the user's emotions into consideration.

[0477] Step 23:

[0478] The server generates responses and support information and sends them to the device.

[0479] Step 24:

[0480] The device displays answers and support information to the user.

[0481] Step 25:

[0482] The user accepts the support offered and contacts the company again if necessary.

[0483] In this way, it becomes possible to provide personalized responses based on user preferences as well as emotional data, providing a more personalized travel experience and real-time support.

[0484] Example 2

[0485] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0486] Conventional travel planning systems have difficulty providing plans that fully take into account the user's preferences and emotions, and they also lack the ability to provide satisfactory real-time support. This means that users cannot receive appropriate support tailored to their individual needs and emotions, which can lead to a decrease in satisfaction during their trip.

[0487] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for learning user information and emotion data, means for generating a travel plan based on the user information and emotion data, and means for providing real-time support based on the emotion data. This makes it possible to provide a personalized travel plan based on the user's preferences and emotions and comprehensive support during the trip.

[0488] "User Information" refers to a user's profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget).

[0489] "Emotion data" refers to information about the emotional state recognized by the emotion engine through the user's facial expressions and voice.

[0490] "Server" refers to the central system that receives, stores, and analyzes user information and sentiment data, generates travel plans, and provides real-time support.

[0491] "Means of learning" refers to the algorithms and programs that allow the server to analyze user information and emotional data, understand the relationships between them, and use them to make future suggestions.

[0492] "Means for generating travel plans" refers to algorithms or programs for creating optimal travel plans based on travel conditions, user information, and emotional data.

[0493] "Means of providing real-time support" refers to systems and functions that take emotional data into account when a user makes an inquiry while traveling and immediately provide appropriate assistance or information.

[0494] "Means for sending feedback to the server" refers to the function of sending user ratings and correction requests from the terminal to the server, which then receives and analyzes them.

[0495] "User device" refers to the device (smartphone, tablet, PC, etc.) used by a user to enter or confirm travel plans, provide feedback, or receive real-time support.

[0496] The present invention is an AI concierge system that proposes personalized travel plans based on the user's preferences and emotions and provides support in real time. Specific embodiments are described below.

[0497] System Configuration

[0498] This system includes hardware and software such as user terminals, servers, databases, emotion engines, chatbots, translation services, and external collaboration services.

[0499] A user device is a device such as a smartphone, tablet, or PC that allows a user to operate an application to enter information, check travel plans, and provide feedback.

[0500] The server is a core system built on a cloud service that receives, stores, and analyzes user information and emotion data. Specific technologies that can be used include Amazon Web Services (AWS®) and Microsoft® Azure®.

[0501] The database is a server-managed storage system that stores user information and emotion data. For example, a relational database such as MySQL or PostgreSQL can be used.

[0502] The emotion engine has the ability to analyze the user's facial expressions and voice to generate emotion data, using machine learning algorithms and voice analysis software (e.g., Google® Cloud Speech-to-Text API).

[0503] A chatbot is an interface for users to make inquiries or request support during their trip, and responds in real time using natural language processing (NLP) techniques, such as Dialogflow and the Microsoft Bot Framework.

[0504] The translation service is a function that converts user inquiries into different languages, and can use the Google Translate API, etc.

[0505] External linkage services are a means of linking with external services such as taxi booking apps and restaurant reservation systems, allowing for immediate responses to user requests.

[0506] System Operation

[0507] When users download the app and launch it for the first time, they enter their profile information and travel preferences, and the emotion engine recognizes the user's facial expressions and voice to collect emotional data.

[0508] The device sends the input information and emotion data to the server, securely using the HTTPS protocol.

[0509] The server stores the received user information and emotion data in a database and learns the relationship between them. Specifically, it analyzes the data using a generative AI model (e.g., TensorFlow).

[0510] The user inputs the desired travel conditions (departure date, destination, travel mode, etc.) This information is also sent to the server via the terminal.

[0511] The server analyzes the travel conditions and the user's preference and emotional data to generate an optimal travel plan. For example, if the user wants to relax based on emotional data, it will select a plan that includes a quiet beach and a spa.

[0512] The terminal displays the generated travel plan to the user, who can review the plan and provide feedback.

[0513] The server re-analyzes the received feedback and emotional data and modifies the plan as needed, providing a final, customized plan.

[0514] If a user needs assistance during their trip, they can contact the chatbot, and the emotion engine will detect the user's stress level and provide assistance accordingly.

[0515] The server allows the chatbot to analyze the inquiry content and emotional data to generate the optimal response, and also provides comprehensive real-time support by linking with external services (e.g., translation services and taxi booking apps).

[0516] Specific examples

[0517] For new users

[0518] The user launches the app for the first time and enters their profile information and travel preferences. The emotion engine recognizes facial expressions and voice and collects emotional data. The device sends the information and emotional data to the server, which stores the information and emotional data in a database and performs learning. The user enters their desired travel conditions and the device sends them to the server. The server generates an optimal travel plan based on the conditions and emotional data. The device displays the travel plan, and the user reviews the plan and provides feedback. The server analyzes the feedback and emotional data and modifies the plan as necessary.

[0519] Prompt Sentence Examples

[0520] "I'm planning a week-long trip to Tokyo. I'd like to enjoy good food and relaxing activities. I'm on a moderate budget. Can you recommend a recommended itinerary?"

[0521] In this way, the system of the present invention can provide personalized travel plans and real-time support based on the user's preferences and emotions.

[0522] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0523] Step 1:

[0524] A user downloads the app and launches it for the first time. They enter their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget). The emotion engine recognizes emotions through facial expressions and voice, and collects emotion data.

[0525] Input: User information, emotion data

[0526] Output: User information and emotion data are stored on the device.

[0527] Step 2:

[0528] The device sends the entered user information and emotion data to the server, securely using the HTTPS protocol.

[0529] Input: User information, emotion data

[0530] Output: User information and emotion data received by the server

[0531] Step 3:

[0532] The server stores the received user information and emotion data in a database. Specifically, a relational database such as MySQL or PostgreSQL can be used. Furthermore, the data is analyzed using a generative AI model (e.g., TensorFlow) to learn the association between the emotion data and user information.

[0533] Input: User information and emotion data received by the server

[0534] Output: Parsed data stored in a database

[0535] Step 4:

[0536] The user inputs the desired travel conditions (departure date, destination, travel mode, etc.) This information is also sent to the server via the terminal.

[0537] Input: Travel conditions

[0538] Output: The travel conditions are sent to the server.

[0539] Step 5:

[0540] The server analyzes the travel conditions and the user's preference and emotional data to generate the optimal travel plan. For example, a user who wants to relax will be offered a plan that includes a quiet beach or spa.

[0541] Input: Travel conditions, user preference data, emotional data

[0542] Output: Generated itinerary

[0543] Step 6:

[0544] The terminal displays the travel plan sent from the server to the user.

[0545] Input: Generated itinerary

[0546] Output: The itinerary displayed to the user

[0547] Step 7:

[0548] The user reviews the proposed itinerary and provides corrections and feedback as needed, such as adding or removing specific activities or adjusting the budget.

[0549] Input: Feedback on travel plans

[0550] Output: The modification request is sent from the terminal to the server.

[0551] Step 8:

[0552] The server re-analyzes the received feedback and emotion data and modifies the plan as necessary, which is then sent back to the device for confirmation by the user.

[0553] Input: Feedback, emotion data

[0554] Output: Modified itinerary

[0555] Step 9:

[0556] The user confirms the confirmed travel plan and gives final approval, which the terminal notifies the server.

[0557] Input: Final travel plan approval

[0558] Output: Authorization data is sent to the server

[0559] Step 10:

[0560] If a user needs assistance during their trip, they can contact the chatbot, and the emotion engine will detect the user's stress level and provide assistance accordingly.

[0561] Input: Inquiry details, emotion data

[0562] Output: The device sends the query and emotion data to the server.

[0563] Step 11:

[0564] The server analyzes the inquiry content and emotional data, and the chatbot generates the optimal response. If necessary, it connects with translation services or taxi booking apps to provide support that takes the user's emotions into consideration.

[0565] Input: Inquiry details, emotion data

[0566] Output: Generated answers, supporting information provided

[0567] Step 12:

[0568] The device displays responses and support information from the server to the user.

[0569] Input: Support Information

[0570] Output: Support information displayed to the user

[0571] At each step, it can provide a personalized travel plan based on the user's preferences and emotions, providing comprehensive support during the trip.

[0572] (Application example 2)

[0573] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0574] Conventional travel plan suggestion systems make suggestions based on the user's preferences, but do not take into account the user's emotional state. This means that they lack support that is in line with the user's emotions and needs, which change in real time. Furthermore, they often cannot provide effective support to reduce stress and anxiety during travel.

[0575] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for inputting user information, a means for transmitting the user information to the server, a means for the server to learn the user information, a means for the server to generate an itinerary based on the user information, a means for the server to transmit the generated itinerary to a user terminal, a means for the user terminal to display the itinerary, a means for transmitting feedback from the user to the server, an emotion recognition means for collecting and recognizing the user's emotions, a means for adjusting the itinerary based on the user's emotions, and a real-time support means for performing the emotion recognition in real time and providing appropriate support. This enables the proposal of a personalized itinerary that takes into account the user's emotions and preferences. Furthermore, by recognizing the user's emotions in real time during the trip and providing appropriate support, it is possible to increase user satisfaction and reduce stress and anxiety.

[0576] A "means for inputting user information" is a device, software, or combination thereof that provides an interface for a user to input their profile information and preferences.

[0577] The "means for transmitting the user information to the server" refers to a device, software, or a combination thereof for transmitting the input user information to the server via a network.

[0578] "Means by which the server learns the user information" refers to an algorithm, device, software, or combination thereof that analyzes and stores the user information received by the server and makes personalized suggestions based on that information.

[0579] The "means by which the server generates a travel plan based on the user information" refers to an algorithm, device, software, or combination thereof for creating an optimal travel plan based on the user's preferences and emotional state.

[0580] The "means for transmitting the travel plan generated by the server to the user terminal" refers to a device, software, or a combination thereof for transmitting the generated travel plan to the user terminal via a network.

[0581] The "means for the user terminal to display the itinerary" refers to a device, software, or a combination thereof for visually or audibly presenting the itinerary received by the user terminal to the user.

[0582] The "means for transmitting the user feedback to the server" refers to a device, software, or a combination thereof for transmitting the user-provided feedback to the server.

[0583] The "emotion recognition means for collecting and recognizing the user's emotions" refers to a device, software, or a combination thereof for collecting and analyzing the user's facial expressions, voice, and other physiological data to recognize the user's emotional state.

[0584] The "means for adjusting the travel plan based on the user's emotions" refers to an algorithm, device, software, or a combination thereof for adjusting the optimal travel plan in real time based on the user's emotional data.

[0585] "Real-time support means for performing the emotion recognition in real time and providing appropriate support" refers to a device, software, or a combination thereof for monitoring a user's emotions in real time and providing immediate support or suggestions in response.

[0586] This invention is a system that proposes optimal travel plans based on the user's preferences and emotions and provides support in real time. The system collects and analyzes user information, monitors the user's emotions in real time using emotion recognition technology, and provides appropriate support.

[0587] System Configuration

[0588] This system consists of a user terminal, a server, an emotion recognition device, and a real-time support device.

[0589] User information collection and emotion recognition

[0590] user:

[0591] Users download the app and launch it for the first time, entering their profile information (name, age, gender) and preferences (favorite meals, activities they are interested in, budget).

[0592] Device:

[0593] The device transmits this user information and emotional data collected through the user's facial expressions and voice to the server.

[0594] server:

[0595] The server stores the received user information and emotion data in a database and learns from this data, thereby learning the relationship between user preferences and emotions.

[0596] Generate a travel plan

[0597] user:

[0598] The user inputs the desired travel conditions (departure date, destination, travel type, etc.).

[0599] Device:

[0600] The terminal transmits the input travel conditions to the server.

[0601] server:

[0602] The server analyzes the travel conditions and learned user preferences and emotional data to generate an optimal travel plan. Based on the emotional data, it selects activities that will relax the user and events that will excite them.

[0603] Device:

[0604] The terminal displays the travel plan sent from the server to the user.

[0605] user:

[0606] The user reviews the proposed itinerary and provides corrections and feedback as needed.

[0607] Customization and Real-Time Support

[0608] user:

[0609] Users request specific requests or changes (e.g., specific restaurant reservations, changes to transportation).

[0610] Device:

[0611] The terminal transmits the user's request and emotion data to the server.

[0612] server:

[0613] The server analyzes the request along with the emotional data and proposes optimal customization options. Taking the emotional data into consideration, the server adjusts the plan to best suit the user.

[0614] Device:

[0615] The terminal displays the final customized plan to the user.

[0616] user:

[0617] The user reviews and confirms the final plan.

[0618] Real-time support during your trip

[0619] user:

[0620] Users can contact the chatbot if they get lost while traveling or in an emergency.

[0621] Device:

[0622] The terminal transmits the inquiry content and emotion data to the server.

[0623] server:

[0624] The server analyzes the inquiry content and emotional data to generate the optimal response, and if necessary, connects with translation services or taxi booking apps to provide support that takes the user's emotions into consideration.

[0625] Device:

[0626] The device displays answers and support information to the user.

[0627] user:

[0628] The user will accept the support provided and contact us again if necessary.

[0629] Hardware and software used

[0630] Hardware:

[0631] Cameras and microphones onboard the self-driving vehicle to collect facial expressions and voice recordings of the user.

[0632] User devices (smartphones, etc.): To check travel plans and receive real-time support.

[0633] software:

[0634] Emotion Engine: Recognizes emotions by analyzing the user's facial expressions and voice.

[0635] Route Planner: Generates optimal travel plans based on user preferences and sentiment data.

[0636] Real-Time Support: Analyzes emotional data in real time while traveling and provides support such as changing music or adjusting the route.

[0637] Specific examples

[0638] If a user feels like relaxing on a weekday, the route planner will suggest a low-stress route and generate a plan that includes a stop at a scenic park along the way. Real-time support will play relaxing music when the user's mood changes and provide information about events the user can enjoy.

[0639] Example prompt sentence:

[0640] Enter your user's profile information and preferences, then use facial expressions and voice to collect emotional data and suggest a low-stress, relaxing driving route.

[0641] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0642] Step 1:

[0643] Entering user information

[0644] User: The user downloads and launches the app, then enters their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget).

[0645] Input: Profile information and preferences

[0646] Output: The input data is saved on the device and sent to the server.

[0647] Step 2:

[0648] Collecting Emotional Data

[0649] Device: The device uses the camera and microphone built into the device to collect the user's facial expressions and voice.

[0650] Input: User's facial expression and voice data

[0651] Output: The collected data is sent to the emotion recognizer.

[0652] Step 3:

[0653] Emotional Data Analysis

[0654] Server: The server uses an emotion engine to analyze the transmitted facial expression and voice data and identify the user's emotional state.

[0655] Input: facial expression data and voice data

[0656] Output: Recognized emotion data (e.g., joy, stress)

[0657] Step 4:

[0658] Store user information and emotional data

[0659] Server: The server stores user profile information, preferences, and sentiment data in a database.

[0660] Input: Profile information, preferences, recognized emotion data

[0661] Output: Consolidated data stored in a database

[0662] Step 5:

[0663] Enter travel conditions

[0664] User: The user inputs the desired travel conditions (departure date, destination, travel mode, etc.).

[0665] Input: Travel conditions

[0666] Output: The input data is saved on the device and sent to the server.

[0667] Step 6:

[0668] Generate a travel plan

[0669] Server: The server analyzes travel conditions, user preferences, and emotional data, and runs algorithms to generate optimal travel plans.

[0670] Input: Travel conditions, preferences, emotional data

[0671] Output: Generated itinerary

[0672] Step 7:

[0673] View travel plans

[0674] Terminal: The terminal displays the generated itinerary to the user.

[0675] Input: Travel Plan

[0676] Output: Display of itinerary

[0677] Step 8:

[0678] Providing and submitting feedback

[0679] User: The user reviews the itinerary and provides corrections and feedback as needed.

[0680] Device: The device sends feedback to the server.

[0681] Input: User feedback

[0682] Output: Feedback sent to the server

[0683] Step 9:

[0684] Generate a customized plan

[0685] Server: The server analyzes the feedback and sentiment data and generates a final customized itinerary.

[0686] Input: Feedback, emotion data

[0687] Output: A customized itinerary

[0688] Step 10:

[0689] View your customized plan

[0690] Terminal: The terminal displays the final customized itinerary to the user.

[0691] Input: Customized Travel Plan

[0692] Output: Display of customized itinerary

[0693] Step 11:

[0694] Real-time support during your trip

[0695] User: The user will use the chatbot to request assistance or queries during their journey.

[0696] Terminal: The terminal sends the query and emotion data to the server.

[0697] Server: The server analyzes the inquiry content and emotion data to generate the most appropriate answer or support. If necessary, it connects with translation services or other external applications to provide support that takes the user's emotions into consideration.

[0698] Input: User query and emotion data

[0699] Output: Answers and supporting information

[0700] Step 12:

[0701] View support information

[0702] Terminal: The terminal displays answers and support information sent from the server to the user.

[0703] Input: Answers and supporting information

[0704] Output: Displayed answers and supporting information

[0705] This allows users to experience the most suitable trip according to their preferences and emotions, reducing stress and anxiety while traveling and providing a highly satisfying travel experience.

[0706] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0707] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0708] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0709] [Second embodiment]

[0710] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0711] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0712] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0713] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0714] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0715] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0716] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0717] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0718] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0719] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0720] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0721] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0722] This invention is an AI concierge system that proposes personalized travel plans based on the user's preferences. The program processing of this system is explained in detail below in natural language.

[0723] 1. Collection of User Information

[0724] User

[0725] A user downloads the app and launches it for the first time.

[0726] Enter your profile information (name, age, gender) and travel preferences (favorite meals, activities you're interested in, budget).

[0727] Terminal

[0728] The input information is sent to the server.

[0729] server

[0730] The received user information is stored in a database and trained by a machine learning algorithm, which then creates a dataset to suggest optimal travel plans for the user.

[0731] 2. Generate a travel plan

[0732] User

[0733] Enter the conditions of your desired trip (departure date, destination, travel method, etc.).

[0734] Terminal

[0735] The entered conditions are sent to the server.

[0736] server

[0737] The system analyzes the received information and uses machine learning algorithms to generate a travel plan based on the user's preferences, taking into account weather information, local event information, budget, and other factors to create the optimal plan.

[0738] Terminal

[0739] The travel plan sent from the server is displayed to the user.

[0740] User

[0741] Review the proposed itinerary and provide corrections and feedback as needed.

[0742] 3. In-depth support and customization

[0743] User

[0744] Request specific requests or changes (e.g., a specific restaurant reservation, change of transportation).

[0745] Terminal

[0746] Sends the request to the server.

[0747] server

[0748] We analyze your request, suggest possible customizations, and make any necessary reservations and arrangements.

[0749] Terminal

[0750] Display the final customized plan to the user.

[0751] User

[0752] Review and finalize the plan.

[0753] 4. Real-time support during your trip

[0754] User

[0755] If you get lost or have any other inquiries while you're there, you can contact them through the chatbot.

[0756] Terminal

[0757] The query is sent to the server.

[0758] server

[0759] The chatbot analyzes the inquiry and generates the most appropriate answer, linking with translation services and taxi booking apps as needed.

[0760] Terminal

[0761] Display answers and support information to users.

[0762] User

[0763] Accept the support offered and contact us again if necessary.

[0764] Specific examples

[0765] For new users

[0766] 1. A user launches the app for the first time and enters their profile information and travel preferences.

[0767] 2. The device sends this information to the server.

[0768] 3. The server stores the information in a database and uses machine learning algorithms to learn from it.

[0769] 4. The user enters the desired travel conditions, and the terminal sends them to the server.

[0770] 5. The server generates an optimal travel plan and sends it to the device.

[0771] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[0772] 7. The server receives the feedback, modifies the plan as needed, and provides the final plan to the user.

[0773] Real-time consultation during your trip

[0774] 1. A user gets lost in the area and makes an inquiry through a chatbot.

[0775] 2. The device sends the query to the server.

[0776] 3. The server generates the best answer and, if necessary, connects with a translation service or taxi booking app.

[0777] 4. The device displays answers and support information to the user.

[0778] 5. The user accepts the support offered and contacts us again if necessary.

[0779] In this way, the system of the present invention proposes travel plans based on the user's preferences and provides comprehensive support during the trip, allowing the user to easily realize the ideal travel experience.

[0780] The processing flow will be explained below.

[0781] Step 1:

[0782] A user downloads the app and launches it for the first time.

[0783] Step 2:

[0784] The device displays an interface for entering the user's profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget).

[0785] Step 3:

[0786] The user enters the required information and presses the send button.

[0787] Step 4:

[0788] The terminal transmits the input information to the server.

[0789] Step 5:

[0790] The server stores the received user information in a database.

[0791] Step 6:

[0792] The server runs machine learning algorithms based on the stored user information to learn the user's preferences.

[0793] Step 7:

[0794] The user inputs the conditions of the desired trip (departure date, destination, travel mode, etc.).

[0795] Step 8:

[0796] The terminal transmits the input travel conditions to the server.

[0797] Step 9:

[0798] The server analyzes the travel conditions received and compares them with the user's preference data.

[0799] Step 10:

[0800] The server uses deep learning algorithms to generate the optimal travel plan (destinations, activities, accommodation, etc.).

[0801] Step 11:

[0802] The server transmits the generated travel plan to the terminal.

[0803] Step 12:

[0804] The terminal displays the travel plan sent from the server to the user.

[0805] Step 13:

[0806] The user reviews the proposed itinerary and enters corrections or feedback as necessary.

[0807] Step 14:

[0808] The device sends the user's feedback to the server.

[0809] Step 15:

[0810] The server analyzes the feedback and modifies the travel plan.

[0811] Step 16:

[0812] The server sends the final revised itinerary to the terminal.

[0813] Step 17:

[0814] The device displays the final revised plan to the user.

[0815] Step 18:

[0816] The user reviews and confirms the final plan.

[0817] Step 19:

[0818] If a user gets lost or needs information while in the area, they can make inquiries through the chatbot function.

[0819] Step 20:

[0820] The terminal sends the inquiry to the server.

[0821] Step 21:

[0822] The server uses a chatbot to analyze the inquiry and generate the most appropriate answer.

[0823] Step 22:

[0824] The server will connect with translation services and taxi booking apps as needed.

[0825] Step 23:

[0826] The server generates responses and support information and sends them to the device.

[0827] Step 24:

[0828] The device displays answers and support information to the user.

[0829] Step 25:

[0830] The user accepts the support offered and contacts the company again if necessary.

[0831] Through this series of steps, users can receive suggestions for optimal travel plans and receive support based on their requests while traveling with peace of mind.

[0832] Example 1

[0833] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0834] Conventional travel plan suggestion systems have difficulty in providing advanced customization based on users' preferences and individual needs, and do not provide sufficient real-time support during the trip. Furthermore, it is difficult to provide support in conjunction with external translation services or taxi booking services. This makes it difficult for users to achieve their ideal travel experience.

[0835] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0836] In this invention, the server includes a means for storing user information in a database and learning using a machine learning algorithm, a means for generating a travel plan based on the user's preferences, and a means for analyzing travel conditions and generating an optimal travel plan. This allows for personalized travel plans based on the user's preferences and needs. It can also accommodate customization requests and provide real-time support on-site, allowing users to enjoy an ideal travel experience.

[0837] "User Information" means information including a User's profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget).

[0838] "Server" means a computer system that stores and analyzes information submitted by users and generates travel plans.

[0839] A "database" is a structured collection of data for storing user information and travel plans.

[0840] A "machine learning algorithm" is a computational method for analyzing data based on user preferences and building predictive models.

[0841] A "travel plan" is a plan that includes travel schedules and suggestions generated based on the user's preferences and conditions.

[0842] "Travel conditions" refers to information such as the departure date, destination, and travel mode desired by the user when traveling.

[0843] A "customization request" is a request by a user to communicate specific wishes or modifications to the system.

[0844] "Real-time support" is a service that provides immediate support and assistance to problems and inquiries users may encounter while traveling.

[0845] A "chatbot" is a program that uses natural language processing technology to automatically respond to user questions and requests.

[0846] "External linkage services" are support functions provided by the system in collaboration with external services, such as translation services and taxi booking services.

[0847] This invention is an AI concierge system that proposes personalized travel plans based on the user's preferences. The program processing of this system is described in detail below.

[0848] Collection of User Information

[0849] When a user first launches the app on their smartphone or tablet, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget). The device sends this information to the server. The server stores the received user information in a database and uses machine learning algorithms (such as Scikit-Learn or TensorFlow) in Python to train the database. This information is used to create a dataset that can be used to suggest optimal travel plans to the user.

[0850] Generate a travel plan

[0851] The user enters the conditions of their desired trip (departure date, destination, travel mode, etc.) into the app. The device sends these conditions to the server. The server analyzes the received conditions and generates an optimal travel plan based on the user's preference data and travel conditions. In doing so, it uses weather information APIs and local event information APIs and also takes budget into consideration. The server sends the generated travel plan to the user's device, which then displays it to the user. The user can review the displayed travel plan and provide feedback if necessary.

[0852] In-depth support and customization

[0853] Users can request specific requests or changes through the app. For example, they can input requests to make a specific restaurant reservation or change their transportation method. The device sends these requests to the server. The server analyzes the request, suggests possible customizations, and makes the necessary reservations and arrangements. This allows for customization that meets the user's individual needs. The final customized plan is generated and displayed on the device. The user reviews and confirms the final plan.

[0854] Real-time support during your trip

[0855] If a user gets lost or needs other support while traveling, they can make an inquiry using the chatbot function within the app. The device sends the inquiry to the server. The server uses the chatbot to analyze the inquiry and generate the most appropriate answer. If necessary, it also connects with translation service APIs and taxi dispatch app APIs. This allows the user to receive the necessary support in real time. The generated answer and support information are displayed on the device, and the user can check and respond.

[0856] Specific examples

[0857] For new users

[0858] 1. A user launches the app for the first time and enters their profile information and travel preferences.

[0859] 2. The device sends this information to the server.

[0860] 3. The server stores the information in a database and uses machine learning algorithms to learn from it.

[0861] 4. The user enters the desired travel conditions, and the terminal sends them to the server.

[0862] 5. The server generates an optimal travel plan and sends it to the device.

[0863] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[0864] 7. The server receives the feedback, modifies the plan as needed, and provides the final plan to the user.

[0865] Real-time consultation during your trip

[0866] 1. A user gets lost in the area and makes an inquiry through a chatbot.

[0867] 2. The device sends the query to the server.

[0868] 3. The server generates the best answer and, if necessary, connects with a translation service or taxi booking app.

[0869] 4. The device displays answers and support information to the user.

[0870] 5. The user accepts the support offered and contacts us again if necessary.

[0871] The system allows users to create optimal travel plans tailored to their preferences and receive comprehensive support during their trip.

[0872] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0873] Program processing flow

[0874] Step 1: Enter your user information

[0875] input:

[0876] When a user launches the app for the first time, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget).

[0877] explanation:

[0878] A user downloads and launches the app on their smartphone or tablet.

[0879] Enter information such as your name, age, gender, and travel preferences into the in-app form.

[0880] The user presses the "Submit" button to confirm the entered information.

[0881] output:

[0882] A dataset of user profile information and preference information

[0883] Step 2: Submit your information

[0884] input:

[0885] User information entered in step 1

[0886] explanation:

[0887] The device sends the user's input information to the backend server via an HTTP POST request.

[0888] Information is transmitted using SSL / TLS encrypted communications.

[0889] output:

[0890] The server receives the user information.

[0891] Step 3: Database storage and machine learning

[0892] input:

[0893] The dataset of user information received in step 2

[0894] explanation:

[0895] The server stores the received user information in a database.

[0896] The database used is MySQL.

[0897] The server uses Python to analyze the received data using machine learning algorithms such as Scikit-Learn and TensorFlow, and performs learning.

[0898] A user preference profile is generated from the learning results.

[0899] output:

[0900] User preference profile

[0901] Step 4: Enter your travel requirements

[0902] input:

[0903] User's desired travel conditions (departure date, destination, travel type, etc.)

[0904] explanation:

[0905] The user enters their travel requirements into a dedicated form within the app.

[0906] Also consider options such as those for families with children or for active outdoorsy people.

[0907] After the user has entered all the required information, he clicks "Submit."

[0908] output:

[0909] Dataset of user desired travel conditions

[0910] Step 5: Sending conditions

[0911] input:

[0912] The data set of travel conditions entered in step 4

[0913] explanation:

[0914] The terminal sends the travel conditions entered by the user to the server via an HTTP POST request.

[0915] Communications are encrypted using SSL / TLS.

[0916] output:

[0917] Travel conditions received

[0918] Step 6: Create a travel plan

[0919] input:

[0920] User information and travel conditions collected in Steps 2 and 4

[0921] explanation:

[0922] The server generates the optimal travel plan based on the user's preference profile and travel conditions using weather information APIs and local event information APIs.

[0923] Travel plans are automatically generated using machine learning algorithms (such as TensorFlow).

[0924] Adjust your plan taking into account your budget.

[0925] output:

[0926] The perfect travel plan for you

[0927] Step 7: Submit and view your plan

[0928] input:

[0929] The itinerary generated in step 6

[0930] explanation:

[0931] The server sends the generated travel plan to the terminal as an HTTP response.

[0932] The device displays the received travel plan to the user.

[0933] output:

[0934] The itinerary displayed to the user

[0935] Step 8: Provide feedback

[0936] input:

[0937] User feedback information

[0938] explanation:

[0939] The user checks the proposed travel plan and, if necessary, inputs changes or corrections as feedback.

[0940] The terminal transmits the feedback information to the server.

[0941] output:

[0942] Feedback information dataset

[0943] Step 9: Submit a customization request

[0944] input:

[0945] Customization Request Information

[0946] explanation:

[0947] The user inputs a specific request (e.g., a specific restaurant reservation, a change of transportation method).

[0948] The terminal sends these requests to the server.

[0949] output:

[0950] Customization request information dataset

[0951] Step 10: Customization and Final Plan Generation

[0952] input:

[0953] Customization request information received in step 9

[0954] explanation:

[0955] The server analyzes the received request and adds any necessary customizations to the itinerary.

[0956] Make any necessary reservations and arrangements and generate your final itinerary.

[0957] output:

[0958] A customized ultimate itinerary

[0959] Step 11: Submit and review your final plan

[0960] input:

[0961] The final itinerary generated in step 10

[0962] explanation:

[0963] The server sends the final customized itinerary to the terminal as an HTTP response.

[0964] The device displays the final plan received to the user.

[0965] The user reviews and confirms the final plan.

[0966] output:

[0967] Final confirmed travel plans

[0968] Step 12: Real-time support during your trip

[0969] input:

[0970] Real-time support inquiries from users

[0971] explanation:

[0972] Users can use chatbots to make inquiries while traveling.

[0973] The terminal sends the inquiry to the server.

[0974] The server analyzes the data using a chatbot and generates the most appropriate answer.

[0975] If necessary, it will also link with translation service APIs and taxi booking app APIs.

[0976] The device displays generated answers and supporting information to the user.

[0977] output:

[0978] Responses to inquiries and support information

[0979] In this way, users can create the perfect travel plan tailored to their preferences and receive comprehensive support during their trip.

[0980] (Application example 1)

[0981] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0982] While conventional travel plan suggestion systems can provide personalized plans based on user preferences, they lack a way for users to visually visualize the atmosphere and locations of their travel destinations. Furthermore, providing real-time support during a trip requires users to use multiple apps simultaneously, which is inconvenient for users. There is a need to solve these problems and realize a more comprehensive travel experience.

[0983] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0984] In this invention, the server includes means for inputting user information, means for transmitting the user information to the server, means for the server to learn the user information, means for the server to generate a travel plan based on the user information, means for the server to transmit the generated travel plan to a user terminal, means for the user terminal to display the travel plan, means for transmitting feedback from the user to the server, means for providing real-time support, and means for providing a virtual reality tour based on the travel plan, thereby enabling the user to visually experience the suggested travel destinations in virtual reality and receive comprehensive real-time support, thereby enabling the user to plan and execute a more satisfying trip.

[0985] "User Information" means data relating to a user's profile information, travel preferences and travel requirements.

[0986] The "server" is a computer system that receives user information, learns and analyzes it, and generates and transmits the optimal travel plan.

[0987] "User terminal" means an electronic device, typically a smartphone or tablet, used to input user information and receive and display travel plans and support information.

[0988] "Virtual reality tour" is a feature that allows users to visually experience the atmosphere and locations of their travel destinations through virtual reality technology.

[0989] A "trip plan" is a plan that includes a suggested travel itinerary and destination details generated based on a user's preferences and conditions.

[0990] "Real-time support" is a support service that provides immediate response and assistance to users' inquiries and problems while traveling.

[0991] A "machine learning algorithm" is an algorithm that learns patterns based on collected user information and generates travel plans based on the results of that learning.

[0992] In order to put the present invention into practice, it is necessary to build a system in which multiple pieces of hardware and software work together. A specific embodiment of this system is shown below.

[0993] 1. Collection and storage of user information

[0994] When a user downloads and launches the application for the first time, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget). This information is sent from the device (e.g., smartphone) used to the server. The server stores the received user information in a database and uses machine learning algorithms to learn from it. This step forms the base dataset for providing personalized travel plans based on the user information.

[0995] 2. Generate a travel plan

[0996] When a user inputs the conditions for their desired travel plan (departure date, destination, travel mode, etc.), these conditions are also sent to the server via their device. The server analyzes the received conditions and uses a machine learning algorithm to generate a travel plan based on the user's preferences. This also takes into account factors such as weather information, local event information, and budget. The generated travel plan is sent from the server to the user's device and displayed to the user.

[0997] 3. Providing VR tours

[0998] After users confirm the proposed itinerary, they can experience a virtual reality (VR) tour of specific tourist spots or points. This function is available by installing an application on a VR headset or smartphone. For example, if a user selects a plan with "Kyoto" as the destination, they can virtually visit Kyoto's famous tourist spots.

[0999] 4. Real-time support during your trip

[1000] If a user gets lost or needs emergency assistance while traveling, they can receive real-time support using the chatbot within the application. When the user enters their inquiry, it is sent from the device to the server, which uses machine learning algorithms to generate the best answer. If necessary, it can also be linked to a translation service or a taxi booking app.

[1001] 5. Specific Examples

[1002] For new users, the flow is as follows:

[1003] 1. A user launches the app for the first time and enters their profile information and travel preferences.

[1004] 2. The device sends this information to the server.

[1005] 3. The server stores the information in a database and uses machine learning algorithms to learn from it.

[1006] 4. The user enters the desired travel conditions, and the device sends them to the server.

[1007] 5. The server generates an optimal travel plan and sends it to the device.

[1008] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[1009] 7. The server receives the feedback, modifies the plan as needed, and provides the final plan to the user.

[1010] As an example of real-time support during travel, here is a specific example of a user getting lost and making an inquiry through a chatbot:

[1011] 1. A user gets lost in the area and makes an inquiry through a chatbot.

[1012] 2. The device sends the query to the server.

[1013] 3. The server generates the best answer and, if necessary, connects with a translation service or taxi booking app.

[1014] 4. The device displays answers and support information to the user.

[1015] 5. The user accepts the support provided and contacts us again if necessary.

[1016] Examples of prompts include:

[1017] "After users download the app and launch it for the first time, they enter the following information:

[1018] Name: User A

[1019] Age: 30

[1020] Gender: Male

[1021] Favorite food: Japanese food

[1022] Interested in: Hiking

[1023] Budget: 50,000 yen

[1024] Next, enter your travel plan requirements:

[1025] Departure date: July 1, 2024

[1026] Destination: Kyoto

[1027] Travel type: Solo travel

[1028] By following these prompts, each function of the system will function properly, providing users with a personalized travel experience.

[1029] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1030] Step 1:

[1031] Collection of User Information

[1032] When a user launches the application for the first time, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget). The device sends this information to the server, which stores the received user information in a database and begins learning using machine learning algorithms. This process forms the base data set for suggesting personalized travel plans to the user.

[1033] Input: User profile information and travel preference data

[1034] Output: User information stored in the database

[1035] Step 2:

[1036] Enter travel plan conditions

[1037] The user inputs the travel plan conditions (departure date, destination, travel mode, etc.). This information is sent to the server via the device. The server analyzes the received conditions and uses machine learning algorithms to generate a travel plan based on the user's preferences. This also takes into account factors such as weather information, local event information, and budget.

[1038] Input: User's travel plan conditions

[1039] Output: Parsed itinerary data

[1040] Step 3:

[1041] Generate and display travel plans

[1042] The server sends the generated travel plan to the user's device, which then displays the plan to the user. The user reviews the proposed plan and provides feedback if necessary. This feedback is sent from the device to the server, which then revises the plan based on that feedback. The final plan is then provided to the user.

[1043] Input: Server-generated travel plan data, user feedback

[1044] Output: Finalized itinerary

[1045] Step 4:

[1046] Providing virtual reality tours

[1047] After reviewing the proposed itinerary, users can experience a virtual reality (VR) tour of specific tourist spots or points. VR content is sent to the device, and users can visually experience the content with a VR headset or smartphone. For example, if a user selects "Kyoto" as their destination, they can virtually visit Kyoto's famous tourist spots.

[1048] Input: Proposed itinerary and related VR content

[1049] Output: The virtual reality tour the user experiences

[1050] Step 5:

[1051] Real-time support during your trip

[1052] If a user gets lost or needs emergency assistance while traveling, they can receive real-time support from a chatbot within the application. When the user enters their inquiry, it is sent from the device to the server. The server uses a machine learning algorithm to generate the optimal answer and sends it to the device. If necessary, it can also be linked to a translation service or a taxi booking app.

[1053] Input: User's support request

[1054] Output: Answers and support information from the chatbot

[1055] The above steps will create a system that allows users to create personalized travel plans, experience virtual reality tours, and receive real-time support throughout their trip.

[1056] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1057] This invention is an AI concierge system that proposes personalized travel plans based on the user's preferences and emotions and provides real-time support. The system incorporates an emotion engine and has the function of adjusting plans and support content according to the user's emotions.

[1058] User information collection and emotion recognition

[1059] User

[1060] Download the app and launch it for the first time.

[1061] Enter your profile information (name, age, gender) and travel preferences (favorite meals, activities you're interested in, budget). The app is equipped with an emotion engine that recognizes the user's emotions through facial expressions and voice.

[1062] Terminal

[1063] The input information and emotion data are sent to the server.

[1064] server

[1065] The received user information and emotion data are stored in a database, which allows the system to learn the relationship between user preferences and emotions.

[1066] Generate a travel plan

[1067] User

[1068] Enter the conditions of your desired trip (departure date, destination, travel method, etc.).

[1069] Terminal

[1070] The entered travel conditions are sent to the server.

[1071] server

[1072] The system analyzes the user's preference and emotional data along with travel conditions to generate an optimal travel plan. Based on the emotional data, it selects activities that will relax the user and events that will excite them.

[1073] Terminal

[1074] The travel plan sent from the server is displayed to the user.

[1075] User

[1076] Review the proposed itinerary and provide corrections and feedback as needed.

[1077] Customization and Real-Time Support

[1078] User

[1079] Request specific requests or changes (e.g., a specific restaurant reservation, change of transportation).

[1080] Terminal

[1081] Send the user's request and emotion data to the server.

[1082] server

[1083] Analyzes the request along with emotional data and proposes optimal customization. Taking emotional data into consideration, adjusts plans to ensure user satisfaction.

[1084] Terminal

[1085] Display the final customized plan to the user.

[1086] User

[1087] Review and confirm the final plan.

[1088] Real-time support during your trip

[1089] User

[1090] If you get lost or have an emergency, you can contact us through the chatbot.

[1091] Terminal

[1092] The inquiry content and emotion data are sent to the server.

[1093] server

[1094] The chatbot analyzes the inquiry content and emotional data to generate the most appropriate response, and if necessary, connects with translation services or taxi booking apps to provide support that takes the user's emotions into consideration.

[1095] Terminal

[1096] Display answers and support information to users.

[1097] User

[1098] Accept the support offered and follow up if necessary.

[1099] Specific examples

[1100] For new users

[1101] 1. A user launches the app for the first time and enters their profile information and travel preferences. The emotion engine recognizes the user's facial expressions and voice to collect emotional data.

[1102] 2. The device sends information and emotion data to the server.

[1103] 3. The server stores the information and emotion data in a database and performs learning.

[1104] 4. The user enters the desired travel conditions, and the terminal sends them to the server.

[1105] 5. The server generates an optimal travel plan based on the conditions and emotion data.

[1106] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[1107] 7. The server analyzes the feedback and sentiment data, modifies the plan as needed, and provides the final plan.

[1108] Real-time consultation during your trip

[1109] 1. When a user gets lost in the area and makes an inquiry through the chatbot, the emotion engine detects the user's stress level.

[1110] 2. The device sends the inquiry and emotion data to the server.

[1111] 3. The server analyzes the inquiry and emotional data to generate the optimal response. If necessary, it connects with translation services or taxi booking apps to provide support that reduces the user's stress.

[1112] 4. The device displays answers and support information to the user.

[1113] 5. The user accepts the support offered and contacts us again if necessary.

[1114] In this way, the system of the present invention incorporating an emotion engine can provide personalized travel plans that take into account not only the user's preferences but also their emotions, providing comprehensive support during the trip.

[1115] The processing flow will be explained below.

[1116] Step 1:

[1117] A user downloads the app and launches it for the first time.

[1118] Step 2:

[1119] The device asks the user to enter their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget).

[1120] Step 3:

[1121] The user enters their profile information and travel preferences and hits the submit button.

[1122] Step 4:

[1123] The terminal transmits the input information to the server.

[1124] Step 5:

[1125] The server stores the received user information in a database.

[1126] Step 6:

[1127] The device uses an emotion engine to collect emotional data from the user's facial expressions and voice.

[1128] Step 7:

[1129] The device transmits the emotion data to the server.

[1130] Step 8:

[1131] The server runs a machine learning algorithm based on user information and emotional data to learn the relationship between user preferences and emotions.

[1132] Step 9:

[1133] The user inputs the conditions of the desired trip (departure date, destination, travel mode, etc.).

[1134] Step 10:

[1135] The terminal transmits the input travel conditions to the server.

[1136] Step 11:

[1137] The server analyzes the travel conditions received and generates an optimal travel plan based on them. Based on the emotional data, it selects activities that will relax or excite the user.

[1138] Step 12:

[1139] The server sends the generated travel plan to the user's terminal.

[1140] Step 13:

[1141] The device displays the travel plan to the user.

[1142] Step 14:

[1143] The user reviews the proposed itinerary and enters corrections or feedback as necessary.

[1144] Step 15:

[1145] The device sends the user's feedback to the server.

[1146] Step 16:

[1147] The server analyzes the feedback and modifies the travel plan, taking into account the emotional data.

[1148] Step 17:

[1149] The server sends the final revised itinerary to the terminal.

[1150] Step 18:

[1151] The device displays the final revised plan to the user.

[1152] Step 19:

[1153] The user reviews and confirms the final plan.

[1154] Step 20:

[1155] If a user gets lost or stressed while in the area, they can contact the chatbot to inquire.

[1156] Step 20A:

[1157] The device monitors the user's emotions in real time and transmits the emotion data along with the query to the server.

[1158] Step 21:

[1159] The server uses a chatbot to analyze the inquiry content and emotional data and generate the most appropriate answer.

[1160] Step 22:

[1161] The server will connect with translation services and taxi booking apps as needed to provide support that takes the user's emotions into consideration.

[1162] Step 23:

[1163] The server generates responses and support information and sends them to the device.

[1164] Step 24:

[1165] The device displays answers and support information to the user.

[1166] Step 25:

[1167] The user accepts the support offered and contacts the company again if necessary.

[1168] In this way, it becomes possible to provide personalized responses based on user preferences as well as emotional data, providing a more personalized travel experience and real-time support.

[1169] Example 2

[1170] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1171] Conventional travel planning systems have difficulty providing plans that fully take into account the user's preferences and emotions, and they also lack the ability to provide satisfactory real-time support. This means that users cannot receive appropriate support tailored to their individual needs and emotions, which can lead to a decrease in satisfaction during their trip.

[1172] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for learning user information and emotion data, means for generating a travel plan based on the user information and emotion data, and means for providing real-time support based on the emotion data. This makes it possible to provide a personalized travel plan based on the user's preferences and emotions and comprehensive support during the trip.

[1173] "User Information" refers to a user's profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget).

[1174] "Emotion data" refers to information about the emotional state recognized by the emotion engine through the user's facial expressions and voice.

[1175] "Server" refers to the central system that receives, stores, and analyzes user information and sentiment data, generates travel plans, and provides real-time support.

[1176] "Means of learning" refers to the algorithms and programs that allow the server to analyze user information and emotional data, understand the relationships between them, and use them to make future suggestions.

[1177] "Means for generating travel plans" refers to algorithms or programs for creating optimal travel plans based on travel conditions, user information, and emotional data.

[1178] "Means of providing real-time support" refers to systems and functions that take emotional data into account when a user makes an inquiry while traveling and immediately provide appropriate assistance or information.

[1179] "Means for sending feedback to the server" refers to the function of sending user ratings and correction requests from the terminal to the server, which then receives and analyzes them.

[1180] "User device" refers to the device (smartphone, tablet, PC, etc.) used by a user to enter or confirm travel plans, provide feedback, or receive real-time support.

[1181] The present invention is an AI concierge system that proposes personalized travel plans based on the user's preferences and emotions and provides support in real time. Specific embodiments are described below.

[1182] System Configuration

[1183] This system includes hardware and software such as user terminals, servers, databases, emotion engines, chatbots, translation services, and external collaboration services.

[1184] A user device is a device such as a smartphone, tablet, or PC that allows a user to operate an application to enter information, check travel plans, and provide feedback.

[1185] The server is built on a cloud service and is the core system that receives, stores, and analyzes user information and emotion data. Specific technologies that can be used include Amazon Web Services (AWS) and Microsoft Azure.

[1186] The database is a server-managed storage system that stores user information and emotion data. For example, a relational database such as MySQL or PostgreSQL can be used.

[1187] The emotion engine has the ability to analyze the user's facial expressions and voice to generate emotion data, which can be achieved using machine learning algorithms or voice analysis software (e.g., Google Cloud Speech-to-Text API).

[1188] A chatbot is an interface for users to make inquiries or request support during their trip, and responds in real time using natural language processing (NLP) techniques, such as Dialogflow and the Microsoft Bot Framework.

[1189] The translation service is a function that converts user inquiries into different languages, and can use the Google Translate API, etc.

[1190] External linkage services are a means of linking with external services such as taxi booking apps and restaurant reservation systems, allowing for immediate responses to user requests.

[1191] System Operation

[1192] When users download the app and launch it for the first time, they enter their profile information and travel preferences, and the emotion engine recognizes the user's facial expressions and voice to collect emotional data.

[1193] The device sends the input information and emotion data to the server, securely using the HTTPS protocol.

[1194] The server stores the received user information and emotion data in a database and learns the relationship between them. Specifically, it analyzes the data using a generative AI model (e.g., TensorFlow).

[1195] The user inputs the desired travel conditions (departure date, destination, travel mode, etc.) This information is also sent to the server via the terminal.

[1196] The server analyzes the travel conditions and the user's preference and emotional data to generate an optimal travel plan. For example, if the user wants to relax based on emotional data, it will select a plan that includes a quiet beach and a spa.

[1197] The terminal displays the generated travel plan to the user, who can review the plan and provide feedback.

[1198] The server re-analyzes the received feedback and emotional data and modifies the plan as needed, providing a final, customized plan.

[1199] If a user needs assistance during their trip, they can contact the chatbot, and the emotion engine will detect the user's stress level and provide assistance accordingly.

[1200] The server allows the chatbot to analyze the inquiry content and emotional data to generate the optimal response, and also provides comprehensive real-time support by linking with external services (e.g., translation services and taxi booking apps).

[1201] Specific examples

[1202] For new users

[1203] The user launches the app for the first time and enters their profile information and travel preferences. The emotion engine recognizes facial expressions and voice and collects emotional data. The device sends the information and emotional data to the server, which stores the information and emotional data in a database and performs learning. The user enters their desired travel conditions and the device sends them to the server. The server generates an optimal travel plan based on the conditions and emotional data. The device displays the travel plan, and the user reviews the plan and provides feedback. The server analyzes the feedback and emotional data and modifies the plan as necessary.

[1204] Prompt Sentence Examples

[1205] "I'm planning a week-long trip to Tokyo. I'd like to enjoy good food and relaxing activities. I'm on a moderate budget. Can you recommend a recommended itinerary?"

[1206] In this way, the system of the present invention can provide personalized travel plans and real-time support based on the user's preferences and emotions.

[1207] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1208] Step 1:

[1209] A user downloads the app and launches it for the first time. They enter their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget). The emotion engine recognizes emotions through facial expressions and voice, and collects emotion data.

[1210] Input: User information, emotion data

[1211] Output: User information and emotion data are stored on the device.

[1212] Step 2:

[1213] The device sends the entered user information and emotion data to the server, securely using the HTTPS protocol.

[1214] Input: User information, emotion data

[1215] Output: User information and emotion data received by the server

[1216] Step 3:

[1217] The server stores the received user information and emotion data in a database. Specifically, a relational database such as MySQL or PostgreSQL can be used. Furthermore, the data is analyzed using a generative AI model (e.g., TensorFlow) to learn the association between the emotion data and user information.

[1218] Input: User information and emotion data received by the server

[1219] Output: Parsed data stored in a database

[1220] Step 4:

[1221] The user inputs the desired travel conditions (departure date, destination, travel mode, etc.) This information is also sent to the server via the terminal.

[1222] Input: Travel conditions

[1223] Output: The travel conditions are sent to the server.

[1224] Step 5:

[1225] The server analyzes the travel conditions and the user's preference and emotional data to generate the optimal travel plan. For example, a user who wants to relax will be offered a plan that includes a quiet beach or spa.

[1226] Input: Travel conditions, user preference data, emotional data

[1227] Output: Generated itinerary

[1228] Step 6:

[1229] The terminal displays the travel plan sent from the server to the user.

[1230] Input: Generated itinerary

[1231] Output: The itinerary displayed to the user

[1232] Step 7:

[1233] The user reviews the proposed itinerary and provides corrections and feedback as needed, such as adding or removing specific activities or adjusting the budget.

[1234] Input: Feedback on travel plans

[1235] Output: The modification request is sent from the terminal to the server.

[1236] Step 8:

[1237] The server re-analyzes the received feedback and emotion data and modifies the plan as necessary, which is then sent back to the device for confirmation by the user.

[1238] Input: Feedback, emotion data

[1239] Output: Modified itinerary

[1240] Step 9:

[1241] The user confirms the confirmed travel plan and gives final approval, which the terminal notifies the server.

[1242] Input: Final travel plan approval

[1243] Output: Authorization data is sent to the server

[1244] Step 10:

[1245] If a user needs assistance during their trip, they can contact the chatbot, and the emotion engine will detect the user's stress level and provide assistance accordingly.

[1246] Input: Inquiry details, emotion data

[1247] Output: The device sends the query and emotion data to the server.

[1248] Step 11:

[1249] The server analyzes the inquiry content and emotional data, and the chatbot generates the optimal response. If necessary, it connects with translation services or taxi booking apps to provide support that takes the user's emotions into consideration.

[1250] Input: Inquiry details, emotion data

[1251] Output: Generated answers, supporting information provided

[1252] Step 12:

[1253] The device displays responses and support information from the server to the user.

[1254] Input: Support Information

[1255] Output: Support information displayed to the user

[1256] At each step, it can provide a personalized travel plan based on the user's preferences and emotions, providing comprehensive support during the trip.

[1257] (Application example 2)

[1258] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1259] Conventional travel plan suggestion systems make suggestions based on the user's preferences, but do not take into account the user's emotional state. This means that they lack support that is in line with the user's emotions and needs, which change in real time. Furthermore, they often cannot provide effective support to reduce stress and anxiety during travel.

[1260] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for inputting user information, a means for transmitting the user information to the server, a means for the server to learn the user information, a means for the server to generate an itinerary based on the user information, a means for the server to transmit the generated itinerary to a user terminal, a means for the user terminal to display the itinerary, a means for transmitting feedback from the user to the server, an emotion recognition means for collecting and recognizing the user's emotions, a means for adjusting the itinerary based on the user's emotions, and a real-time support means for performing the emotion recognition in real time and providing appropriate support. This enables the proposal of a personalized itinerary that takes into account the user's emotions and preferences. Furthermore, by recognizing the user's emotions in real time during the trip and providing appropriate support, it is possible to increase user satisfaction and reduce stress and anxiety.

[1261] A "means for inputting user information" is a device, software, or combination thereof that provides an interface for a user to input their profile information and preferences.

[1262] The "means for transmitting the user information to the server" refers to a device, software, or a combination thereof for transmitting the input user information to the server via a network.

[1263] "Means by which the server learns the user information" refers to an algorithm, device, software, or combination thereof that analyzes and stores the user information received by the server and makes personalized suggestions based on that information.

[1264] The "means by which the server generates a travel plan based on the user information" refers to an algorithm, device, software, or combination thereof for creating an optimal travel plan based on the user's preferences and emotional state.

[1265] The "means for transmitting the travel plan generated by the server to the user terminal" refers to a device, software, or a combination thereof for transmitting the generated travel plan to the user terminal via a network.

[1266] The "means for the user terminal to display the itinerary" refers to a device, software, or a combination thereof for visually or audibly presenting the itinerary received by the user terminal to the user.

[1267] The "means for transmitting the user feedback to the server" refers to a device, software, or a combination thereof for transmitting the user-provided feedback to the server.

[1268] The "emotion recognition means for collecting and recognizing the user's emotions" refers to a device, software, or a combination thereof for collecting and analyzing the user's facial expressions, voice, and other physiological data to recognize the user's emotional state.

[1269] The "means for adjusting the travel plan based on the user's emotions" refers to an algorithm, device, software, or a combination thereof for adjusting the optimal travel plan in real time based on the user's emotional data.

[1270] "Real-time support means for performing the emotion recognition in real time and providing appropriate support" refers to a device, software, or a combination thereof for monitoring a user's emotions in real time and providing immediate support or suggestions in response.

[1271] This invention is a system that proposes optimal travel plans based on the user's preferences and emotions and provides support in real time. The system collects and analyzes user information, monitors the user's emotions in real time using emotion recognition technology, and provides appropriate support.

[1272] System Configuration

[1273] This system consists of a user terminal, a server, an emotion recognition device, and a real-time support device.

[1274] User information collection and emotion recognition

[1275] user:

[1276] Users download the app and launch it for the first time, entering their profile information (name, age, gender) and preferences (favorite meals, activities they are interested in, budget).

[1277] Device:

[1278] The device transmits this user information and emotional data collected through the user's facial expressions and voice to the server.

[1279] server:

[1280] The server stores the received user information and emotion data in a database and learns from this data, thereby learning the relationship between user preferences and emotions.

[1281] Generate a travel plan

[1282] user:

[1283] The user inputs the desired travel conditions (departure date, destination, travel type, etc.).

[1284] Device:

[1285] The terminal transmits the input travel conditions to the server.

[1286] server:

[1287] The server analyzes the travel conditions and learned user preferences and emotional data to generate an optimal travel plan. Based on the emotional data, it selects activities that will relax the user and events that will excite them.

[1288] Device:

[1289] The terminal displays the travel plan sent from the server to the user.

[1290] user:

[1291] The user reviews the proposed itinerary and provides corrections and feedback as needed.

[1292] Customization and Real-Time Support

[1293] user:

[1294] Users request specific requests or changes (e.g., specific restaurant reservations, changes to transportation).

[1295] Device:

[1296] The terminal transmits the user's request and emotion data to the server.

[1297] server:

[1298] The server analyzes the request along with the emotional data and proposes optimal customization options. Taking the emotional data into consideration, the server adjusts the plan to best suit the user.

[1299] Device:

[1300] The terminal displays the final customized plan to the user.

[1301] user:

[1302] The user reviews and confirms the final plan.

[1303] Real-time support during your trip

[1304] user:

[1305] Users can contact the chatbot if they get lost while traveling or in an emergency.

[1306] Device:

[1307] The terminal transmits the inquiry content and emotion data to the server.

[1308] server:

[1309] The server analyzes the inquiry content and emotional data to generate the optimal response, and if necessary, connects with translation services or taxi booking apps to provide support that takes the user's emotions into consideration.

[1310] Device:

[1311] The device displays answers and support information to the user.

[1312] user:

[1313] The user will accept the support provided and contact us again if necessary.

[1314] Hardware and software used

[1315] Hardware:

[1316] Cameras and microphones onboard the self-driving vehicle to collect facial expressions and voice recordings of the user.

[1317] User devices (smartphones, etc.): To check travel plans and receive real-time support.

[1318] software:

[1319] Emotion Engine: Recognizes emotions by analyzing the user's facial expressions and voice.

[1320] Route Planner: Generates optimal travel plans based on user preferences and sentiment data.

[1321] Real-Time Support: Analyzes emotional data in real time while traveling and provides support such as changing music or adjusting the route.

[1322] Specific examples

[1323] If a user feels like relaxing on a weekday, the route planner will suggest a low-stress route and generate a plan that includes a stop at a scenic park along the way. Real-time support will play relaxing music when the user's mood changes and provide information about events the user can enjoy.

[1324] Example prompt sentence:

[1325] Enter your user's profile information and preferences, then use facial expressions and voice to collect emotional data and suggest a low-stress, relaxing driving route.

[1326] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1327] Step 1:

[1328] Entering user information

[1329] User: The user downloads and launches the app, then enters their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget).

[1330] Input: Profile information and preferences

[1331] Output: The input data is saved on the device and sent to the server.

[1332] Step 2:

[1333] Collecting Emotional Data

[1334] Device: The device uses the camera and microphone built into the device to collect the user's facial expressions and voice.

[1335] Input: User's facial expression and voice data

[1336] Output: The collected data is sent to the emotion recognizer.

[1337] Step 3:

[1338] Emotional Data Analysis

[1339] Server: The server uses an emotion engine to analyze the transmitted facial expression and voice data and identify the user's emotional state.

[1340] Input: facial expression data and voice data

[1341] Output: Recognized emotion data (e.g., joy, stress)

[1342] Step 4:

[1343] Store user information and emotional data

[1344] Server: The server stores user profile information, preferences, and sentiment data in a database.

[1345] Input: Profile information, preferences, recognized emotion data

[1346] Output: Consolidated data stored in a database

[1347] Step 5:

[1348] Enter travel conditions

[1349] User: The user inputs the desired travel conditions (departure date, destination, travel mode, etc.).

[1350] Input: Travel conditions

[1351] Output: The input data is saved on the device and sent to the server.

[1352] Step 6:

[1353] Generate a travel plan

[1354] Server: The server analyzes travel conditions, user preferences, and emotional data, and runs algorithms to generate optimal travel plans.

[1355] Input: Travel conditions, preferences, emotional data

[1356] Output: Generated itinerary

[1357] Step 7:

[1358] View travel plans

[1359] Terminal: The terminal displays the generated itinerary to the user.

[1360] Input: Travel Plan

[1361] Output: Display of itinerary

[1362] Step 8:

[1363] Providing and submitting feedback

[1364] User: The user reviews the itinerary and provides corrections and feedback as needed.

[1365] Device: The device sends feedback to the server.

[1366] Input: User feedback

[1367] Output: Feedback sent to the server

[1368] Step 9:

[1369] Generate a customized plan

[1370] Server: The server analyzes the feedback and sentiment data and generates a final customized itinerary.

[1371] Input: Feedback, emotion data

[1372] Output: A customized itinerary

[1373] Step 10:

[1374] View your customized plan

[1375] Terminal: The terminal displays the final customized itinerary to the user.

[1376] Input: Customized Travel Plan

[1377] Output: Display of customized itinerary

[1378] Step 11:

[1379] Real-time support during your trip

[1380] User: The user will use the chatbot to request assistance or queries during their journey.

[1381] Terminal: The terminal sends the query and emotion data to the server.

[1382] Server: The server analyzes the inquiry content and emotion data to generate the most appropriate answer or support. If necessary, it connects with translation services or other external applications to provide support that takes the user's emotions into consideration.

[1383] Input: User query and emotion data

[1384] Output: Answers and supporting information

[1385] Step 12:

[1386] View support information

[1387] Terminal: The terminal displays answers and support information sent from the server to the user.

[1388] Input: Answers and supporting information

[1389] Output: Displayed answers and supporting information

[1390] This allows users to experience the most suitable trip according to their preferences and emotions, reducing stress and anxiety while traveling and providing a highly satisfying travel experience.

[1391] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1392] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1393] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1394] [Third embodiment]

[1395] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1396] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1397] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1398] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1399] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1400] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1401] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1402] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1403] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1404] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1405] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1406] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1407] This invention is an AI concierge system that proposes personalized travel plans based on the user's preferences. The program processing of this system is explained in detail below in natural language.

[1408] 1. Collection of User Information

[1409] User

[1410] A user downloads the app and launches it for the first time.

[1411] Enter your profile information (name, age, gender) and travel preferences (favorite meals, activities you're interested in, budget).

[1412] Terminal

[1413] The input information is sent to the server.

[1414] server

[1415] The received user information is stored in a database and trained by a machine learning algorithm, which then creates a dataset to suggest optimal travel plans for the user.

[1416] 2. Generate a travel plan

[1417] User

[1418] Enter the conditions of your desired trip (departure date, destination, travel method, etc.).

[1419] Terminal

[1420] The entered conditions are sent to the server.

[1421] server

[1422] The system analyzes the received information and uses machine learning algorithms to generate a travel plan based on the user's preferences, taking into account weather information, local event information, budget, and other factors to create the optimal plan.

[1423] Terminal

[1424] The travel plan sent from the server is displayed to the user.

[1425] User

[1426] Review the proposed itinerary and provide corrections and feedback as needed.

[1427] 3. In-depth support and customization

[1428] User

[1429] Request specific requests or changes (e.g., a specific restaurant reservation, change of transportation).

[1430] Terminal

[1431] Sends the request to the server.

[1432] server

[1433] We analyze your request, suggest possible customizations, and make any necessary reservations and arrangements.

[1434] Terminal

[1435] Display the final customized plan to the user.

[1436] User

[1437] Review and finalize the plan.

[1438] 4. Real-time support during your trip

[1439] User

[1440] If you get lost or have any other inquiries while you're there, you can contact them through the chatbot.

[1441] Terminal

[1442] The query is sent to the server.

[1443] server

[1444] The chatbot analyzes the inquiry and generates the most appropriate answer, linking with translation services and taxi booking apps as needed.

[1445] Terminal

[1446] Display answers and support information to users.

[1447] User

[1448] Accept the support offered and contact us again if necessary.

[1449] Specific examples

[1450] For new users

[1451] 1. A user launches the app for the first time and enters their profile information and travel preferences.

[1452] 2. The device sends this information to the server.

[1453] 3. The server stores the information in a database and uses machine learning algorithms to learn from it.

[1454] 4. The user enters the desired travel conditions, and the terminal sends them to the server.

[1455] 5. The server generates an optimal travel plan and sends it to the device.

[1456] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[1457] 7. The server receives the feedback, modifies the plan as needed, and provides the final plan to the user.

[1458] Real-time consultation during your trip

[1459] 1. A user gets lost in the area and makes an inquiry through a chatbot.

[1460] 2. The device sends the query to the server.

[1461] 3. The server generates the best answer and, if necessary, connects with a translation service or taxi booking app.

[1462] 4. The device displays answers and support information to the user.

[1463] 5. The user accepts the support offered and contacts us again if necessary.

[1464] In this way, the system of the present invention proposes travel plans based on the user's preferences and provides comprehensive support during the trip, allowing the user to easily realize the ideal travel experience.

[1465] The processing flow will be explained below.

[1466] Step 1:

[1467] A user downloads the app and launches it for the first time.

[1468] Step 2:

[1469] The device displays an interface for entering the user's profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget).

[1470] Step 3:

[1471] The user enters the required information and presses the send button.

[1472] Step 4:

[1473] The terminal transmits the input information to the server.

[1474] Step 5:

[1475] The server stores the received user information in a database.

[1476] Step 6:

[1477] The server runs machine learning algorithms based on the stored user information to learn the user's preferences.

[1478] Step 7:

[1479] The user inputs the conditions of the desired trip (departure date, destination, travel mode, etc.).

[1480] Step 8:

[1481] The terminal transmits the input travel conditions to the server.

[1482] Step 9:

[1483] The server analyzes the travel conditions received and compares them with the user's preference data.

[1484] Step 10:

[1485] The server uses deep learning algorithms to generate the optimal travel plan (destinations, activities, accommodation, etc.).

[1486] Step 11:

[1487] The server transmits the generated travel plan to the terminal.

[1488] Step 12:

[1489] The terminal displays the travel plan sent from the server to the user.

[1490] Step 13:

[1491] The user reviews the proposed itinerary and enters corrections or feedback as necessary.

[1492] Step 14:

[1493] The device sends the user's feedback to the server.

[1494] Step 15:

[1495] The server analyzes the feedback and modifies the travel plan.

[1496] Step 16:

[1497] The server sends the final revised itinerary to the terminal.

[1498] Step 17:

[1499] The device displays the final revised plan to the user.

[1500] Step 18:

[1501] The user reviews and confirms the final plan.

[1502] Step 19:

[1503] If a user gets lost or needs information while in the area, they can make inquiries through the chatbot function.

[1504] Step 20:

[1505] The terminal sends the inquiry to the server.

[1506] Step 21:

[1507] The server uses a chatbot to analyze the inquiry and generate the most appropriate answer.

[1508] Step 22:

[1509] The server will connect with translation services and taxi booking apps as needed.

[1510] Step 23:

[1511] The server generates responses and support information and sends them to the device.

[1512] Step 24:

[1513] The device displays answers and support information to the user.

[1514] Step 25:

[1515] The user accepts the support offered and contacts the company again if necessary.

[1516] Through this series of steps, users can receive suggestions for optimal travel plans and receive support based on their requests while traveling with peace of mind.

[1517] Example 1

[1518] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1519] Conventional travel plan suggestion systems have difficulty in providing advanced customization based on users' preferences and individual needs, and do not provide sufficient real-time support during the trip. Furthermore, it is difficult to provide support in conjunction with external translation services or taxi booking services. This makes it difficult for users to achieve their ideal travel experience.

[1520] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1521] In this invention, the server includes a means for storing user information in a database and learning using a machine learning algorithm, a means for generating a travel plan based on the user's preferences, and a means for analyzing travel conditions and generating an optimal travel plan. This allows for personalized travel plans based on the user's preferences and needs. It can also accommodate customization requests and provide real-time support on-site, allowing users to enjoy an ideal travel experience.

[1522] "User Information" means information including a User's profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget).

[1523] "Server" means a computer system that stores and analyzes information submitted by users and generates travel plans.

[1524] A "database" is a structured collection of data for storing user information and travel plans.

[1525] A "machine learning algorithm" is a computational method for analyzing data based on user preferences and building predictive models.

[1526] A "travel plan" is a plan that includes travel schedules and suggestions generated based on the user's preferences and conditions.

[1527] "Travel conditions" refers to information such as the departure date, destination, and travel mode desired by the user when traveling.

[1528] A "customization request" is a request by a user to communicate specific wishes or modifications to the system.

[1529] "Real-time support" is a service that provides immediate support and assistance to problems and inquiries users may encounter while traveling.

[1530] A "chatbot" is a program that uses natural language processing technology to automatically respond to user questions and requests.

[1531] "External linkage services" are support functions provided by the system in collaboration with external services, such as translation services and taxi booking services.

[1532] This invention is an AI concierge system that proposes personalized travel plans based on the user's preferences. The program processing of this system is described in detail below.

[1533] Collection of User Information

[1534] When a user first launches the app on their smartphone or tablet, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget). The device sends this information to the server. The server stores the received user information in a database and uses machine learning algorithms (such as Scikit-Learn or TensorFlow) in Python to train the database. This information is used to create a dataset that can be used to suggest optimal travel plans to the user.

[1535] Generate a travel plan

[1536] The user enters the conditions of their desired trip (departure date, destination, travel mode, etc.) into the app. The device sends these conditions to the server. The server analyzes the received conditions and generates an optimal travel plan based on the user's preference data and travel conditions. In doing so, it uses weather information APIs and local event information APIs and also takes budget into consideration. The server sends the generated travel plan to the user's device, which then displays it to the user. The user can review the displayed travel plan and provide feedback if necessary.

[1537] In-depth support and customization

[1538] Users can request specific requests or changes through the app. For example, they can input requests to make a specific restaurant reservation or change their transportation method. The device sends these requests to the server. The server analyzes the request, suggests possible customizations, and makes the necessary reservations and arrangements. This allows for customization that meets the user's individual needs. The final customized plan is generated and displayed on the device. The user reviews and confirms the final plan.

[1539] Real-time support during your trip

[1540] If a user gets lost or needs other support while traveling, they can make an inquiry using the chatbot function within the app. The device sends the inquiry to the server. The server uses the chatbot to analyze the inquiry and generate the most appropriate answer. If necessary, it also connects with translation service APIs and taxi dispatch app APIs. This allows the user to receive the necessary support in real time. The generated answer and support information are displayed on the device, and the user can check and respond.

[1541] Specific examples

[1542] For new users

[1543] 1. A user launches the app for the first time and enters their profile information and travel preferences.

[1544] 2. The device sends this information to the server.

[1545] 3. The server stores the information in a database and uses machine learning algorithms to learn from it.

[1546] 4. The user enters the desired travel conditions, and the terminal sends them to the server.

[1547] 5. The server generates an optimal travel plan and sends it to the device.

[1548] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[1549] 7. The server receives the feedback, modifies the plan as needed, and provides the final plan to the user.

[1550] Real-time consultation during your trip

[1551] 1. A user gets lost in the area and makes an inquiry through a chatbot.

[1552] 2. The device sends the query to the server.

[1553] 3. The server generates the best answer and, if necessary, connects with a translation service or taxi booking app.

[1554] 4. The device displays answers and support information to the user.

[1555] 5. The user accepts the support offered and contacts us again if necessary.

[1556] The system allows users to create optimal travel plans tailored to their preferences and receive comprehensive support during their trip.

[1557] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1558] Program processing flow

[1559] Step 1: Enter your user information

[1560] input:

[1561] When a user launches the app for the first time, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget).

[1562] explanation:

[1563] A user downloads and launches the app on their smartphone or tablet.

[1564] Enter information such as your name, age, gender, and travel preferences into the in-app form.

[1565] The user presses the "Submit" button to confirm the entered information.

[1566] output:

[1567] A dataset of user profile information and preference information

[1568] Step 2: Submit your information

[1569] input:

[1570] User information entered in step 1

[1571] explanation:

[1572] The device sends the user's input information to the backend server via an HTTP POST request.

[1573] Information is transmitted using SSL / TLS encrypted communications.

[1574] output:

[1575] The server receives the user information.

[1576] Step 3: Database storage and machine learning

[1577] input:

[1578] The dataset of user information received in step 2

[1579] explanation:

[1580] The server stores the received user information in a database.

[1581] The database used is MySQL.

[1582] The server uses Python to analyze the received data using machine learning algorithms such as Scikit-Learn and TensorFlow, and performs learning.

[1583] A user preference profile is generated from the learning results.

[1584] output:

[1585] User preference profile

[1586] Step 4: Enter your travel requirements

[1587] input:

[1588] User's desired travel conditions (departure date, destination, travel type, etc.)

[1589] explanation:

[1590] The user enters their travel requirements into a dedicated form within the app.

[1591] Also consider options such as those for families with children or for active outdoorsy people.

[1592] After the user has entered all the required information, he clicks "Submit."

[1593] output:

[1594] Dataset of user desired travel conditions

[1595] Step 5: Sending conditions

[1596] input:

[1597] The data set of travel conditions entered in step 4

[1598] explanation:

[1599] The terminal sends the travel conditions entered by the user to the server via an HTTP POST request.

[1600] Communications are encrypted using SSL / TLS.

[1601] output:

[1602] Travel conditions received

[1603] Step 6: Create a travel plan

[1604] input:

[1605] User information and travel conditions collected in Steps 2 and 4

[1606] explanation:

[1607] The server generates the optimal travel plan based on the user's preference profile and travel conditions using weather information APIs and local event information APIs.

[1608] Travel plans are automatically generated using machine learning algorithms (such as TensorFlow).

[1609] Adjust your plan taking into account your budget.

[1610] output:

[1611] The perfect travel plan for you

[1612] Step 7: Submit and view your plan

[1613] input:

[1614] The itinerary generated in step 6

[1615] explanation:

[1616] The server sends the generated travel plan to the terminal as an HTTP response.

[1617] The device displays the received travel plan to the user.

[1618] output:

[1619] The itinerary displayed to the user

[1620] Step 8: Provide feedback

[1621] input:

[1622] User feedback information

[1623] explanation:

[1624] The user checks the proposed travel plan and, if necessary, inputs changes or corrections as feedback.

[1625] The terminal transmits the feedback information to the server.

[1626] output:

[1627] Feedback information dataset

[1628] Step 9: Submit a customization request

[1629] input:

[1630] Customization Request Information

[1631] explanation:

[1632] The user inputs a specific request (e.g., a specific restaurant reservation, a change of transportation method).

[1633] The terminal sends these requests to the server.

[1634] output:

[1635] Customization request information dataset

[1636] Step 10: Customization and Final Plan Generation

[1637] input:

[1638] Customization request information received in step 9

[1639] explanation:

[1640] The server analyzes the received request and adds any necessary customizations to the itinerary.

[1641] Make any necessary reservations and arrangements and generate your final itinerary.

[1642] output:

[1643] A customized ultimate itinerary

[1644] Step 11: Submit and review your final plan

[1645] input:

[1646] The final itinerary generated in step 10

[1647] explanation:

[1648] The server sends the final customized itinerary to the terminal as an HTTP response.

[1649] The device displays the final plan received to the user.

[1650] The user reviews and confirms the final plan.

[1651] output:

[1652] Final confirmed travel plans

[1653] Step 12: Real-time support during your trip

[1654] input:

[1655] Real-time support inquiries from users

[1656] explanation:

[1657] Users can use chatbots to make inquiries while traveling.

[1658] The terminal sends the inquiry to the server.

[1659] The server analyzes the data using a chatbot and generates the most appropriate answer.

[1660] If necessary, it will also link with translation service APIs and taxi booking app APIs.

[1661] The device displays generated answers and supporting information to the user.

[1662] output:

[1663] Responses to inquiries and support information

[1664] In this way, users can create the perfect travel plan tailored to their preferences and receive comprehensive support during their trip.

[1665] (Application example 1)

[1666] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1667] While conventional travel plan suggestion systems can provide personalized plans based on user preferences, they lack a way for users to visually visualize the atmosphere and locations of their travel destinations. Furthermore, providing real-time support during a trip requires users to use multiple apps simultaneously, which is inconvenient for users. There is a need to solve these problems and realize a more comprehensive travel experience.

[1668] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1669] In this invention, the server includes means for inputting user information, means for transmitting the user information to the server, means for the server to learn the user information, means for the server to generate a travel plan based on the user information, means for the server to transmit the generated travel plan to a user terminal, means for the user terminal to display the travel plan, means for transmitting feedback from the user to the server, means for providing real-time support, and means for providing a virtual reality tour based on the travel plan, thereby enabling the user to visually experience the suggested travel destinations in virtual reality and receive comprehensive real-time support, thereby enabling the user to plan and execute a more satisfying trip.

[1670] "User Information" means data relating to a user's profile information, travel preferences and travel requirements.

[1671] The "server" is a computer system that receives user information, learns and analyzes it, and generates and transmits the optimal travel plan.

[1672] "User terminal" means an electronic device, typically a smartphone or tablet, used to input user information and receive and display travel plans and support information.

[1673] "Virtual reality tour" is a feature that allows users to visually experience the atmosphere and locations of their travel destinations through virtual reality technology.

[1674] A "trip plan" is a plan that includes a suggested travel itinerary and destination details generated based on a user's preferences and conditions.

[1675] "Real-time support" is a support service that provides immediate response and assistance to users' inquiries and problems while traveling.

[1676] A "machine learning algorithm" is an algorithm that learns patterns based on collected user information and generates travel plans based on the results of that learning.

[1677] In order to put the present invention into practice, it is necessary to build a system in which multiple pieces of hardware and software work together. A specific embodiment of this system is shown below.

[1678] 1. Collection and storage of user information

[1679] When a user downloads and launches the application for the first time, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget). This information is sent from the device (e.g., smartphone) used to the server. The server stores the received user information in a database and uses machine learning algorithms to learn from it. This step forms the base dataset for providing personalized travel plans based on the user information.

[1680] 2. Generate a travel plan

[1681] When a user inputs the conditions for their desired travel plan (departure date, destination, travel mode, etc.), these conditions are also sent to the server via their device. The server analyzes the received conditions and uses a machine learning algorithm to generate a travel plan based on the user's preferences. This also takes into account factors such as weather information, local event information, and budget. The generated travel plan is sent from the server to the user's device and displayed to the user.

[1682] 3. Providing VR tours

[1683] After users confirm the proposed itinerary, they can experience a virtual reality (VR) tour of specific tourist spots or points. This function is available by installing an application on a VR headset or smartphone. For example, if a user selects a plan with "Kyoto" as the destination, they can virtually visit Kyoto's famous tourist spots.

[1684] 4. Real-time support during your trip

[1685] If a user gets lost or needs emergency assistance while traveling, they can receive real-time support using the chatbot within the application. When the user enters their inquiry, it is sent from the device to the server, which uses machine learning algorithms to generate the best answer. If necessary, it can also be linked to a translation service or a taxi booking app.

[1686] 5. Specific Examples

[1687] For new users, the flow is as follows:

[1688] 1. A user launches the app for the first time and enters their profile information and travel preferences.

[1689] 2. The device sends this information to the server.

[1690] 3. The server stores the information in a database and uses machine learning algorithms to learn from it.

[1691] 4. The user enters the desired travel conditions, and the device sends them to the server.

[1692] 5. The server generates an optimal travel plan and sends it to the device.

[1693] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[1694] 7. The server receives the feedback, modifies the plan as needed, and provides the final plan to the user.

[1695] As an example of real-time support during travel, here is a specific example of a user getting lost and making an inquiry through a chatbot:

[1696] 1. A user gets lost in the area and makes an inquiry through a chatbot.

[1697] 2. The device sends the query to the server.

[1698] 3. The server generates the best answer and, if necessary, connects with a translation service or taxi booking app.

[1699] 4. The device displays answers and support information to the user.

[1700] 5. The user accepts the support provided and contacts us again if necessary.

[1701] Examples of prompts include:

[1702] "After users download the app and launch it for the first time, they enter the following information:

[1703] Name: User A

[1704] Age: 30

[1705] Gender: Male

[1706] Favorite food: Japanese food

[1707] Interested in: Hiking

[1708] Budget: 50,000 yen

[1709] Next, enter your travel plan requirements:

[1710] Departure date: July 1, 2024

[1711] Destination: Kyoto

[1712] Travel type: Solo travel

[1713] By following these prompts, each function of the system will function properly, providing users with a personalized travel experience.

[1714] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1715] Step 1:

[1716] Collection of User Information

[1717] When a user launches the application for the first time, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget). The device sends this information to the server, which stores the received user information in a database and begins learning using machine learning algorithms. This process forms the base data set for suggesting personalized travel plans to the user.

[1718] Input: User profile information and travel preference data

[1719] Output: User information stored in the database

[1720] Step 2:

[1721] Enter travel plan conditions

[1722] The user inputs the travel plan conditions (departure date, destination, travel mode, etc.). This information is sent to the server via the device. The server analyzes the received conditions and uses machine learning algorithms to generate a travel plan based on the user's preferences. This also takes into account factors such as weather information, local event information, and budget.

[1723] Input: User's travel plan conditions

[1724] Output: Parsed itinerary data

[1725] Step 3:

[1726] Generate and display travel plans

[1727] The server sends the generated travel plan to the user's device, which then displays the plan to the user. The user reviews the proposed plan and provides feedback if necessary. This feedback is sent from the device to the server, which then revises the plan based on that feedback. The final plan is then provided to the user.

[1728] Input: Server-generated travel plan data, user feedback

[1729] Output: Finalized itinerary

[1730] Step 4:

[1731] Providing virtual reality tours

[1732] After reviewing the proposed itinerary, users can experience a virtual reality (VR) tour of specific tourist spots or points. VR content is sent to the device, and users can visually experience the content with a VR headset or smartphone. For example, if a user selects "Kyoto" as their destination, they can virtually visit Kyoto's famous tourist spots.

[1733] Input: Proposed itinerary and related VR content

[1734] Output: The virtual reality tour the user experiences

[1735] Step 5:

[1736] Real-time support during your trip

[1737] If a user gets lost or needs emergency assistance while traveling, they can receive real-time support from a chatbot within the application. When the user enters their inquiry, it is sent from the device to the server. The server uses a machine learning algorithm to generate the optimal answer and sends it to the device. If necessary, it can also be linked to a translation service or a taxi booking app.

[1738] Input: User's support request

[1739] Output: Answers and support information from the chatbot

[1740] The above steps will create a system that allows users to create personalized travel plans, experience virtual reality tours, and receive real-time support throughout their trip.

[1741] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1742] This invention is an AI concierge system that proposes personalized travel plans based on the user's preferences and emotions and provides real-time support. The system incorporates an emotion engine and has the function of adjusting plans and support content according to the user's emotions.

[1743] User information collection and emotion recognition

[1744] User

[1745] Download the app and launch it for the first time.

[1746] Enter your profile information (name, age, gender) and travel preferences (favorite meals, activities you're interested in, budget). The app is equipped with an emotion engine that recognizes the user's emotions through facial expressions and voice.

[1747] Terminal

[1748] The input information and emotion data are sent to the server.

[1749] server

[1750] The received user information and emotion data are stored in a database, which allows the system to learn the relationship between user preferences and emotions.

[1751] Generate a travel plan

[1752] User

[1753] Enter the conditions of your desired trip (departure date, destination, travel method, etc.).

[1754] Terminal

[1755] The entered travel conditions are sent to the server.

[1756] server

[1757] The system analyzes the user's preference and emotional data along with travel conditions to generate an optimal travel plan. Based on the emotional data, it selects activities that will relax the user and events that will excite them.

[1758] Terminal

[1759] The travel plan sent from the server is displayed to the user.

[1760] User

[1761] Review the proposed itinerary and provide corrections and feedback as needed.

[1762] Customization and Real-Time Support

[1763] User

[1764] Request specific requests or changes (e.g., a specific restaurant reservation, change of transportation).

[1765] Terminal

[1766] Send the user's request and emotion data to the server.

[1767] server

[1768] Analyzes the request along with emotional data and proposes optimal customization. Taking emotional data into consideration, adjusts plans to ensure user satisfaction.

[1769] Terminal

[1770] Display the final customized plan to the user.

[1771] User

[1772] Review and confirm the final plan.

[1773] Real-time support during your trip

[1774] User

[1775] If you get lost or have an emergency, you can contact us through the chatbot.

[1776] Terminal

[1777] The inquiry content and emotion data are sent to the server.

[1778] server

[1779] The chatbot analyzes the inquiry content and emotional data to generate the most appropriate response, and if necessary, connects with translation services or taxi booking apps to provide support that takes the user's emotions into consideration.

[1780] Terminal

[1781] Display answers and support information to users.

[1782] User

[1783] Accept the support offered and follow up if necessary.

[1784] Specific examples

[1785] For new users

[1786] 1. A user launches the app for the first time and enters their profile information and travel preferences. The emotion engine recognizes the user's facial expressions and voice to collect emotional data.

[1787] 2. The device sends information and emotion data to the server.

[1788] 3. The server stores the information and emotion data in a database and performs learning.

[1789] 4. The user enters the desired travel conditions, and the terminal sends them to the server.

[1790] 5. The server generates an optimal travel plan based on the conditions and emotion data.

[1791] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[1792] 7. The server analyzes the feedback and sentiment data, modifies the plan as needed, and provides the final plan.

[1793] Real-time consultation during your trip

[1794] 1. When a user gets lost in the area and makes an inquiry through the chatbot, the emotion engine detects the user's stress level.

[1795] 2. The device sends the inquiry and emotion data to the server.

[1796] 3. The server analyzes the inquiry and emotional data to generate the optimal response. If necessary, it connects with translation services or taxi booking apps to provide support that reduces the user's stress.

[1797] 4. The device displays answers and support information to the user.

[1798] 5. The user accepts the support offered and contacts us again if necessary.

[1799] In this way, the system of the present invention incorporating an emotion engine can provide personalized travel plans that take into account not only the user's preferences but also their emotions, providing comprehensive support during the trip.

[1800] The processing flow will be explained below.

[1801] Step 1:

[1802] A user downloads the app and launches it for the first time.

[1803] Step 2:

[1804] The device asks the user to enter their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget).

[1805] Step 3:

[1806] The user enters their profile information and travel preferences and hits the submit button.

[1807] Step 4:

[1808] The terminal transmits the input information to the server.

[1809] Step 5:

[1810] The server stores the received user information in a database.

[1811] Step 6:

[1812] The device uses an emotion engine to collect emotional data from the user's facial expressions and voice.

[1813] Step 7:

[1814] The device transmits the emotion data to the server.

[1815] Step 8:

[1816] The server runs a machine learning algorithm based on user information and emotional data to learn the relationship between user preferences and emotions.

[1817] Step 9:

[1818] The user inputs the conditions of the desired trip (departure date, destination, travel mode, etc.).

[1819] Step 10:

[1820] The terminal transmits the input travel conditions to the server.

[1821] Step 11:

[1822] The server analyzes the travel conditions received and generates an optimal travel plan based on them. Based on the emotional data, it selects activities that will relax or excite the user.

[1823] Step 12:

[1824] The server sends the generated travel plan to the user's terminal.

[1825] Step 13:

[1826] The device displays the travel plan to the user.

[1827] Step 14:

[1828] The user reviews the proposed itinerary and enters corrections or feedback as necessary.

[1829] Step 15:

[1830] The device sends the user's feedback to the server.

[1831] Step 16:

[1832] The server analyzes the feedback and modifies the travel plan, taking into account the emotional data.

[1833] Step 17:

[1834] The server sends the final revised itinerary to the terminal.

[1835] Step 18:

[1836] The device displays the final revised plan to the user.

[1837] Step 19:

[1838] The user reviews and confirms the final plan.

[1839] Step 20:

[1840] If a user gets lost or stressed while in the area, they can contact the chatbot to inquire.

[1841] Step 20A:

[1842] The device monitors the user's emotions in real time and transmits the emotion data along with the query to the server.

[1843] Step 21:

[1844] The server uses a chatbot to analyze the inquiry content and emotional data and generate the most appropriate answer.

[1845] Step 22:

[1846] The server will connect with translation services and taxi booking apps as needed to provide support that takes the user's emotions into consideration.

[1847] Step 23:

[1848] The server generates responses and support information and sends them to the device.

[1849] Step 24:

[1850] The device displays answers and support information to the user.

[1851] Step 25:

[1852] The user accepts the support offered and contacts the company again if necessary.

[1853] In this way, it becomes possible to provide personalized responses based on user preferences as well as emotional data, providing a more personalized travel experience and real-time support.

[1854] Example 2

[1855] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1856] Conventional travel planning systems have difficulty providing plans that fully take into account the user's preferences and emotions, and they also lack the ability to provide satisfactory real-time support. This means that users cannot receive appropriate support tailored to their individual needs and emotions, which can lead to a decrease in satisfaction during their trip.

[1857] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for learning user information and emotion data, means for generating a travel plan based on the user information and emotion data, and means for providing real-time support based on the emotion data. This makes it possible to provide a personalized travel plan based on the user's preferences and emotions and comprehensive support during the trip.

[1858] "User Information" refers to a user's profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget).

[1859] "Emotion data" refers to information about the emotional state recognized by the emotion engine through the user's facial expressions and voice.

[1860] "Server" refers to the central system that receives, stores, and analyzes user information and sentiment data, generates travel plans, and provides real-time support.

[1861] "Means of learning" refers to the algorithms and programs that allow the server to analyze user information and emotional data, understand the relationships between them, and use them to make future suggestions.

[1862] "Means for generating travel plans" refers to algorithms or programs for creating optimal travel plans based on travel conditions, user information, and emotional data.

[1863] "Means of providing real-time support" refers to systems and functions that take emotional data into account when a user makes an inquiry while traveling and immediately provide appropriate assistance or information.

[1864] "Means for sending feedback to the server" refers to the function of sending user ratings and correction requests from the terminal to the server, which then receives and analyzes them.

[1865] "User device" refers to the device (smartphone, tablet, PC, etc.) used by a user to enter or confirm travel plans, provide feedback, or receive real-time support.

[1866] The present invention is an AI concierge system that proposes personalized travel plans based on the user's preferences and emotions and provides support in real time. Specific embodiments are described below.

[1867] System Configuration

[1868] This system includes hardware and software such as user terminals, servers, databases, emotion engines, chatbots, translation services, and external collaboration services.

[1869] A user device is a device such as a smartphone, tablet, or PC that allows a user to operate an application to enter information, check travel plans, and provide feedback.

[1870] The server is built on a cloud service and is the core system that receives, stores, and analyzes user information and emotion data. Specific technologies that can be used include Amazon Web Services (AWS) and Microsoft Azure.

[1871] The database is a server-managed storage system that stores user information and emotion data. For example, a relational database such as MySQL or PostgreSQL can be used.

[1872] The emotion engine has the ability to analyze the user's facial expressions and voice to generate emotion data, which can be achieved using machine learning algorithms or voice analysis software (e.g., Google Cloud Speech-to-Text API).

[1873] A chatbot is an interface for users to make inquiries or request support during their trip, and responds in real time using natural language processing (NLP) techniques, such as Dialogflow and the Microsoft Bot Framework.

[1874] The translation service is a function that converts user inquiries into different languages, and can use the Google Translate API, etc.

[1875] External linkage services are a means of linking with external services such as taxi booking apps and restaurant reservation systems, allowing for immediate responses to user requests.

[1876] System Operation

[1877] When users download the app and launch it for the first time, they enter their profile information and travel preferences, and the emotion engine recognizes the user's facial expressions and voice to collect emotional data.

[1878] The device sends the input information and emotion data to the server, securely using the HTTPS protocol.

[1879] The server stores the received user information and emotion data in a database and learns the relationship between them. Specifically, it analyzes the data using a generative AI model (e.g., TensorFlow).

[1880] The user inputs the desired travel conditions (departure date, destination, travel mode, etc.) This information is also sent to the server via the terminal.

[1881] The server analyzes the travel conditions and the user's preference and emotional data to generate an optimal travel plan. For example, if the user wants to relax based on emotional data, it will select a plan that includes a quiet beach and a spa.

[1882] The terminal displays the generated travel plan to the user, who can review the plan and provide feedback.

[1883] The server re-analyzes the received feedback and emotional data and modifies the plan as needed, providing a final, customized plan.

[1884] If a user needs assistance during their trip, they can contact the chatbot, and the emotion engine will detect the user's stress level and provide assistance accordingly.

[1885] The server allows the chatbot to analyze the inquiry content and emotional data to generate the optimal response, and also provides comprehensive real-time support by linking with external services (e.g., translation services and taxi booking apps).

[1886] Specific examples

[1887] For new users

[1888] The user launches the app for the first time and enters their profile information and travel preferences. The emotion engine recognizes facial expressions and voice and collects emotional data. The device sends the information and emotional data to the server, which stores the information and emotional data in a database and performs learning. The user enters their desired travel conditions and the device sends them to the server. The server generates an optimal travel plan based on the conditions and emotional data. The device displays the travel plan, and the user reviews the plan and provides feedback. The server analyzes the feedback and emotional data and modifies the plan as necessary.

[1889] Prompt Sentence Examples

[1890] "I'm planning a week-long trip to Tokyo. I'd like to enjoy good food and relaxing activities. I'm on a moderate budget. Can you recommend a recommended itinerary?"

[1891] In this way, the system of the present invention can provide personalized travel plans and real-time support based on the user's preferences and emotions.

[1892] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1893] Step 1:

[1894] A user downloads the app and launches it for the first time. They enter their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget). The emotion engine recognizes emotions through facial expressions and voice, and collects emotion data.

[1895] Input: User information, emotion data

[1896] Output: User information and emotion data are stored on the device.

[1897] Step 2:

[1898] The device sends the entered user information and emotion data to the server, securely using the HTTPS protocol.

[1899] Input: User information, emotion data

[1900] Output: User information and emotion data received by the server

[1901] Step 3:

[1902] The server stores the received user information and emotion data in a database. Specifically, a relational database such as MySQL or PostgreSQL can be used. Furthermore, the data is analyzed using a generative AI model (e.g., TensorFlow) to learn the association between the emotion data and user information.

[1903] Input: User information and emotion data received by the server

[1904] Output: Parsed data stored in a database

[1905] Step 4:

[1906] The user inputs the desired travel conditions (departure date, destination, travel mode, etc.) This information is also sent to the server via the terminal.

[1907] Input: Travel conditions

[1908] Output: The travel conditions are sent to the server.

[1909] Step 5:

[1910] The server analyzes the travel conditions and the user's preference and emotional data to generate the optimal travel plan. For example, a user who wants to relax will be offered a plan that includes a quiet beach or spa.

[1911] Input: Travel conditions, user preference data, emotional data

[1912] Output: Generated itinerary

[1913] Step 6:

[1914] The terminal displays the travel plan sent from the server to the user.

[1915] Input: Generated itinerary

[1916] Output: The itinerary displayed to the user

[1917] Step 7:

[1918] The user reviews the proposed itinerary and provides corrections and feedback as needed, such as adding or removing specific activities or adjusting the budget.

[1919] Input: Feedback on travel plans

[1920] Output: The modification request is sent from the terminal to the server.

[1921] Step 8:

[1922] The server re-analyzes the received feedback and emotion data and modifies the plan as necessary, which is then sent back to the device for confirmation by the user.

[1923] Input: Feedback, emotion data

[1924] Output: Modified itinerary

[1925] Step 9:

[1926] The user confirms the confirmed travel plan and gives final approval, which the terminal notifies the server.

[1927] Input: Final travel plan approval

[1928] Output: Authorization data is sent to the server

[1929] Step 10:

[1930] If a user needs assistance during their trip, they can contact the chatbot, and the emotion engine will detect the user's stress level and provide assistance accordingly.

[1931] Input: Inquiry details, emotion data

[1932] Output: The device sends the query and emotion data to the server.

[1933] Step 11:

[1934] The server analyzes the inquiry content and emotional data, and the chatbot generates the optimal response. If necessary, it connects with translation services or taxi booking apps to provide support that takes the user's emotions into consideration.

[1935] Input: Inquiry details, emotion data

[1936] Output: Generated answers, supporting information provided

[1937] Step 12:

[1938] The device displays responses and support information from the server to the user.

[1939] Input: Support Information

[1940] Output: Support information displayed to the user

[1941] At each step, it can provide a personalized travel plan based on the user's preferences and emotions, providing comprehensive support during the trip.

[1942] (Application example 2)

[1943] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1944] Conventional travel plan suggestion systems make suggestions based on the user's preferences, but do not take into account the user's emotional state. This means that they lack support that is in line with the user's emotions and needs, which change in real time. Furthermore, they often cannot provide effective support to reduce stress and anxiety during travel.

[1945] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for inputting user information, a means for transmitting the user information to the server, a means for the server to learn the user information, a means for the server to generate an itinerary based on the user information, a means for the server to transmit the generated itinerary to a user terminal, a means for the user terminal to display the itinerary, a means for transmitting feedback from the user to the server, an emotion recognition means for collecting and recognizing the user's emotions, a means for adjusting the itinerary based on the user's emotions, and a real-time support means for performing the emotion recognition in real time and providing appropriate support. This enables the proposal of a personalized itinerary that takes into account the user's emotions and preferences. Furthermore, by recognizing the user's emotions in real time during the trip and providing appropriate support, it is possible to increase user satisfaction and reduce stress and anxiety.

[1946] A "means for inputting user information" is a device, software, or combination thereof that provides an interface for a user to input their profile information and preferences.

[1947] The "means for transmitting the user information to the server" refers to a device, software, or a combination thereof for transmitting the input user information to the server via a network.

[1948] "Means by which the server learns the user information" refers to an algorithm, device, software, or combination thereof that analyzes and stores the user information received by the server and makes personalized suggestions based on that information.

[1949] The "means by which the server generates a travel plan based on the user information" refers to an algorithm, device, software, or combination thereof for creating an optimal travel plan based on the user's preferences and emotional state.

[1950] The "means for transmitting the travel plan generated by the server to the user terminal" refers to a device, software, or a combination thereof for transmitting the generated travel plan to the user terminal via a network.

[1951] The "means for the user terminal to display the itinerary" refers to a device, software, or a combination thereof for visually or audibly presenting the itinerary received by the user terminal to the user.

[1952] The "means for transmitting the user feedback to the server" refers to a device, software, or a combination thereof for transmitting the user-provided feedback to the server.

[1953] The "emotion recognition means for collecting and recognizing the user's emotions" refers to a device, software, or a combination thereof for collecting and analyzing the user's facial expressions, voice, and other physiological data to recognize the user's emotional state.

[1954] The "means for adjusting the travel plan based on the user's emotions" refers to an algorithm, device, software, or a combination thereof for adjusting the optimal travel plan in real time based on the user's emotional data.

[1955] "Real-time support means for performing the emotion recognition in real time and providing appropriate support" refers to a device, software, or a combination thereof for monitoring a user's emotions in real time and providing immediate support or suggestions in response.

[1956] This invention is a system that proposes optimal travel plans based on the user's preferences and emotions and provides support in real time. The system collects and analyzes user information, monitors the user's emotions in real time using emotion recognition technology, and provides appropriate support.

[1957] System Configuration

[1958] This system consists of a user terminal, a server, an emotion recognition device, and a real-time support device.

[1959] User information collection and emotion recognition

[1960] user:

[1961] Users download the app and launch it for the first time, entering their profile information (name, age, gender) and preferences (favorite meals, activities they are interested in, budget).

[1962] Device:

[1963] The device transmits this user information and emotional data collected through the user's facial expressions and voice to the server.

[1964] server:

[1965] The server stores the received user information and emotion data in a database and learns from this data, thereby learning the relationship between user preferences and emotions.

[1966] Generate a travel plan

[1967] user:

[1968] The user inputs the desired travel conditions (departure date, destination, travel type, etc.).

[1969] Device:

[1970] The terminal transmits the input travel conditions to the server.

[1971] server:

[1972] The server analyzes the travel conditions and learned user preferences and emotional data to generate an optimal travel plan. Based on the emotional data, it selects activities that will relax the user and events that will excite them.

[1973] Device:

[1974] The terminal displays the travel plan sent from the server to the user.

[1975] user:

[1976] The user reviews the proposed itinerary and provides corrections and feedback as needed.

[1977] Customization and Real-Time Support

[1978] user:

[1979] Users request specific requests or changes (e.g., specific restaurant reservations, changes to transportation).

[1980] Device:

[1981] The terminal transmits the user's request and emotion data to the server.

[1982] server:

[1983] The server analyzes the request along with the emotional data and proposes optimal customization options. Taking the emotional data into consideration, the server adjusts the plan to best suit the user.

[1984] Device:

[1985] The terminal displays the final customized plan to the user.

[1986] user:

[1987] The user reviews and confirms the final plan.

[1988] Real-time support during your trip

[1989] user:

[1990] Users can contact the chatbot if they get lost while traveling or in an emergency.

[1991] Device:

[1992] The terminal transmits the inquiry content and emotion data to the server.

[1993] server:

[1994] The server analyzes the inquiry content and emotional data to generate the optimal response, and if necessary, connects with translation services or taxi booking apps to provide support that takes the user's emotions into consideration.

[1995] Device:

[1996] The device displays answers and support information to the user.

[1997] user:

[1998] The user will accept the support provided and contact us again if necessary.

[1999] Hardware and software used

[2000] Hardware:

[2001] Cameras and microphones onboard the self-driving vehicle to collect facial expressions and voice recordings of the user.

[2002] User devices (smartphones, etc.): To check travel plans and receive real-time support.

[2003] software:

[2004] Emotion Engine: Recognizes emotions by analyzing the user's facial expressions and voice.

[2005] Route Planner: Generates optimal travel plans based on user preferences and sentiment data.

[2006] Real-Time Support: Analyzes emotional data in real time while traveling and provides support such as changing music or adjusting the route.

[2007] Specific examples

[2008] If a user feels like relaxing on a weekday, the route planner will suggest a low-stress route and generate a plan that includes a stop at a scenic park along the way. Real-time support will play relaxing music when the user's mood changes and provide information about events the user can enjoy.

[2009] Example prompt sentence:

[2010] Enter your user's profile information and preferences, then use facial expressions and voice to collect emotional data and suggest a low-stress, relaxing driving route.

[2011] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2012] Step 1:

[2013] Entering user information

[2014] User: The user downloads and launches the app, then enters their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget).

[2015] Input: Profile information and preferences

[2016] Output: The input data is saved on the device and sent to the server.

[2017] Step 2:

[2018] Collecting Emotional Data

[2019] Device: The device uses the camera and microphone built into the device to collect the user's facial expressions and voice.

[2020] Input: User's facial expression and voice data

[2021] Output: The collected data is sent to the emotion recognizer.

[2022] Step 3:

[2023] Emotional Data Analysis

[2024] Server: The server uses an emotion engine to analyze the transmitted facial expression and voice data and identify the user's emotional state.

[2025] Input: facial expression data and voice data

[2026] Output: Recognized emotion data (e.g., joy, stress)

[2027] Step 4:

[2028] Store user information and emotional data

[2029] Server: The server stores user profile information, preferences, and sentiment data in a database.

[2030] Input: Profile information, preferences, recognized emotion data

[2031] Output: Consolidated data stored in a database

[2032] Step 5:

[2033] Enter travel conditions

[2034] User: The user inputs the desired travel conditions (departure date, destination, travel mode, etc.).

[2035] Input: Travel conditions

[2036] Output: The input data is saved on the device and sent to the server.

[2037] Step 6:

[2038] Generate a travel plan

[2039] Server: The server analyzes travel conditions, user preferences, and emotional data, and runs algorithms to generate optimal travel plans.

[2040] Input: Travel conditions, preferences, emotional data

[2041] Output: Generated itinerary

[2042] Step 7:

[2043] View travel plans

[2044] Terminal: The terminal displays the generated itinerary to the user.

[2045] Input: Travel Plan

[2046] Output: Display of itinerary

[2047] Step 8:

[2048] Providing and submitting feedback

[2049] User: The user reviews the itinerary and provides corrections and feedback as needed.

[2050] Device: The device sends feedback to the server.

[2051] Input: User feedback

[2052] Output: Feedback sent to the server

[2053] Step 9:

[2054] Generate a customized plan

[2055] Server: The server analyzes the feedback and sentiment data and generates a final customized itinerary.

[2056] Input: Feedback, emotion data

[2057] Output: A customized itinerary

[2058] Step 10:

[2059] View your customized plan

[2060] Terminal: The terminal displays the final customized itinerary to the user.

[2061] Input: Customized Travel Plan

[2062] Output: Display of customized itinerary

[2063] Step 11:

[2064] Real-time support during your trip

[2065] User: The user will use the chatbot to request assistance or queries during their journey.

[2066] Terminal: The terminal sends the query and emotion data to the server.

[2067] Server: The server analyzes the inquiry content and emotion data to generate the most appropriate answer or support. If necessary, it connects with translation services or other external applications to provide support that takes the user's emotions into consideration.

[2068] Input: User query and emotion data

[2069] Output: Answers and supporting information

[2070] Step 12:

[2071] View support information

[2072] Terminal: The terminal displays answers and support information sent from the server to the user.

[2073] Input: Answers and supporting information

[2074] Output: Displayed answers and supporting information

[2075] This allows users to experience the most suitable trip according to their preferences and emotions, reducing stress and anxiety while traveling and providing a highly satisfying travel experience.

[2076] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2077] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2078] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2079] [Fourth embodiment]

[2080] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2081] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2082] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2083] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2084] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2085] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2086] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2087] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2088] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2089] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2090] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2091] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2092] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2093] This invention is an AI concierge system that proposes personalized travel plans based on the user's preferences. The program processing of this system is explained in detail below in natural language.

[2094] 1. Collection of User Information

[2095] User

[2096] A user downloads the app and launches it for the first time.

[2097] Enter your profile information (name, age, gender) and travel preferences (favorite meals, activities you're interested in, budget).

[2098] Terminal

[2099] The input information is sent to the server.

[2100] server

[2101] The received user information is stored in a database and trained by a machine learning algorithm, which then creates a dataset to suggest optimal travel plans for the user.

[2102] 2. Generate a travel plan

[2103] User

[2104] Enter the conditions of your desired trip (departure date, destination, travel method, etc.).

[2105] Terminal

[2106] The entered conditions are sent to the server.

[2107] server

[2108] The system analyzes the received information and uses machine learning algorithms to generate a travel plan based on the user's preferences, taking into account weather information, local event information, budget, and other factors to create the optimal plan.

[2109] Terminal

[2110] The travel plan sent from the server is displayed to the user.

[2111] User

[2112] Review the proposed itinerary and provide corrections and feedback as needed.

[2113] 3. In-depth support and customization

[2114] User

[2115] Request specific requests or changes (e.g., a specific restaurant reservation, change of transportation).

[2116] Terminal

[2117] Sends the request to the server.

[2118] server

[2119] We analyze your request, suggest possible customizations, and make any necessary reservations and arrangements.

[2120] Terminal

[2121] Display the final customized plan to the user.

[2122] User

[2123] Review and finalize the plan.

[2124] 4. Real-time support during your trip

[2125] User

[2126] If you get lost or have any other inquiries while you're there, you can contact them through the chatbot.

[2127] Terminal

[2128] The query is sent to the server.

[2129] server

[2130] The chatbot analyzes the inquiry and generates the most appropriate answer, linking with translation services and taxi booking apps as needed.

[2131] Terminal

[2132] Display answers and support information to users.

[2133] User

[2134] Accept the support offered and contact us again if necessary.

[2135] Specific examples

[2136] For new users

[2137] 1. A user launches the app for the first time and enters their profile information and travel preferences.

[2138] 2. The device sends this information to the server.

[2139] 3. The server stores the information in a database and uses machine learning algorithms to learn from it.

[2140] 4. The user enters the desired travel conditions, and the terminal sends them to the server.

[2141] 5. The server generates an optimal travel plan and sends it to the device.

[2142] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[2143] 7. The server receives the feedback, modifies the plan as needed, and provides the final plan to the user.

[2144] Real-time consultation during your trip

[2145] 1. A user gets lost in the area and makes an inquiry through a chatbot.

[2146] 2. The device sends the query to the server.

[2147] 3. The server generates the best answer and, if necessary, connects with a translation service or taxi booking app.

[2148] 4. The device displays answers and support information to the user.

[2149] 5. The user accepts the support offered and contacts us again if necessary.

[2150] In this way, the system of the present invention proposes travel plans based on the user's preferences and provides comprehensive support during the trip, allowing the user to easily realize the ideal travel experience.

[2151] The processing flow will be explained below.

[2152] Step 1:

[2153] A user downloads the app and launches it for the first time.

[2154] Step 2:

[2155] The device displays an interface for entering the user's profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget).

[2156] Step 3:

[2157] The user enters the required information and presses the send button.

[2158] Step 4:

[2159] The terminal transmits the input information to the server.

[2160] Step 5:

[2161] The server stores the received user information in a database.

[2162] Step 6:

[2163] The server runs machine learning algorithms based on the stored user information to learn the user's preferences.

[2164] Step 7:

[2165] The user inputs the conditions of the desired trip (departure date, destination, travel mode, etc.).

[2166] Step 8:

[2167] The terminal transmits the input travel conditions to the server.

[2168] Step 9:

[2169] The server analyzes the travel conditions received and compares them with the user's preference data.

[2170] Step 10:

[2171] The server uses deep learning algorithms to generate the optimal travel plan (destinations, activities, accommodation, etc.).

[2172] Step 11:

[2173] The server transmits the generated travel plan to the terminal.

[2174] Step 12:

[2175] The terminal displays the travel plan sent from the server to the user.

[2176] Step 13:

[2177] The user reviews the proposed itinerary and enters corrections or feedback as necessary.

[2178] Step 14:

[2179] The device sends the user's feedback to the server.

[2180] Step 15:

[2181] The server analyzes the feedback and modifies the travel plan.

[2182] Step 16:

[2183] The server sends the final revised itinerary to the terminal.

[2184] Step 17:

[2185] The device displays the final revised plan to the user.

[2186] Step 18:

[2187] The user reviews and confirms the final plan.

[2188] Step 19:

[2189] If a user gets lost or needs information while in the area, they can make inquiries through the chatbot function.

[2190] Step 20:

[2191] The terminal sends the inquiry to the server.

[2192] Step 21:

[2193] The server uses a chatbot to analyze the inquiry and generate the most appropriate answer.

[2194] Step 22:

[2195] The server will connect with translation services and taxi booking apps as needed.

[2196] Step 23:

[2197] The server generates responses and support information and sends them to the device.

[2198] Step 24:

[2199] The device displays answers and support information to the user.

[2200] Step 25:

[2201] The user accepts the support offered and contacts the company again if necessary.

[2202] Through this series of steps, users can receive suggestions for optimal travel plans and receive support based on their requests while traveling with peace of mind.

[2203] Example 1

[2204] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2205] Conventional travel plan suggestion systems have difficulty in providing advanced customization based on users' preferences and individual needs, and do not provide sufficient real-time support during the trip. Furthermore, it is difficult to provide support in conjunction with external translation services or taxi booking services. This makes it difficult for users to achieve their ideal travel experience.

[2206] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2207] In this invention, the server includes a means for storing user information in a database and learning using a machine learning algorithm, a means for generating a travel plan based on the user's preferences, and a means for analyzing travel conditions and generating an optimal travel plan. This allows for personalized travel plans based on the user's preferences and needs. It can also accommodate customization requests and provide real-time support on-site, allowing users to enjoy an ideal travel experience.

[2208] "User Information" means information including a User's profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget).

[2209] "Server" means a computer system that stores and analyzes information submitted by users and generates travel plans.

[2210] A "database" is a structured collection of data for storing user information and travel plans.

[2211] A "machine learning algorithm" is a computational method for analyzing data based on user preferences and building predictive models.

[2212] A "travel plan" is a plan that includes travel schedules and suggestions generated based on the user's preferences and conditions.

[2213] "Travel conditions" refers to information such as the departure date, destination, and travel mode desired by the user when traveling.

[2214] A "customization request" is a request by a user to communicate specific wishes or modifications to the system.

[2215] "Real-time support" is a service that provides immediate support and assistance to problems and inquiries users may encounter while traveling.

[2216] A "chatbot" is a program that uses natural language processing technology to automatically respond to user questions and requests.

[2217] "External linkage services" are support functions provided by the system in collaboration with external services, such as translation services and taxi booking services.

[2218] This invention is an AI concierge system that proposes personalized travel plans based on the user's preferences. The program processing of this system is described in detail below.

[2219] Collection of User Information

[2220] When a user first launches the app on their smartphone or tablet, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget). The device sends this information to the server. The server stores the received user information in a database and uses machine learning algorithms (such as Scikit-Learn or TensorFlow) in Python to train the database. This information is used to create a dataset that can be used to suggest optimal travel plans to the user.

[2221] Generate a travel plan

[2222] The user enters the conditions of their desired trip (departure date, destination, travel mode, etc.) into the app. The device sends these conditions to the server. The server analyzes the received conditions and generates an optimal travel plan based on the user's preference data and travel conditions. In doing so, it uses weather information APIs and local event information APIs and also takes budget into consideration. The server sends the generated travel plan to the user's device, which then displays it to the user. The user can review the displayed travel plan and provide feedback if necessary.

[2223] In-depth support and customization

[2224] Users can request specific requests or changes through the app. For example, they can input requests to make a specific restaurant reservation or change their transportation method. The device sends these requests to the server. The server analyzes the request, suggests possible customizations, and makes the necessary reservations and arrangements. This allows for customization that meets the user's individual needs. The final customized plan is generated and displayed on the device. The user reviews and confirms the final plan.

[2225] Real-time support during your trip

[2226] If a user gets lost or needs other support while traveling, they can make an inquiry using the chatbot function within the app. The device sends the inquiry to the server. The server uses the chatbot to analyze the inquiry and generate the most appropriate answer. If necessary, it also connects with translation service APIs and taxi dispatch app APIs. This allows the user to receive the necessary support in real time. The generated answer and support information are displayed on the device, and the user can check and respond.

[2227] Specific examples

[2228] For new users

[2229] 1. A user launches the app for the first time and enters their profile information and travel preferences.

[2230] 2. The device sends this information to the server.

[2231] 3. The server stores the information in a database and uses machine learning algorithms to learn from it.

[2232] 4. The user enters the desired travel conditions, and the terminal sends them to the server.

[2233] 5. The server generates an optimal travel plan and sends it to the device.

[2234] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[2235] 7. The server receives the feedback, modifies the plan as needed, and provides the final plan to the user.

[2236] Real-time consultation during your trip

[2237] 1. A user gets lost in the area and makes an inquiry through a chatbot.

[2238] 2. The device sends the query to the server.

[2239] 3. The server generates the best answer and, if necessary, connects with a translation service or taxi booking app.

[2240] 4. The device displays answers and support information to the user.

[2241] 5. The user accepts the support offered and contacts us again if necessary.

[2242] The system allows users to create optimal travel plans tailored to their preferences and receive comprehensive support during their trip.

[2243] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2244] Program processing flow

[2245] Step 1: Enter your user information

[2246] input:

[2247] When a user launches the app for the first time, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget).

[2248] explanation:

[2249] A user downloads and launches the app on their smartphone or tablet.

[2250] Enter information such as your name, age, gender, and travel preferences into the in-app form.

[2251] The user presses the "Submit" button to confirm the entered information.

[2252] output:

[2253] A dataset of user profile information and preference information

[2254] Step 2: Submit your information

[2255] input:

[2256] User information entered in step 1

[2257] explanation:

[2258] The device sends the user's input information to the backend server via an HTTP POST request.

[2259] Information is transmitted using SSL / TLS encrypted communications.

[2260] output:

[2261] The server receives the user information.

[2262] Step 3: Database storage and machine learning

[2263] input:

[2264] The dataset of user information received in step 2

[2265] explanation:

[2266] The server stores the received user information in a database.

[2267] The database used is MySQL.

[2268] The server uses Python to analyze the received data using machine learning algorithms such as Scikit-Learn and TensorFlow, and performs learning.

[2269] A user preference profile is generated from the learning results.

[2270] output:

[2271] User preference profile

[2272] Step 4: Enter your travel requirements

[2273] input:

[2274] User's desired travel conditions (departure date, destination, travel type, etc.)

[2275] explanation:

[2276] The user enters their travel requirements into a dedicated form within the app.

[2277] Also consider options such as those for families with children or for active outdoorsy people.

[2278] After the user has entered all the required information, he clicks "Submit."

[2279] output:

[2280] Dataset of user desired travel conditions

[2281] Step 5: Sending conditions

[2282] input:

[2283] The data set of travel conditions entered in step 4

[2284] explanation:

[2285] The terminal sends the travel conditions entered by the user to the server via an HTTP POST request.

[2286] Communications are encrypted using SSL / TLS.

[2287] output:

[2288] Travel conditions received

[2289] Step 6: Create a travel plan

[2290] input:

[2291] User information and travel conditions collected in Steps 2 and 4

[2292] explanation:

[2293] The server generates the optimal travel plan based on the user's preference profile and travel conditions using weather information APIs and local event information APIs.

[2294] Travel plans are automatically generated using machine learning algorithms (such as TensorFlow).

[2295] Adjust your plan taking into account your budget.

[2296] output:

[2297] The perfect travel plan for you

[2298] Step 7: Submit and view your plan

[2299] input:

[2300] The itinerary generated in step 6

[2301] explanation:

[2302] The server sends the generated travel plan to the terminal as an HTTP response.

[2303] The device displays the received travel plan to the user.

[2304] output:

[2305] The itinerary displayed to the user

[2306] Step 8: Provide feedback

[2307] input:

[2308] User feedback information

[2309] explanation:

[2310] The user checks the proposed travel plan and, if necessary, inputs changes or corrections as feedback.

[2311] The terminal transmits the feedback information to the server.

[2312] output:

[2313] Feedback information dataset

[2314] Step 9: Submit a customization request

[2315] input:

[2316] Customization Request Information

[2317] explanation:

[2318] The user inputs a specific request (e.g., a specific restaurant reservation, a change of transportation method).

[2319] The terminal sends these requests to the server.

[2320] output:

[2321] Customization request information dataset

[2322] Step 10: Customization and Final Plan Generation

[2323] input:

[2324] Customization request information received in step 9

[2325] explanation:

[2326] The server analyzes the received request and adds any necessary customizations to the itinerary.

[2327] Make any necessary reservations and arrangements and generate your final itinerary.

[2328] output:

[2329] A customized ultimate itinerary

[2330] Step 11: Submit and review your final plan

[2331] input:

[2332] The final itinerary generated in step 10

[2333] explanation:

[2334] The server sends the final customized itinerary to the terminal as an HTTP response.

[2335] The device displays the final plan received to the user.

[2336] The user reviews and confirms the final plan.

[2337] output:

[2338] Final confirmed travel plans

[2339] Step 12: Real-time support during your trip

[2340] input:

[2341] Real-time support inquiries from users

[2342] explanation:

[2343] Users can use chatbots to make inquiries while traveling.

[2344] The terminal sends the inquiry to the server.

[2345] The server analyzes the data using a chatbot and generates the most appropriate answer.

[2346] If necessary, it will also link with translation service APIs and taxi booking app APIs.

[2347] The device displays generated answers and supporting information to the user.

[2348] output:

[2349] Responses to inquiries and support information

[2350] In this way, users can create the perfect travel plan tailored to their preferences and receive comprehensive support during their trip.

[2351] (Application example 1)

[2352] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2353] While conventional travel plan suggestion systems can provide personalized plans based on user preferences, they lack a way for users to visually visualize the atmosphere and locations of their travel destinations. Furthermore, providing real-time support during a trip requires users to use multiple apps simultaneously, which is inconvenient for users. There is a need to solve these problems and realize a more comprehensive travel experience.

[2354] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2355] In this invention, the server includes means for inputting user information, means for transmitting the user information to the server, means for the server to learn the user information, means for the server to generate a travel plan based on the user information, means for the server to transmit the generated travel plan to a user terminal, means for the user terminal to display the travel plan, means for transmitting feedback from the user to the server, means for providing real-time support, and means for providing a virtual reality tour based on the travel plan, thereby enabling the user to visually experience the suggested travel destinations in virtual reality and receive comprehensive real-time support, thereby enabling the user to plan and execute a more satisfying trip.

[2356] "User Information" means data relating to a user's profile information, travel preferences and travel requirements.

[2357] The "server" is a computer system that receives user information, learns and analyzes it, and generates and transmits the optimal travel plan.

[2358] "User terminal" means an electronic device, typically a smartphone or tablet, used to input user information and receive and display travel plans and support information.

[2359] "Virtual reality tour" is a feature that allows users to visually experience the atmosphere and locations of their travel destinations through virtual reality technology.

[2360] A "trip plan" is a plan that includes a suggested travel itinerary and destination details generated based on a user's preferences and conditions.

[2361] "Real-time support" is a support service that provides immediate response and assistance to users' inquiries and problems while traveling.

[2362] A "machine learning algorithm" is an algorithm that learns patterns based on collected user information and generates travel plans based on the results of that learning.

[2363] In order to put the present invention into practice, it is necessary to build a system in which multiple pieces of hardware and software work together. A specific embodiment of this system is shown below.

[2364] 1. Collection and storage of user information

[2365] When a user downloads and launches the application for the first time, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget). This information is sent from the device (e.g., smartphone) used to the server. The server stores the received user information in a database and uses machine learning algorithms to learn from it. This step forms the base dataset for providing personalized travel plans based on the user information.

[2366] 2. Generate a travel plan

[2367] When a user inputs the conditions for their desired travel plan (departure date, destination, travel mode, etc.), these conditions are also sent to the server via their device. The server analyzes the received conditions and uses a machine learning algorithm to generate a travel plan based on the user's preferences. This also takes into account factors such as weather information, local event information, and budget. The generated travel plan is sent from the server to the user's device and displayed to the user.

[2368] 3. Providing VR tours

[2369] After users confirm the proposed itinerary, they can experience a virtual reality (VR) tour of specific tourist spots or points. This function is available by installing an application on a VR headset or smartphone. For example, if a user selects a plan with "Kyoto" as the destination, they can virtually visit Kyoto's famous tourist spots.

[2370] 4. Real-time support during your trip

[2371] If a user gets lost or needs emergency assistance while traveling, they can receive real-time support using the chatbot within the application. When the user enters their inquiry, it is sent from the device to the server, which uses machine learning algorithms to generate the best answer. If necessary, it can also be linked to a translation service or a taxi booking app.

[2372] 5. Specific Examples

[2373] For new users, the flow is as follows:

[2374] 1. A user launches the app for the first time and enters their profile information and travel preferences.

[2375] 2. The device sends this information to the server.

[2376] 3. The server stores the information in a database and uses machine learning algorithms to learn from it.

[2377] 4. The user enters the desired travel conditions, and the device sends them to the server.

[2378] 5. The server generates an optimal travel plan and sends it to the device.

[2379] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[2380] 7. The server receives the feedback, modifies the plan as needed, and provides the final plan to the user.

[2381] As an example of real-time support during travel, here is a specific example of a user getting lost and making an inquiry through a chatbot:

[2382] 1. A user gets lost in the area and makes an inquiry through a chatbot.

[2383] 2. The device sends the query to the server.

[2384] 3. The server generates the best answer and, if necessary, connects with a translation service or taxi booking app.

[2385] 4. The device displays answers and support information to the user.

[2386] 5. The user accepts the support provided and contacts us again if necessary.

[2387] Examples of prompts include:

[2388] "After users download the app and launch it for the first time, they enter the following information:

[2389] Name: User A

[2390] Age: 30

[2391] Gender: Male

[2392] Favorite food: Japanese food

[2393] Interested in: Hiking

[2394] Budget: 50,000 yen

[2395] Next, enter your travel plan requirements:

[2396] Departure date: July 1, 2024

[2397] Destination: Kyoto

[2398] Travel type: Solo travel

[2399] By following these prompts, each function of the system will function properly, providing users with a personalized travel experience.

[2400] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2401] Step 1:

[2402] Collection of User Information

[2403] When a user launches the application for the first time, they enter their profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget). The device sends this information to the server, which stores the received user information in a database and begins learning using machine learning algorithms. This process forms the base data set for suggesting personalized travel plans to the user.

[2404] Input: User profile information and travel preference data

[2405] Output: User information stored in the database

[2406] Step 2:

[2407] Enter travel plan conditions

[2408] The user inputs the travel plan conditions (departure date, destination, travel mode, etc.). This information is sent to the server via the device. The server analyzes the received conditions and uses machine learning algorithms to generate a travel plan based on the user's preferences. This also takes into account factors such as weather information, local event information, and budget.

[2409] Input: User's travel plan conditions

[2410] Output: Parsed itinerary data

[2411] Step 3:

[2412] Generate and display travel plans

[2413] The server sends the generated travel plan to the user's device, which then displays the plan to the user. The user reviews the proposed plan and provides feedback if necessary. This feedback is sent from the device to the server, which then revises the plan based on that feedback. The final plan is then provided to the user.

[2414] Input: Server-generated travel plan data, user feedback

[2415] Output: Finalized itinerary

[2416] Step 4:

[2417] Providing virtual reality tours

[2418] After reviewing the proposed itinerary, users can experience a virtual reality (VR) tour of specific tourist spots or points. VR content is sent to the device, and users can visually experience the content with a VR headset or smartphone. For example, if a user selects "Kyoto" as their destination, they can virtually visit Kyoto's famous tourist spots.

[2419] Input: Proposed itinerary and related VR content

[2420] Output: The virtual reality tour the user experiences

[2421] Step 5:

[2422] Real-time support during your trip

[2423] If a user gets lost or needs emergency assistance while traveling, they can receive real-time support from a chatbot within the application. When the user enters their inquiry, it is sent from the device to the server. The server uses a machine learning algorithm to generate the optimal answer and sends it to the device. If necessary, it can also be linked to a translation service or a taxi booking app.

[2424] Input: User's support request

[2425] Output: Answers and support information from the chatbot

[2426] The above steps will create a system that allows users to create personalized travel plans, experience virtual reality tours, and receive real-time support throughout their trip.

[2427] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2428] This invention is an AI concierge system that proposes personalized travel plans based on the user's preferences and emotions and provides real-time support. The system incorporates an emotion engine and has the function of adjusting plans and support content according to the user's emotions.

[2429] User information collection and emotion recognition

[2430] User

[2431] Download the app and launch it for the first time.

[2432] Enter your profile information (name, age, gender) and travel preferences (favorite meals, activities you're interested in, budget). The app is equipped with an emotion engine that recognizes the user's emotions through facial expressions and voice.

[2433] Terminal

[2434] The input information and emotion data are sent to the server.

[2435] server

[2436] The received user information and emotion data are stored in a database, which allows the system to learn the relationship between user preferences and emotions.

[2437] Generate a travel plan

[2438] User

[2439] Enter the conditions of your desired trip (departure date, destination, travel method, etc.).

[2440] Terminal

[2441] The entered travel conditions are sent to the server.

[2442] server

[2443] The system analyzes the user's preference and emotional data along with travel conditions to generate an optimal travel plan. Based on the emotional data, it selects activities that will relax the user and events that will excite them.

[2444] Terminal

[2445] The travel plan sent from the server is displayed to the user.

[2446] User

[2447] Review the proposed itinerary and provide corrections and feedback as needed.

[2448] Customization and Real-Time Support

[2449] User

[2450] Request specific requests or changes (e.g., a specific restaurant reservation, change of transportation).

[2451] Terminal

[2452] Send the user's request and emotion data to the server.

[2453] server

[2454] Analyzes the request along with emotional data and proposes optimal customization. Taking emotional data into consideration, adjusts plans to ensure user satisfaction.

[2455] Terminal

[2456] Display the final customized plan to the user.

[2457] User

[2458] Review and confirm the final plan.

[2459] Real-time support during your trip

[2460] User

[2461] If you get lost or have an emergency, you can contact us through the chatbot.

[2462] Terminal

[2463] The inquiry content and emotion data are sent to the server.

[2464] server

[2465] The chatbot analyzes the inquiry content and emotional data to generate the most appropriate response, and if necessary, connects with translation services or taxi booking apps to provide support that takes the user's emotions into consideration.

[2466] Terminal

[2467] Display answers and support information to users.

[2468] User

[2469] Accept the support offered and follow up if necessary.

[2470] Specific examples

[2471] For new users

[2472] 1. A user launches the app for the first time and enters their profile information and travel preferences. The emotion engine recognizes the user's facial expressions and voice to collect emotional data.

[2473] 2. The device sends information and emotion data to the server.

[2474] 3. The server stores the information and emotion data in a database and performs learning.

[2475] 4. The user enters the desired travel conditions, and the terminal sends them to the server.

[2476] 5. The server generates an optimal travel plan based on the conditions and emotion data.

[2477] 6. The device displays the travel plan to the user, who reviews the plan and provides feedback.

[2478] 7. The server analyzes the feedback and sentiment data, modifies the plan as needed, and provides the final plan.

[2479] Real-time consultation during your trip

[2480] 1. When a user gets lost in the area and makes an inquiry through the chatbot, the emotion engine detects the user's stress level.

[2481] 2. The device sends the inquiry and emotion data to the server.

[2482] 3. The server analyzes the inquiry and emotional data to generate the optimal response. If necessary, it connects with translation services or taxi booking apps to provide support that reduces the user's stress.

[2483] 4. The device displays answers and support information to the user.

[2484] 5. The user accepts the support offered and contacts us again if necessary.

[2485] In this way, the system of the present invention incorporating an emotion engine can provide personalized travel plans that take into account not only the user's preferences but also their emotions, providing comprehensive support during the trip.

[2486] The processing flow will be explained below.

[2487] Step 1:

[2488] A user downloads the app and launches it for the first time.

[2489] Step 2:

[2490] The device asks the user to enter their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget).

[2491] Step 3:

[2492] The user enters their profile information and travel preferences and hits the submit button.

[2493] Step 4:

[2494] The terminal transmits the input information to the server.

[2495] Step 5:

[2496] The server stores the received user information in a database.

[2497] Step 6:

[2498] The device uses an emotion engine to collect emotional data from the user's facial expressions and voice.

[2499] Step 7:

[2500] The device transmits the emotion data to the server.

[2501] Step 8:

[2502] The server runs a machine learning algorithm based on user information and emotional data to learn the relationship between user preferences and emotions.

[2503] Step 9:

[2504] The user inputs the conditions of the desired trip (departure date, destination, travel mode, etc.).

[2505] Step 10:

[2506] The terminal transmits the input travel conditions to the server.

[2507] Step 11:

[2508] The server analyzes the travel conditions received and generates an optimal travel plan based on them. Based on the emotional data, it selects activities that will relax or excite the user.

[2509] Step 12:

[2510] The server sends the generated travel plan to the user's terminal.

[2511] Step 13:

[2512] The device displays the travel plan to the user.

[2513] Step 14:

[2514] The user reviews the proposed itinerary and enters corrections or feedback as necessary.

[2515] Step 15:

[2516] The device sends the user's feedback to the server.

[2517] Step 16:

[2518] The server analyzes the feedback and modifies the travel plan, taking into account the emotional data.

[2519] Step 17:

[2520] The server sends the final revised itinerary to the terminal.

[2521] Step 18:

[2522] The device displays the final revised plan to the user.

[2523] Step 19:

[2524] The user reviews and confirms the final plan.

[2525] Step 20:

[2526] If a user gets lost or stressed while in the area, they can contact the chatbot to inquire.

[2527] Step 20A:

[2528] The device monitors the user's emotions in real time and transmits the emotion data along with the query to the server.

[2529] Step 21:

[2530] The server uses a chatbot to analyze the inquiry content and emotional data and generate the most appropriate answer.

[2531] Step 22:

[2532] The server will connect with translation services and taxi booking apps as needed to provide support that takes the user's emotions into consideration.

[2533] Step 23:

[2534] The server generates responses and support information and sends them to the device.

[2535] Step 24:

[2536] The device displays answers and support information to the user.

[2537] Step 25:

[2538] The user accepts the support offered and contacts the company again if necessary.

[2539] In this way, it becomes possible to provide personalized responses based on user preferences as well as emotional data, providing a more personalized travel experience and real-time support.

[2540] Example 2

[2541] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2542] Conventional travel planning systems have difficulty providing plans that fully take into account the user's preferences and emotions, and they also lack the ability to provide satisfactory real-time support. This means that users cannot receive appropriate support tailored to their individual needs and emotions, which can lead to a decrease in satisfaction during their trip.

[2543] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for learning user information and emotion data, means for generating a travel plan based on the user information and emotion data, and means for providing real-time support based on the emotion data. This makes it possible to provide a personalized travel plan based on the user's preferences and emotions and comprehensive support during the trip.

[2544] "User Information" refers to a user's profile information (name, age, gender) and travel preferences (favorite meals, activities of interest, budget).

[2545] "Emotion data" refers to information about the emotional state recognized by the emotion engine through the user's facial expressions and voice.

[2546] "Server" refers to the central system that receives, stores, and analyzes user information and sentiment data, generates travel plans, and provides real-time support.

[2547] "Means of learning" refers to the algorithms and programs that allow the server to analyze user information and emotional data, understand the relationships between them, and use them to make future suggestions.

[2548] "Means for generating travel plans" refers to algorithms or programs for creating optimal travel plans based on travel conditions, user information, and emotional data.

[2549] "Means of providing real-time support" refers to systems and functions that take emotional data into account when a user makes an inquiry while traveling and immediately provide appropriate assistance or information.

[2550] "Means for sending feedback to the server" refers to the function of sending user ratings and correction requests from the terminal to the server, which then receives and analyzes them.

[2551] "User device" refers to the device (smartphone, tablet, PC, etc.) used by a user to enter or confirm travel plans, provide feedback, or receive real-time support.

[2552] The present invention is an AI concierge system that proposes personalized travel plans based on the user's preferences and emotions and provides support in real time. Specific embodiments are described below.

[2553] System Configuration

[2554] This system includes hardware and software such as user terminals, servers, databases, emotion engines, chatbots, translation services, and external collaboration services.

[2555] A user device is a device such as a smartphone, tablet, or PC that allows a user to operate an application to enter information, check travel plans, and provide feedback.

[2556] The server is built on a cloud service and is the core system that receives, stores, and analyzes user information and emotion data. Specific technologies that can be used include Amazon Web Services (AWS) and Microsoft Azure.

[2557] The database is a server-managed storage system that stores user information and emotion data. For example, a relational database such as MySQL or PostgreSQL can be used.

[2558] The emotion engine has the ability to analyze the user's facial expressions and voice to generate emotion data, which can be achieved using machine learning algorithms or voice analysis software (e.g., Google Cloud Speech-to-Text API).

[2559] A chatbot is an interface for users to make inquiries or request support during their trip, and responds in real time using natural language processing (NLP) techniques, such as Dialogflow and the Microsoft Bot Framework.

[2560] The translation service is a function that converts user inquiries into different languages, and can use the Google Translate API, etc.

[2561] External linkage services are a means of linking with external services such as taxi booking apps and restaurant reservation systems, allowing for immediate responses to user requests.

[2562] System Operation

[2563] When users download the app and launch it for the first time, they enter their profile information and travel preferences, and the emotion engine recognizes the user's facial expressions and voice to collect emotional data.

[2564] The device sends the input information and emotion data to the server, securely using the HTTPS protocol.

[2565] The server stores the received user information and emotion data in a database and learns the relationship between them. Specifically, it analyzes the data using a generative AI model (e.g., TensorFlow).

[2566] The user inputs the desired travel conditions (departure date, destination, travel mode, etc.) This information is also sent to the server via the terminal.

[2567] The server analyzes the travel conditions and the user's preference and emotional data to generate an optimal travel plan. For example, if the user wants to relax based on emotional data, it will select a plan that includes a quiet beach and a spa.

[2568] The terminal displays the generated travel plan to the user, who can review the plan and provide feedback.

[2569] The server re-analyzes the received feedback and emotional data and modifies the plan as needed, providing a final, customized plan.

[2570] If a user needs assistance during their trip, they can contact the chatbot, and the emotion engine will detect the user's stress level and provide assistance accordingly.

[2571] The server allows the chatbot to analyze the inquiry content and emotional data to generate the optimal response, and also provides comprehensive real-time support by linking with external services (e.g., translation services and taxi booking apps).

[2572] Specific examples

[2573] For new users

[2574] The user launches the app for the first time and enters their profile information and travel preferences. The emotion engine recognizes facial expressions and voice and collects emotional data. The device sends the information and emotional data to the server, which stores the information and emotional data in a database and performs learning. The user enters their desired travel conditions and the device sends them to the server. The server generates an optimal travel plan based on the conditions and emotional data. The device displays the travel plan, and the user reviews the plan and provides feedback. The server analyzes the feedback and emotional data and modifies the plan as necessary.

[2575] Prompt Sentence Examples

[2576] "I'm planning a week-long trip to Tokyo. I'd like to enjoy good food and relaxing activities. I'm on a moderate budget. Can you recommend a recommended itinerary?"

[2577] In this way, the system of the present invention can provide personalized travel plans and real-time support based on the user's preferences and emotions.

[2578] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2579] Step 1:

[2580] A user downloads the app and launches it for the first time. They enter their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget). The emotion engine recognizes emotions through facial expressions and voice, and collects emotion data.

[2581] Input: User information, emotion data

[2582] Output: User information and emotion data are stored on the device.

[2583] Step 2:

[2584] The device sends the entered user information and emotion data to the server, securely using the HTTPS protocol.

[2585] Input: User information, emotion data

[2586] Output: User information and emotion data received by the server

[2587] Step 3:

[2588] The server stores the received user information and emotion data in a database. Specifically, a relational database such as MySQL or PostgreSQL can be used. Furthermore, the data is analyzed using a generative AI model (e.g., TensorFlow) to learn the association between the emotion data and user information.

[2589] Input: User information and emotion data received by the server

[2590] Output: Parsed data stored in a database

[2591] Step 4:

[2592] The user inputs the desired travel conditions (departure date, destination, travel mode, etc.) This information is also sent to the server via the terminal.

[2593] Input: Travel conditions

[2594] Output: The travel conditions are sent to the server.

[2595] Step 5:

[2596] The server analyzes the travel conditions and the user's preference and emotional data to generate the optimal travel plan. For example, a user who wants to relax will be offered a plan that includes a quiet beach or spa.

[2597] Input: Travel conditions, user preference data, emotional data

[2598] Output: Generated itinerary

[2599] Step 6:

[2600] The terminal displays the travel plan sent from the server to the user.

[2601] Input: Generated itinerary

[2602] Output: The itinerary displayed to the user

[2603] Step 7:

[2604] The user reviews the proposed itinerary and provides corrections and feedback as needed, such as adding or removing specific activities or adjusting the budget.

[2605] Input: Feedback on travel plans

[2606] Output: The modification request is sent from the terminal to the server.

[2607] Step 8:

[2608] The server re-analyzes the received feedback and emotion data and modifies the plan as necessary, which is then sent back to the device for confirmation by the user.

[2609] Input: Feedback, emotion data

[2610] Output: Modified itinerary

[2611] Step 9:

[2612] The user confirms the confirmed travel plan and gives final approval, which the terminal notifies the server.

[2613] Input: Final travel plan approval

[2614] Output: Authorization data is sent to the server

[2615] Step 10:

[2616] If a user needs assistance during their trip, they can contact the chatbot, and the emotion engine will detect the user's stress level and provide assistance accordingly.

[2617] Input: Inquiry details, emotion data

[2618] Output: The device sends the query and emotion data to the server.

[2619] Step 11:

[2620] The server analyzes the inquiry content and emotional data, and the chatbot generates the optimal response. If necessary, it connects with translation services or taxi booking apps to provide support that takes the user's emotions into consideration.

[2621] Input: Inquiry details, emotion data

[2622] Output: Generated answers, supporting information provided

[2623] Step 12:

[2624] The device displays responses and support information from the server to the user.

[2625] Input: Support Information

[2626] Output: Support information displayed to the user

[2627] At each step, it can provide a personalized travel plan based on the user's preferences and emotions, providing comprehensive support during the trip.

[2628] (Application example 2)

[2629] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2630] Conventional travel plan suggestion systems make suggestions based on the user's preferences, but do not take into account the user's emotional state. This means that they lack support that is in line with the user's emotions and needs, which change in real time. Furthermore, they often cannot provide effective support to reduce stress and anxiety during travel.

[2631] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for inputting user information, a means for transmitting the user information to the server, a means for the server to learn the user information, a means for the server to generate an itinerary based on the user information, a means for the server to transmit the generated itinerary to a user terminal, a means for the user terminal to display the itinerary, a means for transmitting feedback from the user to the server, an emotion recognition means for collecting and recognizing the user's emotions, a means for adjusting the itinerary based on the user's emotions, and a real-time support means for performing the emotion recognition in real time and providing appropriate support. This enables the proposal of a personalized itinerary that takes into account the user's emotions and preferences. Furthermore, by recognizing the user's emotions in real time during the trip and providing appropriate support, it is possible to increase user satisfaction and reduce stress and anxiety.

[2632] A "means for inputting user information" is a device, software, or combination thereof that provides an interface for a user to input their profile information and preferences.

[2633] The "means for transmitting the user information to the server" refers to a device, software, or a combination thereof for transmitting the input user information to the server via a network.

[2634] "Means by which the server learns the user information" refers to an algorithm, device, software, or combination thereof that analyzes and stores the user information received by the server and makes personalized suggestions based on that information.

[2635] The "means by which the server generates a travel plan based on the user information" refers to an algorithm, device, software, or combination thereof for creating an optimal travel plan based on the user's preferences and emotional state.

[2636] The "means for transmitting the travel plan generated by the server to the user terminal" refers to a device, software, or a combination thereof for transmitting the generated travel plan to the user terminal via a network.

[2637] The "means for the user terminal to display the itinerary" refers to a device, software, or a combination thereof for visually or audibly presenting the itinerary received by the user terminal to the user.

[2638] The "means for transmitting the user feedback to the server" refers to a device, software, or a combination thereof for transmitting the user-provided feedback to the server.

[2639] The "emotion recognition means for collecting and recognizing the user's emotions" refers to a device, software, or a combination thereof for collecting and analyzing the user's facial expressions, voice, and other physiological data to recognize the user's emotional state.

[2640] The "means for adjusting the travel plan based on the user's emotions" refers to an algorithm, device, software, or a combination thereof for adjusting the optimal travel plan in real time based on the user's emotional data.

[2641] "Real-time support means for performing the emotion recognition in real time and providing appropriate support" refers to a device, software, or a combination thereof for monitoring a user's emotions in real time and providing immediate support or suggestions in response.

[2642] This invention is a system that proposes optimal travel plans based on the user's preferences and emotions and provides support in real time. The system collects and analyzes user information, monitors the user's emotions in real time using emotion recognition technology, and provides appropriate support.

[2643] System Configuration

[2644] This system consists of a user terminal, a server, an emotion recognition device, and a real-time support device.

[2645] User information collection and emotion recognition

[2646] user:

[2647] Users download the app and launch it for the first time, entering their profile information (name, age, gender) and preferences (favorite meals, activities they are interested in, budget).

[2648] Device:

[2649] The device transmits this user information and emotional data collected through the user's facial expressions and voice to the server.

[2650] server:

[2651] The server stores the received user information and emotion data in a database and learns from this data, thereby learning the relationship between user preferences and emotions.

[2652] Generate a travel plan

[2653] user:

[2654] The user inputs the desired travel conditions (departure date, destination, travel type, etc.).

[2655] Device:

[2656] The terminal transmits the input travel conditions to the server.

[2657] server:

[2658] The server analyzes the travel conditions and learned user preferences and emotional data to generate an optimal travel plan. Based on the emotional data, it selects activities that will relax the user and events that will excite them.

[2659] Device:

[2660] The terminal displays the travel plan sent from the server to the user.

[2661] user:

[2662] The user reviews the proposed itinerary and provides corrections and feedback as needed.

[2663] Customization and Real-Time Support

[2664] user:

[2665] Users request specific requests or changes (e.g., specific restaurant reservations, changes to transportation).

[2666] Device:

[2667] The terminal transmits the user's request and emotion data to the server.

[2668] server:

[2669] The server analyzes the request along with the emotional data and proposes optimal customization options. Taking the emotional data into consideration, the server adjusts the plan to best suit the user.

[2670] Device:

[2671] The terminal displays the final customized plan to the user.

[2672] user:

[2673] The user reviews and confirms the final plan.

[2674] Real-time support during your trip

[2675] user:

[2676] Users can contact the chatbot if they get lost while traveling or in an emergency.

[2677] Device:

[2678] The terminal transmits the inquiry content and emotion data to the server.

[2679] server:

[2680] The server analyzes the inquiry content and emotional data to generate the optimal response, and if necessary, connects with translation services or taxi booking apps to provide support that takes the user's emotions into consideration.

[2681] Device:

[2682] The device displays answers and support information to the user.

[2683] user:

[2684] The user will accept the support provided and contact us again if necessary.

[2685] Hardware and software used

[2686] Hardware:

[2687] Cameras and microphones onboard the self-driving vehicle to collect facial expressions and voice recordings of the user.

[2688] User devices (smartphones, etc.): To check travel plans and receive real-time support.

[2689] software:

[2690] Emotion Engine: Recognizes emotions by analyzing the user's facial expressions and voice.

[2691] Route Planner: Generates optimal travel plans based on user preferences and sentiment data.

[2692] Real-Time Support: Analyzes emotional data in real time while traveling and provides support such as changing music or adjusting the route.

[2693] Specific examples

[2694] If a user feels like relaxing on a weekday, the route planner will suggest a low-stress route and generate a plan that includes a stop at a scenic park along the way. Real-time support will play relaxing music when the user's mood changes and provide information about events the user can enjoy.

[2695] Example prompt sentence:

[2696] Enter your user's profile information and preferences, then use facial expressions and voice to collect emotional data and suggest a low-stress, relaxing driving route.

[2697] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2698] Step 1:

[2699] Entering user information

[2700] User: The user downloads and launches the app, then enters their profile information (name, age, gender) and travel preferences (favorite meals, activities they are interested in, budget).

[2701] Input: Profile information and preferences

[2702] Output: The input data is saved on the device and sent to the server.

[2703] Step 2:

[2704] Collecting Emotional Data

[2705] Device: The device uses the camera and microphone built into the device to collect the user's facial expressions and voice.

[2706] Input: User's facial expression and voice data

[2707] Output: The collected data is sent to the emotion recognizer.

[2708] Step 3:

[2709] Emotional Data Analysis

[2710] Server: The server uses an emotion engine to analyze the transmitted facial expression and voice data and identify the user's emotional state.

[2711] Input: facial expression data and voice data

[2712] Output: Recognized emotion data (e.g., joy, stress)

[2713] Step 4:

[2714] Store user information and emotional data

[2715] Server: The server stores user profile information, preferences, and sentiment data in a database.

[2716] Input: Profile information, preferences, recognized emotion data

[2717] Output: Consolidated data stored in a database

[2718] Step 5:

[2719] Enter travel conditions

[2720] User: The user inputs the desired travel conditions (departure date, destination, travel mode, etc.).

[2721] Input: Travel conditions

[2722] Output: The input data is saved on the device and sent to the server.

[2723] Step 6:

[2724] Generate a travel plan

[2725] Server: The server analyzes travel conditions, user preferences, and emotional data, and runs algorithms to generate optimal travel plans.

[2726] Input: Travel conditions, preferences, emotional data

[2727] Output: Generated itinerary

[2728] Step 7:

[2729] View travel plans

[2730] Terminal: The terminal displays the generated itinerary to the user.

[2731] Input: Travel Plan

[2732] Output: Display of itinerary

[2733] Step 8:

[2734] Providing and submitting feedback

[2735] User: The user reviews the itinerary and provides corrections and feedback as needed.

[2736] Device: The device sends feedback to the server.

[2737] Input: User feedback

[2738] Output: Feedback sent to the server

[2739] Step 9:

[2740] Generate a customized plan

[2741] Server: The server analyzes the feedback and sentiment data and generates a final customized itinerary.

[2742] Input: Feedback, emotion data

[2743] Output: A customized itinerary

[2744] Step 10:

[2745] View your customized plan

[2746] Terminal: The terminal displays the final customized itinerary to the user.

[2747] Input: Customized Travel Plan

[2748] Output: Display of customized itinerary

[2749] Step 11:

[2750] Real-time support during your trip

[2751] User: The user will use the chatbot to request assistance or queries during their journey.

[2752] Terminal: The terminal sends the query and emotion data to the server.

[2753] Server: The server analyzes the inquiry content and emotion data to generate the most appropriate answer or support. If necessary, it connects with translation services or other external applications to provide support that takes the user's emotions into consideration.

[2754] Input: User query and emotion data

[2755] Output: Answers and supporting information

[2756] Step 12:

[2757] View support information

[2758] Terminal: The terminal displays answers and support information sent from the server to the user.

[2759] Input: Answers and supporting information

[2760] Output: Displayed answers and supporting information

[2761] This allows users to experience the most suitable trip according to their preferences and emotions, reducing stress and anxiety while traveling and providing a highly satisfying travel experience.

[2762] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2763] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2764] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2765] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2766] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2767] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2768] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2769] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2770] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2771] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2772] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2773] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2774] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2775] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2776] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of th...

Claims

1. a means for inputting user information; means for transmitting the user information to a server; means for the server to learn the user information; means for the server to generate a travel plan based on the user information; means for transmitting the travel plan generated by the server to a user terminal; means for displaying the travel plan on the user terminal; means for transmitting feedback from the user to a server; a means to provide real-time support; A system including:

2. The system of claim 1 , wherein the suggestions are based on past travel history.

3. The system of claim 1 , wherein the system provides support in cooperation with a translation service or other external service.

Images