system

Abstract

We provide the system. [Solution] A means for receiving travel preference information from users and analyzing said preference information to identify the user's preferences, A means for generating multiple travel destinations and activities based on the user's preferences, A means for searching for transportation and accommodation for the aforementioned multiple travel destinations and activities, A means of presenting generated travel plans to users and enabling them to select and customize them, A means to finalize the travel plan and send related information, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern busy lives, when an individual makes a travel plan, they need to spend a significant amount of time and effort collecting, comparing, and finding the optimal option among a lot of information. In such a situation, it is easy to feel stressed from the initial stage of the trip, which may also affect the satisfaction of the trip itself. Furthermore, there is a need for a mechanism to efficiently perform planning according to the diverse preferences of individual users.

Means for Solving the Problems

[0005] This invention provides a system that receives and analyzes user preference information to generate travel destinations and activities tailored to individual preferences. Based on the analysis results, the system searches for optimal transportation and accommodation options, obtaining information in real time. Furthermore, it presents the generated travel plan to the user, allowing for easy selection and customization. By taking past travel history into consideration, it enables a more personalized travel experience. In this way, it reduces the time and effort burden of travel planning, providing users with a stress-free planning process.

[0006] "User" refers to an individual who uses the system to plan their trip.

[0007] "Desired information" refers to information that users enter regarding the purpose and conditions of their trip, including destination, budget, duration, and desired activities.

[0008] "Preferences" refer to characteristics related to the user's likes and interests, and serve as a criterion for appropriately suggesting travel options.

[0009] "Travel destinations" refers to a selection of travel destinations suggested based on the user's wishes and preferences.

[0010] "Activities" refer to activities and experiences that can be enjoyed during a trip, and include specific plans that can be carried out at the chosen destination.

[0011] "Transportation" refers to the method of transport used by a user to get to their destination in a travel plan, and includes things like air travel and trains.

[0012] "Accommodation facilities" are facilities that provide places for travelers to stay during their trip, and include hotels and resorts.

[0013] "Plan generation" refers to the process of creating an overall travel schedule and details for the user based on the analyzed information.

[0014] "Customization" refers to the act of adjusting the travel plan presented by the system according to the preferences and requirements of the user.

[0015] "Past travel history" refers to information regarding the trips the user has taken previously, and is used as a reference for considering individual preferences in generating a new plan.

Brief Description of Drawings

[0016] [Figure 1] It is a conceptual diagram showing an example of the configuration of the data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of the data processing device and the smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of the data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of the data processing device and the smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of the data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of the data processing device and the headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of the data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of the data processing device and the robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of a data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention relates to a system that reduces the time and effort users spend planning their trips. The system primarily includes user input of desired information, analysis of preferences, generation and presentation of travel plans, searching for transportation and accommodations, and finally confirming the plan and transmitting the information.

[0038] First, the user enters their travel preferences via their device. This includes potential destinations, budget, travel duration, desired activities, and specific food preferences. The server receives this information and analyzes it using advanced natural language processing technology. The server identifies the user's preferences and selects suitable travel destinations and activities. This analysis takes into account the user's past travel history to provide more personalized suggestions.

[0039] Next, the server searches for suitable transportation and accommodation based on the collected information. Here, it obtains real-time pricing and availability information from reliable partners. This allows users to have detailed options based on the latest data.

[0040] The server then presents the generated travel plan to the user via their terminal. The user reviews this plan and customizes it based on their preferences and requirements. The customized plan is then analyzed again by the system and finalized as the travel plan.

[0041] Finally, if the user is satisfied with the plan, they complete the booking on their device and enter their payment information. Once this is done, the server sends the user a detailed itinerary and related information, and the travel plan is complete.

[0042] For example, if a user wants to take a summer beach trip, they enter this into their device. The server analyzes the request and suggests destinations such as "Okinawa" or "Hawaii," and then presents the best flights and hotels within their budget. The user reviews this and makes customizations, such as adding "marine activities" and "local cuisine" in Okinawa. As a result, all details are finalized based on a satisfactory plan and sent to the user.

[0043] In this way, the present invention efficiently generates optimal travel plans for users and provides a stress-free travel planning experience.

[0044] The following describes the processing flow.

[0045] Step 1:

[0046] The user uses a device to enter their travel preferences, including potential destinations, budget, travel duration, desired activities, and food preferences. Once the user has finished entering the information, the device sends it to the server.

[0047] Step 2:

[0048] The server analyzes the user's preferences received. This uses advanced natural language processing technology to quickly understand the user's tastes and desires. Furthermore, the server also refers to the user's registration information and past travel history to identify their preferences.

[0049] Step 3:

[0050] Based on the analysis results, the server generates travel destinations and activities best suited to the user's preferences. This process involves creating options that consider the characteristics of the destinations, the experiences they offer, and the user's budget.

[0051] Step 4:

[0052] The server searches for suitable transportation and accommodation for the generated candidate locations and activities. During this process, it secures the latest pricing and availability information from reliable providers to create the most efficient plan.

[0053] Step 5:

[0054] The device presents the user with a generated travel plan. Here, the available travel destinations, transportation options, and accommodations are displayed in detail. The user can select items from the presented plan and customize it as needed.

[0055] Step 6:

[0056] The user reviews their customized plan through their device, and if they are satisfied with the travel details, they confirm the booking. Once the booking is confirmed, the device verifies with the server and prompts the user to enter payment information.

[0057] Step 7:

[0058] After the server approves the payment, it prepares the final itinerary and related documents and sends them to the user's device. This completes the user's travel planning process.

[0059] (Example 1)

[0060] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0061] Modern travel planning requires considerable time and effort to select destinations and activities that meet individual preferences from a vast amount of information. Furthermore, real-time information updates are difficult, making it challenging to create a plan that perfectly matches budget and preferences. Additionally, the lack of personalized suggestions based on past travel history makes creating highly satisfying travel plans a significant challenge.

[0062] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0063] In this invention, the server includes means for receiving travel preference information from a user and analyzing the preference information using natural language processing to identify the user's preferences; means for generating multiple travel destination candidates and activities using a generative model based on the generated preference data; and means for acquiring and searching real-time information on transportation and accommodation for the multiple candidates. This enables users to efficiently obtain personalized travel plans and create highly satisfying travel plans while saving time and effort.

[0064] A "user" refers to an individual or group that uses the system to plan a trip.

[0065] "Desired information" refers to the user's requests regarding their trip, such as destination, budget, duration, activities, and food preferences.

[0066] "Natural language processing" is a technology that enables computers to understand, interpret, and generate natural language used by humans.

[0067] "Preference data" refers to information about a user's preferences and tendencies, obtained based on their past behavior and choices.

[0068] A "generative model" is a machine learning model used to analyze data and create new information or options.

[0069] "Travel destination candidates" refers to multiple travel destinations suggested by the system based on the user's preferences and conditions.

[0070] An "activity" refers to a specific action or event that a traveler wants to experience during their trip.

[0071] "Means of transportation" refers to the modes of transport and routes that users can choose to reach their destination when traveling.

[0072] "Accommodation facilities" refer to facilities such as hotels, inns, and guesthouses where travelers stay at their destination.

[0073] "Real-time" means processing data instantly and providing the latest information.

[0074] To implement this invention, it is necessary to configure software that uses an information processing system to assist users in planning their travels. This system uses a server and a terminal, which are the main components, to effectively analyze the user's input information and generate personalized travel plans.

[0075] Users input their desired travel information using their devices. These devices include smartphones and computers, and the interface is designed for ease of use. Users can input their desired information, such as destination, budget, duration, desired activities, and dietary preferences, in natural language. This allows for intuitive and rapid data entry.

[0076] The server receives the input request information and performs analysis using natural language processing technology. This analysis utilizes advanced text analysis libraries such as "spaCy" to identify user preferences and conditions. The analyzed data is combined with the user's past behavioral history and stored as preference data.

[0077] Based on preference data, the server uses a generative AI model to create optimal travel destinations and activities for the user. This generative model utilizes machine learning techniques. A specific example is a generative model using Python, which generates personalized travel plans.

[0078] Next, the server retrieves real-time information on transportation and accommodation based on recent data. This uses APIs from travel information services. For example, the "Amadeus API" is available to retrieve transportation information, providing accurate fares and seat availability.

[0079] Users can review the travel itinerary presented through their device and customize it to their preferences. For example, they can change specific activities within the proposed travel destination or modify accommodations directly on their device. The customized content is then sent to the server and set as the final travel plan.

[0080] In the final stage, the user confirms their travel plan and completes the booking process on their device. Once payment is complete, the server sends the user a detailed travel plan and necessary travel information.

[0081] As a concrete example, a user can enter a prompt message such as, "Please suggest the best travel plan based on the following conditions: destination is a city for spring sightseeing, budget is under 150,000 yen, duration is 3 days, desired activities are cultural experiences, and preferred cuisine is French." Based on this information, the system will suggest a travel plan suitable for the user.

[0082] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0083] Step 1:

[0084] Users input their travel preferences through a terminal. This includes specific conditions such as potential destinations, budget, travel duration, desired activities, and dietary preferences. The entered preferences are transmitted from the terminal to the server as digital data. The terminal checks the format of the entered data and ensures data accuracy by requesting confirmation from the user if there are any ambiguities.

[0085] Step 2:

[0086] The server receives the desired information sent from the terminal and analyzes it using advanced natural language processing techniques. This analysis includes using a text analysis library to identify keywords and contexts related to the user's preferences and travel objectives. The preference data generated by the analysis is stored as the user's profile and used for further processing by a generative AI model.

[0087] Step 3:

[0088] The server uses a generative AI model to generate multiple travel destination options and activities based on preference data obtained through analysis. The generative model combines preference data with previously collected pattern data to construct a travel plan that matches the user's wishes. The generated plan is saved in digital format and prepared for use in the next step.

[0089] Step 4:

[0090] The server collects real-time information on transportation and accommodation based on the generated travel destinations and activities. This process uses configured APIs to interact with external databases and retrieve the latest fares and availability. The retrieved information is then integrated into a plan best suited to the user.

[0091] Step 5:

[0092] The server sends the integrated travel plan to the device and presents it to the user. The user can review the travel plan received on the device and customize activities, accommodations, and budget as needed. The customized information is sent from the device to the server, which re-parses and updates it.

[0093] Step 6:

[0094] The user confirms their final, satisfactory travel plan and completes the booking process on their device. The device provides an interface for entering payment data, such as credit card information, and securely transmits it to the server. The server verifies the payment and confirms all data.

[0095] Step 7:

[0096] The server transmits confirmed travel information to the terminal, providing the user with detailed itinerary and booking confirmations. This allows the user to complete all travel arrangements and proceed with their plans smoothly.

[0097] (Application Example 1)

[0098] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] When planning a trip, travelers face numerous challenges, including the difficulty of choosing a suitable destination from many options and the confusion and hassle involved in comparing transportation and accommodations. A particular challenge is the lack of visual information, which makes it difficult to concretely imagine the atmosphere and activities of a destination, resulting in anxiety and indecision.

[0100] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0101] In this invention, the server includes means for receiving travel preference information from a user and analyzing said preference information to identify the user's preferences; means for generating multiple travel destinations and activities based on the user's preferences; and means for presenting the travel plan in three dimensions using visual equipment, allowing the user to experience it interactively. This enables the user to concretely experience the travel plan visually and make decisions about the plan with greater confidence.

[0102] A "user" is an individual who uses this system when planning a trip and inputs their wishes and preferences.

[0103] "Travel preference information" refers to information entered by the user, including potential travel destinations, budget, travel duration, desired activities, and food preferences.

[0104] "Preferences" refer to the likes and interests that can be inferred from a user's past behavior and choices.

[0105] A "travel destination" refers to a geographical location or region that the user wishes to visit.

[0106] "Activities" refer to events, tours, and other activities that can be experienced at a travel destination.

[0107] "Transportation" refers to the means of getting to a travel destination, and includes airplanes, trains, buses, etc.

[0108] "Accommodation" refers to hotels, inns, or other lodging facilities where travelers stay during their trip.

[0109] "Visual devices" are devices that allow users to visually receive digital information, and include smart glasses and head-mounted displays.

[0110] "Presenting in three dimensions" means displaying information to users in a three-dimensional and spatial manner through visual devices.

[0111] The system for realizing this invention consists of a terminal where the user inputs travel preferences, a server that analyzes the information and generates a travel plan, and a visual device that visually displays the generated plan. Specifically, it is implemented in the following stages.

[0112] Users enter their travel preferences using devices such as smartphones or tablets. This includes desired destinations, budget, travel duration, preferred activities, and dietary preferences. This information, transmitted from the device, is received by the server.

[0113] The server analyzes the received data using advanced natural language processing (NLP) techniques. Specifically, Google Cloud Natural Language (a registered trademark) is used. Based on the analysis, the user's preferences are identified, and personalized travel destinations and activities are suggested based on their past travel experiences.

[0114] This proposed travel plan obtains the latest information on transportation and accommodations and presents it to the user in three dimensions via visual devices. Specifically, smart glasses such as Microsoft HoloLens® are used, and three-dimensional visual information is generated using 3D modeling tools such as Unity.

[0115] Users can review and customize travel details based on visual information. Through this process, an optimized travel plan is created for the user. By experiencing the travel plan visually, users gain a realistic image of the trip and become more confident in their final decision.

[0116] An example of a prompt message is: "Analyze the user's preference data to generate a visually appealing and personalized travel plan, providing interactive information that can be experienced through smart glasses. For example, display potential beach destinations in Okinawa in 3D and suggest additional activities."

[0117] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0118] Step 1:

[0119] The user enters their travel preferences via their device. They specify their desired destination, budget, travel duration, desired activities, and dietary preferences. The entered data is then transmitted from the device to the server.

[0120] Step 2:

[0121] The server analyzes the requested information it receives. First, it uses natural language processing (NLP) techniques to convert user input into structured data. Google Cloud Natural Language is used for this process. The output of the analysis identifies the user's preferences.

[0122] Step 3:

[0123] The server generates suitable travel destinations and activities based on the user's preferences. It refers to past travel history and acquired preference information to generate a list of travel destinations and activities. This information is then presented to the user in a personalized manner.

[0124] Step 4:

[0125] The server searches for transportation and accommodation options related to potential travel destinations and activities. It uses travel information APIs (e.g., Skyscanner API, Booking.com API) to obtain real-time pricing information. The retrieved data is then ready to be presented to the user as up-to-date transportation and accommodation options.

[0126] Step 5:

[0127] The server-generated travel plan is presented to the user in three dimensions through visual devices. Using smart glasses such as Microsoft HoloLens, the 3D model created with Unity is displayed. This allows the user to visually experience the atmosphere of the trip in a concrete way.

[0128] Step 6:

[0129] Users customize their travel plans based on visual information. They add activities and adjust their travel itinerary through the smart glasses interface. This customization is then sent back to the server, and the final travel plan is confirmed.

[0130] Step 7:

[0131] The server finalizes the travel plan and sends detailed related information to the user. The finalized plan includes all booking information and schedules and is transferred to the user's device. The user then uses this information to complete the final travel arrangements.

[0132] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0133] This invention relates to a system for providing more personalized travel plans to users when they are planning a trip, by taking into account their emotions at the time. This system has a function that uses an emotion engine to recognize the user's emotions and adjust the travel plan based on those emotions.

[0134] The user first enters their travel preferences through the device. During this input process, their current emotional state is recorded by selecting sentences or options that express their feelings. If the device is equipped with a camera and microphone, it is also possible to analyze emotions in real time from facial expressions and tone of voice.

[0135] The server processes the received information, and the emotion engine analyzes the user's emotions. Specifically, it uses natural language processing and image analysis to read emotions from text, audio, and images. Based on this analysis, it selects travel destinations and activities that match the user's emotions. For example, a user who wants to relax might be suggested a quiet resort and spa activities.

[0136] Next, the server uses the sentiment analysis results and user preferences to search for the most suitable transportation and accommodation options. This search is performed by referencing real-time data and providing information on rates and availability.

[0137] The server then presents the user with a travel plan tailored to their emotions via the terminal. The user can review and customize the plan, particularly by choosing the option that best suits their feelings from a variety of choices.

[0138] Finally, if the user is satisfied with the plan, they will confirm the booking and make the necessary payment through their device. The confirmed plan will then be sent to the user from the server as final itinerary information.

[0139] For example, if a user is planning a vacation while feeling tired from work, the server will emphasize options that prioritize relaxation and suggest suitable resorts and refreshing activities. This approach allows users to plan the perfect trip tailored to their mood at the time and enjoy a fulfilling experience.

[0140] This system aims to improve user satisfaction in travel planning and provide a more comfortable and fulfilling experience by utilizing emotion analysis technology.

[0141] The following describes the processing flow.

[0142] Step 1:

[0143] The user enters their travel preferences via their device. During this process, options and comments are displayed in a question-and-answer format to understand the user's emotions. If the device has a camera and microphone, facial recognition and voice tone analysis are used to further capture the user's emotions.

[0144] Step 2:

[0145] The server receives user input information and emotion data. The server's emotion engine analyzes the user's emotional state using natural language processing and image and voice analysis techniques. Emotions are then labeled (e.g., want to relax, want adventure).

[0146] Step 3:

[0147] Based on analyzed emotions and preferences, the server generates suitable travel destinations and activities for the user. For example, it suggests beach resorts and hot springs for users who want to relax, and mountain trekking and city exploration for adventure-oriented users.

[0148] Step 4:

[0149] The server searches for transportation and accommodation based on the generated candidate locations and activities. This search involves referencing real-time data to obtain results that include the best rates and availability information.

[0150] Step 5:

[0151] The device presents the user with a travel plan based on sentiment analysis results. The user can review this plan and further customize individual items. For example, the user can adjust the start time and dates of activities.

[0152] Step 6:

[0153] The user reviews the final plan on their device, and if they are satisfied, they confirm the booking. They are then prompted to enter payment information, and the user completes the necessary payment for the booking.

[0154] Step 7:

[0155] The server receives the booking confirmation and prepares the final itinerary and related documents. This information is then delivered to the user via their terminal, completing the travel plan.

[0156] This process is designed to effectively suggest travel plans that are adapted to the user's emotional state, thereby improving overall satisfaction with the travel experience.

[0157] (Example 2)

[0158] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0159] When planning a trip, simply offering suggestions based on preferences is insufficient to maximize user satisfaction. It is necessary to propose personalized travel plans that take into account the user's emotions at the time. Furthermore, selecting the optimal transportation and accommodation options based on real-time information is crucial. Combining past travel history with emotions is expected to lead to even more accurate planning.

[0160] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0161] In this invention, the server includes means for receiving and analyzing travel preference information and emotional data from a user to identify the user's preferences and emotions; means for generating a plurality of potential travel destinations and activities based on the user's preferences and emotions; and means for searching for transportation methods and accommodations for the plurality of potential travel destinations and activities. This makes it possible to propose a more satisfying travel plan that comprehensively takes into account the user's preferences and emotions.

[0162] "Users" refer to people who use this system to plan their trips.

[0163] "Travel preference information" refers to information such as the user's desired destination, itinerary, budget, and purpose of travel when planning a trip.

[0164] "Emotional data" refers to data that indicates a user's emotional state, and is extracted from text, audio, images, and other sources.

[0165] "Preferences" refers to information about a user's interests and preferences.

[0166] "Travel destinations" refers to multiple travel destinations suggested to the user.

[0167] "Activities" refer to recreational activities, events, and other activities that users can participate in during their trip.

[0168] "Method of transportation" refers to the means of transport during a trip.

[0169] "Accommodation facilities" refer to facilities where travelers stay during their trip.

[0170] "Past travel history" refers to records of trips the user has taken in the past.

[0171] "Acquiring in real time" refers to the process of obtaining the latest information online at the present time.

[0172] This invention is a system that takes into account the user's emotions at the time when planning a trip, providing a more personalized plan. Specifically, it uses an emotion engine to analyze the user's emotions and optimizes the travel plan based on the results. The main elements of this system are the terminal used by the user, a server that analyzes the data, and an algorithm that creates the plan based on the analysis results.

[0173] The user first enters information about their travel plans through their device. This input includes basic information such as destination, dates, and budget, as well as questions that reflect their emotions. If the device has a camera and microphone, these can be used to record the user's facial expressions and tone of voice in real time and send this emotion data to the server.

[0174] The server analyzes the user's emotions using natural language processing and image analysis technologies based on the received information. A software called an emotion engine plays a crucial role in this analysis, extracting emotional states from text, audio, and images and representing them as numerical values ​​or categories. This emotion data is combined with the user's preferences and past travel history to generate travel plans.

[0175] The algorithm incorporates analyzed emotional data to generate multiple travel destinations and activities that align with the user's preferences. For example, if it determines that the user is seeking "relaxation," it can suggest a plan that includes a quiet, nature-rich resort or a spa experience.

[0176] Specific examples of prompts from this system include "Suggest relaxing travel destinations for a user who is tired from work" and "List the most suitable activities for the user based on their current mood."

[0177] In this way, the system aims to provide a more satisfying travel plan by comprehensively considering the user's preferences and emotions. As a result, users can receive the optimal travel plan tailored to their mood at the time, allowing them to have a fulfilling experience.

[0178] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0179] Step 1:

[0180] Users use their devices to answer questions that reflect basic travel information and their emotions. This process includes inputting travel destination, dates, budget, and other details. Additionally, if the device has a camera and microphone, the user's facial expressions and voice tone are recorded. This input data is sent to a server as emotional data.

[0181] Step 2:

[0182] The server analyzes travel preference information and emotional data received from the terminal. Text information received as input is analyzed using natural language processing technology, voice data is processed by voice analysis software, and facial expression data is analyzed using image analysis technology. As a result of these analyses, numerical or categorical data indicating the user's emotional state is output.

[0183] Step 3:

[0184] The server generates multiple travel destinations and activities based on analyzed sentiment data and user preference information. A generative AI model is used, receiving prompts such as "Suggest the best travel destination based on the user's sentiment." This generates an appropriate travel plan. This output allows for suggestions such as quiet resorts and spas for users seeking relaxation.

[0185] Step 4:

[0186] The server searches for transportation and accommodation options for suggested travel destinations and activities by referencing a real-time updated database. The search query includes detailed criteria associated with the travel plan, and checks for prices and availability. The output provides detailed information such as costs and available accommodations.

[0187] Step 5:

[0188] The server sends an optimized travel plan to the user's device and makes suggestions. The user reviews the plan details on their device and customizes it as needed, such as changing activities or selecting transportation. The customized information is then sent back to the server.

[0189] Step 6:

[0190] If the user is satisfied with the travel plan, they will confirm the booking and make payment via their device. The confirmed travel plan information is verified by the server and sent to the user as the final itinerary. As output, the user will receive the final confirmed travel schedule.

[0191] (Application Example 2)

[0192] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0193] In travel planning, there is a challenge in providing personalized plans that take into account the emotional state of the user. Furthermore, in the retail experience, there is a lack of effective product and service suggestions that consider the user's emotions. Therefore, there is a need to provide the optimal travel and consumption experience for users and improve their satisfaction.

[0194] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0195] In this invention, the server includes means for receiving travel information from a user and analyzing the information to identify the user's personal preferences; means for generating multiple destinations and activity content based on the user's preferences; and means for analyzing the user's emotions in real time to select and present appropriate products and services in order to improve the consumer experience at physical stores. This makes it possible to provide personalized travel plans that take into account the user's emotional state and preferences, and to improve the optimal consumer experience at physical stores.

[0196] A "user" is an individual who uses this system to customize their travel plans and consumption experiences.

[0197] "Travel information" refers to detailed information about the trip desired by the user, including destination, duration, budget, preferences, etc.

[0198] "Preferences" refers to information that indicates a user's personal preferences and priorities regarding travel.

[0199] "Destination" refers to the geographical location of the place a user plans to visit when planning a trip.

[0200] "Activities" refer to events and activities that users participate in during their travels or while inside physical stores, and include sightseeing, leisure, shopping, and more.

[0201] "Means of transportation" refers to the means of transport used by users to travel between destinations, and includes airplanes, trains, buses, etc.

[0202] "Accommodation facilities" refer to places where travelers stay during their trip, and include hotels, hostels, and vacation rentals.

[0203] A "physical store" refers to a physical store where consumers actually visit to purchase or experience goods and services.

[0204] "Emotions" refers to the psychological state that users experience during the process of planning a trip or engaging in consumer activities.

[0205] "Products" refer to specific items or services provided to customers at physical stores.

[0206] "Service" refers to beneficial actions or support other than the goods provided within a physical store, and includes customer service, advice, and promotions.

[0207] The system that realizes this application is configured as a program that provides personalized experiences through user travel information and sentiment analysis at physical stores. The server receives travel information from users, analyzes it to identify user preferences, generates destinations and activities based on the analyzed information, and further analyzes sentiment at physical stores in real time to suggest appropriate products and services.

[0208] The terminal uses devices such as smartphones and smart glasses to acquire the user's facial expressions and voice data and send it to the server. Based on this data, the server utilizes emotion analysis technologies such as Google Cloud Vision API and Amazon Rekognition to identify the user's emotional state in real time. Based on these results, it can suggest products and services that are best suited to the user. For example, if the user is feeling stressed, the application will suggest products that help them relax. This suggestion is notified to the user in real time via their smartphone or smart glasses.

[0209] As a concrete example, when a user visits a physical store during a holiday trip, the device can analyze the user's facial expressions and suggest a relaxing aromatherapy candle. An example of a prompt to the generating AI model in this case would be, "Analyze the user's emotions from their facial expressions and voice, and suggest the most suitable products and services in real time." In this way, real-time emotion analysis and product suggestions can significantly improve the user experience.

[0210] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0211] Step 1:

[0212] The terminal receives travel preference information from the user as input. This includes information such as destination, duration, budget, and preferences. This information is temporarily stored on the terminal before being sent to the server.

[0213] Step 2:

[0214] The server receives travel information from the terminal as input and performs data analysis to identify the user's preferences. This analysis includes using natural language processing techniques to extract keywords related to travel preferences from the text. The output is data that identifies the user's preferences.

[0215] Step 3:

[0216] The server generates multiple destinations and activities for the user based on identified preferences. This involves using a trained model to select the best suggestions from a database of past travel history. The output is a list of recommended destinations and activities for the user.

[0217] Step 4:

[0218] The device captures the user's facial expressions and voice when they visit a physical store. This serves as input, and real-time capture is performed on the device. This data is then sent to a server for emotion analysis.

[0219] Step 5:

[0220] The server receives real-time facial and audio data sent from the terminal as input and performs emotion analysis using the Google Cloud Vision API and Amazon Rekognition. As a result of the analysis, data identifying the user's current emotional state is output.

[0221] Step 6:

[0222] The server uses a generative AI model that selects products and services suitable for the user based on the results of emotion analysis. This design uses the prompt "Analyze the user's emotions from their facial expressions and voice, and suggest the most suitable products and services in real time" as the prompt for the generative model, resulting in the output of a list of appropriate products and services.

[0223] Step 7:

[0224] The device receives information about recommended products and services from the server and presents it to the user. The user can view the recommended products and services through the device. This information presentation allows the user to have a consumption experience that is tailored to their emotional state.

[0225] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0226] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0227] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0228] [Second Embodiment]

[0229] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0230] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0231] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0232] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0233] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0234] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0235] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0236] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0237] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0238] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0239] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0240] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0241] This invention relates to a system that reduces the time and effort users spend planning their trips. The system primarily includes user input of desired information, analysis of preferences, generation and presentation of travel plans, searching for transportation and accommodations, and finally confirming the plan and transmitting the information.

[0242] First, the user enters their travel preferences via their device. This includes potential destinations, budget, travel duration, desired activities, and specific food preferences. The server receives this information and analyzes it using advanced natural language processing technology. The server identifies the user's preferences and selects suitable travel destinations and activities. This analysis takes into account the user's past travel history to provide more personalized suggestions.

[0243] Next, the server searches for suitable transportation and accommodation based on the collected information. Here, it obtains real-time pricing and availability information from reliable partners. This allows users to have detailed options based on the latest data.

[0244] The server then presents the generated travel plan to the user via their terminal. The user reviews this plan and customizes it based on their preferences and requirements. The customized plan is then analyzed again by the system and finalized as the travel plan.

[0245] Finally, if the user is satisfied with the plan, they complete the booking on their device and enter their payment information. Once this is done, the server sends the user a detailed itinerary and related information, and the travel plan is complete.

[0246] For example, if a user wants to take a summer beach trip, they enter this into their device. The server analyzes the request and suggests destinations such as "Okinawa" or "Hawaii," and then presents the best flights and hotels within their budget. The user reviews this and makes customizations, such as adding "marine activities" and "local cuisine" in Okinawa. As a result, all details are finalized based on a satisfactory plan and sent to the user.

[0247] In this way, the present invention efficiently generates optimal travel plans for users and provides a stress-free travel planning experience.

[0248] The following describes the processing flow.

[0249] Step 1:

[0250] The user uses a device to enter their travel preferences, including potential destinations, budget, travel duration, desired activities, and food preferences. Once the user has finished entering the information, the device sends it to the server.

[0251] Step 2:

[0252] The server analyzes the user's preferences received. This uses advanced natural language processing technology to quickly understand the user's tastes and desires. Furthermore, the server also refers to the user's registration information and past travel history to identify their preferences.

[0253] Step 3:

[0254] Based on the analysis results, the server generates travel destinations and activities best suited to the user's preferences. This process involves creating options that consider the characteristics of the destinations, the experiences they offer, and the user's budget.

[0255] Step 4:

[0256] The server searches for suitable transportation and accommodation for the generated candidate locations and activities. During this process, it secures the latest pricing and availability information from reliable providers to create the most efficient plan.

[0257] Step 5:

[0258] The device presents the user with a generated travel plan. Here, the available travel destinations, transportation options, and accommodations are displayed in detail. The user can select items from the presented plan and customize it as needed.

[0259] Step 6:

[0260] The user reviews their customized plan through their device, and if they are satisfied with the travel details, they confirm the booking. Once the booking is confirmed, the device verifies with the server and prompts the user to enter payment information.

[0261] Step 7:

[0262] After the server approves the payment, it prepares the final itinerary and related documents and sends them to the user's device. This completes the user's travel planning process.

[0263] (Example 1)

[0264] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0265] Modern travel planning requires considerable time and effort to select destinations and activities that meet individual preferences from a vast amount of information. Furthermore, real-time information updates are difficult, making it challenging to create a plan that perfectly matches budget and preferences. Additionally, the lack of personalized suggestions based on past travel history makes creating highly satisfying travel plans a significant challenge.

[0266] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0267] In this invention, the server includes means for receiving travel preference information from a user and analyzing the preference information using natural language processing to identify the user's preferences; means for generating multiple travel destination candidates and activities using a generative model based on the generated preference data; and means for acquiring and searching real-time information on transportation and accommodation for the multiple candidates. This enables users to efficiently obtain personalized travel plans and create highly satisfying travel plans while saving time and effort.

[0268] A "user" refers to an individual or group that uses the system to plan a trip.

[0269] "Desired information" refers to the user's requests regarding their trip, such as destination, budget, duration, activities, and food preferences.

[0270] "Natural language processing" is a technology that enables computers to understand, interpret, and generate natural language used by humans.

[0271] "Preference data" refers to information about a user's preferences and tendencies, obtained based on their past behavior and choices.

[0272] A "generative model" is a machine learning model used to analyze data and create new information or options.

[0273] "Travel destination candidates" refers to multiple travel destinations suggested by the system based on the user's preferences and conditions.

[0274] An "activity" refers to a specific action or event that a traveler wants to experience during their trip.

[0275] "Means of transportation" refers to the modes of transport and routes that users can choose to reach their destination when traveling.

[0276] "Accommodation facilities" refer to facilities such as hotels, inns, and guesthouses where travelers stay at their destination.

[0277] "Real-time" means processing data instantly and providing the latest information.

[0278] To implement this invention, it is necessary to configure software that uses an information processing system to assist users in planning their travels. This system uses a server and a terminal, which are the main components, to effectively analyze the user's input information and generate personalized travel plans.

[0279] The user inputs the desired travel information using a terminal. The terminal can be a smartphone, a personal computer, etc., and the interface is designed to be user-friendly. The user can input desired information such as the travel destination, budget, travel period, desired activities, and food preferences in natural language. This enables intuitive and rapid input.

[0280] The server receives the input desired information and performs analysis using natural language processing technology. For this analysis, by leveraging an advanced text analysis library such as "spaCy", the user's preferences and conditions can be identified. The analyzed data is combined with the user's past behavior history and retained as preference data.

[0281] Based on the preference data, the server uses a generative AI model to create travel destination candidates and activities optimal for the user. This generative model utilizes machine learning technology. As a specific example, a generative model using Python can be cited, and this generates a personalized travel plan.

[0282] Next, based on the latest data, the server obtains information on means of transportation and accommodation facilities in real time. For this, the API of a travel information providing service is used. For example, "Amadeus API" can be utilized to obtain transportation means information, and accurate fares and seat availability are provided.

[0283] The user checks the travel content presented through the terminal and can customize it according to their preferences. For example, specific activities for travel destination candidates or changes to the accommodation can be made on the terminal. The customized content is sent to the server and set as the final travel plan.

[0284] In the final stage, the user finalizes the travel plan on the terminal and conducts the reservation procedure. When the payment is completed, the server sends the user a detailed travel plan and the necessary travel information.

[0285] As a specific example, a user can input a prompt sentence such as "Please propose an optimal travel plan based on the following conditions: the travel destination is spring city sightseeing, the budget is within 150,000 yen, the duration is 3 days, the desired activity is cultural experience, and the meal is French cuisine". Based on this information, the system proposes a travel plan suitable for the user.

[0286] The flow of the specific process in Example 1 will be described using FIG. 11.

[0287] Step 1:

[0288] The user inputs travel wish information through the terminal. This includes specific conditions such as candidate travel destinations, budget, travel duration, desired activities, and food preferences. The input wish information is transmitted from the terminal to the server as digital data. The terminal checks the format of the input data and requests confirmation from the user if there are ambiguous parts to ensure the accuracy of the data.

[0289] Step 2:

[0290] ]> The server receives the wish information transmitted from the terminal and analyzes it using advanced natural language processing technology. This analysis includes processes such as using a text analysis library to identify keywords and contexts related to the user's preferences and travel purposes. The preference data generated by the analysis is stored as the user's profile and utilized for further processing by the generation AI model.

[0291] Step 3:

[0292] The server utilizes the generation AI model based on the preference data obtained from the analysis to generate multiple travel destination candidates and activities. The generation model combines the preference data and pattern data collected in the past to construct a travel plan that meets the user's wishes. The generated plan is saved in digital form and prepared for use in the next step.

[0293] Step 4:

[0294] The server collects real-time information on transportation and accommodation based on the generated travel destinations and activities. This process uses configured APIs to interact with external databases and retrieve the latest fares and availability. The retrieved information is then integrated into a plan best suited to the user.

[0295] Step 5:

[0296] The server sends the integrated travel plan to the device and presents it to the user. The user can review the travel plan received on the device and customize activities, accommodations, and budget as needed. The customized information is sent from the device to the server, which re-parses and updates it.

[0297] Step 6:

[0298] The user confirms their final, satisfactory travel plan and completes the booking process on their device. The device provides an interface for entering payment data, such as credit card information, and securely transmits it to the server. The server verifies the payment and confirms all data.

[0299] Step 7:

[0300] The server transmits confirmed travel information to the terminal, providing the user with detailed itinerary and booking confirmations. This allows the user to complete all travel arrangements and proceed with their plans smoothly.

[0301] (Application Example 1)

[0302] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0303] When making a travel plan, the difficulties faced by users include determining a suitable travel destination from many options and the confusion and effort involved in comparing transportation means and accommodation facilities. In particular, due to the lack of visual information, it is difficult to specifically imagine the atmosphere and activities of the travel destination, resulting in anxiety and confusion, which is an issue.

[0304] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0305] In this invention, the server includes means for receiving travel wish information from a user, analyzing the wish information to identify the user's preferences, means for generating a plurality of travel destinations and activities based on the user's preferences, and means for presenting the travel plan three-dimensionally using a visual device so that the user can experience it interactively. As a result, the user can visually and specifically experience the travel plan and make a more confident decision.

[0306] A "user" is an individual who uses this system when making a travel plan and inputs their wishes and preferences.

[0307] "Travel wish information" is information including candidates for travel destinations, budgets, travel periods, desired activities, food preferences, etc. input by the user.

[0308] "Preferences" refer to the likes and interests inferred based on the user's past behaviors and choices.

[0309] A "travel destination" is a geographical location or area that the user wishes to visit.

[0310] An "activity" is an event, tour, activity, etc. that can be experienced at the travel destination.

[0311] A "transportation means" is a means used for moving to the travel destination and includes airplanes, trains, buses, etc.

[0312] "Accommodation" refers to hotels, inns, or other lodging facilities where travelers stay during their trip.

[0313] "Visual devices" are devices that allow users to visually receive digital information, and include smart glasses and head-mounted displays.

[0314] "Presenting in three dimensions" means displaying information to users in a three-dimensional and spatial manner through visual devices.

[0315] The system for realizing this invention consists of a terminal where the user inputs travel preferences, a server that analyzes the information and generates a travel plan, and a visual device that visually displays the generated plan. Specifically, it is implemented in the following stages.

[0316] Users enter their travel preferences using devices such as smartphones or tablets. This includes desired destinations, budget, travel duration, preferred activities, and dietary preferences. This information, transmitted from the device, is received by the server.

[0317] The server analyzes the received data using advanced natural language processing (NLP) techniques. Specifically, Google Cloud Natural Language is used. Based on the analysis, the user's preferences are identified, and personalized travel destinations and activities are suggested based on their past travel experiences.

[0318] This proposed travel plan will acquire the latest information on transportation and accommodations and present it to the user in three dimensions via visual devices. Specifically, smart glasses such as Microsoft HoloLens will be used, and three-dimensional visual information will be generated using 3D modeling tools such as Unity.

[0319] Users can review and customize travel details based on visual information. Through this process, an optimized travel plan is created for the user. By experiencing the travel plan visually, users gain a realistic image of the trip and become more confident in their final decision.

[0320] An example of a prompt message is: "Analyze the user's preference data to generate a visually appealing and personalized travel plan, providing interactive information that can be experienced through smart glasses. For example, display potential beach destinations in Okinawa in 3D and suggest additional activities."

[0321] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0322] Step 1:

[0323] The user enters their travel preferences via their device. They specify their desired destination, budget, travel duration, desired activities, and dietary preferences. The entered data is then transmitted from the device to the server.

[0324] Step 2:

[0325] The server analyzes the requested information it receives. First, it uses natural language processing (NLP) techniques to convert user input into structured data. Google Cloud Natural Language is used for this process. The output of the analysis identifies the user's preferences.

[0326] Step 3:

[0327] The server generates suitable travel destinations and activities based on the user's preferences. It refers to past travel history and acquired preference information to generate a list of travel destinations and activities. This information is then presented to the user in a personalized manner.

[0328] Step 4:

[0329] The server searches for transportation and accommodation options related to potential travel destinations and activities. It uses travel information APIs (e.g., Skyscanner API, Booking.com API) to obtain real-time pricing information. The retrieved data is then ready to be presented to the user as up-to-date transportation and accommodation options.

[0330] Step 5:

[0331] The server-generated travel plan is presented to the user in three dimensions through visual devices. Using smart glasses such as Microsoft HoloLens, the 3D model created with Unity is displayed. This allows the user to visually experience the atmosphere of the trip in a concrete way.

[0332] Step 6:

[0333] Users customize their travel plans based on visual information. They add activities and adjust their travel itinerary through the smart glasses interface. This customization is then sent back to the server, and the final travel plan is confirmed.

[0334] Step 7:

[0335] The server finalizes the travel plan and sends detailed related information to the user. The finalized plan includes all booking information and schedules and is transferred to the user's device. The user then uses this information to complete the final travel arrangements.

[0336] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0337] This invention relates to a system for providing more personalized travel plans to users when they are planning a trip, by taking into account their emotions at the time. This system has a function that uses an emotion engine to recognize the user's emotions and adjust the travel plan based on those emotions.

[0338] The user first enters their travel preferences through the device. During this input process, their current emotional state is recorded by selecting sentences or options that express their feelings. If the device is equipped with a camera and microphone, it is also possible to analyze emotions in real time from facial expressions and tone of voice.

[0339] The server processes the received information, and the emotion engine analyzes the user's emotions. Specifically, it uses natural language processing and image analysis to read emotions from text, audio, and images. Based on this analysis, it selects travel destinations and activities that match the user's emotions. For example, a user who wants to relax might be suggested a quiet resort and spa activities.

[0340] Next, the server uses the sentiment analysis results and user preferences to search for the most suitable transportation and accommodation options. This search is performed by referencing real-time data and providing information on rates and availability.

[0341] The server then presents the user with a travel plan tailored to their emotions via the terminal. The user can review and customize the plan, particularly by choosing the option that best suits their feelings from a variety of choices.

[0342] Finally, if the user is satisfied with the plan, they will confirm the booking and make the necessary payment through their device. The confirmed plan will then be sent to the user from the server as final itinerary information.

[0343] For example, if a user is planning a vacation while feeling tired from work, the server will emphasize options that prioritize relaxation and suggest suitable resorts and refreshing activities. This approach allows users to plan the perfect trip tailored to their mood at the time and enjoy a fulfilling experience.

[0344] This system aims to improve user satisfaction in travel planning and provide a more comfortable and fulfilling experience by utilizing emotion analysis technology.

[0345] The following describes the processing flow.

[0346] Step 1:

[0347] The user enters their travel preferences via their device. During this process, options and comments are displayed in a question-and-answer format to understand the user's emotions. If the device has a camera and microphone, facial recognition and voice tone analysis are used to further capture the user's emotions.

[0348] Step 2:

[0349] The server receives user input information and emotion data. The server's emotion engine analyzes the user's emotional state using natural language processing and image and voice analysis techniques. Emotions are then labeled (e.g., want to relax, want adventure).

[0350] Step 3:

[0351] Based on analyzed emotions and preferences, the server generates suitable travel destinations and activities for the user. For example, it suggests beach resorts and hot springs for users who want to relax, and mountain trekking and city exploration for adventure-oriented users.

[0352] Step 4:

[0353] The server searches for transportation and accommodation based on the generated candidate locations and activities. This search involves referencing real-time data to obtain results that include the best rates and availability information.

[0354] Step 5:

[0355] The device presents the user with a travel plan based on sentiment analysis results. The user can review this plan and further customize individual items. For example, the user can adjust the start time and dates of activities.

[0356] Step 6:

[0357] The user reviews the final plan on their device, and if they are satisfied, they confirm the booking. They are then prompted to enter payment information, and the user completes the necessary payment for the booking.

[0358] Step 7:

[0359] The server receives the booking confirmation and prepares the final itinerary and related documents. This information is then delivered to the user via their terminal, completing the travel plan.

[0360] This process is designed to effectively suggest travel plans that are adapted to the user's emotional state, thereby improving overall satisfaction with the travel experience.

[0361] (Example 2)

[0362] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0363] When planning a trip, simply offering suggestions based on preferences is insufficient to maximize user satisfaction. It is necessary to propose personalized travel plans that take into account the user's emotions at the time. Furthermore, selecting the optimal transportation and accommodation options based on real-time information is crucial. Combining past travel history with emotions is expected to lead to even more accurate planning.

[0364] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0365] In this invention, the server includes means for receiving and analyzing travel preference information and emotional data from a user to identify the user's preferences and emotions; means for generating a plurality of potential travel destinations and activities based on the user's preferences and emotions; and means for searching for transportation methods and accommodations for the plurality of potential travel destinations and activities. This makes it possible to propose a more satisfying travel plan that comprehensively takes into account the user's preferences and emotions.

[0366] "Users" refer to people who use this system to plan their trips.

[0367] "Travel preference information" refers to information such as the user's desired destination, itinerary, budget, and purpose of travel when planning a trip.

[0368] "Emotional data" refers to data that indicates a user's emotional state, and is extracted from text, audio, images, and other sources.

[0369] "Preferences" refers to information about a user's interests and preferences.

[0370] "Travel destinations" refers to multiple travel destinations suggested to the user.

[0371] "Activities" refer to recreational activities, events, and other activities that users can participate in during their trip.

[0372] "Method of transportation" refers to the means of transport during a trip.

[0373] "Accommodation facilities" refer to facilities where travelers stay during their trip.

[0374] "Past travel history" refers to records of trips the user has taken in the past.

[0375] "Acquiring in real time" refers to the process of obtaining the latest information online at the present time.

[0376] This invention is a system that takes into account the user's emotions at the time when planning a trip, providing a more personalized plan. Specifically, it uses an emotion engine to analyze the user's emotions and optimizes the travel plan based on the results. The main elements of this system are the terminal used by the user, a server that analyzes the data, and an algorithm that creates the plan based on the analysis results.

[0377] The user first enters information about their travel plans through their device. This input includes basic information such as destination, dates, and budget, as well as questions that reflect their emotions. If the device has a camera and microphone, these can be used to record the user's facial expressions and tone of voice in real time and send this emotion data to the server.

[0378] The server analyzes the user's emotions using natural language processing and image analysis technologies based on the received information. A software called an emotion engine plays a crucial role in this analysis, extracting emotional states from text, audio, and images and representing them as numerical values ​​or categories. This emotion data is combined with the user's preferences and past travel history to generate travel plans.

[0379] The algorithm incorporates analyzed emotional data to generate multiple travel destinations and activities that align with the user's preferences. For example, if it determines that the user is seeking "relaxation," it can suggest a plan that includes a quiet, nature-rich resort or a spa experience.

[0380] Specific examples of prompts from this system include "Suggest relaxing travel destinations for a user who is tired from work" and "List the most suitable activities for the user based on their current mood."

[0381] In this way, the system aims to provide a more satisfying travel plan by comprehensively considering the user's preferences and emotions. As a result, users can receive the optimal travel plan tailored to their mood at the time, allowing them to have a fulfilling experience.

[0382] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0383] Step 1:

[0384] Users use their devices to answer questions that reflect basic travel information and their emotions. This process includes inputting travel destination, dates, budget, and other details. Additionally, if the device has a camera and microphone, the user's facial expressions and voice tone are recorded. This input data is sent to a server as emotional data.

[0385] Step 2:

[0386] The server analyzes travel preference information and emotional data received from the terminal. Text information received as input is analyzed using natural language processing technology, voice data is processed by voice analysis software, and facial expression data is analyzed using image analysis technology. As a result of these analyses, numerical or categorical data indicating the user's emotional state is output.

[0387] Step 3:

[0388] The server generates multiple travel destinations and activities based on analyzed sentiment data and user preference information. A generative AI model is used, receiving prompts such as "Suggest the best travel destination based on the user's sentiment." This generates an appropriate travel plan. This output allows for suggestions such as quiet resorts and spas for users seeking relaxation.

[0389] Step 4:

[0390] The server searches for transportation and accommodation options for suggested travel destinations and activities by referencing a real-time updated database. The search query includes detailed criteria associated with the travel plan, and checks for prices and availability. The output provides detailed information such as costs and available accommodations.

[0391] Step 5:

[0392] The server sends an optimized travel plan to the user's device and makes suggestions. The user reviews the plan details on their device and customizes it as needed, such as changing activities or selecting transportation. The customized information is then sent back to the server.

[0393] Step 6:

[0394] If the user is satisfied with the travel plan, they will confirm the booking and make payment via their device. The confirmed travel plan information is verified by the server and sent to the user as the final itinerary. As output, the user will receive the final confirmed travel schedule.

[0395] (Application Example 2)

[0396] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0397] In travel planning, there is a challenge in providing personalized plans that take into account the emotional state of the user. Furthermore, in the retail experience, there is a lack of effective product and service suggestions that consider the user's emotions. Therefore, there is a need to provide the optimal travel and consumption experience for users and improve their satisfaction.

[0398] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0399] In this invention, the server includes means for receiving travel information from a user and analyzing the information to identify the user's personal preferences; means for generating multiple destinations and activity content based on the user's preferences; and means for analyzing the user's emotions in real time to select and present appropriate products and services in order to improve the consumer experience at physical stores. This makes it possible to provide personalized travel plans that take into account the user's emotional state and preferences, and to improve the optimal consumer experience at physical stores.

[0400] A "user" is an individual who uses this system to customize their travel plans and consumption experiences.

[0401] "Travel information" refers to detailed information about the trip desired by the user, including destination, duration, budget, preferences, etc.

[0402] "Preferences" refers to information that indicates a user's personal preferences and priorities regarding travel.

[0403] "Destination" refers to the geographical location of the place a user plans to visit when planning a trip.

[0404] "Activities" refer to events and activities that users participate in during their travels or while inside physical stores, and include sightseeing, leisure, shopping, and more.

[0405] "Means of transportation" refers to the means of transport used by users to travel between destinations, and includes airplanes, trains, buses, etc.

[0406] "Accommodation facilities" refer to places where travelers stay during their trip, and include hotels, hostels, and vacation rentals.

[0407] A "physical store" refers to a physical store where consumers actually visit to purchase or experience goods and services.

[0408] "Emotions" refers to the psychological state that users experience during the process of planning a trip or engaging in consumer activities.

[0409] "Products" refer to specific items or services provided to customers at physical stores.

[0410] "Service" refers to beneficial actions or support other than the goods provided within a physical store, and includes customer service, advice, and promotions.

[0411] The system that realizes this application is configured as a program that provides personalized experiences through user travel information and sentiment analysis at physical stores. The server receives travel information from users, analyzes it to identify user preferences, generates destinations and activities based on the analyzed information, and further analyzes sentiment at physical stores in real time to suggest appropriate products and services.

[0412] The terminal uses devices such as smartphones and smart glasses to acquire the user's facial expressions and voice data and send it to the server. Based on this data, the server utilizes emotion analysis technologies such as Google Cloud Vision API and Amazon Rekognition to identify the user's emotional state in real time. Based on these results, it can suggest products and services that are best suited to the user. For example, if the user is feeling stressed, the application will suggest products that help them relax. This suggestion is notified to the user in real time via their smartphone or smart glasses.

[0413] As a concrete example, when a user visits a physical store during a holiday trip, the device can analyze the user's facial expressions and suggest a relaxing aromatherapy candle. An example of a prompt to the generating AI model in this case would be, "Analyze the user's emotions from their facial expressions and voice, and suggest the most suitable products and services in real time." In this way, real-time emotion analysis and product suggestions can significantly improve the user experience.

[0414] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0415] Step 1:

[0416] The terminal receives travel preference information from the user as input. This includes information such as destination, duration, budget, and preferences. This information is temporarily stored on the terminal before being sent to the server.

[0417] Step 2:

[0418] The server receives travel information from the terminal as input and performs data analysis to identify the user's preferences. This analysis includes using natural language processing techniques to extract keywords related to travel preferences from the text. The output is data that identifies the user's preferences.

[0419] Step 3:

[0420] The server generates multiple destinations and activities for the user based on identified preferences. This involves using a trained model to select the best suggestions from a database of past travel history. The output is a list of recommended destinations and activities for the user.

[0421] Step 4:

[0422] The device captures the user's facial expressions and voice when they visit a physical store. This serves as input, and real-time capture is performed on the device. This data is then sent to a server for emotion analysis.

[0423] Step 5:

[0424] The server receives real-time facial and audio data sent from the terminal as input and performs emotion analysis using the Google Cloud Vision API and Amazon Rekognition. As a result of the analysis, data identifying the user's current emotional state is output.

[0425] Step 6:

[0426] The server uses a generative AI model that selects products and services suitable for the user based on the results of emotion analysis. This design uses the prompt "Analyze the user's emotions from their facial expressions and voice, and suggest the most suitable products and services in real time" as the prompt for the generative model, resulting in the output of a list of appropriate products and services.

[0427] Step 7:

[0428] The device receives information about recommended products and services from the server and presents it to the user. The user can view the recommended products and services through the device. This information presentation allows the user to have a consumption experience that is tailored to their emotional state.

[0429] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0430] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0431] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0432] [Third Embodiment]

[0433] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0434] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0435] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0436] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0437] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0438] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0439] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0440] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0441] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0442] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0443] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0444] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0445] This invention relates to a system that reduces the time and effort users spend planning their trips. The system primarily includes user input of desired information, analysis of preferences, generation and presentation of travel plans, searching for transportation and accommodations, and finally confirming the plan and transmitting the information.

[0446] First, the user enters their travel preferences via their device. This includes potential destinations, budget, travel duration, desired activities, and specific food preferences. The server receives this information and analyzes it using advanced natural language processing technology. The server identifies the user's preferences and selects suitable travel destinations and activities. This analysis takes into account the user's past travel history to provide more personalized suggestions.

[0447] Next, the server searches for suitable transportation and accommodation based on the collected information. Here, it obtains real-time pricing and availability information from reliable partners. This allows users to have detailed options based on the latest data.

[0448] The server then presents the generated travel plan to the user via their terminal. The user reviews this plan and customizes it based on their preferences and requirements. The customized plan is then analyzed again by the system and finalized as the travel plan.

[0449] Finally, if the user is satisfied with the plan, they complete the booking on their device and enter their payment information. Once this is done, the server sends the user a detailed itinerary and related information, and the travel plan is complete.

[0450] For example, if a user wants to take a summer beach trip, they enter this into their device. The server analyzes the request and suggests destinations such as "Okinawa" or "Hawaii," and then presents the best flights and hotels within their budget. The user reviews this and makes customizations, such as adding "marine activities" and "local cuisine" in Okinawa. As a result, all details are finalized based on a satisfactory plan and sent to the user.

[0451] In this way, the present invention efficiently generates optimal travel plans for users and provides a stress-free travel planning experience.

[0452] The following describes the processing flow.

[0453] Step 1:

[0454] The user uses a device to enter their travel preferences, including potential destinations, budget, travel duration, desired activities, and food preferences. Once the user has finished entering the information, the device sends it to the server.

[0455] Step 2:

[0456] The server analyzes the user's preferences received. This uses advanced natural language processing technology to quickly understand the user's tastes and desires. Furthermore, the server also refers to the user's registration information and past travel history to identify their preferences.

[0457] Step 3:

[0458] Based on the analysis results, the server generates travel destinations and activities best suited to the user's preferences. This process involves creating options that consider the characteristics of the destinations, the experiences they offer, and the user's budget.

[0459] Step 4:

[0460] The server searches for suitable transportation and accommodation for the generated candidate locations and activities. During this process, it secures the latest pricing and availability information from reliable providers to create the most efficient plan.

[0461] Step 5:

[0462] The device presents the user with a generated travel plan. Here, the available travel destinations, transportation options, and accommodations are displayed in detail. The user can select items from the presented plan and customize it as needed.

[0463] Step 6:

[0464] The user reviews their customized plan through their device, and if they are satisfied with the travel details, they confirm the booking. Once the booking is confirmed, the device verifies with the server and prompts the user to enter payment information.

[0465] Step 7:

[0466] After the server approves the payment, it prepares the final itinerary and related documents and sends them to the user's device. This completes the user's travel planning process.

[0467] (Example 1)

[0468] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0469] Modern travel planning requires considerable time and effort to select destinations and activities that meet individual preferences from a vast amount of information. Furthermore, real-time information updates are difficult, making it challenging to create a plan that perfectly matches budget and preferences. Additionally, the lack of personalized suggestions based on past travel history makes creating highly satisfying travel plans a significant challenge.

[0470] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0471] In this invention, the server includes means for receiving travel preference information from a user and analyzing the preference information using natural language processing to identify the user's preferences; means for generating multiple travel destination candidates and activities using a generative model based on the generated preference data; and means for acquiring and searching real-time information on transportation and accommodation for the multiple candidates. This enables users to efficiently obtain personalized travel plans and create highly satisfying travel plans while saving time and effort.

[0472] A "user" refers to an individual or group that uses the system to plan a trip.

[0473] "Desired information" refers to the user's requests regarding their trip, such as destination, budget, duration, activities, and food preferences.

[0474] "Natural language processing" is a technology that enables computers to understand, interpret, and generate natural language used by humans.

[0475] "Preference data" refers to information about a user's preferences and tendencies, obtained based on their past behavior and choices.

[0476] A "generative model" is a machine learning model used to analyze data and create new information or options.

[0477] "Travel destination candidates" refers to multiple travel destinations suggested by the system based on the user's preferences and conditions.

[0478] An "activity" refers to a specific action or event that a traveler wants to experience during their trip.

[0479] "Means of transportation" refers to the modes of transport and routes that users can choose to reach their destination when traveling.

[0480] "Accommodation facilities" refer to facilities such as hotels, inns, and guesthouses where travelers stay at their destination.

[0481] "Real-time" means processing data instantly and providing the latest information.

[0482] To implement this invention, it is necessary to configure software that uses an information processing system to assist users in planning their travels. This system uses a server and a terminal, which are the main components, to effectively analyze the user's input information and generate personalized travel plans.

[0483] Users input their desired travel information using their devices. These devices include smartphones and computers, and the interface is designed for ease of use. Users can input their desired information, such as destination, budget, duration, desired activities, and dietary preferences, in natural language. This allows for intuitive and rapid data entry.

[0484] The server receives the input request information and performs analysis using natural language processing technology. This analysis utilizes advanced text analysis libraries such as "spaCy" to identify user preferences and conditions. The analyzed data is combined with the user's past behavioral history and stored as preference data.

[0485] Based on preference data, the server uses a generative AI model to create optimal travel destinations and activities for the user. This generative model utilizes machine learning techniques. A specific example is a generative model using Python, which generates personalized travel plans.

[0486] Next, the server retrieves real-time information on transportation and accommodation based on recent data. This uses APIs from travel information services. For example, the "Amadeus API" is available to retrieve transportation information, providing accurate fares and seat availability.

[0487] Users can review the travel itinerary presented through their device and customize it to their preferences. For example, they can change specific activities within the proposed travel destination or modify accommodations directly on their device. The customized content is then sent to the server and set as the final travel plan.

[0488] In the final stage, the user confirms their travel plan and completes the booking process on their device. Once payment is complete, the server sends the user a detailed travel plan and necessary travel information.

[0489] As a concrete example, a user can enter a prompt message such as, "Please suggest the best travel plan based on the following conditions: destination is a city for spring sightseeing, budget is under 150,000 yen, duration is 3 days, desired activities are cultural experiences, and preferred cuisine is French." Based on this information, the system will suggest a travel plan suitable for the user.

[0490] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0491] Step 1:

[0492] Users input their travel preferences through a terminal. This includes specific conditions such as potential destinations, budget, travel duration, desired activities, and dietary preferences. The entered preferences are transmitted from the terminal to the server as digital data. The terminal checks the format of the entered data and ensures data accuracy by requesting confirmation from the user if there are any ambiguities.

[0493] Step 2:

[0494] The server receives the desired information sent from the terminal and analyzes it using advanced natural language processing techniques. This analysis includes using a text analysis library to identify keywords and contexts related to the user's preferences and travel objectives. The preference data generated by the analysis is stored as the user's profile and used for further processing by a generative AI model.

[0495] Step 3:

[0496] The server uses a generative AI model to generate multiple travel destination options and activities based on preference data obtained through analysis. The generative model combines preference data with previously collected pattern data to construct a travel plan that matches the user's wishes. The generated plan is saved in digital format and prepared for use in the next step.

[0497] Step 4:

[0498] The server collects real-time information on transportation and accommodation based on the generated travel destinations and activities. This process uses configured APIs to interact with external databases and retrieve the latest fares and availability. The retrieved information is then integrated into a plan best suited to the user.

[0499] Step 5:

[0500] The server sends the integrated travel plan to the device and presents it to the user. The user can review the travel plan received on the device and customize activities, accommodations, and budget as needed. The customized information is sent from the device to the server, which re-parses and updates it.

[0501] Step 6:

[0502] The user confirms their final, satisfactory travel plan and completes the booking process on their device. The device provides an interface for entering payment data, such as credit card information, and securely transmits it to the server. The server verifies the payment and confirms all data.

[0503] Step 7:

[0504] The server transmits confirmed travel information to the terminal, providing the user with detailed itinerary and booking confirmations. This allows the user to complete all travel arrangements and proceed with their plans smoothly.

[0505] (Application Example 1)

[0506] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0507] When planning a trip, travelers face numerous challenges, including the difficulty of choosing a suitable destination from many options and the confusion and hassle involved in comparing transportation and accommodations. A particular challenge is the lack of visual information, which makes it difficult to concretely imagine the atmosphere and activities of a destination, resulting in anxiety and indecision.

[0508] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0509] In this invention, the server includes means for receiving travel preference information from a user and analyzing said preference information to identify the user's preferences; means for generating multiple travel destinations and activities based on the user's preferences; and means for presenting the travel plan in three dimensions using visual equipment, allowing the user to experience it interactively. This enables the user to concretely experience the travel plan visually and make decisions about the plan with greater confidence.

[0510] A "user" is an individual who uses this system when planning a trip and inputs their wishes and preferences.

[0511] "Travel preference information" refers to information entered by the user, including potential travel destinations, budget, travel duration, desired activities, and food preferences.

[0512] "Preferences" refer to the likes and interests that can be inferred from a user's past behavior and choices.

[0513] A "travel destination" refers to a geographical location or region that the user wishes to visit.

[0514] "Activities" refer to events, tours, and other activities that can be experienced at a travel destination.

[0515] "Transportation" refers to the means of getting to a travel destination, and includes airplanes, trains, buses, etc.

[0516] "Accommodation" refers to hotels, inns, or other lodging facilities where travelers stay during their trip.

[0517] "Visual devices" are devices that allow users to visually receive digital information, and include smart glasses and head-mounted displays.

[0518] "Presenting in three dimensions" means displaying information to users in a three-dimensional and spatial manner through visual devices.

[0519] The system for realizing this invention consists of a terminal where the user inputs travel preferences, a server that analyzes the information and generates a travel plan, and a visual device that visually displays the generated plan. Specifically, it is implemented in the following stages.

[0520] Users enter their travel preferences using devices such as smartphones or tablets. This includes desired destinations, budget, travel duration, preferred activities, and dietary preferences. This information, transmitted from the device, is received by the server.

[0521] The server analyzes the received data using advanced natural language processing (NLP) techniques. Specifically, Google Cloud Natural Language is used. Based on the analysis, the user's preferences are identified, and personalized travel destinations and activities are suggested based on their past travel experiences.

[0522] This proposed travel plan will acquire the latest information on transportation and accommodations and present it to the user in three dimensions via visual devices. Specifically, smart glasses such as Microsoft HoloLens will be used, and three-dimensional visual information will be generated using 3D modeling tools such as Unity.

[0523] Users can review and customize travel details based on visual information. Through this process, an optimized travel plan is created for the user. By experiencing the travel plan visually, users gain a realistic image of the trip and become more confident in their final decision.

[0524] An example of a prompt message is: "Analyze the user's preference data to generate a visually appealing and personalized travel plan, providing interactive information that can be experienced through smart glasses. For example, display potential beach destinations in Okinawa in 3D and suggest additional activities."

[0525] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0526] Step 1:

[0527] The user enters their travel preferences via their device. They specify their desired destination, budget, travel duration, desired activities, and dietary preferences. The entered data is then transmitted from the device to the server.

[0528] Step 2:

[0529] The server analyzes the requested information it receives. First, it uses natural language processing (NLP) techniques to convert user input into structured data. Google Cloud Natural Language is used for this process. The output of the analysis identifies the user's preferences.

[0530] Step 3:

[0531] The server generates suitable travel destinations and activities based on the user's preferences. It refers to past travel history and acquired preference information to generate a list of travel destinations and activities. This information is then presented to the user in a personalized manner.

[0532] Step 4:

[0533] The server searches for transportation and accommodation options related to potential travel destinations and activities. It uses travel information APIs (e.g., Skyscanner API, Booking.com API) to obtain real-time pricing information. The retrieved data is then ready to be presented to the user as up-to-date transportation and accommodation options.

[0534] Step 5:

[0535] The server-generated travel plan is presented to the user in three dimensions through visual devices. Using smart glasses such as Microsoft HoloLens, the 3D model created with Unity is displayed. This allows the user to visually experience the atmosphere of the trip in a concrete way.

[0536] Step 6:

[0537] Users customize their travel plans based on visual information. They add activities and adjust their travel itinerary through the smart glasses interface. This customization is then sent back to the server, and the final travel plan is confirmed.

[0538] Step 7:

[0539] The server finalizes the travel plan and sends detailed related information to the user. The finalized plan includes all booking information and schedules and is transferred to the user's device. The user then uses this information to complete the final travel arrangements.

[0540] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0541] This invention relates to a system for providing more personalized travel plans to users when they are planning a trip, by taking into account their emotions at the time. This system has a function that uses an emotion engine to recognize the user's emotions and adjust the travel plan based on those emotions.

[0542] The user first enters their travel preferences through the device. During this input process, their current emotional state is recorded by selecting sentences or options that express their feelings. If the device is equipped with a camera and microphone, it is also possible to analyze emotions in real time from facial expressions and tone of voice.

[0543] The server processes the received information, and the emotion engine analyzes the user's emotions. Specifically, it uses natural language processing and image analysis to read emotions from text, audio, and images. Based on this analysis, it selects travel destinations and activities that match the user's emotions. For example, a user who wants to relax might be suggested a quiet resort and spa activities.

[0544] Next, the server uses the sentiment analysis results and user preferences to search for the most suitable transportation and accommodation options. This search is performed by referencing real-time data and providing information on rates and availability.

[0545] The server then presents the user with a travel plan tailored to their emotions via the terminal. The user can review and customize the plan, particularly by choosing the option that best suits their feelings from a variety of choices.

[0546] Finally, if the user is satisfied with the plan, they will confirm the booking and make the necessary payment through their device. The confirmed plan will then be sent to the user from the server as final itinerary information.

[0547] For example, if a user is planning a vacation while feeling tired from work, the server will emphasize options that prioritize relaxation and suggest suitable resorts and refreshing activities. This approach allows users to plan the perfect trip tailored to their mood at the time and enjoy a fulfilling experience.

[0548] This system aims to improve user satisfaction in travel planning and provide a more comfortable and fulfilling experience by utilizing emotion analysis technology.

[0549] The following describes the processing flow.

[0550] Step 1:

[0551] The user enters their travel preferences via their device. During this process, options and comments are displayed in a question-and-answer format to understand the user's emotions. If the device has a camera and microphone, facial recognition and voice tone analysis are used to further capture the user's emotions.

[0552] Step 2:

[0553] The server receives user input information and emotion data. The server's emotion engine analyzes the user's emotional state using natural language processing and image and voice analysis techniques. Emotions are then labeled (e.g., want to relax, want adventure).

[0554] Step 3:

[0555] Based on analyzed emotions and preferences, the server generates suitable travel destinations and activities for the user. For example, it suggests beach resorts and hot springs for users who want to relax, and mountain trekking and city exploration for adventure-oriented users.

[0556] Step 4:

[0557] The server searches for transportation and accommodation based on the generated candidate locations and activities. This search involves referencing real-time data to obtain results that include the best rates and availability information.

[0558] Step 5:

[0559] The device presents the user with a travel plan based on sentiment analysis results. The user can review this plan and further customize individual items. For example, the user can adjust the start time and dates of activities.

[0560] Step 6:

[0561] The user reviews the final plan on their device, and if they are satisfied, they confirm the booking. They are then prompted to enter payment information, and the user completes the necessary payment for the booking.

[0562] Step 7:

[0563] The server receives the booking confirmation and prepares the final itinerary and related documents. This information is then delivered to the user via their terminal, completing the travel plan.

[0564] This process is designed to effectively suggest travel plans that are adapted to the user's emotional state, thereby improving overall satisfaction with the travel experience.

[0565] (Example 2)

[0566] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0567] When planning a trip, simply offering suggestions based on preferences is insufficient to maximize user satisfaction. It is necessary to propose personalized travel plans that take into account the user's emotions at the time. Furthermore, selecting the optimal transportation and accommodation options based on real-time information is crucial. Combining past travel history with emotions is expected to lead to even more accurate planning.

[0568] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0569] In this invention, the server includes means for receiving and analyzing travel preference information and emotional data from a user to identify the user's preferences and emotions; means for generating a plurality of potential travel destinations and activities based on the user's preferences and emotions; and means for searching for transportation methods and accommodations for the plurality of potential travel destinations and activities. This makes it possible to propose a more satisfying travel plan that comprehensively takes into account the user's preferences and emotions.

[0570] "Users" refer to people who use this system to plan their trips.

[0571] "Travel preference information" refers to information such as the user's desired destination, itinerary, budget, and purpose of travel when planning a trip.

[0572] "Emotional data" refers to data that indicates a user's emotional state, and is extracted from text, audio, images, and other sources.

[0573] "Preferences" refers to information about a user's interests and preferences.

[0574] "Travel destinations" refers to multiple travel destinations suggested to the user.

[0575] "Activities" refer to recreational activities, events, and other activities that users can participate in during their trip.

[0576] "Method of transportation" refers to the means of transport during a trip.

[0577] "Accommodation facilities" refer to facilities where travelers stay during their trip.

[0578] "Past travel history" refers to records of trips the user has taken in the past.

[0579] "Acquiring in real time" refers to the process of obtaining the latest information online at the present time.

[0580] This invention is a system that takes into account the user's emotions at the time when planning a trip, providing a more personalized plan. Specifically, it uses an emotion engine to analyze the user's emotions and optimizes the travel plan based on the results. The main elements of this system are the terminal used by the user, a server that analyzes the data, and an algorithm that creates the plan based on the analysis results.

[0581] The user first enters information about their travel plans through their device. This input includes basic information such as destination, dates, and budget, as well as questions that reflect their emotions. If the device has a camera and microphone, these can be used to record the user's facial expressions and tone of voice in real time and send this emotion data to the server.

[0582] The server analyzes the user's emotions using natural language processing and image analysis technologies based on the received information. A software called an emotion engine plays a crucial role in this analysis, extracting emotional states from text, audio, and images and representing them as numerical values ​​or categories. This emotion data is combined with the user's preferences and past travel history to generate travel plans.

[0583] The algorithm incorporates analyzed emotional data to generate multiple travel destinations and activities that align with the user's preferences. For example, if it determines that the user is seeking "relaxation," it can suggest a plan that includes a quiet, nature-rich resort or a spa experience.

[0584] Specific examples of prompts from this system include "Suggest relaxing travel destinations for a user who is tired from work" and "List the most suitable activities for the user based on their current mood."

[0585] In this way, the system aims to provide a more satisfying travel plan by comprehensively considering the user's preferences and emotions. As a result, users can receive the optimal travel plan tailored to their mood at the time, allowing them to have a fulfilling experience.

[0586] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0587] Step 1:

[0588] Users use their devices to answer questions that reflect basic travel information and their emotions. This process includes inputting travel destination, dates, budget, and other details. Additionally, if the device has a camera and microphone, the user's facial expressions and voice tone are recorded. This input data is sent to a server as emotional data.

[0589] Step 2:

[0590] The server analyzes travel preference information and emotional data received from the terminal. Text information received as input is analyzed using natural language processing technology, voice data is processed by voice analysis software, and facial expression data is analyzed using image analysis technology. As a result of these analyses, numerical or categorical data indicating the user's emotional state is output.

[0591] Step 3:

[0592] The server generates multiple travel destinations and activities based on analyzed sentiment data and user preference information. A generative AI model is used, receiving prompts such as "Suggest the best travel destination based on the user's sentiment." This generates an appropriate travel plan. This output allows for suggestions such as quiet resorts and spas for users seeking relaxation.

[0593] Step 4:

[0594] The server searches for transportation and accommodation options for suggested travel destinations and activities by referencing a real-time updated database. The search query includes detailed criteria associated with the travel plan, and checks for prices and availability. The output provides detailed information such as costs and available accommodations.

[0595] Step 5:

[0596] The server sends an optimized travel plan to the user's device and makes suggestions. The user reviews the plan details on their device and customizes it as needed, such as changing activities or selecting transportation. The customized information is then sent back to the server.

[0597] Step 6:

[0598] If the user is satisfied with the travel plan, they will confirm the booking and make payment via their device. The confirmed travel plan information is verified by the server and sent to the user as the final itinerary. As output, the user will receive the final confirmed travel schedule.

[0599] (Application Example 2)

[0600] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0601] In travel planning, there is a challenge in providing personalized plans that take into account the emotional state of the user. Furthermore, in the retail experience, there is a lack of effective product and service suggestions that consider the user's emotions. Therefore, there is a need to provide the optimal travel and consumption experience for users and improve their satisfaction.

[0602] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0603] In this invention, the server includes means for receiving travel information from a user and analyzing the information to identify the user's personal preferences; means for generating multiple destinations and activity content based on the user's preferences; and means for analyzing the user's emotions in real time to select and present appropriate products and services in order to improve the consumer experience at physical stores. This makes it possible to provide personalized travel plans that take into account the user's emotional state and preferences, and to improve the optimal consumer experience at physical stores.

[0604] A "user" is an individual who uses this system to customize their travel plans and consumption experiences.

[0605] "Travel information" refers to detailed information about the trip desired by the user, including destination, duration, budget, preferences, etc.

[0606] "Preferences" refers to information that indicates a user's personal preferences and priorities regarding travel.

[0607] "Destination" refers to the geographical location of the place a user plans to visit when planning a trip.

[0608] "Activities" refer to events and activities that users participate in during their travels or while inside physical stores, and include sightseeing, leisure, shopping, and more.

[0609] "Means of transportation" refers to the means of transport used by users to travel between destinations, and includes airplanes, trains, buses, etc.

[0610] "Accommodation facilities" refer to places where travelers stay during their trip, and include hotels, hostels, and vacation rentals.

[0611] A "physical store" refers to a physical store where consumers actually visit to purchase or experience goods and services.

[0612] "Emotions" refers to the psychological state that users experience during the process of planning a trip or engaging in consumer activities.

[0613] "Products" refer to specific items or services provided to customers at physical stores.

[0614] "Service" refers to beneficial actions or support other than the goods provided within a physical store, and includes customer service, advice, and promotions.

[0615] The system that realizes this application is configured as a program that provides personalized experiences through user travel information and sentiment analysis at physical stores. The server receives travel information from users, analyzes it to identify user preferences, generates destinations and activities based on the analyzed information, and further analyzes sentiment at physical stores in real time to suggest appropriate products and services.

[0616] The terminal uses devices such as smartphones and smart glasses to acquire the user's facial expressions and voice data and send it to the server. Based on this data, the server utilizes emotion analysis technologies such as Google Cloud Vision API and Amazon Rekognition to identify the user's emotional state in real time. Based on these results, it can suggest products and services that are best suited to the user. For example, if the user is feeling stressed, the application will suggest products that help them relax. This suggestion is notified to the user in real time via their smartphone or smart glasses.

[0617] As a concrete example, when a user visits a physical store during a holiday trip, the device can analyze the user's facial expressions and suggest a relaxing aromatherapy candle. An example of a prompt to the generating AI model in this case would be, "Analyze the user's emotions from their facial expressions and voice, and suggest the most suitable products and services in real time." In this way, real-time emotion analysis and product suggestions can significantly improve the user experience.

[0618] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0619] Step 1:

[0620] The terminal receives travel preference information from the user as input. This includes information such as destination, duration, budget, and preferences. This information is temporarily stored on the terminal before being sent to the server.

[0621] Step 2:

[0622] The server receives travel information from the terminal as input and performs data analysis to identify the user's preferences. This analysis includes using natural language processing techniques to extract keywords related to travel preferences from the text. The output is data that identifies the user's preferences.

[0623] Step 3:

[0624] The server generates multiple destinations and activities for the user based on identified preferences. This involves using a trained model to select the best suggestions from a database of past travel history. The output is a list of recommended destinations and activities for the user.

[0625] Step 4:

[0626] The device captures the user's facial expressions and voice when they visit a physical store. This serves as input, and real-time capture is performed on the device. This data is then sent to a server for emotion analysis.

[0627] Step 5:

[0628] The server receives real-time facial and audio data sent from the terminal as input and performs emotion analysis using the Google Cloud Vision API and Amazon Rekognition. As a result of the analysis, data identifying the user's current emotional state is output.

[0629] Step 6:

[0630] The server uses a generative AI model that selects products and services suitable for the user based on the results of emotion analysis. This design uses the prompt "Analyze the user's emotions from their facial expressions and voice, and suggest the most suitable products and services in real time" as the prompt for the generative model, resulting in the output of a list of appropriate products and services.

[0631] Step 7:

[0632] The device receives information about recommended products and services from the server and presents it to the user. The user can view the recommended products and services through the device. This information presentation allows the user to have a consumption experience that is tailored to their emotional state.

[0633] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0634] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0635] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0636] [Fourth Embodiment]

[0637] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0638] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0639] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0640] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0641] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0642] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0643] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0644] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0645] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0646] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0647] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0648] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0649] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0650] This invention relates to a system that reduces the time and effort users spend planning their trips. The system primarily includes user input of desired information, analysis of preferences, generation and presentation of travel plans, searching for transportation and accommodations, and finally confirming the plan and transmitting the information.

[0651] First, the user enters their travel preferences via their device. This includes potential destinations, budget, travel duration, desired activities, and specific food preferences. The server receives this information and analyzes it using advanced natural language processing technology. The server identifies the user's preferences and selects suitable travel destinations and activities. This analysis takes into account the user's past travel history to provide more personalized suggestions.

[0652] Next, the server searches for suitable transportation and accommodation based on the collected information. Here, it obtains real-time pricing and availability information from reliable partners. This allows users to have detailed options based on the latest data.

[0653] The server then presents the generated travel plan to the user via their terminal. The user reviews this plan and customizes it based on their preferences and requirements. The customized plan is then analyzed again by the system and finalized as the travel plan.

[0654] Finally, if the user is satisfied with the plan, they complete the booking on their device and enter their payment information. Once this is done, the server sends the user a detailed itinerary and related information, and the travel plan is complete.

[0655] For example, if a user wants to take a summer beach trip, they enter this into their device. The server analyzes the request and suggests destinations such as "Okinawa" or "Hawaii," and then presents the best flights and hotels within their budget. The user reviews this and makes customizations, such as adding "marine activities" and "local cuisine" in Okinawa. As a result, all details are finalized based on a satisfactory plan and sent to the user.

[0656] In this way, the present invention efficiently generates optimal travel plans for users and provides a stress-free travel planning experience.

[0657] The following describes the processing flow.

[0658] Step 1:

[0659] The user uses a device to enter their travel preferences, including potential destinations, budget, travel duration, desired activities, and food preferences. Once the user has finished entering the information, the device sends it to the server.

[0660] Step 2:

[0661] The server analyzes the user's preferences received. This uses advanced natural language processing technology to quickly understand the user's tastes and desires. Furthermore, the server also refers to the user's registration information and past travel history to identify their preferences.

[0662] Step 3:

[0663] Based on the analysis results, the server generates travel destinations and activities best suited to the user's preferences. This process involves creating options that consider the characteristics of the destinations, the experiences they offer, and the user's budget.

[0664] Step 4:

[0665] The server searches for suitable transportation and accommodation for the generated candidate locations and activities. During this process, it secures the latest pricing and availability information from reliable providers to create the most efficient plan.

[0666] Step 5:

[0667] The device presents the user with a generated travel plan. Here, the available travel destinations, transportation options, and accommodations are displayed in detail. The user can select items from the presented plan and customize it as needed.

[0668] Step 6:

[0669] The user reviews their customized plan through their device, and if they are satisfied with the travel details, they confirm the booking. Once the booking is confirmed, the device verifies with the server and prompts the user to enter payment information.

[0670] Step 7:

[0671] After the server approves the payment, it prepares the final itinerary and related documents and sends them to the user's device. This completes the user's travel planning process.

[0672] (Example 1)

[0673] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0674] Modern travel planning requires considerable time and effort to select destinations and activities that meet individual preferences from a vast amount of information. Furthermore, real-time information updates are difficult, making it challenging to create a plan that perfectly matches budget and preferences. Additionally, the lack of personalized suggestions based on past travel history makes creating highly satisfying travel plans a significant challenge.

[0675] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0676] In this invention, the server includes means for receiving travel preference information from a user and analyzing the preference information using natural language processing to identify the user's preferences; means for generating multiple travel destination candidates and activities using a generative model based on the generated preference data; and means for acquiring and searching real-time information on transportation and accommodation for the multiple candidates. This enables users to efficiently obtain personalized travel plans and create highly satisfying travel plans while saving time and effort.

[0677] A "user" refers to an individual or group that uses the system to plan a trip.

[0678] "Desired information" refers to the user's requests regarding their trip, such as destination, budget, duration, activities, and food preferences.

[0679] "Natural language processing" is a technology that enables computers to understand, interpret, and generate natural language used by humans.

[0680] "Preference data" refers to information about a user's preferences and tendencies, obtained based on their past behavior and choices.

[0681] A "generative model" is a machine learning model used to analyze data and create new information or options.

[0682] "Travel destination candidates" refers to multiple travel destinations suggested by the system based on the user's preferences and conditions.

[0683] An "activity" refers to a specific action or event that a traveler wants to experience during their trip.

[0684] "Means of transportation" refers to the modes of transport and routes that users can choose to reach their destination when traveling.

[0685] "Accommodation facilities" refer to facilities such as hotels, inns, and guesthouses where travelers stay at their destination.

[0686] "Real-time" means processing data instantly and providing the latest information.

[0687] To implement this invention, it is necessary to configure software that uses an information processing system to assist users in planning their travels. This system uses a server and a terminal, which are the main components, to effectively analyze the user's input information and generate personalized travel plans.

[0688] Users input their desired travel information using their devices. These devices include smartphones and computers, and the interface is designed for ease of use. Users can input their desired information, such as destination, budget, duration, desired activities, and dietary preferences, in natural language. This allows for intuitive and rapid data entry.

[0689] The server receives the input request information and performs analysis using natural language processing technology. This analysis utilizes advanced text analysis libraries such as "spaCy" to identify user preferences and conditions. The analyzed data is combined with the user's past behavioral history and stored as preference data.

[0690] Based on preference data, the server uses a generative AI model to create optimal travel destinations and activities for the user. This generative model utilizes machine learning techniques. A specific example is a generative model using Python, which generates personalized travel plans.

[0691] Next, the server retrieves real-time information on transportation and accommodation based on recent data. This uses APIs from travel information services. For example, the "Amadeus API" is available to retrieve transportation information, providing accurate fares and seat availability.

[0692] Users can review the travel itinerary presented through their device and customize it to their preferences. For example, they can change specific activities within the proposed travel destination or modify accommodations directly on their device. The customized content is then sent to the server and set as the final travel plan.

[0693] In the final stage, the user confirms their travel plan and completes the booking process on their device. Once payment is complete, the server sends the user a detailed travel plan and necessary travel information.

[0694] As a concrete example, a user can enter a prompt message such as, "Please suggest the best travel plan based on the following conditions: destination is a city for spring sightseeing, budget is under 150,000 yen, duration is 3 days, desired activities are cultural experiences, and preferred cuisine is French." Based on this information, the system will suggest a travel plan suitable for the user.

[0695] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0696] Step 1:

[0697] Users input their travel preferences through a terminal. This includes specific conditions such as potential destinations, budget, travel duration, desired activities, and dietary preferences. The entered preferences are transmitted from the terminal to the server as digital data. The terminal checks the format of the entered data and ensures data accuracy by requesting confirmation from the user if there are any ambiguities.

[0698] Step 2:

[0699] The server receives the desired information sent from the terminal and analyzes it using advanced natural language processing techniques. This analysis includes using a text analysis library to identify keywords and contexts related to the user's preferences and travel objectives. The preference data generated by the analysis is stored as the user's profile and used for further processing by a generative AI model.

[0700] Step 3:

[0701] The server uses a generative AI model to generate multiple travel destination options and activities based on preference data obtained through analysis. The generative model combines preference data with previously collected pattern data to construct a travel plan that matches the user's wishes. The generated plan is saved in digital format and prepared for use in the next step.

[0702] Step 4:

[0703] The server collects real-time information on transportation and accommodation based on the generated travel destinations and activities. This process uses configured APIs to interact with external databases and retrieve the latest fares and availability. The retrieved information is then integrated into a plan best suited to the user.

[0704] Step 5:

[0705] The server sends the integrated travel plan to the device and presents it to the user. The user can review the travel plan received on the device and customize activities, accommodations, and budget as needed. The customized information is sent from the device to the server, which re-parses and updates it.

[0706] Step 6:

[0707] The user confirms their final, satisfactory travel plan and completes the booking process on their device. The device provides an interface for entering payment data, such as credit card information, and securely transmits it to the server. The server verifies the payment and confirms all data.

[0708] Step 7:

[0709] The server transmits confirmed travel information to the terminal, providing the user with detailed itinerary and booking confirmations. This allows the user to complete all travel arrangements and proceed with their plans smoothly.

[0710] (Application Example 1)

[0711] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0712] When planning a trip, travelers face numerous challenges, including the difficulty of choosing a suitable destination from many options and the confusion and hassle involved in comparing transportation and accommodations. A particular challenge is the lack of visual information, which makes it difficult to concretely imagine the atmosphere and activities of a destination, resulting in anxiety and indecision.

[0713] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0714] In this invention, the server includes means for receiving travel preference information from a user and analyzing said preference information to identify the user's preferences; means for generating multiple travel destinations and activities based on the user's preferences; and means for presenting the travel plan in three dimensions using visual equipment, allowing the user to experience it interactively. This enables the user to concretely experience the travel plan visually and make decisions about the plan with greater confidence.

[0715] A "user" is an individual who uses this system when planning a trip and inputs their wishes and preferences.

[0716] "Travel preference information" refers to information entered by the user, including potential travel destinations, budget, travel duration, desired activities, and food preferences.

[0717] "Preferences" refer to the likes and interests that can be inferred from a user's past behavior and choices.

[0718] A "travel destination" refers to a geographical location or region that the user wishes to visit.

[0719] "Activities" refer to events, tours, and other activities that can be experienced at a travel destination.

[0720] "Transportation" refers to the means of getting to a travel destination, and includes airplanes, trains, buses, etc.

[0721] "Accommodation" refers to hotels, inns, or other lodging facilities where travelers stay during their trip.

[0722] "Visual devices" are devices that allow users to visually receive digital information, and include smart glasses and head-mounted displays.

[0723] "Presenting in three dimensions" means displaying information to users in a three-dimensional and spatial manner through visual devices.

[0724] The system for realizing this invention consists of a terminal where the user inputs travel preferences, a server that analyzes the information and generates a travel plan, and a visual device that visually displays the generated plan. Specifically, it is implemented in the following stages.

[0725] Users enter their travel preferences using devices such as smartphones or tablets. This includes desired destinations, budget, travel duration, preferred activities, and dietary preferences. This information, transmitted from the device, is received by the server.

[0726] The server analyzes the received data using advanced natural language processing (NLP) techniques. Specifically, Google Cloud Natural Language is used. Based on the analysis, the user's preferences are identified, and personalized travel destinations and activities are suggested based on their past travel experiences.

[0727] This proposed travel plan will acquire the latest information on transportation and accommodations and present it to the user in three dimensions via visual devices. Specifically, smart glasses such as Microsoft HoloLens will be used, and three-dimensional visual information will be generated using 3D modeling tools such as Unity.

[0728] Users can review and customize travel details based on visual information. Through this process, an optimized travel plan is created for the user. By experiencing the travel plan visually, users gain a realistic image of the trip and become more confident in their final decision.

[0729] An example of a prompt message is: "Analyze the user's preference data to generate a visually appealing and personalized travel plan, providing interactive information that can be experienced through smart glasses. For example, display potential beach destinations in Okinawa in 3D and suggest additional activities."

[0730] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0731] Step 1:

[0732] The user enters their travel preferences via their device. They specify their desired destination, budget, travel duration, desired activities, and dietary preferences. The entered data is then transmitted from the device to the server.

[0733] Step 2:

[0734] The server analyzes the requested information it receives. First, it uses natural language processing (NLP) techniques to convert user input into structured data. Google Cloud Natural Language is used for this process. The output of the analysis identifies the user's preferences.

[0735] Step 3:

[0736] The server generates suitable travel destinations and activities based on the user's preferences. It refers to past travel history and acquired preference information to generate a list of travel destinations and activities. This information is then presented to the user in a personalized manner.

[0737] Step 4:

[0738] The server searches for transportation and accommodation options related to potential travel destinations and activities. It uses travel information APIs (e.g., Skyscanner API, Booking.com API) to obtain real-time pricing information. The retrieved data is then ready to be presented to the user as up-to-date transportation and accommodation options.

[0739] Step 5:

[0740] The server-generated travel plan is presented to the user in three dimensions through visual devices. Using smart glasses such as Microsoft HoloLens, the 3D model created with Unity is displayed. This allows the user to visually experience the atmosphere of the trip in a concrete way.

[0741] Step 6:

[0742] Users customize their travel plans based on visual information. They add activities and adjust their travel itinerary through the smart glasses interface. This customization is then sent back to the server, and the final travel plan is confirmed.

[0743] Step 7:

[0744] The server finalizes the travel plan and sends detailed related information to the user. The finalized plan includes all booking information and schedules and is transferred to the user's device. The user then uses this information to complete the final travel arrangements.

[0745] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0746] This invention relates to a system for providing more personalized travel plans to users when they are planning a trip, by taking into account their emotions at the time. This system has a function that uses an emotion engine to recognize the user's emotions and adjust the travel plan based on those emotions.

[0747] The user first enters their travel preferences through the device. During this input process, their current emotional state is recorded by selecting sentences or options that express their feelings. If the device is equipped with a camera and microphone, it is also possible to analyze emotions in real time from facial expressions and tone of voice.

[0748] The server processes the received information, and the emotion engine analyzes the user's emotions. Specifically, it uses natural language processing and image analysis to read emotions from text, audio, and images. Based on this analysis, it selects travel destinations and activities that match the user's emotions. For example, a user who wants to relax might be suggested a quiet resort and spa activities.

[0749] Next, the server uses the sentiment analysis results and user preferences to search for the most suitable transportation and accommodation options. This search is performed by referencing real-time data and providing information on rates and availability.

[0750] The server then presents the user with a travel plan tailored to their emotions via the terminal. The user can review and customize the plan, particularly by choosing the option that best suits their feelings from a variety of choices.

[0751] Finally, if the user is satisfied with the plan, they will confirm the booking and make the necessary payment through their device. The confirmed plan will then be sent to the user from the server as final itinerary information.

[0752] For example, if a user is planning a vacation while feeling tired from work, the server will emphasize options that prioritize relaxation and suggest suitable resorts and refreshing activities. This approach allows users to plan the perfect trip tailored to their mood at the time and enjoy a fulfilling experience.

[0753] This system aims to improve user satisfaction in travel planning and provide a more comfortable and fulfilling experience by utilizing emotion analysis technology.

[0754] The following describes the processing flow.

[0755] Step 1:

[0756] The user enters their travel preferences via their device. During this process, options and comments are displayed in a question-and-answer format to understand the user's emotions. If the device has a camera and microphone, facial recognition and voice tone analysis are used to further capture the user's emotions.

[0757] Step 2:

[0758] The server receives user input information and emotion data. The server's emotion engine analyzes the user's emotional state using natural language processing and image and voice analysis techniques. Emotions are then labeled (e.g., want to relax, want adventure).

[0759] Step 3:

[0760] Based on analyzed emotions and preferences, the server generates suitable travel destinations and activities for the user. For example, it suggests beach resorts and hot springs for users who want to relax, and mountain trekking and city exploration for adventure-oriented users.

[0761] Step 4:

[0762] The server searches for transportation and accommodation based on the generated candidate locations and activities. This search involves referencing real-time data to obtain results that include the best rates and availability information.

[0763] Step 5:

[0764] The device presents the user with a travel plan based on sentiment analysis results. The user can review this plan and further customize individual items. For example, the user can adjust the start time and dates of activities.

[0765] Step 6:

[0766] The user reviews the final plan on their device, and if they are satisfied, they confirm the booking. They are then prompted to enter payment information, and the user completes the necessary payment for the booking.

[0767] Step 7:

[0768] The server receives the booking confirmation and prepares the final itinerary and related documents. This information is then delivered to the user via their terminal, completing the travel plan.

[0769] This process is designed to effectively suggest travel plans that are adapted to the user's emotional state, thereby improving overall satisfaction with the travel experience.

[0770] (Example 2)

[0771] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0772] When planning a trip, simply offering suggestions based on preferences is insufficient to maximize user satisfaction. It is necessary to propose personalized travel plans that take into account the user's emotions at the time. Furthermore, selecting the optimal transportation and accommodation options based on real-time information is crucial. Combining past travel history with emotions is expected to lead to even more accurate planning.

[0773] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0774] In this invention, the server includes means for receiving and analyzing travel preference information and emotional data from a user to identify the user's preferences and emotions; means for generating a plurality of potential travel destinations and activities based on the user's preferences and emotions; and means for searching for transportation methods and accommodations for the plurality of potential travel destinations and activities. This makes it possible to propose a more satisfying travel plan that comprehensively takes into account the user's preferences and emotions.

[0775] "Users" refer to people who use this system to plan their trips.

[0776] "Travel preference information" refers to information such as the user's desired destination, itinerary, budget, and purpose of travel when planning a trip.

[0777] "Emotional data" refers to data that indicates a user's emotional state, and is extracted from text, audio, images, and other sources.

[0778] "Preferences" refers to information about a user's interests and preferences.

[0779] "Travel destinations" refers to multiple travel destinations suggested to the user.

[0780] "Activities" refer to recreational activities, events, and other activities that users can participate in during their trip.

[0781] "Method of transportation" refers to the means of transport during a trip.

[0782] "Accommodation facilities" refer to facilities where travelers stay during their trip.

[0783] "Past travel history" refers to records of trips the user has taken in the past.

[0784] "Acquiring in real time" refers to the process of obtaining the latest information online at the present time.

[0785] This invention is a system that takes into account the user's emotions at the time when planning a trip, providing a more personalized plan. Specifically, it uses an emotion engine to analyze the user's emotions and optimizes the travel plan based on the results. The main elements of this system are the terminal used by the user, a server that analyzes the data, and an algorithm that creates the plan based on the analysis results.

[0786] The user first enters information about their travel plans through their device. This input includes basic information such as destination, dates, and budget, as well as questions that reflect their emotions. If the device has a camera and microphone, these can be used to record the user's facial expressions and tone of voice in real time and send this emotion data to the server.

[0787] The server analyzes the user's emotions using natural language processing and image analysis technologies based on the received information. A software called an emotion engine plays a crucial role in this analysis, extracting emotional states from text, audio, and images and representing them as numerical values ​​or categories. This emotion data is combined with the user's preferences and past travel history to generate travel plans.

[0788] The algorithm incorporates analyzed emotional data to generate multiple travel destinations and activities that align with the user's preferences. For example, if it determines that the user is seeking "relaxation," it can suggest a plan that includes a quiet, nature-rich resort or a spa experience.

[0789] Specific examples of prompts from this system include "Suggest relaxing travel destinations for a user who is tired from work" and "List the most suitable activities for the user based on their current mood."

[0790] In this way, the system aims to provide a more satisfying travel plan by comprehensively considering the user's preferences and emotions. As a result, users can receive the optimal travel plan tailored to their mood at the time, allowing them to have a fulfilling experience.

[0791] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0792] Step 1:

[0793] Users use their devices to answer questions that reflect basic travel information and their emotions. This process includes inputting travel destination, dates, budget, and other details. Additionally, if the device has a camera and microphone, the user's facial expressions and voice tone are recorded. This input data is sent to a server as emotional data.

[0794] Step 2:

[0795] The server analyzes travel preference information and emotional data received from the terminal. Text information received as input is analyzed using natural language processing technology, voice data is processed by voice analysis software, and facial expression data is analyzed using image analysis technology. As a result of these analyses, numerical or categorical data indicating the user's emotional state is output.

[0796] Step 3:

[0797] The server generates multiple travel destinations and activities based on analyzed sentiment data and user preference information. A generative AI model is used, receiving prompts such as "Suggest the best travel destination based on the user's sentiment." This generates an appropriate travel plan. This output allows for suggestions such as quiet resorts and spas for users seeking relaxation.

[0798] Step 4:

[0799] The server searches for transportation and accommodation options for suggested travel destinations and activities by referencing a real-time updated database. The search query includes detailed criteria associated with the travel plan, and checks for prices and availability. The output provides detailed information such as costs and available accommodations.

[0800] Step 5:

[0801] The server sends an optimized travel plan to the user's device and makes suggestions. The user reviews the plan details on their device and customizes it as needed, such as changing activities or selecting transportation. The customized information is then sent back to the server.

[0802] Step 6:

[0803] If the user is satisfied with the travel plan, they will confirm the booking and make payment via their device. The confirmed travel plan information is verified by the server and sent to the user as the final itinerary. As output, the user will receive the final confirmed travel schedule.

[0804] (Application Example 2)

[0805] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0806] In travel planning, there is a challenge in providing personalized plans that take into account the emotional state of the user. Furthermore, in the retail experience, there is a lack of effective product and service suggestions that consider the user's emotions. Therefore, there is a need to provide the optimal travel and consumption experience for users and improve their satisfaction.

[0807] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0808] In this invention, the server includes means for receiving travel information from a user and analyzing the information to identify the user's personal preferences; means for generating multiple destinations and activity content based on the user's preferences; and means for analyzing the user's emotions in real time to select and present appropriate products and services in order to improve the consumer experience at physical stores. This makes it possible to provide personalized travel plans that take into account the user's emotional state and preferences, and to improve the optimal consumer experience at physical stores.

[0809] A "user" is an individual who uses this system to customize their travel plans and consumption experiences.

[0810] "Travel information" refers to detailed information about the trip desired by the user, including destination, duration, budget, preferences, etc.

[0811] "Preferences" refers to information that indicates a user's personal preferences and priorities regarding travel.

[0812] "Destination" refers to the geographical location of the place a user plans to visit when planning a trip.

[0813] "Activities" refer to events and activities that users participate in during their travels or while inside physical stores, and include sightseeing, leisure, shopping, and more.

[0814] "Means of transportation" refers to the means of transport used by users to travel between destinations, and includes airplanes, trains, buses, etc.

[0815] "Accommodation facilities" refer to places where travelers stay during their trip, and include hotels, hostels, and vacation rentals.

[0816] A "physical store" refers to a physical store where consumers actually visit to purchase or experience goods and services.

[0817] "Emotions" refers to the psychological state that users experience during the process of planning a trip or engaging in consumer activities.

[0818] "Products" refer to specific items or services provided to customers at physical stores.

[0819] "Service" refers to beneficial actions or support other than the goods provided within a physical store, and includes customer service, advice, and promotions.

[0820] The system that realizes this application is configured as a program that provides personalized experiences through user travel information and sentiment analysis at physical stores. The server receives travel information from users, analyzes it to identify user preferences, generates destinations and activities based on the analyzed information, and further analyzes sentiment at physical stores in real time to suggest appropriate products and services.

[0821] The terminal uses devices such as smartphones and smart glasses to acquire the user's facial expressions and voice data and send it to the server. Based on this data, the server utilizes emotion analysis technologies such as Google Cloud Vision API and Amazon Rekognition to identify the user's emotional state in real time. Based on these results, it can suggest products and services that are best suited to the user. For example, if the user is feeling stressed, the application will suggest products that help them relax. This suggestion is notified to the user in real time via their smartphone or smart glasses.

[0822] As a concrete example, when a user visits a physical store during a holiday trip, the device can analyze the user's facial expressions and suggest a relaxing aromatherapy candle. An example of a prompt to the generating AI model in this case would be, "Analyze the user's emotions from their facial expressions and voice, and suggest the most suitable products and services in real time." In this way, real-time emotion analysis and product suggestions can significantly improve the user experience.

[0823] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0824] Step 1:

[0825] The terminal receives travel preference information from the user as input. This includes information such as destination, duration, budget, and preferences. This information is temporarily stored on the terminal before being sent to the server.

[0826] Step 2:

[0827] The server receives travel information from the terminal as input and performs data analysis to identify the user's preferences. This analysis includes using natural language processing techniques to extract keywords related to travel preferences from the text. The output is data that identifies the user's preferences.

[0828] Step 3:

[0829] The server generates multiple destinations and activities for the user based on identified preferences. This involves using a trained model to select the best suggestions from a database of past travel history. The output is a list of recommended destinations and activities for the user.

[0830] Step 4:

[0831] The device captures the user's facial expressions and voice when they visit a physical store. This serves as input, and real-time capture is performed on the device. This data is then sent to a server for emotion analysis.

[0832] Step 5:

[0833] The server receives real-time facial and audio data sent from the terminal as input and performs emotion analysis using the Google Cloud Vision API and Amazon Rekognition. As a result of the analysis, data identifying the user's current emotional state is output.

[0834] Step 6:

[0835] The server uses a generative AI model that selects products and services suitable for the user based on the results of emotion analysis. This design uses the prompt "Analyze the user's emotions from their facial expressions and voice, and suggest the most suitable products and services in real time" as the prompt for the generative model, resulting in the output of a list of appropriate products and services.

[0836] Step 7:

[0837] The device receives information about recommended products and services from the server and presents it to the user. The user can view the recommended products and services through the device. This information presentation allows the user to have a consumption experience that is tailored to their emotional state.

[0838] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0839] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0840] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0841] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0842] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0843] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0844] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0845] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0846] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0847] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0848] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0849] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0850] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0851] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0852] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0853] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0854] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0855] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0856] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0857] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0858] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0859] The following is further disclosed regarding the embodiments described above.

[0860] (Claim 1)

[0861] A means for receiving travel preference information from users and analyzing said preference information to identify the user's preferences,

[0862] A means for generating multiple travel destinations and activities based on the user's preferences,

[0863] A means for searching for transportation and accommodation for the aforementioned multiple travel destinations and activities,

[0864] A means of presenting generated travel plans to users and enabling them to select and customize them,

[0865] A means to finalize the travel plan and send related information,

[0866] A system that includes this.

[0867] (Claim 2)

[0868] The system according to claim 1, further comprising means for customizing a travel plan using the user's past travel history.

[0869] (Claim 3)

[0870] The system according to claim 1, further comprising means for obtaining real-time rates for transportation and accommodation.

[0871] "Example 1"

[0872] (Claim 1)

[0873] A means for receiving travel preference information from users and analyzing said preference information using natural language processing to identify the user's preferences,

[0874] A means for generating multiple travel destination candidates and activities using a generative model based on generated preference data,

[0875] A means for obtaining and searching for information on transportation and accommodation in real time for the aforementioned multiple candidates,

[0876] A means of displaying the generated travel details on a terminal, allowing the user to select and adjust them,

[0877] A means of sending the finalized travel details to the user and completing the travel procedures,

[0878] A system that includes this.

[0879] (Claim 2)

[0880] The system according to claim 1, further comprising means for analyzing the user's past behavioral history and personalizing the travel content.

[0881] (Claim 3)

[0882] The system according to claim 1, further comprising means for dynamically obtaining charges for the use of means of transportation and accommodation.

[0883] "Application Example 1"

[0884] (Claim 1)

[0885] A means for receiving travel preference information from users and analyzing said preference information to identify the user's preferences,

[0886] A means for generating multiple travel destinations and activities based on the user's preferences,

[0887] A means for searching for transportation and accommodation for the aforementioned multiple travel destinations and activities,

[0888] A means of presenting the generated travel plan to the user and enabling selection and adjustment,

[0889] A means to finalize the travel plan and send related information,

[0890] A means of presenting travel plans in three dimensions using visual devices, allowing users to experience them interactively,

[0891] A device that includes this.

[0892] (Claim 2)

[0893] The apparatus according to claim 1, further comprising means for adjusting a travel plan using the user's past travel history.

[0894] (Claim 3)

[0895] The apparatus according to claim 1, further comprising means for obtaining transportation and accommodation rates in real time.

[0896] "Example 2 of combining an emotion engine"

[0897] (Claim 1)

[0898] A means for receiving travel preference information from users, and analyzing said preference information and emotional data to identify the user's preferences and emotions,

[0899] A means for generating multiple travel destinations and activities based on the user's preferences and emotions,

[0900] A means for searching for transportation methods and accommodations for the aforementioned multiple travel destinations and activities,

[0901] A means of presenting the generated travel plan to the user and allowing them to select and modify it,

[0902] A means to finalize the travel plan and send related information,

[0903] A system that includes this.

[0904] (Claim 2)

[0905] The system according to claim 1, further comprising means for optimizing a travel plan using the user's past travel history and sentiment data.

[0906] (Claim 3)

[0907] The system according to claim 1, further comprising means for obtaining transportation methods and accommodation rates in real time.

[0908] "Application example 2 when combining with an emotional engine"

[0909] (Claim 1)

[0910] A means of receiving travel information from users and analyzing that information to identify the user's personal preferences,

[0911] A means for generating multiple destinations and activity content based on the user's preferences,

[0912] A means for searching for means of transportation and accommodation for the aforementioned multiple destinations and activities,

[0913] A means of presenting the generated plan to the user and allowing them to select and modify it,

[0914] To improve the consumer experience in physical stores, we need a means to analyze customer emotions in real time and select and present appropriate products and services.

[0915] A means to finalize the travel plan and send relevant information,

[0916] A system that includes this.

[0917] (Claim 2)

[0918] The system according to claim 1, further comprising means for modifying a travel plan using a user's historical travel records.

[0919] (Claim 3)

[0920] The system according to claim 1, further comprising means for obtaining real-time prices for transportation and accommodation. [Explanation of Symbols]

[0921] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for receiving travel preference information from users and analyzing said preference information to identify the user's preferences, A means for generating multiple travel destinations and activities based on the user's preferences, A means for searching for transportation and accommodation for the aforementioned multiple travel destinations and activities, A means of presenting generated travel plans to users and enabling them to select and customize them, A means to finalize the travel plan and send related information, A system that includes this.

2. The system according to claim 1, further comprising means for customizing a travel plan using the user's past travel history.

3. The system according to claim 1, further comprising means for obtaining real-time rates for transportation and accommodation.

Images