system

Abstract

Provide a system. 【Solution means】 Means for inputting traveler information, Means for generating a travel plan based on the traveler information, Means for providing a guiding avatar based on the generated travel plan, Means for obtaining external information and refining the travel plan, Means for providing the generated travel plan and guiding avatar information to the traveler, Means for receiving confirmation and modification of the plan from the traveler, Means for linking the finally determined travel plan to the reservation procedure, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the current situation where the number of foreign tourists visiting Japan is increasing year by year, there are problems such as a shortage of guides and a great deal of trouble in creating travel plans. Tour guides and tour planners are often overwhelmed with multiple reservations and inquiries, and are in a situation where they miss business opportunities due to schedule overlaps and complicated procedures. Therefore, there is a demand for efficiently and accurately providing a personalized travel experience to travelers.

Means for Solving the Problems

[0005] The present invention is a system that includes means for inputting traveler information, means for generating a travel plan based on the traveler information, means for providing a guide avatar based on the generated travel plan, means for acquiring external information to refine the travel plan, means for providing the traveler with information on the generated travel plan and the guide avatar, means for accepting confirmation and modification of the plan from the traveler, and means for linking the finalized travel plan to the booking procedure. This enables the provision of meticulous service to tourists visiting Japan, as well as the provision of efficient and wide-ranging tour experiences.

[0006] The term "traveler" refers to an individual who goes to a travel destination to engage in sightseeing or business activities.

[0007] A "travel plan" refers to an itinerary or schedule of activities that travelers prepare in advance to efficiently carry out their activities at their destination.

[0008] A "guide avatar" refers to a virtual guide character that uses digital technology to provide explanations about tourist destinations and various facilities.

[0009] "External information" refers to data on transportation, weather, events, etc., at the travel destination that is referenced when formulating travel plans.

[0010] "Booking procedures" refer to the series of processes in which travelers make advance arrangements for transportation, accommodation, sightseeing activities, and other things necessary to carry out their travel plans.

[0011] "Means of inputting information" refers to an interface that allows travelers to input their requests and conditions into a digital device.

[0012] "Methods of refinement" refer to the process of acquiring and incorporating additional information to make the initial travel plan more detailed and specific.

[0013] "Means for receiving confirmation and revisions" refers to a method for travelers to communicate their opinions and requests regarding the plan and for those opinions to be reflected in the plan.

[0014] "Generative means" refers to processes and technologies for automatically creating new information and plans based on input data. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention is a system for providing travelers with an efficient and personalized travel experience. The following describes the configurations for implementing this system.

[0037] First, the user enters travel information via their device, such as places they want to visit, activities they are interested in, travel duration, and budget. The device collects this information and sends it to the server.

[0038] Next, the server automatically generates a travel plan based on the information received from the user. This plan generation utilizes a generation AI implemented on the server, which constructs a plan tailored to the user's preferences based on the knowledge and experience of professional guides. The AI ​​can suggest the optimal sightseeing route and schedule according to the user's interests and budget.

[0039] Regarding the generated travel plan, the server retrieves detailed information on tourist destinations, events, transportation options, etc., through external digital services to further refine the plan. This makes it possible to provide travel plans that are always based on the latest and most accurate information.

[0040] Next, the server prepares a guide avatar based on the travel plan. This avatar is responsible for providing visual and audio guidance, conveying explanations of tourist attractions and cultural background information in real time. Users can view the generated travel plan and the avatar's guidance on their device.

[0041] For example, if a user wants to sightsee in Tokyo and requests a plan that includes visiting museums and dining locally, the server will provide a plan that includes museum opening hours, admission fees, and access information. It can also include a suggestion for a popular local restaurant for lunch. If the user has any requests for additions or modifications to this plan, they can do so through their device, and the server will readjust the plan accordingly.

[0042] Finally, once the user confirms their plan, the server integrates it with partnered external booking services to support travel arrangements. This allows users to easily complete their bookings and proceed with their trip smoothly.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The user enters their travel preferences via the device. Specifically, they enter information such as places they want to visit, activities they are interested in, length of stay, and budget. The device collects this data and sends it to the server.

[0046] Step 2:

[0047] The server receives information entered by the user and analyzes it to generate a travel plan. The AI ​​on the server creates a travel plan that meets the user's wishes, based on the knowledge and experience of professional guides. This AI suggests destinations, sightseeing routes, and schedules.

[0048] Step 3:

[0049] The server retrieves tourist destination information, event information, and transportation information from external digital services to improve the accuracy of travel plans. Based on this information, it modifies the travel plan to ensure that the plan is based on the latest data.

[0050] Step 4:

[0051] The server prepares a guide avatar based on the generated travel plan. The avatar has a script that explains detailed information and cultural background of the destinations, and is configured to guide the user visually or audibly.

[0052] Step 5:

[0053] The device displays the travel plan and avatar information received from the server to the user. The user reviews the presented plan, approves it if they are satisfied with the contents, and submits a change request if necessary.

[0054] Step 6:

[0055] If a user requests a change, the server receives the information and readjusts the plan. The server then generates a new travel plan, reflecting the changes, and provides feedback to the user.

[0056] Step 7:

[0057] The server confirms the travel plan that the user has finally approved. If reservations are required as part of the travel process, the server then connects the information to external booking services to assist with the booking arrangements. Through this process, the user completes their travel preparations.

[0058] (Example 1)

[0059] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0060] To provide travelers with effective and personalized travel experiences, it is necessary to efficiently collect and analyze a large amount of information and generate appropriate travel plans. However, conventional systems have problems with the accuracy of information and the flexibility of plans, and have been unable to fully meet the individual needs of travelers.

[0061] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0062] In this invention, the server includes means for analyzing traveler information and automatically generating a travel plan using a generative AI model; means for dynamically acquiring information from external sources and refining the travel plan; and means for presenting the generated travel plan and guidance information to the traveler and accepting modifications according to the traveler's requests. This makes it possible to provide the latest and most flexible travel plan optimized for each traveler's wishes and conditions.

[0063] A "data processing device" is an electronic device that has the function of collecting travelers' input information and transmitting it to a server.

[0064] A "generative AI model" is an artificial intelligence system used to automatically generate optimal travel plans based on traveler information.

[0065] A "guidance output device" is a medium or device for providing information visually and audibly based on a travel plan.

[0066] "External information sources" refer to databases and services that dynamically retrieve information, such as detailed information about tourist destinations and transportation information, in order to improve the accuracy of travel planning.

[0067] A "reservation management system" is a system for executing or managing reservation procedures based on a finalized travel plan.

[0068] "Traveler" refers to an individual or group that plans and executes a trip using this system.

[0069] This invention is a system for providing travelers with personalized travel experiences. Specific embodiments of the system are described below.

[0070] The user first uses a terminal to input travel information such as their desired destination, interesting sightseeing experiences, travel duration, and budget. The terminal organizes this input information and sends it to the server via a secure communication protocol. In this process, the terminal functions as a data processing device.

[0071] The server analyzes data received from the user and automatically generates a travel plan using a generative AI model. The generative AI model uses artificial intelligence technology to simulate the knowledge of a professional guide and build a customized plan tailored to the user's requests. In particular, the server can use prompts such as: "The user wants to visit art museums in Tokyo. Please suggest recommended museums and access information."

[0072] In the generated travel plan, the server dynamically retrieves the latest data on tourist destinations, event information, and transportation options from external sources to improve the accuracy of the plan. This ensures that users are always provided with accurate and up-to-date information.

[0073] Next, the server prepares an avatar as a guidance output device based on the travel plan. This avatar is designed to provide travelers with real-time explanations of tourist attractions and cultural background information through visual and auditory means. Users can view the generated travel plan and the guidance information provided by the avatar via their terminal.

[0074] For example, if a user enters "I want to visit art museums in Tokyo and enjoy local cuisine," the server will provide a detailed plan including museum opening hours, admission fees, and access information. Furthermore, it can also include popular local restaurants as lunch options.

[0075] Finally, once the user confirms their travel plan, the server, acting as a reservation management system, links the plan to affiliated external reservation services. This allows users to complete their travel reservations without cumbersome procedures, ensuring a smooth travel experience.

[0076] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0077] Step 1:

[0078] The user uses a terminal to enter their desired travel destination, activities of interest, duration, and budget. The terminal collects this information and converts it into a digital format. This input data is then sent to the server as foundational information necessary for the next processing step.

[0079] Step 2:

[0080] The server analyzes traveler information received from the terminal. The server utilizes a generative AI model to automatically generate travel plans that match the user's interests and budget. Specifically, the server generates prompt messages and passes them to the AI ​​model to construct a plan optimized for the user's needs. For example, using the prompt message, "The user wants to visit art museums in Tokyo. Please suggest a recommended route," the AI ​​generates a detailed sightseeing route. The output is the generated travel plan.

[0081] Step 3:

[0082] The server adds the latest data from external sources to the generated travel plan. The server retrieves opening hours, event information, and transportation information for tourist attractions and incorporates it into the travel plan. This results in a travel plan based on the most up-to-date details. The output is a refined and up-to-date travel plan.

[0083] Step 4:

[0084] The server prepares a guide avatar based on the travel plan. This avatar is configured to provide travelers with real-time explanations of tourist attractions and cultural background information through visual and audio information. This allows users to receive detailed travel guidance through their devices. The output is the avatar's guide information related to the travel plan.

[0085] Step 5:

[0086] Users view the generated travel plan and avatar-based guidance through their device. If a user wishes to modify the plan, they can send a request to the server via their device. The server then readjusts the travel plan based on this request. The input is the user's modification request, and the output is the updated travel plan.

[0087] Step 6:

[0088] Once a user confirms their travel plans, the server integrates them with a partnered booking management system. The server automatically processes the user's travel booking and performs the necessary procedures, allowing the user to easily complete their reservation. The output is confirmation of the confirmed travel booking.

[0089] (Application Example 1)

[0090] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0091] Traditional travel planning systems sometimes struggle to provide appropriate travel plans tailored to travelers' interests and budgets, and they have limited means for travelers to virtually experience tourist destinations before actually visiting. As a result, travelers may be unable to make informed decisions, leading to decreased satisfaction with their travel experience.

[0092] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0093] In this invention, the server includes means for inputting traveler information, means for generating a travel plan based on the traveler information, and means for visually presenting the travel plan to the user using virtual reality technology. This allows travelers to virtually experience tourist destinations in advance, providing a more satisfying travel experience.

[0094] "Traveler" refers to an individual or group intending to visit a tourist destination.

[0095] "Means of inputting information" refers to the interface used by travelers to input information such as places they want to visit, activities they are interested in, travel duration, and budget into a device.

[0096] "Methods for generating travel plans" refers to the process of creating efficient and personalized travel plans using a generative AI model based on information entered by travelers.

[0097] A "guide avatar" refers to a computer program that provides travelers with real-time visual and audio guidance, including explanations of tourist attractions and cultural background information.

[0098] "Virtual reality technology" refers to technology that presents a computer-generated virtual environment to the user and allows them to experience it in conjunction with the real world.

[0099] "Means of acquiring external information" refers to approaches that involve obtaining detailed information about tourist destinations, event information, and the latest information on transportation from external digital services in order to refine travel plans.

[0100] "Means of visually presenting information to the user" refers to display and projection technologies that utilize virtual reality technology to visually represent travel plans and provide them to the user.

[0101] "Methods for linking with booking procedures" refers to the process of linking the finalized travel plan with external booking services to support smooth arrangements.

[0102] This invention is implemented through a system that provides travelers with a personalized travel experience. The system consists of multiple components, which are described in detail here.

[0103] First, the user enters their travel preferences using a device. The device has the function of collecting information such as places to visit, activities of interest, travel duration, and budget through its interface, and sending it to the server. In this process, smartphones and tablets are commonly used as the device.

[0104] The server generates travel plans based on the collected information. The generative AI model used here automatically constructs optimal sightseeing routes and schedules tailored to the user's interests and budget. The AI's computational processing is based on the knowledge of professional guides, providing highly accurate plans that meet user needs. For this to work, the server needs to have high-performance processing capabilities, and cloud-based server infrastructure is primarily utilized.

[0105] Furthermore, the server acquires external information. It regularly retrieves the latest data from external digital services, such as detailed information on tourist destinations, transportation options, and event information, to further refine the generated plans. Database technology is used for data import and cross-referencing, and it is expected that NoSQL databases such as MongoDB will be primarily used.

[0106] Along with the generated travel plan, a guidance avatar is provided to enable visual and audio guidance. Using virtual reality technology, users can visually confirm their travel plan using smart glasses or a head-mounted display. The user experience is enhanced through guidance in a virtual space.

[0107] This entire process includes a mechanism to accept requests for revisions to travel plans, in case users need to review or make changes. The finalized travel plan can then be linked with an external booking service, allowing travelers to make reservations directly.

[0108] For example, if a user enters a prompt such as, "Please generate a travel plan for Tokyo that includes visiting art museums and enjoying local cuisine. The budget is 50,000 yen, and the itinerary is for two days," a travel plan including a virtual tour of Tokyo's art museums and a guide to local restaurants will be generated, and the process will include an experience through virtual reality technology.

[0109] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0110] Step 1:

[0111] The user enters travel preferences using a terminal. Information such as desired destinations, activities of interest, travel duration, and budget is collected through the input interface and sent to the server. The input data is converted to a specified format and transferred in a format ready for further processing.

[0112] Step 2:

[0113] The server generates a travel plan using a generative AI model based on the received travel preference information. Prompt messages are input to the AI, which constructs sightseeing routes and schedules tailored to the user's interests and preferences. During this process, the AI ​​model analyzes a large amount of travel data to select the most suitable plan. The output travel plan data is converted into an intermediate format to prepare for subsequent processing.

[0114] Step 3:

[0115] The server dynamically retrieves detailed information about tourist destinations from external digital services. It obtains event information and transportation data related to travel plans, and uses this information to add to and refine existing travel plans. This data from external services is received in real time via multiple APIs and integrated into the travel plan data.

[0116] Step 4:

[0117] The server prepares a guide avatar based on the updated travel plan. It generates visual and audio guidance elements and constructs data to present to the user using virtual reality technology. This includes virtual tours of specific tourist spots and explanations of their cultural background. This data is then passed to a module for visualization.

[0118] Step 5:

[0119] Users review the travel plan and avatar information generated via their device and request changes as needed. Changes made by the user are sent to the server, and the plan data is recalculated. A new travel plan reflecting the changes is generated again and finalized as the final plan.

[0120] Step 6:

[0121] The server connects confirmed travel plans to partnered external booking services. Travelers can easily book transportation and accommodations through a platform integrated with specific travel booking systems. Booking information is fed back to the user's device in real time, enabling smooth travel arrangements.

[0122] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0123] This invention is a system for providing travelers with a personalized travel experience, and is particularly characterized by its incorporation of an emotion engine that recognizes the user's emotions. The configuration for implementing this system is shown below.

[0124] First, the user uses their device to enter their travel preferences. After entering information such as the travel destination, places of interest, travel duration, and budget, the device sends this information to the server.

[0125] The server analyzes the information received from the user and uses a generative AI to generate an appropriate travel plan. The travel plan is created by leveraging the knowledge of professional guides while taking into account the user's requests and constraints.

[0126] Furthermore, this system incorporates an emotion engine that analyzes the user's facial expressions and tone of voice to determine their emotions in real time. By taking the user's satisfaction level and expectations into account, the generated travel plan becomes more personalized.

[0127] The server uses external digital services to obtain the latest tourist information, event data, and traffic information, and incorporates this into the travel plan. The plan is adjusted to take the user's emotions into account, refining the travel plan by utilizing feedback from the emotion engine.

[0128] The server then generates a guide avatar based on the travel plan, customizing the guide's style and tone according to the user's mood. The avatar visually and audibly provides detailed information and cultural background of the destinations, conveying the user's expectations for the trip.

[0129] For example, if a user expresses interest in visiting temples and wants to relax during a trip to Kyoto, the server will use an emotion engine to generate a guide avatar that conveys information in a calm tone, suggesting the optimal temple route and heartwarming lunch options to the traveler.

[0130] If the user is satisfied with the plan and ultimately approves the trip, the server connects the necessary information for the booking process to an external service, smoothly supporting the travel preparations. In this way, the present invention is a system that enables the generation and execution of travel plans centered on the user's emotions.

[0131] The following describes the processing flow.

[0132] Step 1:

[0133] The user uses a terminal to enter their travel preferences. This information includes destination, places of interest, travel duration, and budget. The terminal collects the information entered by the user and sends it to the server.

[0134] Step 2:

[0135] The server receives the submitted travel information and uses AI to generate an optimal travel plan for the user. In this process, destinations and activities are selected based on the user's preferences and requirements.

[0136] Step 3:

[0137] The emotion engine analyzes video and audio data obtained through the user's device to recognize the user's emotional state. The emotion engine also analyzes the user's facial expressions and tone of voice to measure satisfaction and interest levels.

[0138] Step 4:

[0139] The server adjusts the travel plan based on the analysis results of the emotion engine. Based on the user's emotions, it optimizes the selection of destinations and the order of activities to provide a plan that meets the user's expectations.

[0140] Step 5:

[0141] The server retrieves tourist information, the latest event data, and traffic information from external digital services and incorporates this information into travel plans. This ensures that detailed plans are always based on the most up-to-date and accurate information.

[0142] Step 6:

[0143] The server generates a guide avatar based on the travel plan. The avatar's guide style is adjusted according to the user's emotions, enabling it to effectively provide detailed information and cultural background about tourist attractions.

[0144] Step 7:

[0145] The device displays updated travel plans and information about the guide avatar to the user. The user reviews this information and, if there are any requests for corrections or additions, enters feedback and sends it from the device to the server.

[0146] Step 8:

[0147] The server receives user feedback, readjusts the travel plan as needed, and then generates the final version. Once the user approves the plan and reservations are required, the server coordinates with external booking services to complete the process. Through this process, users can confidently proceed with their travel preparations.

[0148] (Example 2)

[0149] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0150] In the field of modern travel planning, there is a demand for personalized experiences that meet the individual needs of travelers. However, conventional technologies have faced challenges in providing flexible travel plans that accommodate travelers' emotions and individual preferences. In particular, there has been a lack of technology that can grasp travelers' emotions in real time and optimize travel plans based on them. This invention aims to realize a more satisfying travel experience by providing personalized travel plans that take into account travelers' emotions and individual requests.

[0151] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0152] In this invention, the server includes means for acquiring traveler attribute information and preference information, means for creating a travel plan using a generative AI based on the acquired information, and means for analyzing the user's emotions in real time and reflecting them in the travel plan. This makes it possible to personalize and optimize travel plans to suit the traveler's emotions and individual needs.

[0153] "Traveler" refers to an individual or group that enters or obtains information for the purpose of travel.

[0154] "Attribute information" refers to basic information about travelers, including information about personal characteristics such as name, age, gender, and nationality.

[0155] "Desired information" refers to the conditions, destinations, interests, budget, and duration of travel that travelers seek in their trip, as well as other requirements necessary for travel planning.

[0156] "Generative AI" refers to a program or system that uses artificial intelligence technology to process data and automatically generate travel plans.

[0157] "Real-time analysis" refers to a technology that processes data the moment it is collected and provides results immediately.

[0158] "External information sources" refer to external databases or information services that provide necessary information that the server itself does not possess.

[0159] A "virtual character" is a character generated on a computer that serves to provide information to the user visually and audibly.

[0160] "Emotional analysis" refers to a technology that analyzes a user's facial expressions, voice tone, posture, etc., to detect their emotional state.

[0161] "Reservation procedures" refer to a series of actions taken in advance to secure transportation and accommodation based on a travel plan.

[0162] This invention is a system for providing users with a personalized travel experience. The system begins with the user inputting attribute information and preferences related to their trip using a terminal. The terminal sends the input information to a server. The server uses a generative AI model to analyze the acquired information and create an individually customized travel plan.

[0163] In this system, the server is equipped with an emotion engine that has the ability to analyze the user's facial expressions and tone of voice in real time. The results of the emotion engine's analysis are used to customize travel plans, ensuring that the most desirable experience is provided to the traveler.

[0164] Furthermore, the server collaborates with external information services to dynamically acquire the latest tourism and transportation data, integrating it into the travel plan. This information is stored in a database on the server and used to refine the plan.

[0165] Furthermore, the server generates a virtual character based on the travel plan. This character guides the traveler with information related to the plan, both visually and audibly. The guidance will reflect the user's interests, including details about tourist attractions and their cultural background.

[0166] For example, if a user wants to visit Kyoto and wants to tour temples, the server passes a prompt to the generating AI model saying, "Considering that the user is in a relaxed mood, please suggest a plan to visit quiet temples and tranquil gardens in Kyoto." This allows the server to create a travel plan, including the optimal route, and provide it to the user.

[0167] Finally, once the user approves the travel plan, the server integrates with an external booking system to automatically secure transportation and accommodation. In this way, users can proceed with travel preparations stress-free.

[0168] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0169] Step 1:

[0170] The user uses a terminal to input attribute information and preferences related to their trip. Specifically, they fill in information such as destination, travel duration, activities of interest, and budget in an input form. The terminal generates data packets to format the entered data and send it to the server. The input is information from the user, and the output is the data packets sent to the server.

[0171] Step 2:

[0172] The server analyzes travel information received from the terminal. First, it processes the data to standardize the format for storage in the database. The input is the raw data received from the terminal, and the output is data converted into a format suitable for analysis. To pass it to the generating AI model, it identifies the user's desired conditions and generates prompt statements.

[0173] Step 3:

[0174] The server creates travel plans using a generative AI model. Based on prompts, the generative AI generates a travel plan that best reflects the traveler's preferences. In this process, knowledge data from professional guides is utilized. The input consists of prompts and related database information, while the output is the generated travel plan.

[0175] Step 4:

[0176] The server activates an emotion engine to monitor the user's emotions in real time. This information is obtained by analyzing the user's camera video and voice input. The input is camera video and audio data, and the output is the analyzed emotion information. Based on this information, the travel plan is adjusted.

[0177] Step 5:

[0178] The server connects to external information sources to obtain the latest tourist information, event information, and transportation information. It retrieves information via external APIs and updates the data to improve the accuracy of the travel plan. The input is data from external information sources, and the output is the updated travel plan.

[0179] Step 6:

[0180] The server generates a virtual character based on the travel plan. The character is adapted to the user's emotions and provides visual and auditory guidance through the details of the plan. The input is the travel plan and emotional information, and the output is a user-operable guide avatar.

[0181] Step 7:

[0182] Users can review the travel plan presented through their device and request revisions as needed. The server finalizes the travel plan upon receiving user approval or revision requests. The input is user feedback, and the output is the finalized travel plan.

[0183] Step 8:

[0184] The server processes bookings in conjunction with an external booking system based on the finalized travel plan. The input is the final travel plan, and the output is booking information for secured transportation and accommodation. This process allows users to smoothly prepare for their trip.

[0185] (Application Example 2)

[0186] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0187] Modern travelers are increasingly seeking interactive travel experiences that are tailored to their individual preferences and emotions. However, traditional systems fail to adequately personalize these experiences by considering travelers' feelings, resulting in a lack of appropriate information and emotionally resonant guidance. This leads to challenges in improving travel experience satisfaction and can even decrease satisfaction with the planning and execution of trips.

[0188] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0189] In this invention, the server includes means for inputting traveler information, means for analyzing the user's emotional state, and means for providing a digital guide based on the generated travel plan. This makes it possible to provide a personalized travel experience based on the traveler's emotions.

[0190] A "means for inputting traveler information" refers to a mechanism for collecting detailed information such as travelers' personal preferences, desired destinations, duration, and budget, and using this information to plan their trips.

[0191] A "means for generating travel plans" is a mechanism that automatically creates optimal sightseeing routes and itineraries based on collected traveler information.

[0192] "Methods for analyzing the user's emotional state" refers to technology that recognizes and analyzes the user's emotions in real time from their voice and facial expressions, and reflects this in the travel plan.

[0193] "Means of providing digital guides for informational purposes" refers to technology that generates digital characters to provide travelers with visual and auditory information and background details about their destinations and to guide them.

[0194] "Methods for acquiring external information to refine travel plans" refer to the process of gathering the latest tourist information and traffic conditions from sources such as the internet, and then incorporating them into the travel plan to make it more precise.

[0195] "Means to provide to travelers" refers to technology that instantly provides and makes accessible to travelers the generated travel plans and guide information.

[0196] "Means of adjusting the tone and style of digital guides according to the traveler's emotions" refers to technology that dynamically changes the method and expression of guidance according to the user's emotional state, thereby ensuring a more personalized experience.

[0197] "Methods for integrating with booking procedures" refer to technologies that enable seamless booking of accommodations, transportation, and other services based on the finalized travel plan.

[0198] The system for realizing this invention takes user emotions into consideration and provides a personalized travel experience based on specific conditions. The system consists of the user's terminal, an emotion recognition engine, a generative AI model, and a server that interacts with an external database.

[0199] The user first uses a smart device to input basic information such as travel destination, duration, and budget. This information is sent from the device to the server. Based on this information, the server inputs prompts into an AI model to generate a travel plan.

[0200] The emotion recognition engine, for example using the Affectiva SDK, analyzes the user's voice and facial expressions through smart glasses or a smartphone to determine their emotional state in real time. The results of the emotion analysis are fed back into the user's travel plan, allowing for fine-tuning of the plan.

[0201] The travel plan includes a digital guide generated using a generative AI model, providing users with visual and auditory information about the cultural background of places and activities to do during their visit. Furthermore, the plan's accuracy is improved by incorporating up-to-date traffic and event data from external sources.

[0202] For example, if a user is planning a trip to Kyoto and expresses interest in visiting temples and wants to relax, the emotion engine will detect this state and provide a digital guide in a calm tone via the server. An example of a prompt in this case would be, "The user is currently in a relaxed mood and is looking for information on quiet temples in Kyoto. Please suggest recommended routes and details."

[0203] This system dynamically optimizes the travel experience in response to the user's emotional reactions, providing a more meaningful trip tailored to individual preferences.

[0204] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0205] Step 1:

[0206] The user uses a smart device to input their travel destination and preferences (travel duration, budget, places of interest, etc.) into the terminal. This information is processed by the terminal and sent to the server. The entered data is used by the server as basic data to generate a travel plan.

[0207] Step 2:

[0208] The server creates prompts for use in the AI ​​model based on the traveler information it receives. These prompts reflect the user's preferences and are input into the AI ​​model. The AI ​​processes these prompts and generates a basic travel plan.

[0209] Step 3:

[0210] An emotion recognition engine collects real-time emotional data from the user (analysis results of facial expressions and voice). Using a device such as smart glasses, the analyzed emotional information is sent to a server. The server uses this information to adjust the travel plan based on the user's emotional state.

[0211] Step 4:

[0212] The server retrieves the latest data from external information sources (traffic information, tourist information, event information, etc.) and integrates it into existing travel plans. This process refines and optimizes the plan.

[0213] Step 5:

[0214] The server uses a generative AI model to create and provide digital guides. The digital guides adjust their tone and style according to the user's emotions and provide traveler-related information visually and audibly.

[0215] Step 6:

[0216] The user reviews the provided travel plan and makes any necessary modifications on their device. The modifications are sent to the server, and an updated plan is generated.

[0217] Step 7:

[0218] The finalized travel plan is then linked to the booking process via the server. In this step, accommodation and transportation reservations are completed by the relevant services.

[0219] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0220] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0221] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0222] [Second Embodiment]

[0223] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0224] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0225] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0226] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0227] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0228] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0229] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0230] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0231] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0232] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0233] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0234] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0235] This invention is a system for providing travelers with an efficient and personalized travel experience. The following describes the configurations for implementing this system.

[0236] First, the user enters travel information via their device, such as places they want to visit, activities they are interested in, travel duration, and budget. The device collects this information and sends it to the server.

[0237] Next, the server automatically generates a travel plan based on the information received from the user. This plan generation utilizes a generation AI implemented on the server, which constructs a plan tailored to the user's preferences based on the knowledge and experience of professional guides. The AI ​​can suggest the optimal sightseeing route and schedule according to the user's interests and budget.

[0238] Regarding the generated travel plan, the server retrieves detailed information on tourist destinations, events, transportation options, etc., through external digital services to further refine the plan. This makes it possible to provide travel plans that are always based on the latest and most accurate information.

[0239] Next, the server prepares a guide avatar based on the travel plan. This avatar is responsible for providing visual and audio guidance, conveying explanations of tourist attractions and cultural background information in real time. Users can view the generated travel plan and the avatar's guidance on their device.

[0240] For example, if a user wants to sightsee in Tokyo and requests a plan that includes visiting museums and dining locally, the server will provide a plan that includes museum opening hours, admission fees, and access information. It can also include a suggestion for a popular local restaurant for lunch. If the user has any requests for additions or modifications to this plan, they can do so through their device, and the server will readjust the plan accordingly.

[0241] Finally, once the user confirms their plan, the server integrates it with partnered external booking services to support travel arrangements. This allows users to easily complete their bookings and proceed with their trip smoothly.

[0242] The following describes the processing flow.

[0243] Step 1:

[0244] The user enters their travel preferences via the device. Specifically, they enter information such as places they want to visit, activities they are interested in, length of stay, and budget. The device collects this data and sends it to the server.

[0245] Step 2:

[0246] The server receives information entered by the user and analyzes it to generate a travel plan. The AI ​​on the server creates a travel plan that meets the user's wishes, based on the knowledge and experience of professional guides. This AI suggests destinations, sightseeing routes, and schedules.

[0247] Step 3:

[0248] The server retrieves tourist destination information, event information, and transportation information from external digital services to improve the accuracy of travel plans. Based on this information, it modifies the travel plan to ensure that the plan is based on the latest data.

[0249] Step 4:

[0250] The server prepares a guide avatar based on the generated travel plan. The avatar has a script that explains detailed information and cultural background of the destinations, and is configured to guide the user visually or audibly.

[0251] Step 5:

[0252] The device displays the travel plan and avatar information received from the server to the user. The user reviews the presented plan, approves it if they are satisfied with the contents, and submits a change request if necessary.

[0253] Step 6:

[0254] If a user requests a change, the server receives the information and readjusts the plan. The server then generates a new travel plan, reflecting the changes, and provides feedback to the user.

[0255] Step 7:

[0256] The server confirms the travel plan that the user has finally approved. If reservations are required as part of the travel process, the server then connects the information to external booking services to assist with the booking arrangements. Through this process, the user completes their travel preparations.

[0257] (Example 1)

[0258] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0259] To provide travelers with effective and personalized travel experiences, it is necessary to efficiently collect and analyze a large amount of information and generate appropriate travel plans. However, conventional systems have problems with the accuracy of information and the flexibility of plans, and have been unable to fully meet the individual needs of travelers.

[0260] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0261] In this invention, the server includes means for analyzing traveler information and automatically generating a travel plan using a generative AI model; means for dynamically acquiring information from external sources and refining the travel plan; and means for presenting the generated travel plan and guidance information to the traveler and accepting modifications according to the traveler's requests. This makes it possible to provide the latest and most flexible travel plan optimized for each traveler's wishes and conditions.

[0262] A "data processing device" is an electronic device that has the function of collecting travelers' input information and transmitting it to a server.

[0263] A "generative AI model" is an artificial intelligence system used to automatically generate optimal travel plans based on traveler information.

[0264] A "guidance output device" is a medium or device for providing information visually and audibly based on a travel plan.

[0265] "External information sources" refer to databases and services that dynamically retrieve information, such as detailed information about tourist destinations and transportation information, in order to improve the accuracy of travel planning.

[0266] A "reservation management system" is a system for executing or managing reservation procedures based on a finalized travel plan.

[0267] "Traveler" refers to an individual or group that plans and executes a trip using this system.

[0268] This invention is a system for providing travelers with personalized travel experiences. Specific embodiments of the system are described below.

[0269] The user first uses a terminal to input travel information such as their desired destination, interesting sightseeing experiences, travel duration, and budget. The terminal organizes this input information and sends it to the server via a secure communication protocol. In this process, the terminal functions as a data processing device.

[0270] The server analyzes data received from the user and automatically generates a travel plan using a generative AI model. The generative AI model uses artificial intelligence technology to simulate the knowledge of a professional guide and build a customized plan tailored to the user's requests. In particular, the server can use prompts such as: "The user wants to visit art museums in Tokyo. Please suggest recommended museums and access information."

[0271] In the generated travel plan, the server dynamically retrieves the latest data on tourist destinations, event information, and transportation options from external sources to improve the accuracy of the plan. This ensures that users are always provided with accurate and up-to-date information.

[0272] Next, the server prepares an avatar as a guidance output device based on the travel plan. This avatar is designed to provide travelers with real-time explanations of tourist attractions and cultural background information through visual and auditory means. Users can view the generated travel plan and the guidance information provided by the avatar via their terminal.

[0273] For example, if a user enters "I want to visit art museums in Tokyo and enjoy local cuisine," the server will provide a detailed plan including museum opening hours, admission fees, and access information. Furthermore, it can also include popular local restaurants as lunch options.

[0274] Finally, once the user confirms their travel plan, the server, acting as a reservation management system, links the plan to affiliated external reservation services. This allows users to complete their travel reservations without cumbersome procedures, ensuring a smooth travel experience.

[0275] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0276] Step 1:

[0277] The user uses a terminal to enter their desired travel destination, activities of interest, duration, and budget. The terminal collects this information and converts it into a digital format. This input data is then sent to the server as foundational information necessary for the next processing step.

[0278] Step 2:

[0279] The server analyzes the traveler information received from the terminal. The server utilizes a generative AI model to automatically generate a travel plan that matches the user's interests and budget. Specifically, the server constructs a plan optimized for the user's needs by generating a prompt text and passing it to the AI model. For example, by using a prompt text such as "The user wants to visit a museum in Tokyo. Please propose a recommended route", the AI generates a detailed sightseeing route. The output is the generated travel plan.

[0280] Step 3:

[0281] The server adds the latest data from external information sources to the generated travel plan. The server obtains opening information, event information, and transportation information of tourist destinations and reflects them in the travel plan. As a result, a travel plan based on the latest detailed information is generated. The output is the refined and latest travel plan.

[0282] Step 4:

[0283] The server prepares a guiding avatar based on the travel plan. This avatar is set to provide the traveler with real-time explanations of tourist destinations and cultural background information through visual and audio information. As a result, the user can receive detailed travel guidance through the terminal. The output is the guide information of the avatar related to the travel plan.

[0284] Step 5:

[0285] The user checks the generated travel plan and the guidance by the avatar through the terminal. If the user wishes to modify the plan, a request can be sent to the server via the terminal. The server readjusts the travel plan based on this request. The input is the user's modification request, and the output is the updated travel plan.

[0286] Step 6:

[0287] Once a user confirms their travel plans, the server integrates them with a partnered booking management system. The server automatically processes the user's travel booking and performs the necessary procedures, allowing the user to easily complete their reservation. The output is confirmation of the confirmed travel booking.

[0288] (Application Example 1)

[0289] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0290] Traditional travel planning systems sometimes struggle to provide appropriate travel plans tailored to travelers' interests and budgets, and they have limited means for travelers to virtually experience tourist destinations before actually visiting. As a result, travelers may be unable to make informed decisions, leading to decreased satisfaction with their travel experience.

[0291] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0292] In this invention, the server includes means for inputting traveler information, means for generating a travel plan based on the traveler information, and means for visually presenting the travel plan to the user using virtual reality technology. This allows travelers to virtually experience tourist destinations in advance, providing a more satisfying travel experience.

[0293] "Traveler" refers to an individual or group intending to visit a tourist destination.

[0294] "Means of inputting information" refers to the interface used by travelers to input information such as places they want to visit, activities they are interested in, travel duration, and budget into a device.

[0295] "Methods for generating travel plans" refers to the process of creating efficient and personalized travel plans using a generative AI model based on information entered by travelers.

[0296] A "guide avatar" refers to a computer program that provides travelers with real-time visual and audio guidance, including explanations of tourist attractions and cultural background information.

[0297] "Virtual reality technology" refers to technology that presents a computer-generated virtual environment to the user and allows them to experience it in conjunction with the real world.

[0298] "Means of acquiring external information" refers to approaches that involve obtaining detailed information about tourist destinations, event information, and the latest information on transportation from external digital services in order to refine travel plans.

[0299] "Means of visually presenting information to the user" refers to display and projection technologies that utilize virtual reality technology to visually represent travel plans and provide them to the user.

[0300] "Methods for linking with booking procedures" refers to the process of linking the finalized travel plan with external booking services to support smooth arrangements.

[0301] This invention is implemented through a system that provides travelers with a personalized travel experience. The system consists of multiple components, which are described in detail here.

[0302] First, the user enters their travel preferences using a device. The device has the function of collecting information such as places to visit, activities of interest, travel duration, and budget through its interface, and sending it to the server. In this process, smartphones and tablets are commonly used as the device.

[0303] The server generates a travel plan based on the collected information. The generation AI model used here automatically constructs an optimal sightseeing route and schedule according to the user's interests and budget. The AI's computational processing is based on the knowledge of professional guides, providing a high-precision plan tailored to user needs. For this purpose, the server needs to have high-performance processing capabilities, and mainly cloud-based server infrastructure is utilized.

[0304] Furthermore, the server obtains external information. It regularly acquires the latest data such as detailed information of tourist attractions, transportation means, and event information from external digital services to further refine the generated plan. At this time, database technology is used for data import and cross-reference, and it is mainly assumed that a NoSQL database such as MongoDB is used.

[0305] Along with the generated travel plan, a guiding avatar for realizing visual and audio guidance is provided. Through an application using virtual reality technology, users can visually confirm the travel plan using smart glasses or head-mounted displays. Through the guidance in the virtual space, the user experience is improved.

[0306] In this series of processes, in preparation for the possibility that the user may make confirmations or changes, a mechanism for accepting modification requests for the travel plan is also supported. The finally confirmed travel plan can be directly reserved by the traveler by collaborating with an external reservation service.

[0307] As a specific example, when the user inputs a prompt sentence such as "Please generate a travel plan including visiting art museums and local cuisine in Tokyo. The budget is 50,000 yen and the schedule is for two days.", a travel plan including a virtual tour of art museums in Tokyo and guidance to local restaurants is generated, and an experience through virtual reality technology is provided in the process.

[0308] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0309] Step 1:

[0310] The user enters travel preferences using a terminal. Information such as desired destinations, activities of interest, travel duration, and budget is collected through the input interface and sent to the server. The input data is converted to a specified format and transferred in a format ready for further processing.

[0311] Step 2:

[0312] The server generates a travel plan using a generative AI model based on the received travel preference information. Prompt messages are input to the AI, which constructs sightseeing routes and schedules tailored to the user's interests and preferences. During this process, the AI ​​model analyzes a large amount of travel data to select the most suitable plan. The output travel plan data is converted into an intermediate format to prepare for subsequent processing.

[0313] Step 3:

[0314] The server dynamically retrieves detailed information about tourist destinations from external digital services. It obtains event information and transportation data related to travel plans, and uses this information to add to and refine existing travel plans. This data from external services is received in real time via multiple APIs and integrated into the travel plan data.

[0315] Step 4:

[0316] The server prepares a guide avatar based on the updated travel plan. It generates visual and audio guidance elements and constructs data to present to the user using virtual reality technology. This includes virtual tours of specific tourist spots and explanations of their cultural background. This data is then passed to a module for visualization.

[0317] Step 5:

[0318] Users review the travel plan and avatar information generated via their device and request changes as needed. Changes made by the user are sent to the server, and the plan data is recalculated. A new travel plan reflecting the changes is generated again and finalized as the final plan.

[0319] Step 6:

[0320] The server connects confirmed travel plans to partnered external booking services. Travelers can easily book transportation and accommodations through a platform integrated with specific travel booking systems. Booking information is fed back to the user's device in real time, enabling smooth travel arrangements.

[0321] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0322] This invention is a system for providing travelers with a personalized travel experience, and is particularly characterized by its incorporation of an emotion engine that recognizes the user's emotions. The configuration for implementing this system is shown below.

[0323] First, the user uses their device to enter their travel preferences. After entering information such as the travel destination, places of interest, travel duration, and budget, the device sends this information to the server.

[0324] The server analyzes the information received from the user and uses a generative AI to generate an appropriate travel plan. The travel plan is created by leveraging the knowledge of professional guides while taking into account the user's requests and constraints.

[0325] Furthermore, this system incorporates an emotion engine that analyzes the user's facial expressions and tone of voice to determine their emotions in real time. By taking the user's satisfaction level and expectations into account, the generated travel plan becomes more personalized.

[0326] The server uses external digital services to obtain the latest tourist information, event data, and traffic information, and incorporates this into the travel plan. The plan is adjusted to take the user's emotions into account, refining the travel plan by utilizing feedback from the emotion engine.

[0327] The server then generates a guide avatar based on the travel plan, customizing the guide's style and tone according to the user's mood. The avatar visually and audibly provides detailed information and cultural background of the destinations, conveying the user's expectations for the trip.

[0328] For example, if a user expresses interest in visiting temples and wants to relax during their trip to Kyoto, the server will use an emotion engine to generate a guide avatar that conveys information in a calm tone, suggesting the optimal temple route and heartwarming lunch options to the traveler.

[0329] If the user is satisfied with the plan and ultimately approves the trip, the server connects the necessary information for the booking process to an external service, smoothly supporting the travel preparations. In this way, the present invention is a system that enables the generation and execution of travel plans centered on the user's emotions.

[0330] The following describes the processing flow.

[0331] Step 1:

[0332] The user uses a device to enter their travel preferences. This information includes destination, places of interest, travel duration, and budget. The device collects the information entered by the user and sends it to the server.

[0333] Step 2:

[0334] The server receives the submitted travel information and uses AI to generate an optimal travel plan for the user. In this process, destinations and activities are selected based on the user's preferences and requirements.

[0335] Step 3:

[0336] The emotion engine analyzes video and audio data obtained through the user's device to recognize the user's emotional state. The emotion engine also analyzes the user's facial expressions and tone of voice to measure satisfaction and interest levels.

[0337] Step 4:

[0338] The server adjusts the travel plan based on the analysis results of the emotion engine. Based on the user's emotions, it optimizes the selection of destinations and the order of activities to provide a plan that meets the user's expectations.

[0339] Step 5:

[0340] The server retrieves tourist information, the latest event data, and traffic information from external digital services and incorporates this information into travel plans. This ensures that detailed plans are always based on the most up-to-date and accurate information.

[0341] Step 6:

[0342] The server generates a guide avatar based on the travel plan. The avatar's guide style is adjusted according to the user's emotions, enabling it to effectively provide detailed information and cultural background about tourist attractions.

[0343] Step 7:

[0344] The device displays updated travel plans and information about the guide avatar to the user. The user reviews this information and, if there are any requests for corrections or additions, enters feedback and sends it from the device to the server.

[0345] Step 8:

[0346] The server receives user feedback, readjusts the travel plan as needed, and then generates the final version. Once the user approves the plan and reservations are required, the server coordinates with external booking services to complete the process. Through this process, users can confidently proceed with their travel preparations.

[0347] (Example 2)

[0348] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0349] In the field of modern travel planning, there is a demand for personalized experiences that meet the individual needs of travelers. However, conventional technologies have faced challenges in providing flexible travel plans that accommodate travelers' emotions and individual preferences. In particular, there has been a lack of technology that can grasp travelers' emotions in real time and optimize travel plans based on them. This invention aims to realize a more satisfying travel experience by providing personalized travel plans that take into account travelers' emotions and individual requests.

[0350] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0351] In this invention, the server includes means for acquiring traveler attribute information and preference information, means for creating a travel plan using a generative AI based on the acquired information, and means for analyzing the user's emotions in real time and reflecting them in the travel plan. This makes it possible to personalize and optimize travel plans to suit the traveler's emotions and individual needs.

[0352] "Traveler" refers to an individual or group that enters or obtains information for the purpose of travel.

[0353] "Attribute information" refers to basic information about travelers, including information about personal characteristics such as name, age, gender, and nationality.

[0354] "Desired information" refers to the conditions, destinations, interests, budget, and duration of travel that travelers seek in their trip, as well as other requirements necessary for travel planning.

[0355] "Generative AI" refers to a program or system that uses artificial intelligence technology to process data and automatically generate travel plans.

[0356] "Real-time analysis" refers to a technology that processes data the moment it is collected and provides results immediately.

[0357] "External information sources" refer to external databases or information services that provide necessary information that the server itself does not possess.

[0358] A "virtual character" is a character generated on a computer that serves to provide information to the user visually and audibly.

[0359] "Emotional analysis" refers to a technology that analyzes a user's facial expressions, voice tone, posture, etc., to detect their emotional state.

[0360] "Reservation procedures" refer to a series of actions taken in advance to secure transportation and accommodation based on a travel plan.

[0361] This invention is a system for providing users with a personalized travel experience. The system begins with the user inputting attribute information and preferences related to their trip using a terminal. The terminal sends the input information to a server. The server uses a generative AI model to analyze the acquired information and create an individually customized travel plan.

[0362] In this system, the server is equipped with an emotion engine that has the ability to analyze the user's facial expressions and tone of voice in real time. The results of the emotion engine's analysis are used to customize travel plans, ensuring that the most desirable experience is provided to the traveler.

[0363] Furthermore, the server collaborates with external information services to dynamically acquire the latest tourism and transportation data, integrating it into the travel plan. This information is stored in a database on the server and used to refine the plan.

[0364] Furthermore, the server generates a virtual character based on the travel plan. This character guides the traveler with information related to the plan, both visually and audibly. The guidance will reflect the user's interests, including details about tourist attractions and their cultural background.

[0365] For example, if a user wants to visit Kyoto and wants to tour temples, the server passes a prompt to the generating AI model saying, "Considering that the user is in a relaxed mood, please suggest a plan to visit quiet temples and tranquil gardens in Kyoto." This allows the server to create a travel plan, including the optimal route, and provide it to the user.

[0366] Finally, once the user approves the travel plan, the server integrates with an external booking system to automatically secure transportation and accommodation. In this way, users can proceed with travel preparations stress-free.

[0367] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0368] Step 1:

[0369] The user uses a terminal to input attribute information and preferences related to their trip. Specifically, they fill in information such as destination, travel duration, activities of interest, and budget in an input form. The terminal generates data packets to format the entered data and send it to the server. The input is information from the user, and the output is the data packets sent to the server.

[0370] Step 2:

[0371] The server analyzes travel information received from the terminal. First, it processes the data to standardize the format for storage in the database. The input is the raw data received from the terminal, and the output is data converted into a format suitable for analysis. To pass it to the generating AI model, it identifies the user's desired conditions and generates prompt statements.

[0372] Step 3:

[0373] The server creates travel plans using a generative AI model. Based on prompts, the generative AI generates a travel plan that best reflects the traveler's preferences. In this process, knowledge data from professional guides is utilized. The input consists of prompts and related database information, while the output is the generated travel plan.

[0374] Step 4:

[0375] The server activates an emotion engine to monitor the user's emotions in real time. This information is obtained by analyzing the user's camera video and voice input. The input is camera video and audio data, and the output is the analyzed emotion information. Based on this information, the travel plan is adjusted.

[0376] Step 5:

[0377] The server connects to external information sources to obtain the latest tourist information, event information, and transportation information. It retrieves information via external APIs and updates the data to improve the accuracy of the travel plan. The input is data from external information sources, and the output is the updated travel plan.

[0378] Step 6:

[0379] The server generates a virtual character based on the travel plan. The character is adapted to the user's emotions and provides visual and auditory guidance through the details of the plan. The input is the travel plan and emotional information, and the output is a user-operable guide avatar.

[0380] Step 7:

[0381] Users can review the travel plan presented through their device and request revisions as needed. The server finalizes the travel plan upon receiving user approval or revision requests. The input is user feedback, and the output is the finalized travel plan.

[0382] Step 8:

[0383] The server processes bookings in conjunction with an external booking system based on the finalized travel plan. The input is the final travel plan, and the output is booking information for secured transportation and accommodation. This process allows users to smoothly prepare for their trip.

[0384] (Application Example 2)

[0385] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0386] Modern travelers are increasingly seeking interactive travel experiences that are tailored to their individual preferences and emotions. However, traditional systems fail to adequately personalize these experiences by considering travelers' feelings, resulting in a lack of appropriate information and emotionally resonant guidance. This leads to challenges in improving travel experience satisfaction and can even decrease satisfaction with the planning and execution of trips.

[0387] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0388] In this invention, the server includes means for inputting traveler information, means for analyzing the user's emotional state, and means for providing a digital guide based on the generated travel plan. This makes it possible to provide a personalized travel experience based on the traveler's emotions.

[0389] A "means for inputting traveler information" refers to a mechanism for collecting detailed information such as travelers' personal preferences, desired destinations, duration, and budget, and using this information to plan their trips.

[0390] A "means for generating travel plans" is a mechanism that automatically creates optimal sightseeing routes and itineraries based on collected traveler information.

[0391] "Methods for analyzing the user's emotional state" refers to technology that recognizes and analyzes the user's emotions in real time from their voice and facial expressions, and reflects this in the travel plan.

[0392] "Means of providing digital guides for informational purposes" refers to technology that generates digital characters to provide travelers with visual and auditory information and background details about their destinations and to guide them.

[0393] "Methods for acquiring external information to refine travel plans" refer to the process of gathering the latest tourist information and traffic conditions from sources such as the internet, and then incorporating them into the travel plan to make it more precise.

[0394] "Means to provide to travelers" refers to technology that instantly provides and makes accessible to travelers the generated travel plans and guide information.

[0395] "Means of adjusting the tone and style of digital guides according to the traveler's emotions" refers to technology that dynamically changes the method and expression of guidance according to the user's emotional state, thereby ensuring a more personalized experience.

[0396] "Methods for integrating with booking procedures" refer to technologies that enable seamless booking of accommodations, transportation, and other services based on the finalized travel plan.

[0397] The system for realizing this invention takes user emotions into consideration and provides a personalized travel experience based on specific conditions. The system consists of the user's terminal, an emotion recognition engine, a generative AI model, and a server that interacts with an external database.

[0398] The user first uses a smart device to input basic information such as travel destination, duration, and budget. This information is sent from the device to the server. Based on this information, the server inputs prompts into an AI model to generate a travel plan.

[0399] The emotion recognition engine, for example using the Affectiva SDK, analyzes the user's voice and facial expressions through smart glasses or a smartphone to determine their emotional state in real time. The results of the emotion analysis are fed back into the user's travel plan, allowing for fine-tuning of the plan.

[0400] The travel plan includes a digital guide generated using a generative AI model, providing users with visual and auditory information about the cultural background of places and activities to do during their visit. Furthermore, the plan's accuracy is improved by incorporating up-to-date traffic and event data from external sources.

[0401] For example, if a user is planning a trip to Kyoto and expresses interest in visiting temples and wants to relax, the emotion engine will detect this state and provide a digital guide in a calm tone via the server. An example of a prompt in this case would be, "The user is currently in a relaxed mood and is looking for information on quiet temples in Kyoto. Please suggest recommended routes and details."

[0402] This system dynamically optimizes the travel experience in response to the user's emotional reactions, providing a more meaningful trip tailored to individual preferences.

[0403] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0404] Step 1:

[0405] The user uses a smart device to input their travel destination and preferences (travel duration, budget, places of interest, etc.) into the terminal. This information is processed by the terminal and sent to the server. The entered data is used by the server as basic data to generate a travel plan.

[0406] Step 2:

[0407] The server creates prompts for use in the AI ​​model based on the traveler information it receives. These prompts reflect the user's preferences and are input into the AI ​​model. The AI ​​processes these prompts and generates a basic travel plan.

[0408] Step 3:

[0409] An emotion recognition engine collects real-time emotional data from the user (analysis results of facial expressions and voice). Using a device such as smart glasses, the analyzed emotional information is sent to a server. The server uses this information to adjust the travel plan based on the user's emotional state.

[0410] Step 4:

[0411] The server retrieves the latest data from external information sources (traffic information, tourist information, event information, etc.) and integrates it into existing travel plans. This process refines and optimizes the plan.

[0412] Step 5:

[0413] The server uses a generative AI model to create and provide digital guides. The digital guides adjust their tone and style according to the user's emotions and provide traveler-related information visually and audibly.

[0414] Step 6:

[0415] The user reviews the provided travel plan and makes any necessary modifications on their device. The modifications are sent to the server, and an updated plan is generated.

[0416] Step 7:

[0417] The finalized travel plan is then linked to the booking process via the server. In this step, accommodation and transportation reservations are completed by the relevant services.

[0418] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0419] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0420] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0421] [Third Embodiment]

[0422] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0423] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0424] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0425] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0426] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0427] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0428] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0429] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0430] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0431] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0432] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0433] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0434] This invention is a system for providing travelers with an efficient and personalized travel experience. The following describes the configurations for implementing this system.

[0435] First, the user enters travel information via their device, such as places they want to visit, activities they are interested in, travel duration, and budget. The device collects this information and sends it to the server.

[0436] Next, the server automatically generates a travel plan based on the information received from the user. This plan generation utilizes a generation AI implemented on the server, which constructs a plan tailored to the user's preferences based on the knowledge and experience of professional guides. The AI ​​can suggest the optimal sightseeing route and schedule according to the user's interests and budget.

[0437] Regarding the generated travel plan, the server retrieves detailed information on tourist destinations, events, transportation options, etc., through external digital services to further refine the plan. This makes it possible to provide travel plans that are always based on the latest and most accurate information.

[0438] Next, the server prepares a guide avatar based on the travel plan. This avatar is responsible for providing visual and audio guidance, conveying explanations of tourist attractions and cultural background information in real time. Users can view the generated travel plan and the avatar's guidance on their device.

[0439] For example, if a user wants to sightsee in Tokyo and requests a plan that includes visiting museums and dining locally, the server will provide a plan that includes museum opening hours, admission fees, and access information. It can also include a suggestion for a popular local restaurant for lunch. If the user has any requests for additions or modifications to this plan, they can do so through their device, and the server will readjust the plan accordingly.

[0440] Finally, once the user confirms their plan, the server integrates it with partnered external booking services to support travel arrangements. This allows users to easily complete their bookings and proceed with their trip smoothly.

[0441] The following describes the processing flow.

[0442] Step 1:

[0443] The user enters their travel preferences via the device. Specifically, they enter information such as places they want to visit, activities they are interested in, length of stay, and budget. The device collects this data and sends it to the server.

[0444] Step 2:

[0445] The server receives information entered by the user and analyzes it to generate a travel plan. The AI ​​on the server creates a travel plan that meets the user's wishes, based on the knowledge and experience of professional guides. This AI suggests destinations, sightseeing routes, and schedules.

[0446] Step 3:

[0447] The server retrieves tourist destination information, event information, and transportation information from external digital services to improve the accuracy of travel plans. Based on this information, it modifies the travel plan to ensure that the plan is based on the latest data.

[0448] Step 4:

[0449] The server prepares a guide avatar based on the generated travel plan. The avatar has a script that explains detailed information and cultural background of the destinations, and is configured to guide the user visually or audibly.

[0450] Step 5:

[0451] The device displays the travel plan and avatar information received from the server to the user. The user reviews the presented plan, approves it if they are satisfied with the contents, and submits a change request if necessary.

[0452] Step 6:

[0453] If a user requests a change, the server receives the information and readjusts the plan. The server then generates a new travel plan, reflecting the changes, and provides feedback to the user.

[0454] Step 7:

[0455] The server confirms the travel plan that the user has finally approved. If reservations are required as part of the travel process, the server then connects the information to external booking services to assist with the booking arrangements. Through this process, the user completes their travel preparations.

[0456] (Example 1)

[0457] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0458] To provide travelers with effective and personalized travel experiences, it is necessary to efficiently collect and analyze a large amount of information and generate appropriate travel plans. However, conventional systems have problems with the accuracy of information and the flexibility of plans, and have been unable to fully meet the individual needs of travelers.

[0459] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0460] In this invention, the server includes means for analyzing traveler information and automatically generating a travel plan using a generative AI model; means for dynamically acquiring information from external sources and refining the travel plan; and means for presenting the generated travel plan and guidance information to the traveler and accepting modifications according to the traveler's requests. This makes it possible to provide the latest and most flexible travel plan optimized for each traveler's wishes and conditions.

[0461] A "data processing device" is an electronic device that has the function of collecting travelers' input information and transmitting it to a server.

[0462] A "generative AI model" is an artificial intelligence system used to automatically generate optimal travel plans based on traveler information.

[0463] A "guidance output device" is a medium or device for providing information visually and audibly based on a travel plan.

[0464] "External information sources" refer to databases and services that dynamically retrieve information, such as detailed information about tourist destinations and transportation information, in order to improve the accuracy of travel planning.

[0465] A "reservation management system" is a system for executing or managing reservation procedures based on a finalized travel plan.

[0466] "Traveler" refers to an individual or group that plans and executes a trip using this system.

[0467] This invention is a system for providing travelers with personalized travel experiences. Specific embodiments of the system are described below.

[0468] The user first uses a terminal to input travel information such as their desired destination, interesting sightseeing experiences, travel duration, and budget. The terminal organizes this input information and sends it to the server via a secure communication protocol. In this process, the terminal functions as a data processing device.

[0469] The server analyzes data received from the user and automatically generates a travel plan using a generative AI model. The generative AI model uses artificial intelligence technology to simulate the knowledge of a professional guide and build a customized plan tailored to the user's requests. In particular, the server can use prompts such as: "The user wants to visit art museums in Tokyo. Please suggest recommended museums and access information."

[0470] In the generated travel plan, the server dynamically retrieves the latest data on tourist destinations, event information, and transportation options from external sources to improve the accuracy of the plan. This ensures that users are always provided with accurate and up-to-date information.

[0471] Next, the server prepares an avatar as a guidance output device based on the travel plan. This avatar is designed to provide travelers with real-time explanations of tourist attractions and cultural background information through visual and auditory means. Users can view the generated travel plan and the guidance information provided by the avatar via their terminal.

[0472] For example, if a user enters "I want to visit art museums in Tokyo and enjoy local cuisine," the server will provide a detailed plan including museum opening hours, admission fees, and access information. Furthermore, it can also include popular local restaurants as lunch options.

[0473] Finally, once the user confirms their travel plan, the server, acting as a reservation management system, links the plan to affiliated external reservation services. This allows users to complete their travel reservations without cumbersome procedures, ensuring a smooth travel experience.

[0474] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0475] Step 1:

[0476] The user uses a terminal to enter their desired travel destination, activities of interest, duration, and budget. The terminal collects this information and converts it into a digital format. This input data is then sent to the server as foundational information necessary for the next processing step.

[0477] Step 2:

[0478] The server analyzes traveler information received from the terminal. The server utilizes a generative AI model to automatically generate travel plans that match the user's interests and budget. Specifically, the server generates prompt messages and passes them to the AI ​​model to construct a plan optimized for the user's needs. For example, using the prompt message, "The user wants to visit art museums in Tokyo. Please suggest a recommended route," the AI ​​generates a detailed sightseeing route. The output is the generated travel plan.

[0479] Step 3:

[0480] The server adds the latest data from external sources to the generated travel plan. The server retrieves opening hours, event information, and transportation information for tourist attractions and incorporates it into the travel plan. This results in a travel plan based on the most up-to-date details. The output is a refined and up-to-date travel plan.

[0481] Step 4:

[0482] The server prepares a guide avatar based on the travel plan. This avatar is configured to provide travelers with real-time explanations of tourist attractions and cultural background information through visual and audio information. This allows users to receive detailed travel guidance through their devices. The output is the avatar's guide information related to the travel plan.

[0483] Step 5:

[0484] Users view the generated travel plan and avatar-based guidance through their device. If a user wishes to modify the plan, they can send a request to the server via their device. The server then readjusts the travel plan based on this request. The input is the user's modification request, and the output is the updated travel plan.

[0485] Step 6:

[0486] Once a user confirms their travel plans, the server integrates them with a partnered booking management system. The server automatically processes the user's travel booking and performs the necessary procedures, allowing the user to easily complete their reservation. The output is confirmation of the confirmed travel booking.

[0487] (Application Example 1)

[0488] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0489] Traditional travel planning systems sometimes struggle to provide appropriate travel plans tailored to travelers' interests and budgets, and they have limited means for travelers to virtually experience tourist destinations before actually visiting. As a result, travelers may be unable to make informed decisions, leading to decreased satisfaction with their travel experience.

[0490] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0491] In this invention, the server includes means for inputting traveler information, means for generating a travel plan based on the traveler information, and means for visually presenting the travel plan to the user using virtual reality technology. This allows travelers to virtually experience tourist destinations in advance, providing a more satisfying travel experience.

[0492] "Traveler" refers to an individual or group intending to visit a tourist destination.

[0493] "Means of inputting information" refers to the interface used by travelers to input information such as places they want to visit, activities they are interested in, travel duration, and budget into a device.

[0494] "Methods for generating travel plans" refers to the process of creating efficient and personalized travel plans using a generative AI model based on information entered by travelers.

[0495] A "guide avatar" refers to a computer program that provides travelers with real-time visual and audio guidance, including explanations of tourist attractions and cultural background information.

[0496] "Virtual reality technology" refers to technology that presents a computer-generated virtual environment to the user and allows them to experience it in conjunction with the real world.

[0497] "Means of acquiring external information" refers to approaches that involve obtaining detailed information about tourist destinations, event information, and the latest information on transportation from external digital services in order to refine travel plans.

[0498] "Means of visually presenting information to the user" refers to display and projection technologies that utilize virtual reality technology to visually represent travel plans and provide them to the user.

[0499] "Methods for linking with booking procedures" refers to the process of linking the finalized travel plan with external booking services to support smooth arrangements.

[0500] This invention is implemented through a system that provides travelers with a personalized travel experience. The system consists of multiple components, which are described in detail here.

[0501] First, the user enters their travel preferences using a device. The device has the function of collecting information such as places to visit, activities of interest, travel duration, and budget through its interface, and sending it to the server. In this process, smartphones and tablets are commonly used as the device.

[0502] The server generates travel plans based on the collected information. The generative AI model used here automatically constructs optimal sightseeing routes and schedules tailored to the user's interests and budget. The AI's computational processing is based on the knowledge of professional guides, providing highly accurate plans that meet user needs. For this to work, the server needs to have high-performance processing capabilities, and cloud-based server infrastructure is primarily utilized.

[0503] Furthermore, the server acquires external information. It regularly retrieves the latest data from external digital services, such as detailed information on tourist destinations, transportation options, and event information, to further refine the generated plans. Database technology is used for data import and cross-referencing, and it is expected that NoSQL databases such as MongoDB will be primarily used.

[0504] Along with the generated travel plan, a guidance avatar is provided to enable visual and audio guidance. Using virtual reality technology, users can visually confirm their travel plan using smart glasses or a head-mounted display. The user experience is enhanced through guidance in a virtual space.

[0505] This entire process includes a mechanism to accept requests for revisions to travel plans, in case users need to review or make changes. The finalized travel plan can then be linked with an external booking service, allowing travelers to make reservations directly.

[0506] For example, if a user enters a prompt such as, "Please generate a travel plan for Tokyo that includes visiting art museums and enjoying local cuisine. The budget is 50,000 yen, and the itinerary is for two days," a travel plan including a virtual tour of Tokyo's art museums and a guide to local restaurants will be generated, and the process will include an experience through virtual reality technology.

[0507] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0508] Step 1:

[0509] The user enters travel preferences using a terminal. Information such as desired destinations, activities of interest, travel duration, and budget is collected through the input interface and sent to the server. The input data is converted to a specified format and transferred in a format ready for further processing.

[0510] Step 2:

[0511] The server generates a travel plan using a generative AI model based on the received travel preference information. Prompt messages are input to the AI, which constructs sightseeing routes and schedules tailored to the user's interests and preferences. During this process, the AI ​​model analyzes a large amount of travel data to select the most suitable plan. The output travel plan data is converted into an intermediate format to prepare for subsequent processing.

[0512] Step 3:

[0513] The server dynamically retrieves detailed information about tourist destinations from external digital services. It obtains event information and transportation data related to travel plans, and uses this information to add to and refine existing travel plans. This data from external services is received in real time via multiple APIs and integrated into the travel plan data.

[0514] Step 4:

[0515] The server prepares a guide avatar based on the updated travel plan. It generates visual and audio guidance elements and constructs data to present to the user using virtual reality technology. This includes virtual tours of specific tourist spots and explanations of their cultural background. This data is then passed to a module for visualization.

[0516] Step 5:

[0517] Users review the travel plan and avatar information generated via their device and request changes as needed. Changes made by the user are sent to the server, and the plan data is recalculated. A new travel plan reflecting the changes is generated again and finalized as the final plan.

[0518] Step 6:

[0519] The server connects confirmed travel plans to partnered external booking services. Travelers can easily book transportation and accommodations through a platform integrated with specific travel booking systems. Booking information is fed back to the user's device in real time, enabling smooth travel arrangements.

[0520] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0521] This invention is a system for providing travelers with a personalized travel experience, and is particularly characterized by its incorporation of an emotion engine that recognizes the user's emotions. The configuration for implementing this system is shown below.

[0522] First, the user uses their device to enter their travel preferences. After entering information such as the travel destination, places of interest, travel duration, and budget, the device sends this information to the server.

[0523] The server analyzes the information received from the user and uses a generative AI to generate an appropriate travel plan. The travel plan is created by leveraging the knowledge of professional guides while taking into account the user's requests and constraints.

[0524] Furthermore, this system incorporates an emotion engine that analyzes the user's facial expressions and tone of voice to determine their emotions in real time. By taking the user's satisfaction level and expectations into account, the generated travel plan becomes more personalized.

[0525] The server uses external digital services to obtain the latest tourist information, event data, and traffic information, and incorporates this into the travel plan. The plan is adjusted to take the user's emotions into account, refining the travel plan by utilizing feedback from the emotion engine.

[0526] The server then generates a guide avatar based on the travel plan, customizing the guide's style and tone according to the user's mood. The avatar visually and audibly provides detailed information and cultural background of the destinations, conveying the user's expectations for the trip.

[0527] For example, if a user expresses interest in visiting temples and wants to relax during a trip to Kyoto, the server will use an emotion engine to generate a guide avatar that conveys information in a calm tone, suggesting the optimal temple route and heartwarming lunch options to the traveler.

[0528] If the user is satisfied with the plan and ultimately approves the trip, the server connects the necessary information for the booking process to an external service, smoothly supporting the travel preparations. In this way, the present invention is a system that enables the generation and execution of travel plans centered on the user's emotions.

[0529] The following describes the processing flow.

[0530] Step 1:

[0531] The user uses a terminal to enter their travel preferences. This information includes destination, places of interest, travel duration, and budget. The terminal collects the information entered by the user and sends it to the server.

[0532] Step 2:

[0533] The server receives the submitted travel information and uses AI to generate an optimal travel plan for the user. In this process, destinations and activities are selected based on the user's preferences and requirements.

[0534] Step 3:

[0535] The emotion engine analyzes video and audio data obtained through the user's device to recognize the user's emotional state. The emotion engine also analyzes the user's facial expressions and tone of voice to measure satisfaction and interest levels.

[0536] Step 4:

[0537] The server adjusts the travel plan based on the analysis results of the emotion engine. Based on the user's emotions, it optimizes the selection of destinations and the order of activities to provide a plan that meets the user's expectations.

[0538] Step 5:

[0539] The server retrieves tourist information, the latest event data, and traffic information from external digital services and incorporates this information into travel plans. This ensures that detailed plans are always based on the most up-to-date and accurate information.

[0540] Step 6:

[0541] The server generates a guide avatar based on the travel plan. The avatar's guide style is adjusted according to the user's emotions, enabling it to effectively provide detailed information and cultural background about tourist attractions.

[0542] Step 7:

[0543] The device displays updated travel plans and information about the guide avatar to the user. The user reviews this information and, if there are any requests for corrections or additions, enters feedback and sends it from the device to the server.

[0544] Step 8:

[0545] The server receives user feedback, readjusts the travel plan as needed, and then generates the final version. Once the user approves the plan and reservations are required, the server coordinates with external booking services to complete the process. Through this process, users can confidently proceed with their travel preparations.

[0546] (Example 2)

[0547] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0548] In the field of modern travel planning, there is a demand for personalized experiences that meet the individual needs of travelers. However, conventional technologies have faced challenges in providing flexible travel plans that accommodate travelers' emotions and individual preferences. In particular, there has been a lack of technology that can grasp travelers' emotions in real time and optimize travel plans based on them. This invention aims to realize a more satisfying travel experience by providing personalized travel plans that take into account travelers' emotions and individual requests.

[0549] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0550] In this invention, the server includes means for acquiring traveler attribute information and preference information, means for creating a travel plan using a generative AI based on the acquired information, and means for analyzing the user's emotions in real time and reflecting them in the travel plan. This makes it possible to personalize and optimize travel plans to suit the traveler's emotions and individual needs.

[0551] "Traveler" refers to an individual or group that enters or obtains information for the purpose of travel.

[0552] "Attribute information" refers to basic information about travelers, including information about personal characteristics such as name, age, gender, and nationality.

[0553] "Desired information" refers to the conditions, destinations, interests, budget, and duration of travel that travelers seek in their trip, as well as other requirements necessary for travel planning.

[0554] "Generative AI" refers to a program or system that uses artificial intelligence technology to process data and automatically generate travel plans.

[0555] "Real-time analysis" refers to a technology that processes data the moment it is collected and provides results immediately.

[0556] "External information sources" refer to external databases or information services that provide necessary information that the server itself does not possess.

[0557] A "virtual character" is a character generated on a computer that serves to provide information to the user visually and audibly.

[0558] "Emotional analysis" refers to a technology that analyzes a user's facial expressions, voice tone, posture, etc., to detect their emotional state.

[0559] "Reservation procedures" refer to a series of actions taken in advance to secure transportation and accommodation based on a travel plan.

[0560] This invention is a system for providing users with a personalized travel experience. The system begins with the user inputting attribute information and preferences related to their trip using a terminal. The terminal sends the input information to a server. The server uses a generative AI model to analyze the acquired information and create an individually customized travel plan.

[0561] In this system, the server is equipped with an emotion engine that has the ability to analyze the user's facial expressions and tone of voice in real time. The results of the emotion engine's analysis are used to customize travel plans, ensuring that the most desirable experience is provided to the traveler.

[0562] Furthermore, the server collaborates with external information services to dynamically acquire the latest tourism and transportation data, integrating it into the travel plan. This information is stored in a database on the server and used to refine the plan.

[0563] Furthermore, the server generates a virtual character based on the travel plan. This character guides the traveler with information related to the plan, both visually and audibly. The guidance will reflect the user's interests, including details about tourist attractions and their cultural background.

[0564] For example, if a user wants to visit Kyoto and wants to tour temples, the server passes a prompt to the generating AI model saying, "Considering that the user is in a relaxed mood, please suggest a plan to visit quiet temples and tranquil gardens in Kyoto." This allows the server to create a travel plan, including the optimal route, and provide it to the user.

[0565] Finally, once the user approves the travel plan, the server integrates with an external booking system to automatically secure transportation and accommodation. In this way, users can proceed with travel preparations stress-free.

[0566] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0567] Step 1:

[0568] The user uses a terminal to input attribute information and preferences related to their trip. Specifically, they fill in information such as destination, travel duration, activities of interest, and budget in an input form. The terminal generates data packets to format the entered data and send it to the server. The input is information from the user, and the output is the data packets sent to the server.

[0569] Step 2:

[0570] The server analyzes travel information received from the terminal. First, it processes the data to standardize the format for storage in the database. The input is the raw data received from the terminal, and the output is data converted into a format suitable for analysis. To pass it to the generating AI model, it identifies the user's desired conditions and generates prompt statements.

[0571] Step 3:

[0572] The server creates travel plans using a generative AI model. Based on prompts, the generative AI generates a travel plan that best reflects the traveler's preferences. In this process, knowledge data from professional guides is utilized. The input consists of prompts and related database information, while the output is the generated travel plan.

[0573] Step 4:

[0574] The server activates an emotion engine to monitor the user's emotions in real time. This information is obtained by analyzing the user's camera video and voice input. The input is camera video and audio data, and the output is the analyzed emotion information. Based on this information, the travel plan is adjusted.

[0575] Step 5:

[0576] The server connects to external information sources to obtain the latest tourist information, event information, and transportation information. It retrieves information via external APIs and updates the data to improve the accuracy of the travel plan. The input is data from external information sources, and the output is the updated travel plan.

[0577] Step 6:

[0578] The server generates a virtual character based on the travel plan. The character is adapted to the user's emotions and provides visual and auditory guidance through the details of the plan. The input is the travel plan and emotional information, and the output is a user-operable guide avatar.

[0579] Step 7:

[0580] Users can review the travel plan presented through their device and request revisions as needed. The server finalizes the travel plan upon receiving user approval or revision requests. The input is user feedback, and the output is the finalized travel plan.

[0581] Step 8:

[0582] The server processes bookings in conjunction with an external booking system based on the finalized travel plan. The input is the final travel plan, and the output is booking information for secured transportation and accommodation. This process allows users to smoothly prepare for their trip.

[0583] (Application Example 2)

[0584] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0585] Modern travelers are increasingly seeking interactive travel experiences that are tailored to their individual preferences and emotions. However, traditional systems fail to adequately personalize these experiences by considering travelers' feelings, resulting in a lack of appropriate information and emotionally resonant guidance. This leads to challenges in improving travel experience satisfaction and can even decrease satisfaction with the planning and execution of trips.

[0586] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0587] In this invention, the server includes means for inputting traveler information, means for analyzing the user's emotional state, and means for providing a digital guide based on the generated travel plan. This makes it possible to provide a personalized travel experience based on the traveler's emotions.

[0588] A "means for inputting traveler information" refers to a mechanism for collecting detailed information such as travelers' personal preferences, desired destinations, duration, and budget, and using this information to plan their trips.

[0589] A "means for generating travel plans" is a mechanism that automatically creates optimal sightseeing routes and itineraries based on collected traveler information.

[0590] "Methods for analyzing the user's emotional state" refers to technology that recognizes and analyzes the user's emotions in real time from their voice and facial expressions, and reflects this in the travel plan.

[0591] "Means of providing digital guides for informational purposes" refers to technology that generates digital characters to provide travelers with visual and auditory information and background details about their destinations and to guide them.

[0592] "Methods for acquiring external information to refine travel plans" refer to the process of gathering the latest tourist information and traffic conditions from sources such as the internet, and then incorporating them into the travel plan to make it more precise.

[0593] "Means to provide to travelers" refers to technology that instantly provides and makes accessible to travelers the generated travel plans and guide information.

[0594] "Means of adjusting the tone and style of digital guides according to the traveler's emotions" refers to technology that dynamically changes the method and expression of guidance according to the user's emotional state, thereby ensuring a more personalized experience.

[0595] "Methods for integrating with booking procedures" refer to technologies that enable seamless booking of accommodations, transportation, and other services based on the finalized travel plan.

[0596] The system for realizing this invention takes user emotions into consideration and provides a personalized travel experience based on specific conditions. The system consists of the user's terminal, an emotion recognition engine, a generative AI model, and a server that interacts with an external database.

[0597] The user first uses a smart device to input basic information such as travel destination, duration, and budget. This information is sent from the device to the server. Based on this information, the server inputs prompts into an AI model to generate a travel plan.

[0598] The emotion recognition engine, for example using the Affectiva SDK, analyzes the user's voice and facial expressions through smart glasses or a smartphone to determine their emotional state in real time. The results of the emotion analysis are fed back into the user's travel plan, allowing for fine-tuning of the plan.

[0599] The travel plan includes a digital guide generated using a generative AI model, providing users with visual and auditory information about the cultural background of places and activities to do during their visit. Furthermore, the plan's accuracy is improved by incorporating up-to-date traffic and event data from external sources.

[0600] For example, if a user is planning a trip to Kyoto and expresses interest in visiting temples and wants to relax, the emotion engine will detect this state and provide a digital guide in a calm tone via the server. An example of a prompt in this case would be, "The user is currently in a relaxed mood and is looking for information on quiet temples in Kyoto. Please suggest recommended routes and details."

[0601] This system dynamically optimizes the travel experience in response to the user's emotional reactions, providing a more meaningful trip tailored to individual preferences.

[0602] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0603] Step 1:

[0604] The user uses a smart device to input their travel destination and preferences (travel duration, budget, places of interest, etc.) into the terminal. This information is processed by the terminal and sent to the server. The entered data is used by the server as basic data to generate a travel plan.

[0605] Step 2:

[0606] The server creates prompts for use in the AI ​​model based on the traveler information it receives. These prompts reflect the user's preferences and are input into the AI ​​model. The AI ​​processes these prompts and generates a basic travel plan.

[0607] Step 3:

[0608] An emotion recognition engine collects real-time emotional data from the user (analysis results of facial expressions and voice). Using a device such as smart glasses, the analyzed emotional information is sent to a server. The server uses this information to adjust the travel plan based on the user's emotional state.

[0609] Step 4:

[0610] The server retrieves the latest data from external information sources (traffic information, tourist information, event information, etc.) and integrates it into existing travel plans. This process refines and optimizes the plan.

[0611] Step 5:

[0612] The server uses a generative AI model to create and provide digital guides. The digital guides adjust their tone and style according to the user's emotions and provide traveler-related information visually and audibly.

[0613] Step 6:

[0614] The user reviews the provided travel plan and makes any necessary modifications on their device. The modifications are sent to the server, and an updated plan is generated.

[0615] Step 7:

[0616] The finalized travel plan is then linked to the booking process via the server. In this step, accommodation and transportation reservations are completed by the relevant services.

[0617] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0618] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0619] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0620] [Fourth Embodiment]

[0621] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0622] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0623] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0624] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0625] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0626] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0627] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0628] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0629] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0630] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0631] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0632] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0633] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0634] This invention is a system for providing travelers with an efficient and personalized travel experience. The following describes the configurations for implementing this system.

[0635] First, the user enters travel information via their device, such as places they want to visit, activities they are interested in, travel duration, and budget. The device collects this information and sends it to the server.

[0636] Next, the server automatically generates a travel plan based on the information received from the user. This plan generation utilizes a generation AI implemented on the server, which constructs a plan tailored to the user's preferences based on the knowledge and experience of professional guides. The AI ​​can suggest the optimal sightseeing route and schedule according to the user's interests and budget.

[0637] Regarding the generated travel plan, the server retrieves detailed information on tourist destinations, events, transportation options, etc., through external digital services to further refine the plan. This makes it possible to provide travel plans that are always based on the latest and most accurate information.

[0638] Next, the server prepares a guide avatar based on the travel plan. This avatar is responsible for providing visual and audio guidance, conveying explanations of tourist attractions and cultural background information in real time. Users can view the generated travel plan and the avatar's guidance on their device.

[0639] For example, if a user wants to sightsee in Tokyo and requests a plan that includes visiting museums and dining locally, the server will provide a plan that includes museum opening hours, admission fees, and access information. It can also include a suggestion for a popular local restaurant for lunch. If the user has any requests for additions or modifications to this plan, they can do so through their device, and the server will readjust the plan accordingly.

[0640] Finally, once the user confirms their plan, the server integrates it with partnered external booking services to support travel arrangements. This allows users to easily complete their bookings and proceed with their trip smoothly.

[0641] The following describes the processing flow.

[0642] Step 1:

[0643] The user enters their travel preferences via the device. Specifically, they enter information such as places they want to visit, activities they are interested in, length of stay, and budget. The device collects this data and sends it to the server.

[0644] Step 2:

[0645] The server receives information entered by the user and analyzes it to generate a travel plan. The AI ​​on the server creates a travel plan that meets the user's wishes, based on the knowledge and experience of professional guides. This AI suggests destinations, sightseeing routes, and schedules.

[0646] Step 3:

[0647] The server retrieves tourist destination information, event information, and transportation information from external digital services to improve the accuracy of travel plans. Based on this information, it modifies the travel plan to ensure that the plan is based on the latest data.

[0648] Step 4:

[0649] The server prepares a guide avatar based on the generated travel plan. The avatar has a script that explains detailed information and cultural background of the destinations, and is configured to guide the user visually or audibly.

[0650] Step 5:

[0651] The device displays the travel plan and avatar information received from the server to the user. The user reviews the presented plan, approves it if they are satisfied with the contents, and submits a change request if necessary.

[0652] Step 6:

[0653] If a user requests a change, the server receives the information and readjusts the plan. The server then generates a new travel plan, reflecting the changes, and provides feedback to the user.

[0654] Step 7:

[0655] The server confirms the travel plan that the user has finally approved. If reservations are required as part of the travel process, the server then connects the information to external booking services to assist with the booking arrangements. Through this process, the user completes their travel preparations.

[0656] (Example 1)

[0657] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0658] To provide travelers with effective and personalized travel experiences, it is necessary to efficiently collect and analyze a large amount of information and generate appropriate travel plans. However, conventional systems have problems with the accuracy of information and the flexibility of plans, and have been unable to fully meet the individual needs of travelers.

[0659] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0660] In this invention, the server includes means for analyzing traveler information and automatically generating a travel plan using a generative AI model; means for dynamically acquiring information from external sources and refining the travel plan; and means for presenting the generated travel plan and guidance information to the traveler and accepting modifications according to the traveler's requests. This makes it possible to provide the latest and most flexible travel plan optimized for each traveler's wishes and conditions.

[0661] A "data processing device" is an electronic device that has the function of collecting travelers' input information and transmitting it to a server.

[0662] A "generative AI model" is an artificial intelligence system used to automatically generate optimal travel plans based on traveler information.

[0663] A "guidance output device" is a medium or device for providing information visually and audibly based on a travel plan.

[0664] "External information sources" refer to databases and services that dynamically retrieve information, such as detailed information about tourist destinations and transportation information, in order to improve the accuracy of travel planning.

[0665] A "reservation management system" is a system for executing or managing reservation procedures based on a finalized travel plan.

[0666] "Traveler" refers to an individual or group that plans and executes a trip using this system.

[0667] This invention is a system for providing travelers with personalized travel experiences. Specific embodiments of the system are described below.

[0668] The user first uses a terminal to input travel information such as their desired destination, interesting sightseeing experiences, travel duration, and budget. The terminal organizes this input information and sends it to the server via a secure communication protocol. In this process, the terminal functions as a data processing device.

[0669] The server analyzes data received from the user and automatically generates a travel plan using a generative AI model. The generative AI model uses artificial intelligence technology to simulate the knowledge of a professional guide and build a customized plan tailored to the user's requests. In particular, the server can use prompts such as: "The user wants to visit art museums in Tokyo. Please suggest recommended museums and access information."

[0670] In the generated travel plan, the server dynamically retrieves the latest data on tourist destinations, event information, and transportation options from external sources to improve the accuracy of the plan. This ensures that users are always provided with accurate and up-to-date information.

[0671] Next, the server prepares an avatar as a guidance output device based on the travel plan. This avatar is designed to provide travelers with real-time explanations of tourist attractions and cultural background information through visual and auditory means. Users can view the generated travel plan and the guidance information provided by the avatar via their terminal.

[0672] For example, if a user enters "I want to visit art museums in Tokyo and enjoy local cuisine," the server will provide a detailed plan including museum opening hours, admission fees, and access information. Furthermore, it can also include popular local restaurants as lunch options.

[0673] Finally, once the user confirms their travel plan, the server, acting as a reservation management system, links the plan to affiliated external reservation services. This allows users to complete their travel reservations without cumbersome procedures, ensuring a smooth travel experience.

[0674] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0675] Step 1:

[0676] The user uses a terminal to enter their desired travel destination, activities of interest, duration, and budget. The terminal collects this information and converts it into a digital format. This input data is then sent to the server as foundational information necessary for the next processing step.

[0677] Step 2:

[0678] The server analyzes traveler information received from the terminal. The server utilizes a generative AI model to automatically generate travel plans that match the user's interests and budget. Specifically, the server generates prompt messages and passes them to the AI ​​model to construct a plan optimized for the user's needs. For example, using the prompt message, "The user wants to visit art museums in Tokyo. Please suggest a recommended route," the AI ​​generates a detailed sightseeing route. The output is the generated travel plan.

[0679] Step 3:

[0680] The server adds the latest data from external sources to the generated travel plan. The server retrieves opening hours, event information, and transportation information for tourist attractions and incorporates it into the travel plan. This results in a travel plan based on the most up-to-date details. The output is a refined and up-to-date travel plan.

[0681] Step 4:

[0682] The server prepares a guide avatar based on the travel plan. This avatar is configured to provide travelers with real-time explanations of tourist attractions and cultural background information through visual and audio information. This allows users to receive detailed travel guidance through their devices. The output is the avatar's guide information related to the travel plan.

[0683] Step 5:

[0684] Users view the generated travel plan and avatar-based guidance through their device. If a user wishes to modify the plan, they can send a request to the server via their device. The server then readjusts the travel plan based on this request. The input is the user's modification request, and the output is the updated travel plan.

[0685] Step 6:

[0686] Once a user confirms their travel plans, the server integrates them with a partnered booking management system. The server automatically processes the user's travel booking and performs the necessary procedures, allowing the user to easily complete their reservation. The output is confirmation of the confirmed travel booking.

[0687] (Application Example 1)

[0688] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0689] Traditional travel planning systems sometimes struggle to provide appropriate travel plans tailored to travelers' interests and budgets, and they have limited means for travelers to virtually experience tourist destinations before actually visiting. As a result, travelers may be unable to make informed decisions, leading to decreased satisfaction with their travel experience.

[0690] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0691] In this invention, the server includes means for inputting traveler information, means for generating a travel plan based on the traveler information, and means for visually presenting the travel plan to the user using virtual reality technology. This allows travelers to virtually experience tourist destinations in advance, providing a more satisfying travel experience.

[0692] "Traveler" refers to an individual or group intending to visit a tourist destination.

[0693] "Means of inputting information" refers to the interface used by travelers to input information such as places they want to visit, activities they are interested in, travel duration, and budget into a device.

[0694] "Methods for generating travel plans" refers to the process of creating efficient and personalized travel plans using a generative AI model based on information entered by travelers.

[0695] A "guide avatar" refers to a computer program that provides travelers with real-time visual and audio guidance, including explanations of tourist attractions and cultural background information.

[0696] "Virtual reality technology" refers to technology that presents a computer-generated virtual environment to the user and allows them to experience it in conjunction with the real world.

[0697] "Means of acquiring external information" refers to approaches that involve obtaining detailed information about tourist destinations, event information, and the latest information on transportation from external digital services in order to refine travel plans.

[0698] "Means of visually presenting information to the user" refers to display and projection technologies that utilize virtual reality technology to visually represent travel plans and provide them to the user.

[0699] "Methods for linking with booking procedures" refers to the process of linking the finalized travel plan with external booking services to support smooth arrangements.

[0700] This invention is implemented through a system that provides travelers with a personalized travel experience. The system consists of multiple components, which are described in detail here.

[0701] First, the user enters their travel preferences using a device. The device has the function of collecting information such as places to visit, activities of interest, travel duration, and budget through its interface, and sending it to the server. In this process, smartphones and tablets are commonly used as the device.

[0702] The server generates travel plans based on the collected information. The generative AI model used here automatically constructs optimal sightseeing routes and schedules tailored to the user's interests and budget. The AI's computational processing is based on the knowledge of professional guides, providing highly accurate plans that meet user needs. For this to work, the server needs to have high-performance processing capabilities, and cloud-based server infrastructure is primarily utilized.

[0703] Furthermore, the server acquires external information. It regularly retrieves the latest data from external digital services, such as detailed information on tourist destinations, transportation options, and event information, to further refine the generated plans. Database technology is used for data import and cross-referencing, and it is expected that NoSQL databases such as MongoDB will be primarily used.

[0704] Along with the generated travel plan, a guidance avatar is provided to enable visual and audio guidance. Using virtual reality technology, users can visually confirm their travel plan using smart glasses or a head-mounted display. The user experience is enhanced through guidance in a virtual space.

[0705] This entire process includes a mechanism to accept requests for revisions to travel plans, in case users need to review or make changes. The finalized travel plan can then be linked with an external booking service, allowing travelers to make reservations directly.

[0706] For example, if a user enters a prompt such as, "Please generate a travel plan for Tokyo that includes visiting art museums and enjoying local cuisine. The budget is 50,000 yen, and the itinerary is for two days," a travel plan including a virtual tour of Tokyo's art museums and a guide to local restaurants will be generated, and the process will include an experience through virtual reality technology.

[0707] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0708] Step 1:

[0709] The user enters travel preferences using a terminal. Information such as desired destinations, activities of interest, travel duration, and budget is collected through the input interface and sent to the server. The input data is converted to a specified format and transferred in a format ready for further processing.

[0710] Step 2:

[0711] The server generates a travel plan using a generative AI model based on the received travel preference information. Prompt messages are input to the AI, which constructs sightseeing routes and schedules tailored to the user's interests and preferences. During this process, the AI ​​model analyzes a large amount of travel data to select the most suitable plan. The output travel plan data is converted into an intermediate format to prepare for subsequent processing.

[0712] Step 3:

[0713] The server dynamically retrieves detailed information about tourist destinations from external digital services. It obtains event information and transportation data related to travel plans, and uses this information to add to and refine existing travel plans. This data from external services is received in real time via multiple APIs and integrated into the travel plan data.

[0714] Step 4:

[0715] The server prepares a guide avatar based on the updated travel plan. It generates visual and audio guidance elements and constructs data to present to the user using virtual reality technology. This includes virtual tours of specific tourist spots and explanations of their cultural background. This data is then passed to a module for visualization.

[0716] Step 5:

[0717] Users review the travel plan and avatar information generated via their device and request changes as needed. Changes made by the user are sent to the server, and the plan data is recalculated. A new travel plan reflecting the changes is generated again and finalized as the final plan.

[0718] Step 6:

[0719] The server connects confirmed travel plans to partnered external booking services. Travelers can easily book transportation and accommodations through a platform integrated with specific travel booking systems. Booking information is fed back to the user's device in real time, enabling smooth travel arrangements.

[0720] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0721] This invention is a system for providing travelers with a personalized travel experience, and is particularly characterized by its incorporation of an emotion engine that recognizes the user's emotions. The configuration for implementing this system is shown below.

[0722] First, the user uses their device to enter their travel preferences. After entering information such as the travel destination, places of interest, travel duration, and budget, the device sends this information to the server.

[0723] The server analyzes the information received from the user and uses a generative AI to generate an appropriate travel plan. The travel plan is created by leveraging the knowledge of professional guides while taking into account the user's requests and constraints.

[0724] Furthermore, this system incorporates an emotion engine that analyzes the user's facial expressions and tone of voice to determine their emotions in real time. By taking the user's satisfaction level and expectations into account, the generated travel plan becomes more personalized.

[0725] The server uses external digital services to obtain the latest tourist information, event data, and traffic information, and incorporates this into the travel plan. The plan is adjusted to take the user's emotions into account, refining the travel plan by utilizing feedback from the emotion engine.

[0726] The server then generates a guide avatar based on the travel plan, customizing the guide's style and tone according to the user's mood. The avatar visually and audibly provides detailed information and cultural background of the destinations, conveying the user's expectations for the trip.

[0727] For example, if a user expresses interest in visiting temples and wants to relax during their trip to Kyoto, the server will use an emotion engine to generate a guide avatar that conveys information in a calm tone, suggesting the optimal temple route and heartwarming lunch options to the traveler.

[0728] If the user is satisfied with the plan and ultimately approves the trip, the server connects the necessary information for the booking process to an external service, smoothly supporting the travel preparations. In this way, the present invention is a system that enables the generation and execution of travel plans centered on the user's emotions.

[0729] The following describes the processing flow.

[0730] Step 1:

[0731] The user uses a device to enter their travel preferences. This information includes destination, places of interest, travel duration, and budget. The device collects the information entered by the user and sends it to the server.

[0732] Step 2:

[0733] The server receives the submitted travel information and uses AI to generate an optimal travel plan for the user. In this process, destinations and activities are selected based on the user's preferences and requirements.

[0734] Step 3:

[0735] The emotion engine analyzes video and audio data obtained through the user's device to recognize the user's emotional state. The emotion engine also analyzes the user's facial expressions and tone of voice to measure satisfaction and interest levels.

[0736] Step 4:

[0737] The server adjusts the travel plan based on the analysis results of the emotion engine. Based on the user's emotions, it optimizes the selection of destinations and the order of activities to provide a plan that meets the user's expectations.

[0738] Step 5:

[0739] The server retrieves tourist information, the latest event data, and traffic information from external digital services and incorporates this information into travel plans. This ensures that detailed plans are always based on the most up-to-date and accurate information.

[0740] Step 6:

[0741] The server generates a guide avatar based on the travel plan. The avatar's guide style is adjusted according to the user's emotions, enabling it to effectively provide detailed information and cultural background about tourist attractions.

[0742] Step 7:

[0743] The device displays updated travel plans and information about the guide avatar to the user. The user reviews this information and, if there are any requests for corrections or additions, enters feedback and sends it from the device to the server.

[0744] Step 8:

[0745] The server receives user feedback, readjusts the travel plan as needed, and then generates the final version. Once the user approves the plan and reservations are required, the server coordinates with external booking services to complete the process. Through this process, users can confidently proceed with their travel preparations.

[0746] (Example 2)

[0747] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0748] In the field of modern travel planning, there is a demand for personalized experiences that meet the individual needs of travelers. However, conventional technologies have faced challenges in providing flexible travel plans that accommodate travelers' emotions and individual preferences. In particular, there has been a lack of technology that can grasp travelers' emotions in real time and optimize travel plans based on them. This invention aims to realize a more satisfying travel experience by providing personalized travel plans that take into account travelers' emotions and individual requests.

[0749] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0750] In this invention, the server includes means for acquiring traveler attribute information and preference information, means for creating a travel plan using a generative AI based on the acquired information, and means for analyzing the user's emotions in real time and reflecting them in the travel plan. This makes it possible to personalize and optimize travel plans to suit the traveler's emotions and individual needs.

[0751] "Traveler" refers to an individual or group that enters or obtains information for the purpose of travel.

[0752] "Attribute information" refers to basic information about travelers, including information about personal characteristics such as name, age, gender, and nationality.

[0753] "Desired information" refers to the conditions, destinations, interests, budget, and duration of travel that travelers seek in their trip, as well as other requirements necessary for travel planning.

[0754] "Generative AI" refers to a program or system that uses artificial intelligence technology to process data and automatically generate travel plans.

[0755] "Real-time analysis" refers to a technology that processes data the moment it is collected and provides results immediately.

[0756] "External information sources" refer to external databases or information services that provide necessary information that the server itself does not possess.

[0757] A "virtual character" is a character generated on a computer that serves to provide information to the user visually and audibly.

[0758] "Emotional analysis" refers to a technology that analyzes a user's facial expressions, voice tone, posture, etc., to detect their emotional state.

[0759] "Reservation procedures" refer to a series of actions taken in advance to secure transportation and accommodation based on a travel plan.

[0760] This invention is a system for providing users with a personalized travel experience. The system begins with the user inputting attribute information and preferences related to their trip using a terminal. The terminal sends the input information to a server. The server uses a generative AI model to analyze the acquired information and create an individually customized travel plan.

[0761] In this system, the server is equipped with an emotion engine that has the ability to analyze the user's facial expressions and tone of voice in real time. The results of the emotion engine's analysis are used to customize travel plans, ensuring that the most desirable experience is provided to the traveler.

[0762] Furthermore, the server collaborates with external information services to dynamically acquire the latest tourism and transportation data, integrating it into the travel plan. This information is stored in a database on the server and used to refine the plan.

[0763] Furthermore, the server generates a virtual character based on the travel plan. This character guides the traveler with information related to the plan, both visually and audibly. The guidance will reflect the user's interests, including details about tourist attractions and their cultural background.

[0764] For example, if a user wants to visit Kyoto and wants to tour temples, the server passes a prompt to the generating AI model saying, "Considering that the user is in a relaxed mood, please suggest a plan to visit quiet temples and tranquil gardens in Kyoto." This allows the server to create a travel plan, including the optimal route, and provide it to the user.

[0765] Finally, once the user approves the travel plan, the server integrates with an external booking system to automatically secure transportation and accommodation. In this way, users can proceed with travel preparations stress-free.

[0766] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0767] Step 1:

[0768] The user uses a terminal to input attribute information and preferences related to their trip. Specifically, they fill in information such as destination, travel duration, activities of interest, and budget in an input form. The terminal generates data packets to format the entered data and send it to the server. The input is information from the user, and the output is the data packets sent to the server.

[0769] Step 2:

[0770] The server analyzes travel information received from the terminal. First, it processes the data to standardize the format for storage in the database. The input is the raw data received from the terminal, and the output is data converted into a format suitable for analysis. To pass it to the generating AI model, it identifies the user's desired conditions and generates prompt statements.

[0771] Step 3:

[0772] The server creates travel plans using a generative AI model. Based on prompts, the generative AI generates a travel plan that best reflects the traveler's preferences. In this process, knowledge data from professional guides is utilized. The input consists of prompts and related database information, while the output is the generated travel plan.

[0773] Step 4:

[0774] The server activates an emotion engine to monitor the user's emotions in real time. This information is obtained by analyzing the user's camera video and voice input. The input is camera video and audio data, and the output is the analyzed emotion information. Based on this information, the travel plan is adjusted.

[0775] Step 5:

[0776] The server connects to external information sources to obtain the latest tourist information, event information, and transportation information. It retrieves information via external APIs and updates the data to improve the accuracy of the travel plan. The input is data from external information sources, and the output is the updated travel plan.

[0777] Step 6:

[0778] The server generates a virtual character based on the travel plan. The character is adapted to the user's emotions and provides visual and auditory guidance through the details of the plan. The input is the travel plan and emotional information, and the output is a user-operable guide avatar.

[0779] Step 7:

[0780] Users can review the travel plan presented through their device and request revisions as needed. The server finalizes the travel plan upon receiving user approval or revision requests. The input is user feedback, and the output is the finalized travel plan.

[0781] Step 8:

[0782] The server processes bookings in conjunction with an external booking system based on the finalized travel plan. The input is the final travel plan, and the output is booking information for secured transportation and accommodation. This process allows users to smoothly prepare for their trip.

[0783] (Application Example 2)

[0784] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0785] Modern travelers are increasingly seeking interactive travel experiences that are tailored to their individual preferences and emotions. However, traditional systems fail to adequately personalize these experiences by considering travelers' feelings, resulting in a lack of appropriate information and emotionally resonant guidance. This leads to challenges in improving travel experience satisfaction and can even decrease satisfaction with the planning and execution of trips.

[0786] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0787] In this invention, the server includes means for inputting traveler information, means for analyzing the user's emotional state, and means for providing a digital guide based on the generated travel plan. This makes it possible to provide a personalized travel experience based on the traveler's emotions.

[0788] A "means for inputting traveler information" refers to a mechanism for collecting detailed information such as travelers' personal preferences, desired destinations, duration, and budget, and using this information to plan their trips.

[0789] A "means for generating travel plans" is a mechanism that automatically creates optimal sightseeing routes and itineraries based on collected traveler information.

[0790] "Methods for analyzing the user's emotional state" refers to technology that recognizes and analyzes the user's emotions in real time from their voice and facial expressions, and reflects this in the travel plan.

[0791] "Means of providing digital guides for informational purposes" refers to technology that generates digital characters to provide travelers with visual and auditory information and background details about their destinations and to guide them.

[0792] "Methods for acquiring external information to refine travel plans" refer to the process of gathering the latest tourist information and traffic conditions from sources such as the internet, and then incorporating them into the travel plan to make it more precise.

[0793] "Means to provide to travelers" refers to technology that instantly provides and makes accessible to travelers the generated travel plans and guide information.

[0794] "Means of adjusting the tone and style of digital guides according to the traveler's emotions" refers to technology that dynamically changes the method and expression of guidance according to the user's emotional state, thereby ensuring a more personalized experience.

[0795] "Methods for integrating with booking procedures" refer to technologies that enable seamless booking of accommodations, transportation, and other services based on the finalized travel plan.

[0796] The system for realizing this invention takes user emotions into consideration and provides a personalized travel experience based on specific conditions. The system consists of the user's terminal, an emotion recognition engine, a generative AI model, and a server that interacts with an external database.

[0797] The user first uses a smart device to input basic information such as travel destination, duration, and budget. This information is sent from the device to the server. Based on this information, the server inputs prompts into an AI model to generate a travel plan.

[0798] The emotion recognition engine, for example using the Affectiva SDK, analyzes the user's voice and facial expressions through smart glasses or a smartphone to determine their emotional state in real time. The results of the emotion analysis are fed back into the user's travel plan, allowing for fine-tuning of the plan.

[0799] The travel plan includes a digital guide generated using a generative AI model, providing users with visual and auditory information about the cultural background of places and activities to do during their visit. Furthermore, the plan's accuracy is improved by incorporating up-to-date traffic and event data from external sources.

[0800] For example, if a user is planning a trip to Kyoto and expresses interest in visiting temples and wants to relax, the emotion engine will detect this state and provide a digital guide in a calm tone via the server. An example of a prompt in this case would be, "The user is currently in a relaxed mood and is looking for information on quiet temples in Kyoto. Please suggest recommended routes and details."

[0801] This system dynamically optimizes the travel experience in response to the user's emotional reactions, providing a more meaningful trip tailored to individual preferences.

[0802] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0803] Step 1:

[0804] The user uses a smart device to input their travel destination and preferences (travel duration, budget, places of interest, etc.) into the terminal. This information is processed by the terminal and sent to the server. The entered data is used by the server as basic data to generate a travel plan.

[0805] Step 2:

[0806] The server creates prompts for use in the AI ​​model based on the traveler information it receives. These prompts reflect the user's preferences and are input into the AI ​​model. The AI ​​processes these prompts and generates a basic travel plan.

[0807] Step 3:

[0808] An emotion recognition engine collects real-time emotional data from the user (analysis results of facial expressions and voice). Using a device such as smart glasses, the analyzed emotional information is sent to a server. The server uses this information to adjust the travel plan based on the user's emotional state.

[0809] Step 4:

[0810] The server retrieves the latest data from external information sources (traffic information, tourist information, event information, etc.) and integrates it into existing travel plans. This process refines and optimizes the plan.

[0811] Step 5:

[0812] The server uses a generative AI model to create and provide digital guides. The digital guides adjust their tone and style according to the user's emotions and provide traveler-related information visually and audibly.

[0813] Step 6:

[0814] The user reviews the provided travel plan and makes any necessary modifications on their device. The modifications are sent to the server, and an updated plan is generated.

[0815] Step 7:

[0816] The finalized travel plan is then linked to the booking process via the server. In this step, accommodation and transportation reservations are completed by the relevant services.

[0817] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0818] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0819] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0820] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0821] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0822] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0823] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0824] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0825] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0826] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0827] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0828] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0829] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0830] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0831] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0832] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0833] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0834] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0835] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0836] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0837] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0838] The following is further disclosed regarding the embodiments described above.

[0839] (Claim 1)

[0840] A means of entering traveler information,

[0841] Means for generating a travel plan based on the information of the aforementioned traveler,

[0842] A means of providing a guide avatar based on the generated travel plan,

[0843] A means of acquiring external information to refine travel plans,

[0844] A means of providing travelers with information on the generated travel plan and the guide avatar,

[0845] A means for travelers to confirm and modify their plans,

[0846] A means of linking the finalized travel plan to the booking process,

[0847] A system that includes this.

[0848] (Claim 2)

[0849] The system according to claim 1, wherein a guide avatar is used to generate a script that includes details of tourist destinations, itinerary guidance, and cultural background information.

[0850] (Claim 3)

[0851] The system according to claim 1, which improves the accuracy of a travel plan by dynamically acquiring external information in the generation of a travel plan.

[0852] "Example 1"

[0853] (Claim 1)

[0854] A means of inputting traveler information via a data processing device,

[0855] A means for analyzing the traveler's information and automatically generating a travel plan using a generative AI model,

[0856] Means for preparing a guidance output device that provides audio and visual information based on a generated travel plan,

[0857] A means of dynamically acquiring information from external sources and refining travel plans,

[0858] A means of presenting the generated travel plan and information to the traveler and accepting modifications upon the traveler's request,

[0859] A means of linking the finalized travel plan to the reservation management system,

[0860] A system that includes this.

[0861] (Claim 2)

[0862] The system according to claim 1, wherein the guidance output device is used to generate guidance information including location information, itinerary guidance, and cultural background.

[0863] (Claim 3)

[0864] The system according to claim 1, which dynamically acquires external information to improve accuracy in the process of generating a travel plan.

[0865] "Application Example 1"

[0866] (Claim 1)

[0867] A means of entering traveler information,

[0868] Means for generating a travel plan based on the information of the aforementioned traveler,

[0869] A means of providing a guide avatar based on the generated travel plan,

[0870] A means of acquiring external information to refine travel plans,

[0871] A means of providing travelers with information on the generated travel plan and the guide avatar,

[0872] A means of visually presenting travel plans to users using virtual reality technology,

[0873] A means for travelers to confirm and modify their plans,

[0874] A means of linking the finalized travel plan to the booking process,

[0875] A system that includes this.

[0876] (Claim 2)

[0877] The system according to claim 1, wherein a guide avatar is used to generate a script that includes details of tourist destinations, itinerary guidance, and cultural background information.

[0878] (Claim 3)

[0879] The system according to claim 1, which improves the accuracy of a travel plan by dynamically acquiring external information in the generation of a travel plan.

[0880] "Example 2 of combining an emotion engine"

[0881] (Claim 1)

[0882] Means for obtaining traveler attribute information and preference information,

[0883] A means of creating a travel plan using a generation AI based on acquired information,

[0884] A means to analyze user emotions in real time and reflect them in travel planning,

[0885] A means of incorporating the latest travel-related data obtained from external sources into the above travel plan,

[0886] A means for generating a virtual character that provides visual and audio guidance based on the above travel plan,

[0887] A means of presenting travel plans to travelers and accepting their approval or modification,

[0888] A method of integrating booking procedures with an external system based on an approved travel plan,

[0889] A system that includes this.

[0890] (Claim 2)

[0891] The system according to claim 1, which adjusts the guidance style of a virtual character based on the traveler's emotional analysis and provides information optimized for the user.

[0892] (Claim 3)

[0893] The system according to claim 1, which dynamically acquires external information including information on airline tickets, accommodations, and tourist destinations, thereby improving the precision of travel planning.

[0894] "Application example 2 when combining with an emotional engine"

[0895] (Claim 1)

[0896] A means of entering traveler information,

[0897] Means for generating a travel plan based on the information of the aforementioned traveler,

[0898] A means of analyzing the user's emotional state,

[0899] A means of providing a digital guide for guidance based on the generated travel plan,

[0900] A means of acquiring external information to refine travel plans,

[0901] A means of providing travelers with information on generated travel plans and digital guides,

[0902] A means for travelers to confirm and modify their plans,

[0903] A means of adjusting the tone and style of the digital guide to suit the emotions of travelers,

[0904] A means of linking the finalized travel plan to the booking process,

[0905] A system that includes this.

[0906] (Claim 2)

[0907] The system according to claim 1, wherein a digital guide for informational purposes is used to generate a script that includes details of tourist attractions, itinerary information, and cultural background information.

[0908] (Claim 3)

[0909] The system according to claim 1, which improves the accuracy of a travel plan by dynamically acquiring external information in the generation of a travel plan. [Explanation of Symbols]

[0910] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of entering traveler information, Means for generating a travel plan based on the information of the aforementioned traveler, A means of providing a guide avatar based on the generated travel plan, A means of acquiring external information to refine travel plans, A means of providing travelers with information on the generated travel plan and the guide avatar, A means for travelers to confirm and modify their plans, A means of linking the finalized travel plan to the booking process, A system that includes this.

2. The system according to claim 1, wherein a guide avatar is used to generate a script that includes details of tourist destinations, itinerary guidance, and cultural background information.

3. The system according to claim 1, which improves the accuracy of a travel plan by dynamically acquiring external information in the generation of a travel plan.

Images