system

Abstract

We provide the system. [Solution] A means for users to input their personal interests and travel preferences using a device, A means by which a server receives and analyzes information obtained from the terminal and generates a travel plan using a generative model, A means of presenting a generated travel plan to the user using virtual guidance technology, A means of adjusting travel plans based on user feedback, A means of providing information related to the adjusted travel plan in conjunction with external data services, A system that includes this.

Description

Technical Field

[0004]

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When foreign tourists plan a trip, there is a problem that it takes a lot of time and effort to make a travel plan that suits their interests. Gathering detailed information for each tourist destination and planning according to individual tourist requirements requires specialized knowledge, which is a great burden for busy users. Also, it may not be possible to obtain local information in real time during the trip, and the tourist experience may be restricted.

Means for Solving the Problems

[0006] "Users" refer to tourists visiting Japan who use the system to plan their trips.

[0007] A "device" is a device used by a user to input information, and includes smartphones, tablets, and personal computers.

[0008] A "server" refers to a central system that receives and analyzes data sent by users and generates travel plans using a generative model.

[0009] A "generative model" refers to an artificial intelligence model that generates travel plans based on user input information and utilizes the knowledge of professional travel guides.

[0010] A "travel plan" refers to a detailed travel plan that includes suggested destinations, activities, and accommodation options based on the user's interests and travel preferences.

[0011] "Virtual guidance technology" refers to technology that presents a generated travel plan to the user through visual and audio means.

[0012] "Feedback" refers to the opinions and requests that users make regarding the travel plan presented.

[0013] "External data services" refer to external information providers with whom you collaborate to provide up-to-date and detailed information related to your travel plans. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by a processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is implemented as a system for providing personalized travel plans to tourists visiting Japan. In this embodiment of the system, the user, terminal, and server cooperate with each other to efficiently construct the travel plan.

[0036] Users input information about tourist destinations and experiences they are interested in using devices such as smartphones or personal computers. This input information specifically includes places they want to visit, the purpose of their trip, their budget, and desired activities. The device organizes the entered information and sends it to the server in an appropriate data format.

[0037] Next, the server receives this information and begins the process of generating a travel plan. Based on the specified conditions, the server uses a generative model to create a list of optimal tourist destinations and activities. The generative model is trained on the knowledge of professional travel guides and can propose a travel plan that best suits the user's needs. Furthermore, this also includes suggestions for accommodations and transportation.

[0038] The generated travel plan is presented to the user through virtual navigation technology. Specifically, it uses a combination of audio guides and visuals to help the user easily understand the plan. The user can review this plan and request changes if necessary.

[0039] The server, upon receiving user feedback, adjusts the travel plan. This involves replanning to better meet the user's preferences. The server also works with external data services to dynamically reflect the latest tourist information and tour options as new information becomes available.

[0040] For example, if a user enters a desire to "experience traditional Japanese culture," the server will generate a travel plan that includes historical sites in Kyoto and Nara. For instance, it might suggest activities such as renting a kimono or participating in a traditional tea ceremony. This plan is then explained through virtual navigation technology, along with a visual representation of the itinerary.

[0041] In this way, the present invention is a system that improves the travel experience of tourists visiting Japan and reduces the time burden in planning their trips.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The user launches a dedicated application on their device and enters travel information such as the purpose of the trip, budget, desired activities, and places they want to visit. The device then organizes this information and prepares to communicate with the server.

[0045] Step 2:

[0046] The terminal sends the entered user information to the server. The transmitted information is formatted according to a specified data format to ensure efficient transmission.

[0047] Step 3:

[0048] The server receives user information from the terminal and begins data analysis. Based on the received information, the server performs analysis to understand the user's preferences and travel conditions. This establishes the requirements necessary for formulating a travel plan.

[0049] Step 4:

[0050] The server activates a generative model and uses the analyzed user information to generate a travel plan that includes appropriate tourist destinations and activities. The generative model leverages the knowledge of professional travel guides that it has learned in advance to suggest the optimal plan.

[0051] Step 5:

[0052] The server prepares to present the generated travel plan to the user using virtual navigation technology. The plan includes planned destinations, schedules, activities, and related information.

[0053] Step 6:

[0054] The terminal uses virtual navigation technology to present travel plans to users visually and audibly. The navigation content is designed to help users easily visualize the trip.

[0055] Step 7:

[0056] The user reviews the presented travel plan and sends any necessary changes or feedback from their device to the server. User feedback is a crucial element in further tailoring the plan to the user.

[0057] Step 8:

[0058] The server receives user feedback and regenerates or adjusts the travel plan. If necessary, it retrieves the latest information from external data services and incorporates it into the plan.

[0059] Step 9:

[0060] The server finalizes the travel plan and provides it to the user via the terminal. The process ends when the user confirms that the finalized plan meets their expectations.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] There is a need to efficiently and effectively provide personalized travel plans to tourists visiting Japan. Traditional guidebooks and fixed tour plans fail to meet the diverse needs of individual tourists, resulting in a decline in the quality of their travel experience. Furthermore, it is difficult to change travel plans or incorporate new information, leading to a significant burden of planning time and effort.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for the user to input their personal interests and travel requests using an information processing device, means for a central processing device to receive and analyze the information obtained from the information processing device and generate a travel plan using a generation AI model, and means for presenting the generated travel plan to the user using virtual guidance technology. This makes it possible to provide flexible and personalized travel plans that meet individual needs.

[0066] "Users" refers to travelers and tourists who use the system, and means individuals with specific travel preferences.

[0067] An "information processing device" is a device that receives input information from a user, and examples include smartphones and personal computers.

[0068] A "central processing unit" is a computer system that receives data from information processing units and processes it, and is responsible for generating travel plans that meet the user's requests.

[0069] A "generative AI model" refers to an algorithm that has been pre-trained with travel-related knowledge and is used to propose the optimal travel plan based on received data.

[0070] "Virtual guidance technology" is a technology that presents generated travel plans to users visually and audibly, and is a means of helping users easily understand the contents of the plan.

[0071] "External information provision services" refer to information sources that allow the central processing unit to acquire tourism information and the latest data from external sources and incorporate them into travel plans.

[0072] This invention is a system that provides personalized travel plans to tourists visiting Japan. This system is realized through the mutual cooperation of users, terminals, and servers.

[0073] Users input their travel preferences using devices such as smartphones or computers. This input includes places they want to visit, the purpose of their trip, their budget, and desired activities. Specifically, users input prompts such as "I want to visit art museums in Tokyo" or "I would like to experience a traditional tea ceremony in Kyoto."

[0074] The terminal organizes the input information, converts it into a communication format, and sends it to the server. The communication protocols and data formats used in this process are optimized to meet the system's needs.

[0075] Upon receiving data from a terminal, the server generates a travel plan using a generative AI model. This model not only lists the most suitable tourist destinations and activities based on the user's preferences, but also suggests accommodations and transportation options. The generative AI model is trained on historical travel data and the knowledge of professional travel guides.

[0076] The generated travel plan is presented to the user using virtual navigation technology. This technology combines audio output and visual display to make the plan intuitively understandable to the user. This information presentation not only helps the user understand the plan but also encourages them to further adjust travel details and provide feedback.

[0077] Therefore, the present invention aims to provide flexible travel plans that meet the diverse needs of tourists visiting Japan and to improve the quality of their travel.

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] Users input information about tourist destinations they want to visit or activities they want to experience using devices such as smartphones or computers. The input is in text format, creating prompt statements such as "I want to visit art museums in Tokyo." This becomes the input data for the system.

[0081] Step 2:

[0082] The terminal collects data entered by the user and organizes it according to its internally configured data format. During this process, the terminal converts the data into formats such as JSON to efficiently transmit it to the server. The converted data then becomes the input sent to the server.

[0083] Step 3:

[0084] The server receives data sent from the terminal and begins analysis. The server activates a generative AI model and generates a list of tourist destinations and activities based on the user's preferences. Through data analysis and model output, a travel plan tailored to the user's needs is generated.

[0085] Step 4:

[0086] The server inputs the generated travel plan into virtual navigation technology and converts it into a presentation format for the user. This conversion process combines audio guidance and visual displays to present the generated plan in an easily understandable way for the user. This information presentation becomes the output to the user.

[0087] Step 5:

[0088] The user reviews the proposed travel plan and submits feedback to request changes as needed. This feedback includes adjustments to the timing of visits and any additional requests, which then become new input data for the server.

[0089] Step 6:

[0090] The server analyzes user feedback and readjusts the travel plan. If necessary, it gathers the latest information from external information services and incorporates it into the plan. This generates updated options to provide users with a better travel plan.

[0091] (Application Example 1)

[0092] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0093] There is a need to efficiently and effectively provide personalized travel plans tailored to the individual interests and preferences of tourists visiting Japan. However, conventional systems provide tourist information unilaterally, making it difficult to consider specific experiences in advance. Furthermore, it has been difficult for users to provide feedback and for the results to be easily reflected. This invention aims to solve these problems and provide a system to improve the quality of the travel experience.

[0094] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0095] In this invention, the server includes means for a user to input their personal interests and travel preferences using an information terminal, means for a data processing device to receive and analyze the information obtained from the information terminal and generate a travel plan using generation technology, and means for presenting the generated travel plan to the user using three-dimensional virtual space technology. This makes it possible to provide a personalized travel plan based on the user's intentions and to virtually experience that plan.

[0096] An "information terminal" is a device used by a user to input their personal interests and travel preferences.

[0097] A "data processing device" is a server device that analyzes data received from an information terminal and generates a travel plan using generation technology.

[0098] "Generative technology" refers to algorithms and models used to create optimal travel plans based on user input.

[0099] "Three-dimensional virtual space technology" is a technology for presenting generated travel plans to users visually and interactively.

[0100] A "voice generation engine" is a technology that provides users with voice guidance on travel plans.

[0101] "Visual display technology" refers to technology used to visually display scenes and information related to travel planning.

[0102] "External information services" refer to external data provision services that are partnered with to provide the latest tourist information and options.

[0103] "Evaluation information" refers to data collected from user feedback and opinions, used to adjust and regenerate travel plans.

[0104] The system for realizing this application example is configured as follows: The user inputs their interests and travel preferences using an information terminal. This terminal could be a smartphone or a personal computer. After this input is complete, the information is received and analyzed by a data processing device (server). This analysis includes generating a travel plan using a generative AI model, automatically creating a travel plan optimized for the user's conditions.

[0105] The generated travel plan is presented to the user using three-dimensional virtual space technology. For this purpose, hardware such as VR goggles and smartphones are used. The software utilizes Unity 3D to build the virtual reality environment and generates the plan using OpenAI® APIs. Information on restaurants, accommodations, and tourist attractions is kept up-to-date by linking with external information services. Furthermore, Google® Cloud's Text-to-Speech API is used as the speech generation engine, allowing the travel plan to be explained in audio.

[0106] Users can review the provided travel plans and experience them in a virtual space beforehand. Wearing VR goggles, users can walk around the virtual tourist destinations unfolding before their eyes, experiencing the actual plan. This allows users to concretely imagine the atmosphere and activities of the places they want to visit before traveling.

[0107] For example, if a prompt such as "Please tell me about beautiful places to see autumn foliage in Japan and their highlights" is input into the AI ​​model, the system will suggest the best autumn foliage spots and related activities for the user. This allows tourists visiting Japan to obtain valuable advance information to fully enjoy the charms of Japan.

[0108] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0109] Step 1:

[0110] Users use an information terminal to input their interests and travel preferences. This input includes information such as desired destinations, budget, and activities of interest. The input data is organized within the terminal and converted into a format that can be sent to the server.

[0111] Step 2:

[0112] The server receives user information sent from the terminal. Based on this information, it performs data analysis and generates a travel plan using a generative AI model. At this stage, tourist destinations, accommodations, and transportation methods that meet the user's conditions are optimized and output as a specific plan.

[0113] Step 3:

[0114] The server presents the generated travel plan to the user using three-dimensional virtual space technology. Unity 3D is used to visually represent the information, outputting the travel plan as detailed visual information. Google Cloud's Text-to-Speech API is used for audio guidance, explaining the plan to the user verbally.

[0115] Step 4:

[0116] Users virtually experience a travel plan presented within a virtual space. This experience includes virtual sightseeing and pre-exploration of activities through VR goggles. The information users gather through this experience will be used for future feedback.

[0117] Step 5:

[0118] Users provide feedback based on their experience in the virtual space. This feedback is sent to the server and used as data to readjust the plan. Based on the received evaluation information, the server updates the travel plan as needed and proposes a new plan.

[0119] Step 6:

[0120] The server connects with external information services to retrieve the latest tourist information relevant to the adjusted travel plan. This information includes the latest event information and newly added activity options. The retrieved data is then provided to the user again, completing the final travel plan.

[0121] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0122] This invention provides a system that offers personalized travel plans to tourists visiting Japan, achieving advanced planning that takes user emotions into account. This system functions through the collaboration of a terminal, server, emotion engine, generative model, and virtual guidance technology.

[0123] Users use the device to input their travel preferences, such as personal interests, places they want to visit, and their budget. The device creates a user profile based on this information and sends it to the server. User feedback and emotions expressed during the input process are sent to the emotion engine via sensors and software on the device.

[0124] The server receives and analyzes information sent by the user. The generative model generates an optimal travel plan based on this analyzed information and the user's emotions recognized by the emotion engine. The generated plan includes destinations, activity selections, and itinerary optimization, with adjustments made according to the user's emotional state.

[0125] The emotion engine analyzes user emotions based on their input and the context of the conversation, and feeds the results back to the server. This feedback is used to adjust the travel plan; for example, if the user is feeling anxious, suggestions for more relaxing plans or increased support will be provided.

[0126] The generated travel plan is presented to the user using virtual navigation technology. The device visually explains the plan to the user through audio and video. This presentation is designed to allow the user to easily understand the plan and proceed with their trip with confidence.

[0127] For example, if a user enters "I would like a relaxing hot spring trip," the server will suggest hot spring resorts and relaxing activities. If the emotion engine recognizes the user's feelings of joy or excitement, surprise elements for further relaxation will be added to the plan. In this way, a travel plan tailored to the user's emotions is formed and provided to the user.

[0128] Through this system, users can enjoy a travel experience that takes their emotions into consideration, reducing the time burden of planning and enabling comfortable and satisfying sightseeing.

[0129] The following describes the processing flow.

[0130] Step 1:

[0131] The user uses a device to input information such as the purpose of their trip, their budget, the tourist destinations they want to visit, and the activities they want to experience. The device then neatly organizes this information and prepares it as data to be sent to the server.

[0132] Step 2:

[0133] The terminal sends the entered data to the server. The information is transmitted using a secure communication protocol, minimizing time delays.

[0134] Step 3:

[0135] The server receives information from the terminal and analyzes it by comparing it with past data in the database. The information is then organized, taking into account user preferences, past travel templates, and general tourism trends.

[0136] Step 4:

[0137] The server uses the analysis results to launch a generative model, which then provides suggestions based on the user's personal data and travel data. The generated travel plan includes details of planned destinations and specific activities, and is designed to best meet the user's preferences.

[0138] Step 5:

[0139] The emotion engine operates using sensors and analysis software within the device to capture emotional data that emerges during user input and responses. This includes facial expressions, voice tone, input speed, and patterns.

[0140] Step 6:

[0141] The emotion engine analyzes the data it acquires to recognize the user's emotions. For example, it determines whether the user is feeling stressed, anxious, or relaxed, and sends the result to the server.

[0142] Step 7:

[0143] The server receives feedback from the emotion engine and adjusts the travel plan. Depending on the recognized emotions, suggestions to enhance relaxation and support options to provide reassurance are added to the plan.

[0144] Step 8:

[0145] The server sends the finalized travel plan to the terminal, which then uses virtual navigation technology to present the plan to the user via audio and video. The user then has the opportunity to review the plan and provide further feedback.

[0146] Step 9:

[0147] The user reviews the plan and gives final approval. They can request further changes if necessary, but the plan is usually finalized at this stage. Once the plan is finalized, the user can begin preparing for their trip.

[0148] (Example 2)

[0149] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0150] In recent years, there has been a growing demand for personalized travel planning that better suits individual needs and circumstances. However, traditional travel planning systems have struggled to generate and adjust plans that reflect user emotions, failing to improve the quality of the user experience. As a result, users may be dissatisfied with their travel plans.

[0151] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0152] In this invention, the server includes means for the user to input personal interests, travel preferences, and emotional data using a terminal; means for the terminal to transmit the user's emotions to an emotion engine via an emotion sensor; and means for the server to receive and analyze the information and emotional data obtained from the terminal and generate an optimal travel plan corresponding to the user's emotions using a generative model. This makes it possible to generate and provide a personalized travel plan that takes the user's emotions into account.

[0153] A "terminal" is an electronic device used by a user to input information and receive processing results.

[0154] An "emotion engine" is software or hardware that analyzes user input information and sensor data to identify their emotional state.

[0155] A "server" is a central processing unit that receives information sent by users, analyzes it, and generates travel plans using a generative model.

[0156] A "generative model" is an algorithm or machine learning model used on a server to construct an optimal travel plan based on user input and sentiment data.

[0157] "Virtual guidance technology" is a technology that presents a generated travel plan to the user through audio and video, providing information visually and aurally.

[0158] An "external information service" is an external information provision system that can connect with the server to provide additional information related to travel planning.

[0159] A "prompt statement" is input information that gives specific instructions to a generative model to produce a desired output result.

[0160] This invention realizes a system that provides personalized travel plans to tourists visiting Japan. Users input data such as their personal interests, places they want to visit, budget, and emotions using a terminal. The terminal has an emotion sensor that extracts emotion data from the user's facial expressions and voice and transmits it to an emotion engine.

[0161] Information transmitted from the terminal is received by the server. The server analyzes the user's input information and emotional data. A generative AI model is used for the analysis, and this model acts as an algorithm to generate an appropriate travel plan based on the user's needs and emotional state. The generated plan includes optimization of destinations, activities, and itinerary, and is adjusted according to the user's emotional state. For example, if the user inputs "I would like a relaxing hot spring trip," the server will suggest relaxing activities in addition to hot spring destinations based on this input. Furthermore, if the user's emotions of joy or excitement are detected, surprise elements for further relaxation will be added to the plan.

[0162] The generated travel plan, sent from the server, is presented to the user on the terminal using virtual navigation technology. This virtual navigation technology includes features that provide information to the user visually and aurally, utilizing both audio and video. This technology allows the user to understand the plan's contents concretely and intuitively. This system allows users to enjoy a travel experience that takes their emotions into consideration, while reducing the time burden of planning and achieving a comfortable and satisfying sightseeing experience.

[0163] An example of a prompt for a generative AI model is, "Generate the optimal relaxing travel plan while taking the user's emotions into consideration." Through this sentence, specific instructions are provided to the generative model.

[0164] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0165] Step 1:

[0166] The user uses the device to input their travel preferences, such as personal interests, places they want to visit, and budget. This input data is then compiled into a user profile by the device. Furthermore, an emotion sensor built into the device monitors the user's facial expressions and voice tone, extracting emotion data. This emotion data is attached to the user profile and sent to the server in the next step. The user profile and emotion data are then sent from the device as output.

[0167] Step 2:

[0168] The user profile and sentiment data generated by the device are sent to the server. The server receives this data, stores it in a database, and then begins analysis. The analysis process generates prompt messages that take into account the user's interests and emotions. These prompt messages have a structure such as, "Generate the optimal relaxation travel plan while considering the user's emotions." The output is a set of prompt messages.

[0169] Step 3:

[0170] The server inputs the generated prompt text into the AI ​​model. Based on the prompt text, the AI ​​model creates an optimal travel plan tailored to the user's emotional state and interests. This process includes selecting destinations and activities, and optimizing the itinerary. The generated travel plan is then output.

[0171] Step 4:

[0172] The server analyzes the generated travel plan and makes necessary adjustments based on the user's emotions. For example, if the user is emotionally unstable, relaxing activities are added to the plan. The adjusted plan is sent to the device. The adjusted travel plan is generated as output.

[0173] Step 5:

[0174] The terminal presents the user with a pre-arranged travel plan received from the server using virtual navigation technology. It visualizes and explains the plan using audio and video technologies. The user can review the plan and provide feedback if necessary. The travel plan is presented to the user in a visually readable format as output.

[0175] (Application Example 2)

[0176] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0177] In travel planning, not only individual preferences and interests but also emotions at the time have a significant influence. However, existing systems have been insufficient in providing travel plans that take users' emotions into account. As a result, optimizing the user experience during travel has been difficult, and innovative methods to increase user satisfaction were needed.

[0178] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0179] In this invention, the server includes means having a function for the user to input their personal interests and travel preferences using an information processing device, means having a function for an information providing device to receive and analyze the information obtained from the information processing device and generate a travel plan using a generation model, and means having a function for presenting the generated travel plan to the user using a virtual guidance means. This makes it possible to provide a travel plan that has been dynamically modified by analyzing the user's emotions in three-dimensional video.

[0180] An "information processing device" is a device that allows users to input their personal interests and travel preferences, and has the function of analyzing that input information.

[0181] An "information provision device" is a device that receives and analyzes data obtained from an information processing device, and serves as the foundation for generating travel plans using a generative model.

[0182] A "generative model" is a mathematical or empirical model used to create the optimal travel plan based on user input and emotional data.

[0183] A "virtual guidance method" is a technology that uses three-dimensional images and audio to present travel plans to users visually and aurally.

[0184] "Emotional analysis" is a technology that uses sensors to determine a user's emotions in real time and incorporates the results into the travel plan.

[0185] "Three-dimensional imagery" is a video technology that provides users with three-dimensional visual information within a virtual space, and is used as part of virtual guidance.

[0186] An "external information exchange system" is a system for communicating with external information resources related to a coordinated travel plan, and its role is to provide users with the necessary information.

[0187] This invention realizes a system that provides personalized travel plans that take into account the user's emotions. When a user inputs their travel preferences, such as the places they want to visit and their budget, using an information processing device, this information is transmitted to an information providing device. The information providing device uses a generative AI model to generate a travel plan based on the user's input information.

[0188] Next, an emotion analysis engine is used to analyze the user's emotions at the time of input. This analysis utilizes the user's facial expressions and voice information collected by sensors. The results of this emotion analysis are reflected in the travel plan; for example, if the system receives feedback that the user wants to relax, a plan including hot springs and relaxation activities will be suggested.

[0189] The generated travel plan is provided to the user using virtual guidance. Specifically, it is possible to visually present the plan content on devices such as smart glasses and virtual display devices by combining three-dimensional images and audio. This allows users to understand the travel plan while receiving visual feedback and to make the most of the information provided in accordance with their emotions.

[0190] For example, if a user communicates via smart glasses that they "want to plan a relaxing trip to Kyoto," the generative model will generate a plan that includes Kyoto's famous sights and meditation opportunities. Based on the user's sentiment analysis, options such as visiting a quiet garden or experiencing healing music may also be added.

[0191] Example prompt: "Generate the optimal travel plan for a user who desires a peaceful Kyoto sightseeing trip, and dynamically adjust it based on their emotional state."

[0192] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0193] Step 1:

[0194] Users enter their travel preferences using a terminal. This information includes places they want to visit, their budget, and categories of interest. This data is temporarily stored on the terminal as a user profile and later transmitted to the information provider.

[0195] Step 2:

[0196] The server, acting as an information provider, receives user profiles and performs analysis using a generative AI model. Based on the input data, the generative model generates a draft of an optimal travel plan. This plan includes destinations and recommended activities.

[0197] Step 3:

[0198] The device collects user emotion data through interaction with the user, using facial recognition sensors and voice analysis software. This emotion data is fed back to the server in real time.

[0199] Step 4:

[0200] The server adjusts the draft travel plan based on the analysis results from the emotion analysis engine. If the user is seeking relaxation, the generating AI model makes adjustments such as adding destinations suitable for relaxation.

[0201] Step 5:

[0202] The server generates the final travel plan and sends it to the terminal via a virtual navigation system. The terminal then presents the plan to the user as three-dimensional images and audio. If the terminal is equipped with smart glasses, the user will see three-dimensional visuals and sounds.

[0203] Step 6:

[0204] The user reviews the presented travel plan and provides feedback. If there are any additional requests or changes, the information is entered again from the device, and the plan is updated through re-analysis on the server.

[0205] Step 7:

[0206] The server retrieves the necessary information through an external information exchange system and presents the final adjustment results to the user. This allows the user to enjoy the most appropriate travel plan, including reservation status, weather information, and real-time traffic information.

[0207] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0208] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0209] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0210] [Second Embodiment]

[0211] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0212] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0213] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0214] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0215] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0216] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0217] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0218] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0219] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0220] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0221] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0222] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0223] This invention is implemented as a system for providing personalized travel plans to tourists visiting Japan. In this embodiment of the system, the user, terminal, and server cooperate with each other to efficiently construct the travel plan.

[0224] Users input information about tourist destinations and experiences they are interested in using devices such as smartphones or personal computers. This input information specifically includes places they want to visit, the purpose of their trip, their budget, and desired activities. The device organizes the entered information and sends it to the server in an appropriate data format.

[0225] Next, the server receives this information and begins the process of generating a travel plan. Based on the specified conditions, the server uses a generative model to create a list of optimal tourist destinations and activities. The generative model is trained on the knowledge of professional travel guides and can propose a travel plan that best suits the user's needs. Furthermore, this also includes suggestions for accommodations and transportation.

[0226] The generated travel plan is presented to the user through virtual navigation technology. Specifically, it uses a combination of audio guides and visuals to help the user easily understand the plan. The user can review this plan and request changes if necessary.

[0227] The server, upon receiving user feedback, adjusts the travel plan. This involves replanning to better meet the user's preferences. The server also works with external data services to dynamically reflect the latest tourist information and tour options as new information becomes available.

[0228] For example, if a user enters a desire to "experience traditional Japanese culture," the server will generate a travel plan that includes historical sites in Kyoto and Nara. For instance, it might suggest activities such as renting a kimono or participating in a traditional tea ceremony. This plan is then explained through virtual navigation technology, along with a visual representation of the itinerary.

[0229] In this way, the present invention is a system that improves the travel experience of tourists visiting Japan and reduces the time burden in planning their trips.

[0230] The following describes the processing flow.

[0231] Step 1:

[0232] The user launches a dedicated application on their device and enters travel information such as the purpose of the trip, budget, desired activities, and places they want to visit. The device then organizes this information and prepares to communicate with the server.

[0233] Step 2:

[0234] The terminal sends the entered user information to the server. The transmitted information is formatted according to a specified data format to ensure efficient transmission.

[0235] Step 3:

[0236] The server receives user information from the terminal and begins data analysis. Based on the received information, the server performs analysis to understand the user's preferences and travel conditions. This establishes the requirements necessary for formulating a travel plan.

[0237] Step 4:

[0238] The server activates a generative model and uses the analyzed user information to generate a travel plan that includes appropriate tourist destinations and activities. The generative model leverages the knowledge of professional travel guides that it has learned in advance to suggest the optimal plan.

[0239] Step 5:

[0240] The server prepares to present the generated travel plan to the user using virtual navigation technology. The plan includes planned destinations, schedules, activities, and related information.

[0241] Step 6:

[0242] The terminal uses virtual navigation technology to present travel plans to users visually and audibly. The navigation content is designed to help users easily visualize the trip.

[0243] Step 7:

[0244] The user reviews the presented travel plan and sends any necessary changes or feedback from their device to the server. User feedback is a crucial element in further tailoring the plan to the user.

[0245] Step 8:

[0246] The server receives user feedback and regenerates or adjusts the travel plan. If necessary, it retrieves the latest information from external data services and incorporates it into the plan.

[0247] Step 9:

[0248] The server finalizes the travel plan and provides it to the user via the terminal. The process ends when the user confirms that the finalized plan meets their expectations.

[0249] (Example 1)

[0250] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0251] There is a need to efficiently and effectively provide personalized travel plans to tourists visiting Japan. Traditional guidebooks and fixed tour plans fail to meet the diverse needs of individual tourists, resulting in a decline in the quality of their travel experience. Furthermore, it is difficult to change travel plans or incorporate new information, leading to a significant burden of planning time and effort.

[0252] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0253] In this invention, the server includes means for the user to input their personal interests and travel requests using an information processing device, means for a central processing device to receive and analyze the information obtained from the information processing device and generate a travel plan using a generation AI model, and means for presenting the generated travel plan to the user using virtual guidance technology. This makes it possible to provide flexible and personalized travel plans that meet individual needs.

[0254] "Users" refers to travelers and tourists who use the system, and means individuals with specific travel preferences.

[0255] An "information processing device" is a device that receives input information from a user, and examples include smartphones and personal computers.

[0256] A "central processing unit" is a computer system that receives data from information processing units and processes it, and is responsible for generating travel plans that meet the user's requests.

[0257] A "generative AI model" refers to an algorithm that has been pre-trained with travel-related knowledge and is used to propose the optimal travel plan based on received data.

[0258] "Virtual guidance technology" is a technology that presents generated travel plans to users visually and audibly, and is a means of helping users easily understand the contents of the plan.

[0259] "External information provision services" refer to information sources that allow the central processing unit to acquire tourism information and the latest data from external sources and incorporate them into travel plans.

[0260] This invention is a system that provides personalized travel plans to tourists visiting Japan. This system is realized through the mutual cooperation of users, terminals, and servers.

[0261] Users input their travel preferences using devices such as smartphones or computers. This input includes places they want to visit, the purpose of their trip, their budget, and desired activities. Specifically, users input prompts such as "I want to visit art museums in Tokyo" or "I would like to experience a traditional tea ceremony in Kyoto."

[0262] The terminal organizes the input information, converts it into a communication format, and sends it to the server. The communication protocols and data formats used in this process are optimized to meet the system's needs.

[0263] Upon receiving data from a terminal, the server generates a travel plan using a generative AI model. This model not only lists the most suitable tourist destinations and activities based on the user's preferences, but also suggests accommodations and transportation options. The generative AI model is trained on historical travel data and the knowledge of professional travel guides.

[0264] The generated travel plan is presented to the user using virtual navigation technology. This technology combines audio output and visual display to make the plan intuitively understandable to the user. This information presentation not only helps the user understand the plan but also encourages them to further adjust travel details and provide feedback.

[0265] Therefore, the present invention aims to provide flexible travel plans that meet the diverse needs of tourists visiting Japan and to improve the quality of their travel.

[0266] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0267] Step 1:

[0268] Users input information about tourist destinations they want to visit or activities they want to experience using devices such as smartphones or computers. The input is in text format, creating prompt statements such as "I want to visit art museums in Tokyo." This becomes the input data for the system.

[0269] Step 2:

[0270] The terminal collects data entered by the user and organizes it according to its internally configured data format. During this process, the terminal converts the data into formats such as JSON to efficiently transmit it to the server. The converted data then becomes the input sent to the server.

[0271] Step 3:

[0272] The server receives data sent from the terminal and begins analysis. The server activates a generative AI model and generates a list of tourist destinations and activities based on the user's preferences. Through data analysis and model output, a travel plan tailored to the user's needs is generated.

[0273] Step 4:

[0274] The server inputs the generated travel plan into virtual navigation technology and converts it into a presentation format for the user. This conversion process combines audio guidance and visual displays to present the generated plan in an easily understandable way for the user. This information presentation becomes the output to the user.

[0275] Step 5:

[0276] The user reviews the proposed travel plan and submits feedback to request changes as needed. This feedback includes adjustments to the timing of visits and any additional requests, which then become new input data for the server.

[0277] Step 6:

[0278] The server analyzes user feedback and readjusts the travel plan. If necessary, it gathers the latest information from external information services and incorporates it into the plan. This generates updated options to provide users with a better travel plan.

[0279] (Application Example 1)

[0280] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0281] There is a demand to efficiently and effectively provide personalized travel plans according to the individual interests and wishes of foreign tourists visiting Japan. However, in conventional systems, the provision of tourism information is one-sided, and it is difficult to pre-examine specific experiences. Furthermore, it has also been difficult for users to provide feedback and easily reflect the results. The present invention aims to solve these problems and provide a system for improving the quality of travel experiences.

[0282] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0283] In this invention, the server includes means for a user to input their personal interests and travel wishes using an information terminal, means for the data processing device to receive and analyze the information obtained from the information terminal and generate a travel plan using generation technology, and means for presenting the generated travel plan to the user using three-dimensional virtual space technology. As a result, it becomes possible to provide a personalized travel plan based on the user's intentions and to virtually experience that plan.

[0284] An "information terminal" is a device used by a user and is a device used to input personal interests and travel wishes.

[0285] A "data processing device" is a server device that has the function of analyzing data received from an information terminal and generating a travel plan using generation technology.

[0286] "Generation technology" refers to algorithms and models for creating an optimal travel plan based on the input information of a user.

[0287] "Three-dimensional virtual space technology" is a technology for presenting generated travel plans to users visually and interactively.

[0288] A "voice generation engine" is a technology that provides users with voice guidance on travel plans.

[0289] "Visual display technology" refers to technology used to visually display scenes and information related to travel planning.

[0290] "External information services" refer to external data provision services that are partnered with to provide the latest tourist information and options.

[0291] "Evaluation information" refers to data collected from user feedback and opinions, used to adjust and regenerate travel plans.

[0292] The system for realizing this application example is configured as follows: The user inputs their interests and travel preferences using an information terminal. This terminal could be a smartphone or a personal computer. After this input is complete, the information is received and analyzed by a data processing device (server). This analysis includes generating a travel plan using a generative AI model, automatically creating a travel plan optimized for the user's conditions.

[0293] The generated travel plan is presented to the user using three-dimensional virtual space technology. For this purpose, hardware such as VR goggles and smartphones are used. The software utilizes Unity 3D to build the virtual reality environment and leverages OpenAI APIs to generate the plan. Information on restaurants, accommodations, and tourist attractions is kept up-to-date by linking with external information services. Furthermore, Google Cloud's Text-to-Speech API is used as the speech generation engine, allowing the travel plan to be explained in audio.

[0294] Users can review the provided travel plans and experience them in a virtual space beforehand. Wearing VR goggles, users can walk around the virtual tourist destinations unfolding before their eyes, experiencing the actual plan. This allows users to concretely imagine the atmosphere and activities of the places they want to visit before traveling.

[0295] For example, if a prompt such as "Please tell me about beautiful places to see autumn foliage in Japan and their highlights" is input into the AI ​​model, the system will suggest the best autumn foliage spots and related activities for the user. This allows tourists visiting Japan to obtain valuable advance information to fully enjoy the charms of Japan.

[0296] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0297] Step 1:

[0298] Users use an information terminal to input their interests and travel preferences. This input includes information such as desired destinations, budget, and activities of interest. The input data is organized within the terminal and converted into a format that can be sent to the server.

[0299] Step 2:

[0300] The server receives user information sent from the terminal. Based on this information, it performs data analysis and generates a travel plan using a generative AI model. At this stage, tourist destinations, accommodations, and transportation methods that meet the user's conditions are optimized and output as a specific plan.

[0301] Step 3:

[0302] The server presents the generated travel plan to the user using three-dimensional virtual space technology. Here, Unity 3D is utilized to visually represent information, and the travel plan is output as detailed visual information. Google Cloud's Text-to-Speech API is used for voice guidance, and the plan details are explained to the user audibly.

[0303] Step 4:

[0304] The user virtually experiences the travel plan presented within the virtual space. This experience includes virtual sightseeing through VR goggles and prior consideration of activities. The information obtained by the user through the experience is useful for the following feedback.

[0305] Step 5:

[0306] The user provides feedback based on the experience within the virtual space. This feedback is sent to the server and used as data for readjusting the plan. Based on the received evaluation information, the server updates the travel plan as necessary and proposes a new plan.

[0307] Step 6:

[0308] The server collaborates with external information services to obtain the latest tourism information related to the adjusted travel plan. This information includes the latest event information and newly added activity options. The acquired data is provided to the user again, resulting in the completion of the final travel plan.

[0309] Furthermore, an emotion engine for estimating the user's emotions may be combined. That is, the specific processing unit 290 may estimate the user's emotions using the emotion recognition model 59 and perform specific processing using the user's emotions.

[0310] This invention provides a system that offers personalized travel plans to tourists visiting Japan, achieving advanced planning that takes user emotions into account. This system functions through the collaboration of a terminal, server, emotion engine, generative model, and virtual guidance technology.

[0311] Users use the device to input their travel preferences, such as personal interests, places they want to visit, and their budget. The device creates a user profile based on this information and sends it to the server. User feedback and emotions expressed during the input process are sent to the emotion engine via sensors and software on the device.

[0312] The server receives and analyzes information sent by the user. The generative model generates an optimal travel plan based on this analyzed information and the user's emotions recognized by the emotion engine. The generated plan includes destinations, activity selections, and itinerary optimization, with adjustments made according to the user's emotional state.

[0313] The emotion engine analyzes user emotions based on their input and the context of the conversation, and feeds the results back to the server. This feedback is used to adjust the travel plan; for example, if the user is feeling anxious, suggestions for more relaxing plans or increased support will be provided.

[0314] The generated travel plan is presented to the user using virtual navigation technology. The device visually explains the plan to the user through audio and video. This presentation is designed to allow the user to easily understand the plan and proceed with their trip with confidence.

[0315] For example, if a user enters "I would like a relaxing hot spring trip," the server will suggest hot spring resorts and relaxing activities. If the emotion engine recognizes the user's feelings of joy or excitement, surprise elements for further relaxation will be added to the plan. In this way, a travel plan tailored to the user's emotions is formed and provided to the user.

[0316] Through this system, users can enjoy a travel experience that takes their emotions into consideration, reducing the time burden of planning and enabling comfortable and satisfying sightseeing.

[0317] The following describes the processing flow.

[0318] Step 1:

[0319] The user uses a device to input information such as the purpose of their trip, their budget, the tourist destinations they want to visit, and the activities they want to experience. The device then neatly organizes this information and prepares it as data to be sent to the server.

[0320] Step 2:

[0321] The terminal sends the entered data to the server. The information is transmitted using a secure communication protocol, minimizing time delays.

[0322] Step 3:

[0323] The server receives information from the terminal and analyzes it by comparing it with past data in the database. The information is then organized, taking into account user preferences, past travel templates, and general tourism trends.

[0324] Step 4:

[0325] The server uses the analysis results to launch a generative model, which then provides suggestions based on the user's personal data and travel data. The generated travel plan includes details of planned destinations and specific activities, and is designed to best meet the user's preferences.

[0326] Step 5:

[0327] The emotion engine operates using sensors and analysis software within the device to capture emotional data that emerges during user input and responses. This includes facial expressions, voice tone, input speed, and patterns.

[0328] Step 6:

[0329] The emotion engine analyzes the data it acquires to recognize the user's emotions. For example, it determines whether the user is feeling stressed, anxious, or relaxed, and sends the result to the server.

[0330] Step 7:

[0331] The server receives feedback from the emotion engine and adjusts the travel plan. Depending on the recognized emotions, suggestions to enhance relaxation and support options to provide reassurance are added to the plan.

[0332] Step 8:

[0333] The server sends the finalized travel plan to the terminal, which then uses virtual navigation technology to present the plan to the user via audio and video. The user then has the opportunity to review the plan and provide further feedback.

[0334] Step 9:

[0335] The user reviews the plan and gives final approval. They can request further changes if necessary, but the plan is usually finalized at this stage. Once the plan is finalized, the user can begin preparing for their trip.

[0336] (Example 2)

[0337] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0338] In recent years, there has been a growing demand for personalized travel planning that better suits individual needs and circumstances. However, traditional travel planning systems have struggled to generate and adjust plans that reflect user emotions, failing to improve the quality of the user experience. As a result, users may be dissatisfied with their travel plans.

[0339] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0340] In this invention, the server includes means for the user to input personal interests, travel preferences, and emotional data using a terminal; means for the terminal to transmit the user's emotions to an emotion engine via an emotion sensor; and means for the server to receive and analyze the information and emotional data obtained from the terminal and generate an optimal travel plan corresponding to the user's emotions using a generative model. This makes it possible to generate and provide a personalized travel plan that takes the user's emotions into account.

[0341] A "terminal" is an electronic device used by a user to input information and receive processing results.

[0342] An "emotion engine" is software or hardware that analyzes user input information and sensor data to identify their emotional state.

[0343] A "server" is a central processing unit that receives information sent by users, analyzes it, and generates travel plans using a generative model.

[0344] A "generative model" is an algorithm or machine learning model used on a server to construct an optimal travel plan based on user input and sentiment data.

[0345] "Virtual guidance technology" is a technology that presents a generated travel plan to the user through audio and video, providing information visually and aurally.

[0346] An "external information service" is an external information provision system that can connect with the server to provide additional information related to travel planning.

[0347] A "prompt statement" is input information that gives specific instructions to a generative model to produce a desired output result.

[0348] This invention realizes a system that provides personalized travel plans to tourists visiting Japan. Users input data such as their personal interests, places they want to visit, budget, and emotions using a terminal. The terminal has an emotion sensor that extracts emotion data from the user's facial expressions and voice and transmits it to an emotion engine.

[0349] Information transmitted from the terminal is received by the server. The server analyzes the user's input information and emotional data. A generative AI model is used for the analysis, and this model acts as an algorithm to generate an appropriate travel plan based on the user's needs and emotional state. The generated plan includes optimization of destinations, activities, and itinerary, and is adjusted according to the user's emotional state. For example, if the user inputs "I would like a relaxing hot spring trip," the server will suggest relaxing activities in addition to hot spring destinations based on this input. Furthermore, if the user's emotions of joy or excitement are detected, surprise elements for further relaxation will be added to the plan.

[0350] The generated travel plan, sent from the server, is presented to the user on the terminal using virtual navigation technology. This virtual navigation technology includes features that provide information to the user visually and aurally, utilizing both audio and video. This technology allows the user to understand the plan's contents concretely and intuitively. This system allows users to enjoy a travel experience that takes their emotions into consideration, while reducing the time burden of planning and achieving a comfortable and satisfying sightseeing experience.

[0351] An example of a prompt for a generative AI model is, "Generate the optimal relaxing travel plan while taking the user's emotions into consideration." Through this sentence, specific instructions are provided to the generative model.

[0352] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0353] Step 1:

[0354] The user uses the device to input their travel preferences, such as personal interests, places they want to visit, and budget. This input data is then compiled into a user profile by the device. Furthermore, an emotion sensor built into the device monitors the user's facial expressions and voice tone, extracting emotion data. This emotion data is attached to the user profile and sent to the server in the next step. The user profile and emotion data are then sent from the device as output.

[0355] Step 2:

[0356] The user profile and sentiment data generated by the device are sent to the server. The server receives this data, stores it in a database, and then begins analysis. The analysis process generates prompt messages that take into account the user's interests and emotions. These prompt messages have a structure such as, "Generate the optimal relaxation travel plan while considering the user's emotions." The output is a set of prompt messages.

[0357] Step 3:

[0358] The server inputs the generated prompt text into the AI ​​model. Based on the prompt text, the AI ​​model creates an optimal travel plan tailored to the user's emotional state and interests. This process includes selecting destinations and activities, and optimizing the itinerary. The generated travel plan is then output.

[0359] Step 4:

[0360] The server analyzes the generated travel plan and makes necessary adjustments based on the user's emotions. For example, if the user is emotionally unstable, relaxing activities are added to the plan. The adjusted plan is sent to the device. The adjusted travel plan is generated as output.

[0361] Step 5:

[0362] The terminal presents the user with a pre-arranged travel plan received from the server using virtual navigation technology. It visualizes and explains the plan using audio and video technologies. The user can review the plan and provide feedback if necessary. The travel plan is presented to the user in a visually readable format as output.

[0363] (Application Example 2)

[0364] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0365] In travel planning, not only individual preferences and interests but also emotions at the time have a significant influence. However, existing systems have been insufficient in providing travel plans that take users' emotions into account. As a result, optimizing the user experience during travel has been difficult, and innovative methods to increase user satisfaction were needed.

[0366] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0367] In this invention, the server includes means having a function for the user to input their personal interests and travel preferences using an information processing device, means having a function for an information providing device to receive and analyze the information obtained from the information processing device and generate a travel plan using a generation model, and means having a function for presenting the generated travel plan to the user using a virtual guidance means. This makes it possible to provide a travel plan that has been dynamically modified by analyzing the user's emotions in three-dimensional video.

[0368] An "information processing device" is a device that allows users to input their personal interests and travel preferences, and has the function of analyzing that input information.

[0369] An "information provision device" is a device that receives and analyzes data obtained from an information processing device, and serves as the foundation for generating travel plans using a generative model.

[0370] A "generative model" is a mathematical or empirical model used to create the optimal travel plan based on user input and emotional data.

[0371] A "virtual guidance method" is a technology that uses three-dimensional images and audio to present travel plans to users visually and aurally.

[0372] "Emotional analysis" is a technology that uses sensors to determine a user's emotions in real time and incorporates the results into the travel plan.

[0373] "Three-dimensional imagery" is a video technology that provides users with three-dimensional visual information within a virtual space, and is used as part of virtual guidance.

[0374] An "external information exchange system" is a system for communicating with external information resources related to a coordinated travel plan, and its role is to provide users with the necessary information.

[0375] This invention realizes a system that provides personalized travel plans that take into account the user's emotions. When a user inputs their travel preferences, such as the places they want to visit and their budget, using an information processing device, this information is transmitted to an information providing device. The information providing device uses a generative AI model to generate a travel plan based on the user's input information.

[0376] Next, an emotion analysis engine is used to analyze the user's emotions at the time of input. This analysis utilizes the user's facial expressions and voice information collected by sensors. The results of this emotion analysis are reflected in the travel plan; for example, if the system receives feedback that the user wants to relax, a plan including hot springs and relaxation activities will be suggested.

[0377] The generated travel plan is provided to the user using virtual guidance. Specifically, it is possible to visually present the plan content on devices such as smart glasses and virtual display devices by combining three-dimensional images and audio. This allows users to understand the travel plan while receiving visual feedback and to make the most of the information provided in accordance with their emotions.

[0378] For example, if a user communicates via smart glasses that they "want to plan a relaxing trip to Kyoto," the generative model will generate a plan that includes Kyoto's famous sights and meditation opportunities. Based on the user's sentiment analysis, options such as visiting a quiet garden or experiencing healing music may also be added.

[0379] Example prompt: "Generate the optimal travel plan for a user who desires a peaceful Kyoto sightseeing trip, and dynamically adjust it based on their emotional state."

[0380] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0381] Step 1:

[0382] Users enter their travel preferences using a terminal. This information includes places they want to visit, their budget, and categories of interest. This data is temporarily stored on the terminal as a user profile and later transmitted to the information provider.

[0383] Step 2:

[0384] The server, acting as an information provider, receives user profiles and performs analysis using a generative AI model. Based on the input data, the generative model generates a draft of an optimal travel plan. This plan includes destinations and recommended activities.

[0385] Step 3:

[0386] The device collects user emotion data through interaction with the user, using facial recognition sensors and voice analysis software. This emotion data is fed back to the server in real time.

[0387] Step 4:

[0388] The server adjusts the draft travel plan based on the analysis results from the emotion analysis engine. If the user is seeking relaxation, the generating AI model makes adjustments such as adding destinations suitable for relaxation.

[0389] Step 5:

[0390] The server generates the final travel plan and sends it to the terminal via a virtual navigation system. The terminal then presents the plan to the user as three-dimensional images and audio. If the terminal is equipped with smart glasses, the user will see three-dimensional visuals and sounds.

[0391] Step 6:

[0392] The user reviews the presented travel plan and provides feedback. If there are any additional requests or changes, the information is entered again from the device, and the plan is updated through re-analysis on the server.

[0393] Step 7:

[0394] The server retrieves the necessary information through an external information exchange system and presents the final adjustment results to the user. This allows the user to enjoy the most appropriate travel plan, including reservation status, weather information, and real-time traffic information.

[0395] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0396] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0397] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0398] [Third Embodiment]

[0399] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0400] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0401] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0402] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0403] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0404] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0405] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0406] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0407] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0408] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0409] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0410] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0411] This invention is implemented as a system for providing personalized travel plans to tourists visiting Japan. In this embodiment of the system, the user, terminal, and server cooperate with each other to efficiently construct the travel plan.

[0412] Users input information about tourist destinations and experiences they are interested in using devices such as smartphones or personal computers. This input information specifically includes places they want to visit, the purpose of their trip, their budget, and desired activities. The device organizes the entered information and sends it to the server in an appropriate data format.

[0413] Next, the server receives this information and begins the process of generating a travel plan. Based on the specified conditions, the server uses a generative model to create a list of optimal tourist destinations and activities. The generative model is trained on the knowledge of professional travel guides and can propose a travel plan that best suits the user's needs. Furthermore, this also includes suggestions for accommodations and transportation.

[0414] The generated travel plan is presented to the user through virtual navigation technology. Specifically, it uses a combination of audio guides and visuals to help the user easily understand the plan. The user can review this plan and request changes if necessary.

[0415] The server, upon receiving user feedback, adjusts the travel plan. This involves replanning to better meet the user's preferences. The server also works with external data services to dynamically reflect the latest tourist information and tour options as new information becomes available.

[0416] For example, if a user enters a desire to "experience traditional Japanese culture," the server will generate a travel plan that includes historical sites in Kyoto and Nara. For instance, it might suggest activities such as renting a kimono or participating in a traditional tea ceremony. This plan is then explained through virtual navigation technology, along with a visual representation of the itinerary.

[0417] In this way, the present invention is a system that improves the travel experience of tourists visiting Japan and reduces the time burden in planning their trips.

[0418] The following describes the processing flow.

[0419] Step 1:

[0420] The user launches a dedicated application on their device and enters travel information such as the purpose of the trip, budget, desired activities, and places they want to visit. The device then organizes this information and prepares to communicate with the server.

[0421] Step 2:

[0422] The terminal sends the entered user information to the server. The transmitted information is formatted according to a specified data format to ensure efficient transmission.

[0423] Step 3:

[0424] The server receives user information from the terminal and begins data analysis. Based on the received information, the server performs analysis to understand the user's preferences and travel conditions. This establishes the requirements necessary for formulating a travel plan.

[0425] Step 4:

[0426] The server activates a generative model and uses the analyzed user information to generate a travel plan that includes appropriate tourist destinations and activities. The generative model leverages the knowledge of professional travel guides that it has learned in advance to suggest the optimal plan.

[0427] Step 5:

[0428] The server prepares to present the generated travel plan to the user using virtual navigation technology. The plan includes planned destinations, schedules, activities, and related information.

[0429] Step 6:

[0430] The terminal uses virtual navigation technology to present travel plans to users visually and audibly. The navigation content is designed to help users easily visualize the trip.

[0431] Step 7:

[0432] The user reviews the presented travel plan and sends any necessary changes or feedback from their device to the server. User feedback is a crucial element in further tailoring the plan to the user.

[0433] Step 8:

[0434] The server receives user feedback and regenerates or adjusts the travel plan. If necessary, it retrieves the latest information from external data services and incorporates it into the plan.

[0435] Step 9:

[0436] The server finalizes the travel plan and provides it to the user via the terminal. The process ends when the user confirms that the finalized plan meets their expectations.

[0437] (Example 1)

[0438] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0439] There is a need to efficiently and effectively provide personalized travel plans to tourists visiting Japan. Traditional guidebooks and fixed tour plans fail to meet the diverse needs of individual tourists, resulting in a decline in the quality of their travel experience. Furthermore, it is difficult to change travel plans or incorporate new information, leading to a significant burden of planning time and effort.

[0440] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0441] In this invention, the server includes means for the user to input their personal interests and travel requests using an information processing device, means for a central processing device to receive and analyze the information obtained from the information processing device and generate a travel plan using a generation AI model, and means for presenting the generated travel plan to the user using virtual guidance technology. This makes it possible to provide flexible and personalized travel plans that meet individual needs.

[0442] "Users" refers to travelers and tourists who use the system, and means individuals with specific travel preferences.

[0443] An "information processing device" is a device that receives input information from a user, and examples include smartphones and personal computers.

[0444] A "central processing unit" is a computer system that receives data from information processing units and processes it, and is responsible for generating travel plans that meet the user's requests.

[0445] A "generative AI model" refers to an algorithm that has been pre-trained with travel-related knowledge and is used to propose the optimal travel plan based on received data.

[0446] "Virtual guidance technology" is a technology that presents generated travel plans to users visually and audibly, and is a means of helping users easily understand the contents of the plan.

[0447] "External information provision services" refer to information sources that allow the central processing unit to acquire tourism information and the latest data from external sources and incorporate them into travel plans.

[0448] This invention is a system that provides personalized travel plans to tourists visiting Japan. This system is realized through the mutual cooperation of users, terminals, and servers.

[0449] Users input their travel preferences using devices such as smartphones or computers. This input includes places they want to visit, the purpose of their trip, their budget, and desired activities. Specifically, users input prompts such as "I want to visit art museums in Tokyo" or "I would like to experience a traditional tea ceremony in Kyoto."

[0450] The terminal organizes the input information, converts it into a communication format, and sends it to the server. The communication protocol and data format used in this process are optimized to meet the system's needs.

[0451] Upon receiving data from a terminal, the server generates a travel plan using a generative AI model. This model not only lists the most suitable tourist destinations and activities based on the user's preferences, but also suggests accommodations and transportation options. The generative AI model is trained on historical travel data and the knowledge of professional travel guides.

[0452] The generated travel plan is presented to the user using virtual navigation technology. This technology combines audio output and visual display to make the plan intuitively understandable to the user. This information presentation not only helps the user understand the plan but also encourages them to further adjust travel details and provide feedback.

[0453] Therefore, the present invention aims to provide flexible travel plans that meet the diverse needs of tourists visiting Japan and to improve the quality of their travel.

[0454] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0455] Step 1:

[0456] Users input information about tourist destinations they want to visit or activities they want to experience using devices such as smartphones or computers. The input is in text format, creating prompt statements such as "I want to visit art museums in Tokyo." This becomes the input data for the system.

[0457] Step 2:

[0458] The terminal collects data entered by the user and organizes it according to its internally configured data format. During this process, the terminal converts the data into formats such as JSON to efficiently transmit it to the server. The converted data then becomes the input sent to the server.

[0459] Step 3:

[0460] The server receives data sent from the terminal and begins analysis. The server activates a generative AI model and generates a list of tourist destinations and activities based on the user's preferences. Through data analysis and model output, a travel plan tailored to the user's needs is generated.

[0461] Step 4:

[0462] The server inputs the generated travel plan into virtual navigation technology and converts it into a presentation format for the user. This conversion process combines audio guidance and visual displays to present the generated plan in an easily understandable way for the user. This information presentation becomes the output to the user.

[0463] Step 5:

[0464] The user reviews the proposed travel plan and submits feedback to request changes as needed. This feedback includes adjustments to the timing of visits and additional requests, which then become new input data for the server.

[0465] Step 6:

[0466] The server analyzes user feedback and readjusts the travel plan. If necessary, it gathers the latest information from external information services and incorporates it into the plan. This generates updated options to provide users with a better travel plan.

[0467] (Application Example 1)

[0468] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0469] There is a need to efficiently and effectively provide personalized travel plans tailored to the individual interests and preferences of tourists visiting Japan. However, conventional systems provide tourist information unilaterally, making it difficult to consider specific experiences in advance. Furthermore, it has been difficult for users to provide feedback and for the results to be easily reflected. This invention aims to solve these problems and provide a system to improve the quality of the travel experience.

[0470] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0471] In this invention, the server includes means for a user to input their personal interests and travel preferences using an information terminal, means for a data processing device to receive and analyze the information obtained from the information terminal and generate a travel plan using generation technology, and means for presenting the generated travel plan to the user using three-dimensional virtual space technology. This makes it possible to provide a personalized travel plan based on the user's intentions and to virtually experience that plan.

[0472] An "information terminal" is a device used by a user to input their personal interests and travel preferences.

[0473] A "data processing device" is a server device that analyzes data received from an information terminal and generates a travel plan using generation technology.

[0474] "Generative technology" refers to algorithms and models used to create optimal travel plans based on user input.

[0475] "Three-dimensional virtual space technology" is a technology for presenting generated travel plans to users visually and interactively.

[0476] A "voice generation engine" is a technology that provides users with voice guidance on travel plans.

[0477] "Visual display technology" refers to technology used to visually display scenes and information related to travel planning.

[0478] "External information services" refer to external data provision services that are partnered with to provide the latest tourist information and options.

[0479] "Evaluation information" refers to data collected from user feedback and opinions, used to adjust and regenerate travel plans.

[0480] The system for realizing this application example is configured as follows: The user inputs their interests and travel preferences using an information terminal. This terminal could be a smartphone or a personal computer. After this input is complete, the information is received and analyzed by a data processing device (server). This analysis includes generating a travel plan using a generative AI model, automatically creating a travel plan optimized for the user's conditions.

[0481] The generated travel plan is presented to the user using three-dimensional virtual space technology. For this purpose, hardware such as VR goggles and smartphones are used. The software utilizes Unity 3D to build the virtual reality environment and leverages OpenAI APIs to generate the plan. Information on restaurants, accommodations, and tourist attractions is kept up-to-date by linking with external information services. Furthermore, Google Cloud's Text-to-Speech API is used as the speech generation engine, allowing the travel plan to be explained in audio.

[0482] Users can review the provided travel plans and experience them in a virtual space beforehand. Wearing VR goggles, users can walk around the virtual tourist destinations unfolding before their eyes, experiencing the actual plan. This allows users to concretely imagine the atmosphere and activities of the places they want to visit before traveling.

[0483] For example, if a prompt such as "Please tell me about beautiful places to see autumn foliage in Japan and their highlights" is input into the AI ​​model, the system will suggest the best autumn foliage spots and related activities for the user. This allows tourists visiting Japan to obtain valuable advance information to fully enjoy the charms of Japan.

[0484] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0485] Step 1:

[0486] Users use an information terminal to input their interests and travel preferences. This input includes information such as desired destinations, budget, and activities of interest. The input data is organized within the terminal and converted into a format that can be sent to the server.

[0487] Step 2:

[0488] The server receives user information sent from the terminal. Based on this information, it performs data analysis and generates a travel plan using a generative AI model. At this stage, tourist destinations, accommodations, and transportation methods that meet the user's conditions are optimized and output as a specific plan.

[0489] Step 3:

[0490] The server presents the generated travel plan to the user using three-dimensional virtual space technology. Unity 3D is used to visually represent the information, outputting the travel plan as detailed visual information. Google Cloud's Text-to-Speech API is used for audio guidance, explaining the plan to the user verbally.

[0491] Step 4:

[0492] Users virtually experience a travel plan presented within a virtual space. This experience includes virtual sightseeing and pre-exploration of activities through VR goggles. The information users gather through this experience will be used for future feedback.

[0493] Step 5:

[0494] Users provide feedback based on their experience in the virtual space. This feedback is sent to the server and used as data to readjust the plan. Based on the received evaluation information, the server updates the travel plan as needed and proposes a new plan.

[0495] Step 6:

[0496] The server connects with external information services to retrieve the latest tourist information relevant to the adjusted travel plan. This information includes the latest event information and newly added activity options. The retrieved data is then provided to the user again, completing the final travel plan.

[0497] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0498] This invention provides a system that offers personalized travel plans to tourists visiting Japan, achieving advanced planning that takes user emotions into account. This system functions through the collaboration of a terminal, server, emotion engine, generative model, and virtual guidance technology.

[0499] Users use the device to input their travel preferences, such as personal interests, places they want to visit, and their budget. The device creates a user profile based on this information and sends it to the server. User feedback and emotions expressed during the input process are sent to the emotion engine via sensors and software on the device.

[0500] The server receives and analyzes information sent by the user. The generative model generates an optimal travel plan based on this analyzed information and the user's emotions recognized by the emotion engine. The generated plan includes destinations, activity selections, and itinerary optimization, with adjustments made according to the user's emotional state.

[0501] The emotion engine analyzes user emotions based on their input and the context of the conversation, and feeds the results back to the server. This feedback is used to adjust the travel plan; for example, if the user is feeling anxious, suggestions for more relaxing plans or increased support will be provided.

[0502] The generated travel plan is presented to the user using virtual navigation technology. The device visually explains the plan to the user through audio and video. This presentation is designed to allow the user to easily understand the plan and proceed with their trip with confidence.

[0503] For example, if a user enters "I would like a relaxing hot spring trip," the server will suggest hot spring resorts and relaxing activities. If the emotion engine recognizes the user's feelings of joy or excitement, surprise elements for further relaxation will be added to the plan. In this way, a travel plan tailored to the user's emotions is formed and provided to the user.

[0504] Through this system, users can enjoy a travel experience that takes their emotions into consideration, reducing the time burden of planning and enabling comfortable and satisfying sightseeing.

[0505] The following describes the processing flow.

[0506] Step 1:

[0507] The user uses a device to input information such as the purpose of their trip, their budget, the tourist destinations they want to visit, and the activities they want to experience. The device then neatly organizes this information and prepares it as data to be sent to the server.

[0508] Step 2:

[0509] The terminal sends the entered data to the server. The information is transmitted using a secure communication protocol, minimizing time delays.

[0510] Step 3:

[0511] The server receives information from the terminal and analyzes it by comparing it with past data in the database. The information is then organized, taking into account user preferences, past travel templates, and general tourism trends.

[0512] Step 4:

[0513] The server uses the analysis results to launch a generative model, which then provides suggestions based on the user's personal data and travel data. The generated travel plan includes details of planned destinations and specific activities, and is designed to best meet the user's preferences.

[0514] Step 5:

[0515] The emotion engine operates using sensors and analysis software within the device to capture emotional data that emerges during user input and responses. This includes facial expressions, voice tone, input speed, and patterns.

[0516] Step 6:

[0517] The emotion engine analyzes the data it acquires to recognize the user's emotions. For example, it determines whether the user is feeling stressed, anxious, or relaxed, and sends the result to the server.

[0518] Step 7:

[0519] The server receives feedback from the emotion engine and adjusts the travel plan. Depending on the recognized emotions, suggestions to enhance relaxation and support options to provide reassurance are added to the plan.

[0520] Step 8:

[0521] The server sends the finalized travel plan to the terminal, which then uses virtual navigation technology to present the plan to the user via audio and video. The user then has the opportunity to review the plan and provide further feedback.

[0522] Step 9:

[0523] The user reviews the plan and gives final approval. They can request further changes if necessary, but the plan is usually finalized at this stage. Once the plan is finalized, the user can begin preparing for their trip.

[0524] (Example 2)

[0525] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0526] In recent years, there has been a growing demand for personalized travel planning that better suits individual needs and circumstances. However, traditional travel planning systems have struggled to generate and adjust plans that reflect user emotions, failing to improve the quality of the user experience. As a result, users may be dissatisfied with their travel plans.

[0527] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0528] In this invention, the server includes means for the user to input personal interests, travel preferences, and emotional data using a terminal; means for the terminal to transmit the user's emotions to an emotion engine via an emotion sensor; and means for the server to receive and analyze the information and emotional data obtained from the terminal and generate an optimal travel plan corresponding to the user's emotions using a generative model. This makes it possible to generate and provide a personalized travel plan that takes the user's emotions into account.

[0529] A "terminal" is an electronic device used by a user to input information and receive processing results.

[0530] An "emotion engine" is software or hardware that analyzes user input information and sensor data to identify their emotional state.

[0531] A "server" is a central processing unit that receives information sent by users, analyzes it, and generates travel plans using a generative model.

[0532] A "generative model" is an algorithm or machine learning model used on a server to construct an optimal travel plan based on user input and sentiment data.

[0533] "Virtual guidance technology" is a technology that presents a generated travel plan to the user through audio and video, providing information visually and aurally.

[0534] An "external information service" is an external information provision system that can connect with the server to provide additional information related to travel planning.

[0535] A "prompt statement" is input information that gives specific instructions to a generative model to produce a desired output result.

[0536] This invention realizes a system that provides personalized travel plans to tourists visiting Japan. Users input data such as their personal interests, places they want to visit, budget, and emotions using a terminal. The terminal has an emotion sensor that extracts emotion data from the user's facial expressions and voice and transmits it to an emotion engine.

[0537] Information transmitted from the terminal is received by the server. The server analyzes the user's input information and emotional data. A generative AI model is used for the analysis, and this model acts as an algorithm to generate an appropriate travel plan based on the user's needs and emotional state. The generated plan includes optimization of destinations, activities, and itinerary, and is adjusted according to the user's emotional state. For example, if the user inputs "I would like a relaxing hot spring trip," the server will suggest relaxing activities in addition to hot spring destinations based on this input. Furthermore, if the user's emotions of joy or excitement are detected, surprise elements for further relaxation will be added to the plan.

[0538] The generated travel plan, sent from the server, is presented to the user on the terminal using virtual navigation technology. This virtual navigation technology includes features that provide information to the user visually and aurally, utilizing both audio and video. This technology allows the user to understand the plan's contents concretely and intuitively. This system allows users to enjoy a travel experience that takes their emotions into consideration, while reducing the time burden of planning and achieving a comfortable and satisfying sightseeing experience.

[0539] An example of a prompt for a generative AI model is, "Generate the optimal relaxing travel plan while taking the user's emotions into consideration." Through this sentence, specific instructions are provided to the generative model.

[0540] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0541] Step 1:

[0542] The user uses the device to input their travel preferences, such as personal interests, places they want to visit, and budget. This input data is then compiled into a user profile by the device. Furthermore, an emotion sensor built into the device monitors the user's facial expressions and voice tone, extracting emotion data. This emotion data is attached to the user profile and sent to the server in the next step. The user profile and emotion data are then sent from the device as output.

[0543] Step 2:

[0544] The user profile and sentiment data generated by the device are sent to the server. The server receives this data, stores it in a database, and then begins analysis. The analysis process generates prompt messages that take into account the user's interests and emotions. These prompt messages have a structure such as, "Generate the optimal relaxation travel plan while considering the user's emotions." The output is a set of prompt messages.

[0545] Step 3:

[0546] The server inputs the generated prompt text into the AI ​​model. Based on the prompt text, the AI ​​model creates an optimal travel plan tailored to the user's emotional state and interests. This process includes selecting destinations and activities, and optimizing the itinerary. The generated travel plan is then output.

[0547] Step 4:

[0548] The server analyzes the generated travel plan and makes necessary adjustments based on the user's emotions. For example, if the user is emotionally unstable, relaxing activities are added to the plan. The adjusted plan is sent to the device. The adjusted travel plan is generated as output.

[0549] Step 5:

[0550] The terminal presents the user with a pre-arranged travel plan received from the server using virtual navigation technology. It visualizes and explains the plan using audio and video technologies. The user can review the plan and provide feedback if necessary. The travel plan is presented to the user in a visually readable format as output.

[0551] (Application Example 2)

[0552] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0553] In travel planning, not only individual preferences and interests but also emotions at the time have a significant influence. However, existing systems have been insufficient in providing travel plans that take users' emotions into account. As a result, optimizing the user experience during travel has been difficult, and innovative methods to increase user satisfaction were needed.

[0554] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0555] In this invention, the server includes means having a function for the user to input their personal interests and travel preferences using an information processing device, means having a function for an information providing device to receive and analyze the information obtained from the information processing device and generate a travel plan using a generation model, and means having a function for presenting the generated travel plan to the user using a virtual guidance means. This makes it possible to provide a travel plan that has been dynamically modified by analyzing the user's emotions in three-dimensional video.

[0556] An "information processing device" is a device that allows users to input their personal interests and travel preferences, and has the function of analyzing that input information.

[0557] An "information provision device" is a device that receives and analyzes data obtained from an information processing device, and serves as the foundation for generating travel plans using a generative model.

[0558] A "generative model" is a mathematical or empirical model used to create the optimal travel plan based on user input and emotional data.

[0559] A "virtual guidance method" is a technology that uses three-dimensional images and audio to present travel plans to users visually and aurally.

[0560] "Emotional analysis" is a technology that uses sensors to determine a user's emotions in real time and incorporates the results into the travel plan.

[0561] "Three-dimensional imagery" is a video technology that provides users with three-dimensional visual information within a virtual space, and is used as part of virtual guidance.

[0562] An "external information exchange system" is a system for communicating with external information resources related to a coordinated travel plan, and its role is to provide users with the necessary information.

[0563] This invention realizes a system that provides personalized travel plans that take into account the user's emotions. When a user inputs their travel preferences, such as the places they want to visit and their budget, using an information processing device, this information is transmitted to an information providing device. The information providing device uses a generative AI model to generate a travel plan based on the user's input information.

[0564] Next, an emotion analysis engine is used to analyze the user's emotions at the time of input. This analysis utilizes the user's facial expressions and voice information collected by sensors. The results of this emotion analysis are reflected in the travel plan; for example, if the system receives feedback that the user wants to relax, a plan including hot springs and relaxation activities will be suggested.

[0565] The generated travel plan is provided to the user using virtual guidance. Specifically, it is possible to visually present the plan content on devices such as smart glasses and virtual display devices by combining three-dimensional images and audio. This allows users to understand the travel plan while receiving visual feedback and to make the most of the information provided in accordance with their emotions.

[0566] For example, if a user communicates via smart glasses that they "want to plan a relaxing trip to Kyoto," the generative model will generate a plan that includes Kyoto's famous sights and meditation opportunities. Based on the user's sentiment analysis, options such as visiting a quiet garden or experiencing healing music may also be added.

[0567] Example prompt: "Generate the optimal travel plan for a user who desires a peaceful Kyoto sightseeing trip, and dynamically adjust it based on their emotional state."

[0568] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0569] Step 1:

[0570] Users enter their travel preferences using a terminal. This information includes places they want to visit, their budget, and categories of interest. This data is temporarily stored on the terminal as a user profile and later transmitted to the information provider.

[0571] Step 2:

[0572] The server, acting as an information provider, receives user profiles and performs analysis using a generative AI model. Based on the input data, the generative model generates a draft of an optimal travel plan. This plan includes destinations and recommended activities.

[0573] Step 3:

[0574] The device collects user emotion data through interaction with the user, using facial recognition sensors and voice analysis software. This emotion data is fed back to the server in real time.

[0575] Step 4:

[0576] The server adjusts the draft travel plan based on the analysis results from the emotion analysis engine. If the user is seeking relaxation, the generating AI model makes adjustments such as adding destinations suitable for relaxation.

[0577] Step 5:

[0578] The server generates the final travel plan and sends it to the terminal via a virtual navigation system. The terminal then presents the plan to the user as three-dimensional images and audio. If the terminal is equipped with smart glasses, the user will see three-dimensional visuals and sounds.

[0579] Step 6:

[0580] The user reviews the presented travel plan and provides feedback. If there are any additional requests or changes, the information is entered again from the device, and the plan is updated through re-analysis on the server.

[0581] Step 7:

[0582] The server retrieves the necessary information through an external information exchange system and presents the final adjustment results to the user. This allows the user to enjoy the most appropriate travel plan, including reservation status, weather information, and real-time traffic information.

[0583] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0584] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0585] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0586] [Fourth Embodiment]

[0587] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0588] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0589] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0590] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0591] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0592] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0593] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0594] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors in the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0595] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0596] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0597] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0598] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0599] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0600] This invention is implemented as a system for providing personalized travel plans to tourists visiting Japan. In this embodiment of the system, the user, terminal, and server cooperate with each other to efficiently construct the travel plan.

[0601] Users input information about tourist destinations and experiences they are interested in using devices such as smartphones or personal computers. This input information specifically includes places they want to visit, the purpose of their trip, their budget, and desired activities. The device organizes the entered information and sends it to the server in an appropriate data format.

[0602] Next, the server receives this information and begins the process of generating a travel plan. Based on the specified conditions, the server uses a generative model to create a list of optimal tourist destinations and activities. The generative model is trained on the knowledge of professional travel guides and can propose a travel plan that best suits the user's needs. Furthermore, this also includes suggestions for accommodations and transportation.

[0603] The generated travel plan is presented to the user through virtual navigation technology. Specifically, it uses a combination of audio guides and visuals to help the user easily understand the plan. The user can review this plan and request changes if necessary.

[0604] The server, upon receiving user feedback, adjusts the travel plan. This involves replanning to better meet the user's preferences. The server also works with external data services to dynamically reflect the latest tourist information and tour options as new information becomes available.

[0605] For example, if a user enters a desire to "experience traditional Japanese culture," the server will generate a travel plan that includes historical sites in Kyoto and Nara. For instance, it might suggest activities such as renting a kimono or participating in a traditional tea ceremony. This plan is then explained through virtual navigation technology, along with a visual representation of the itinerary.

[0606] In this way, the present invention is a system that improves the travel experience of tourists visiting Japan and reduces the time burden in planning their trips.

[0607] The following describes the processing flow.

[0608] Step 1:

[0609] The user launches a dedicated application on their device and enters travel information such as the purpose of the trip, budget, desired activities, and places they want to visit. The device then organizes this information and prepares to communicate with the server.

[0610] Step 2:

[0611] The terminal sends the entered user information to the server. The transmitted information is formatted according to a specified data format to ensure efficient transmission.

[0612] Step 3:

[0613] The server receives user information from the terminal and begins data analysis. Based on the received information, the server performs analysis to understand the user's preferences and travel conditions. This establishes the requirements necessary for formulating a travel plan.

[0614] Step 4:

[0615] The server activates a generative model and uses the analyzed user information to generate a travel plan that includes appropriate tourist destinations and activities. The generative model leverages the knowledge of professional travel guides that it has learned in advance to suggest the optimal plan.

[0616] Step 5:

[0617] The server prepares to present the generated travel plan to the user using virtual navigation technology. The plan includes planned destinations, schedules, activities, and related information.

[0618] Step 6:

[0619] The terminal uses virtual navigation technology to present travel plans to users visually and audibly. The navigation content is designed to help users easily visualize the trip.

[0620] Step 7:

[0621] The user reviews the presented travel plan and sends any necessary changes or feedback from their device to the server. User feedback is a crucial element in further tailoring the plan to the user.

[0622] Step 8:

[0623] The server receives user feedback and regenerates or adjusts the travel plan. If necessary, it retrieves the latest information from external data services and incorporates it into the plan.

[0624] Step 9:

[0625] The server finalizes the travel plan and provides it to the user via the terminal. The process ends when the user confirms that the finalized plan meets their expectations.

[0626] (Example 1)

[0627] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0628] There is a need to efficiently and effectively provide personalized travel plans to tourists visiting Japan. Traditional guidebooks and fixed tour plans fail to meet the diverse needs of individual tourists, resulting in a decline in the quality of their travel experience. Furthermore, it is difficult to change travel plans or incorporate new information, leading to a significant burden of planning time and effort.

[0629] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0630] In this invention, the server includes means for the user to input their personal interests and travel requests using an information processing device, means for a central processing device to receive and analyze the information obtained from the information processing device and generate a travel plan using a generation AI model, and means for presenting the generated travel plan to the user using virtual guidance technology. This makes it possible to provide flexible and personalized travel plans that meet individual needs.

[0631] "Users" refers to travelers and tourists who use the system, and means individuals with specific travel preferences.

[0632] An "information processing device" is a device that receives input information from a user, and examples include smartphones and personal computers.

[0633] A "central processing unit" is a computer system that receives data from information processing units and processes it, and is responsible for generating travel plans that meet the user's requests.

[0634] A "generative AI model" refers to an algorithm that has been pre-trained with travel-related knowledge and is used to propose the optimal travel plan based on received data.

[0635] "Virtual guidance technology" is a technology that presents generated travel plans to users visually and audibly, and is a means of helping users easily understand the contents of the plan.

[0636] "External information provision services" refer to information sources that allow the central processing unit to acquire tourism information and the latest data from external sources and incorporate them into travel plans.

[0637] This invention is a system that provides personalized travel plans to tourists visiting Japan. This system is realized through the mutual cooperation of users, terminals, and servers.

[0638] Users input their travel preferences using devices such as smartphones or computers. This input includes places they want to visit, the purpose of their trip, their budget, and desired activities. Specifically, users input prompts such as "I want to visit art museums in Tokyo" or "I would like to experience a traditional tea ceremony in Kyoto."

[0639] The terminal organizes the input information, converts it into a communication format, and sends it to the server. The communication protocol and data format used in this process are optimized to meet the system's needs.

[0640] Upon receiving data from a terminal, the server generates a travel plan using a generative AI model. This model not only lists the most suitable tourist destinations and activities based on the user's preferences, but also suggests accommodations and transportation options. The generative AI model is trained on historical travel data and the knowledge of professional travel guides.

[0641] The generated travel plan is presented to the user using virtual navigation technology. This technology combines audio output and visual display to make the plan intuitively understandable to the user. This information presentation not only helps the user understand the plan but also encourages them to further adjust travel details and provide feedback.

[0642] Therefore, the present invention aims to provide flexible travel plans that meet the diverse needs of tourists visiting Japan and to improve the quality of their travel.

[0643] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0644] Step 1:

[0645] Users input information about tourist destinations they want to visit or activities they want to experience using devices such as smartphones or computers. The input is in text format, creating prompt statements such as "I want to visit art museums in Tokyo." This becomes the input data for the system.

[0646] Step 2:

[0647] The terminal collects data entered by the user and organizes it according to its internally configured data format. During this process, the terminal converts the data into formats such as JSON to efficiently transmit it to the server. The converted data then becomes the input sent to the server.

[0648] Step 3:

[0649] The server receives data sent from the terminal and begins analysis. The server activates a generative AI model and generates a list of tourist destinations and activities based on the user's preferences. Through data analysis and model output, a travel plan tailored to the user's needs is generated.

[0650] Step 4:

[0651] The server inputs the generated travel plan into virtual navigation technology and converts it into a presentation format for the user. This conversion process combines audio guidance and visual displays to present the generated plan in an easily understandable way for the user. This information presentation becomes the output to the user.

[0652] Step 5:

[0653] The user reviews the proposed travel plan and submits feedback to request changes as needed. This feedback includes adjustments to the timing of visits and additional requests, which then become new input data for the server.

[0654] Step 6:

[0655] The server analyzes user feedback and readjusts the travel plan. If necessary, it gathers the latest information from external information services and incorporates it into the plan. This generates updated options to provide users with a better travel plan.

[0656] (Application Example 1)

[0657] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0658] There is a need to efficiently and effectively provide personalized travel plans tailored to the individual interests and preferences of tourists visiting Japan. However, conventional systems provide tourist information unilaterally, making it difficult to consider specific experiences in advance. Furthermore, it has been difficult for users to provide feedback and for the results to be easily reflected. This invention aims to solve these problems and provide a system to improve the quality of the travel experience.

[0659] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0660] In this invention, the server includes means for a user to input their personal interests and travel preferences using an information terminal, means for a data processing device to receive and analyze the information obtained from the information terminal and generate a travel plan using generation technology, and means for presenting the generated travel plan to the user using three-dimensional virtual space technology. This makes it possible to provide a personalized travel plan based on the user's intentions and to virtually experience that plan.

[0661] An "information terminal" is a device used by a user to input their personal interests and travel preferences.

[0662] A "data processing device" is a server device that analyzes data received from an information terminal and generates a travel plan using generation technology.

[0663] "Generative technology" refers to algorithms and models used to create optimal travel plans based on user input.

[0664] "Three-dimensional virtual space technology" is a technology for presenting generated travel plans to users visually and interactively.

[0665] A "voice generation engine" is a technology that provides users with voice guidance on travel plans.

[0666] "Visual display technology" refers to technology used to visually display scenes and information related to travel planning.

[0667] "External information services" refer to external data provision services that are partnered with to provide the latest tourist information and options.

[0668] "Evaluation information" refers to data collected from user feedback and opinions, used to adjust and regenerate travel plans.

[0669] The system for realizing this application example is configured as follows: The user inputs their interests and travel preferences using an information terminal. This terminal could be a smartphone or a personal computer. After this input is complete, the information is received and analyzed by a data processing device (server). This analysis includes generating a travel plan using a generative AI model, automatically creating a travel plan optimized for the user's conditions.

[0670] The generated travel plan is presented to the user using three-dimensional virtual space technology. For this purpose, hardware such as VR goggles and smartphones are used. The software utilizes Unity 3D to build the virtual reality environment and leverages OpenAI APIs to generate the plan. Information on restaurants, accommodations, and tourist attractions is kept up-to-date by linking with external information services. Furthermore, Google Cloud's Text-to-Speech API is used as the speech generation engine, allowing the travel plan to be explained in audio.

[0671] Users can review the provided travel plans and experience them in a virtual space beforehand. Wearing VR goggles, users can walk around the virtual tourist destinations unfolding before their eyes, experiencing the actual plan. This allows users to concretely imagine the atmosphere and activities of the places they want to visit before traveling.

[0672] For example, if a prompt such as "Please tell me about beautiful places to see autumn foliage in Japan and their highlights" is input into the AI ​​model, the system will suggest the best autumn foliage spots and related activities for the user. This allows tourists visiting Japan to obtain valuable advance information to fully enjoy the charms of Japan.

[0673] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0674] Step 1:

[0675] Users use an information terminal to input their interests and travel preferences. This input includes information such as desired destinations, budget, and activities of interest. The input data is organized within the terminal and converted into a format that can be sent to the server.

[0676] Step 2:

[0677] The server receives user information sent from the terminal. Based on this information, it performs data analysis and generates a travel plan using a generative AI model. At this stage, tourist destinations, accommodations, and transportation methods that meet the user's conditions are optimized and output as a specific plan.

[0678] Step 3:

[0679] The server presents the generated travel plan to the user using three-dimensional virtual space technology. Unity 3D is used to visually represent the information, outputting the travel plan as detailed visual information. Google Cloud's Text-to-Speech API is used for audio guidance, explaining the plan to the user verbally.

[0680] Step 4:

[0681] Users virtually experience a travel plan presented within a virtual space. This experience includes virtual sightseeing and pre-exploration of activities through VR goggles. The information users gather through this experience will be used for future feedback.

[0682] Step 5:

[0683] Users provide feedback based on their experience in the virtual space. This feedback is sent to the server and used as data to readjust the plan. Based on the received evaluation information, the server updates the travel plan as needed and proposes a new plan.

[0684] Step 6:

[0685] The server connects with external information services to retrieve the latest tourist information relevant to the adjusted travel plan. This information includes the latest event information and newly added activity options. The retrieved data is then provided to the user again, completing the final travel plan.

[0686] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0687] This invention provides a system that offers personalized travel plans to tourists visiting Japan, achieving advanced planning that takes user emotions into account. This system functions through the collaboration of a terminal, server, emotion engine, generative model, and virtual guidance technology.

[0688] Users use the device to input their travel preferences, such as personal interests, places they want to visit, and their budget. The device creates a user profile based on this information and sends it to the server. User feedback and emotions expressed during the input process are sent to the emotion engine via sensors and software on the device.

[0689] The server receives and analyzes information sent by the user. The generative model generates an optimal travel plan based on this analyzed information and the user's emotions recognized by the emotion engine. The generated plan includes destinations, activity selections, and itinerary optimization, with adjustments made according to the user's emotional state.

[0690] The emotion engine analyzes user emotions based on their input and the context of the conversation, and feeds the results back to the server. This feedback is used to adjust the travel plan; for example, if the user is feeling anxious, suggestions for more relaxing plans or increased support will be provided.

[0691] The generated travel plan is presented to the user using virtual navigation technology. The device visually explains the plan to the user through audio and video. This presentation is designed to allow the user to easily understand the plan and proceed with their trip with confidence.

[0692] For example, if a user enters "I would like a relaxing hot spring trip," the server will suggest hot spring resorts and relaxing activities. If the emotion engine recognizes the user's feelings of joy or excitement, surprise elements for further relaxation will be added to the plan. In this way, a travel plan tailored to the user's emotions is formed and provided to the user.

[0693] Through this system, users can enjoy a travel experience that takes their emotions into consideration, reducing the time burden of planning and enabling comfortable and satisfying sightseeing.

[0694] The following describes the processing flow.

[0695] Step 1:

[0696] The user uses a device to input information such as the purpose of their trip, their budget, the tourist destinations they want to visit, and the activities they want to experience. The device then neatly organizes this information and prepares it as data to be sent to the server.

[0697] Step 2:

[0698] The terminal sends the entered data to the server. The information is transmitted using a secure communication protocol, minimizing time delays.

[0699] Step 3:

[0700] The server receives information from the terminal and analyzes it by comparing it with past data in the database. The information is then organized, taking into account user preferences, past travel templates, and general tourism trends.

[0701] Step 4:

[0702] The server uses the analysis results to launch a generative model, which then provides suggestions based on the user's personal data and travel data. The generated travel plan includes details of planned destinations and specific activities, and is designed to best meet the user's preferences.

[0703] Step 5:

[0704] The emotion engine operates using sensors and analysis software within the device to capture emotional data that emerges during user input and responses. This includes facial expressions, voice tone, input speed, and patterns.

[0705] Step 6:

[0706] The emotion engine analyzes the acquired data to recognize the user's emotions. For example, it determines whether the user is feeling stressed, anxious, or relaxed, and sends the result to the server.

[0707] Step 7:

[0708] The server receives feedback from the emotion engine and adjusts the travel plan. Depending on the recognized emotions, suggestions to enhance relaxation and support options to provide reassurance are added to the plan.

[0709] Step 8:

[0710] The server sends the finalized travel plan to the terminal, which then uses virtual guidance technology to present the plan to the user via audio and video. The user then has the opportunity to review the plan and provide further feedback.

[0711] Step 9:

[0712] The user reviews the plan and gives final approval. They can request further changes if necessary, but the plan is usually finalized at this stage. Once the plan is finalized, the user can begin preparing for their trip.

[0713] (Example 2)

[0714] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0715] In recent years, there has been a growing demand for personalized travel planning that better suits individual needs and circumstances. However, traditional travel planning systems have struggled to generate and adjust plans that reflect user emotions, failing to improve the quality of the user experience. As a result, users may be dissatisfied with their travel plans.

[0716] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0717] In this invention, the server includes means for the user to input personal interests, travel preferences, and emotional data using a terminal; means for the terminal to transmit the user's emotions to an emotion engine via an emotion sensor; and means for the server to receive and analyze the information and emotional data obtained from the terminal and generate an optimal travel plan corresponding to the user's emotions using a generative model. This makes it possible to generate and provide a personalized travel plan that takes the user's emotions into account.

[0718] A "terminal" is an electronic device used by a user to input information and receive processing results.

[0719] An "emotion engine" is software or hardware that analyzes user input information and sensor data to identify their emotional state.

[0720] A "server" is a central processing unit that receives information sent by users, analyzes it, and generates travel plans using a generative model.

[0721] A "generative model" is an algorithm or machine learning model used on a server to construct an optimal travel plan based on user input and sentiment data.

[0722] "Virtual guidance technology" is a technology that presents a generated travel plan to the user through audio and video, providing information visually and aurally.

[0723] An "external information service" is an external information provision system that can connect with the server to provide additional information related to travel planning.

[0724] A "prompt statement" is input information that gives specific instructions to a generative model to produce a desired output result.

[0725] This invention realizes a system that provides personalized travel plans to tourists visiting Japan. Users input data such as their personal interests, places they want to visit, budget, and emotions using a terminal. The terminal has an emotion sensor that extracts emotion data from the user's facial expressions and voice and transmits it to an emotion engine.

[0726] Information transmitted from the terminal is received by the server. The server analyzes the user's input information and emotional data. A generative AI model is used for the analysis, and this model acts as an algorithm to generate an appropriate travel plan based on the user's needs and emotional state. The generated plan includes optimization of destinations, activities, and itinerary, and is adjusted according to the user's emotional state. For example, if the user inputs "I would like a relaxing hot spring trip," the server will suggest relaxing activities in addition to hot spring destinations based on this input. Furthermore, if the user's emotions of joy or excitement are detected, surprise elements for further relaxation will be added to the plan.

[0727] The generated travel plan, sent from the server, is presented to the user on the terminal using virtual navigation technology. This virtual navigation technology includes features that provide information to the user visually and aurally, utilizing both audio and video. This technology allows the user to understand the plan's contents concretely and intuitively. This system allows users to enjoy a travel experience that takes their emotions into consideration, while reducing the time burden of planning and achieving a comfortable and satisfying sightseeing experience.

[0728] An example of a prompt for a generative AI model is, "Generate the optimal relaxing travel plan while taking the user's emotions into consideration." Through this sentence, specific instructions are provided to the generative model.

[0729] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0730] Step 1:

[0731] The user uses the device to input their travel preferences, such as personal interests, places they want to visit, and budget. This input data is then compiled into a user profile by the device. Furthermore, an emotion sensor built into the device monitors the user's facial expressions and voice tone, extracting emotion data. This emotion data is attached to the user profile and sent to the server in the next step. The user profile and emotion data are then sent from the device as output.

[0732] Step 2:

[0733] The user profile and sentiment data generated by the device are sent to the server. The server receives this data, stores it in a database, and then begins analysis. The analysis process generates prompt messages that take into account the user's interests and emotions. These prompt messages have a structure such as, "Generate the optimal relaxation travel plan while considering the user's emotions." The output is a set of prompt messages.

[0734] Step 3:

[0735] The server inputs the generated prompt text into the AI ​​model. Based on the prompt text, the AI ​​model creates an optimal travel plan tailored to the user's emotional state and interests. This process includes selecting destinations and activities, and optimizing the itinerary. The generated travel plan is then output.

[0736] Step 4:

[0737] The server analyzes the generated travel plan and makes necessary adjustments based on the user's emotions. For example, if the user is emotionally unstable, relaxing activities are added to the plan. The adjusted plan is sent to the device. The adjusted travel plan is generated as output.

[0738] Step 5:

[0739] The terminal presents the user with a pre-arranged travel plan received from the server using virtual navigation technology. It visualizes and explains the plan using audio and video technologies. The user can review the plan and provide feedback if necessary. The travel plan is presented to the user in a visually readable format as output.

[0740] (Application Example 2)

[0741] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0742] In travel planning, not only individual preferences and interests but also emotions at the time have a significant influence. However, existing systems have been insufficient in providing travel plans that take users' emotions into account. As a result, optimizing the user experience during travel has been difficult, and innovative methods to increase user satisfaction were needed.

[0743] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0744] In this invention, the server includes means having a function for the user to input their personal interests and travel preferences using an information processing device, means having a function for an information providing device to receive and analyze the information obtained from the information processing device and generate a travel plan using a generation model, and means having a function for presenting the generated travel plan to the user using a virtual guidance means. This makes it possible to provide a travel plan that has been dynamically modified by analyzing the user's emotions in three-dimensional video.

[0745] An "information processing device" is a device that allows users to input their personal interests and travel preferences, and has the function of analyzing that input information.

[0746] An "information provision device" is a device that receives and analyzes data obtained from an information processing device, and serves as the foundation for generating travel plans using a generative model.

[0747] A "generative model" is a mathematical or empirical model used to create the optimal travel plan based on user input and emotional data.

[0748] A "virtual guidance method" is a technology that uses three-dimensional images and audio to present travel plans to users visually and aurally.

[0749] "Emotional analysis" is a technology that uses sensors to determine a user's emotions in real time and incorporates the results into the travel plan.

[0750] "Three-dimensional imagery" is a video technology that provides users with three-dimensional visual information within a virtual space, and is used as part of virtual guidance.

[0751] An "external information exchange system" is a system for communicating with external information resources related to a coordinated travel plan, and its role is to provide users with the necessary information.

[0752] This invention realizes a system that provides personalized travel plans that take into account the user's emotions. When a user inputs their travel preferences, such as the places they want to visit and their budget, using an information processing device, this information is transmitted to an information providing device. The information providing device uses a generative AI model to generate a travel plan based on the user's input information.

[0753] Next, an emotion analysis engine is used to analyze the user's emotions at the time of input. This analysis utilizes the user's facial expressions and voice information collected by sensors. The results of this emotion analysis are reflected in the travel plan; for example, if the system receives feedback that the user wants to relax, a plan including hot springs and relaxation activities will be suggested.

[0754] The generated travel plan is provided to the user using virtual guidance. Specifically, it is possible to visually present the plan content on devices such as smart glasses and virtual display devices by combining three-dimensional images and audio. This allows users to understand the travel plan while receiving visual feedback and to make the most of the information provided in accordance with their emotions.

[0755] For example, if a user communicates via smart glasses that they "want to plan a relaxing trip to Kyoto," the generative model will generate a plan that includes Kyoto's famous sights and meditation opportunities. Based on the user's sentiment analysis, options such as visiting a quiet garden or experiencing healing music may also be added.

[0756] Example prompt: "Generate the optimal travel plan for a user who desires a peaceful Kyoto sightseeing trip, and dynamically adjust it based on their emotional state."

[0757] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0758] Step 1:

[0759] Users enter their travel preferences using a terminal. This information includes places they want to visit, their budget, and categories of interest. This data is temporarily stored on the terminal as a user profile and later transmitted to the information provider.

[0760] Step 2:

[0761] The server acts as an information provider, receiving user profiles and performing analysis using a generative AI model. Based on the input data, the generative model generates a draft of an optimal travel plan. This plan includes destinations and recommended activities.

[0762] Step 3:

[0763] The device collects user emotion data through interaction with the user, using facial recognition sensors and voice analysis software. This emotion data is fed back to the server in real time.

[0764] Step 4:

[0765] The server adjusts the draft travel plan based on the analysis results from the emotion analysis engine. If the user is seeking relaxation, the generating AI model makes adjustments such as adding destinations suitable for relaxation.

[0766] Step 5:

[0767] The server generates the final travel plan and sends it to the terminal via a virtual navigation system. The terminal then presents the plan to the user as three-dimensional images and audio. If the terminal is equipped with smart glasses, the user will see three-dimensional visuals and sounds.

[0768] Step 6:

[0769] The user reviews the presented travel plan and provides feedback. If there are any additional requests or changes, the information is entered again from the device, and the plan is updated through re-analysis on the server.

[0770] Step 7:

[0771] The server retrieves the necessary information through an external information exchange system and presents the final adjustment results to the user. This allows the user to enjoy the most appropriate travel plan, including reservation status, weather information, and real-time traffic information.

[0772] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0773] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0774] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0775] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0776] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0777] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0778] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0779] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0780] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0781] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0782] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0783] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0784] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0785] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0786] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0787] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0788] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0789] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0790] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0791] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0792] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0793] The following is further disclosed regarding the embodiments described above.

[0794] (Claim 1)

[0795] A means for users to input their personal interests and travel preferences using a device,

[0796] A means by which a server receives and analyzes information obtained from the terminal and generates a travel plan using a generative model,

[0797] A means of presenting a generated travel plan to the user using virtual guidance technology,

[0798] A means of adjusting travel plans based on user feedback,

[0799] A means of providing information related to the adjusted travel plan in conjunction with external data services,

[0800] A system that includes this.

[0801] (Claim 2)

[0802] The system according to claim 1, wherein the virtual guidance technology explains the travel plan using audio and video.

[0803] (Claim 3)

[0804] The system according to claim 1, wherein the generative model regenerates a travel plan based on user feedback.

[0805] "Example 1"

[0806] (Claim 1)

[0807] A means by which users input their personal interests and travel preferences using an information processing device,

[0808] A means for a central processing unit to receive and analyze information obtained from the aforementioned information processing device and generate a travel plan using a generated AI model,

[0809] A means of presenting the generated travel plan to the user using virtual guidance technology,

[0810] A means of adjusting travel plans based on user feedback,

[0811] A means of providing information related to the adjusted travel plan in conjunction with external information provision services,

[0812] A system that includes this.

[0813] (Claim 2)

[0814] The system according to claim 1, wherein the virtual guidance technology explains the travel plan using audio and visual displays.

[0815] (Claim 3)

[0816] The system according to claim 1, wherein the generating AI model regenerates a travel plan based on user feedback.

[0817] "Application Example 1"

[0818] (Claim 1)

[0819] A means for users to input their personal interests and travel preferences using an information terminal,

[0820] A data processing device receives and analyzes information obtained from the aforementioned information terminal, and generates a travel plan using generation technology.

[0821] A means of presenting the generated travel plan to the user using three-dimensional virtual space technology,

[0822] A means of adjusting travel plans based on user ratings,

[0823] A means of providing information related to the coordinated travel plan in conjunction with external information services,

[0824] A means of experiencing a travel plan in advance within a virtual space,

[0825] A system that includes this.

[0826] (Claim 2)

[0827] The system according to claim 1, wherein the three-dimensional virtual space technology explains a travel plan using a voice generation engine and video display technology.

[0828] (Claim 3)

[0829] The system according to claim 1, wherein the generation technology regenerates a travel plan based on user evaluation information, and makes a portion of it available for virtual experience in a virtual space.

[0830] "Example 2 of combining an emotion engine"

[0831] (Claim 1)

[0832] A means by which users input personal interests, travel preferences, and emotional data using a device,

[0833] The terminal includes means for transmitting the user's emotions to an emotion engine via an emotion sensor,

[0834] The server receives and analyzes information and emotional data obtained from the terminal, and generates an optimal travel plan that corresponds to the user's emotions using a generative model.

[0835] A means of presenting a generated travel plan to the user using virtual guidance technology via audio and video,

[0836] A means of adjusting travel plans while taking into account the user's emotional state,

[0837] A means of providing information related to the coordinated travel plan in conjunction with external information services,

[0838] A system that includes this.

[0839] (Claim 2)

[0840] The system according to claim 1, wherein a generative AI model regenerates a travel plan based on the user's emotions.

[0841] (Claim 3)

[0842] The system according to claim 1, wherein the virtual guidance technology presents a travel plan using virtual reality technology.

[0843] "Application example 2 when combining with an emotional engine"

[0844] (Claim 1)

[0845] A means having a function for a user to input their personal interests and travel preferences using an information processing device,

[0846] The information providing device has a function to receive and analyze information obtained from the aforementioned information processing device and generate a travel plan using a generation model.

[0847] A means having the function of presenting the generated travel plan to the user using a virtual guidance means,

[0848] A means of adjusting travel plans based on user feedback,

[0849] A means having the function of providing information related to the adjusted travel plan in conjunction with an external information exchange system,

[0850] A means that analyzes the user's emotions using sensors and has the function of dynamically changing the travel plan using the analysis results,

[0851] A means having a function that visually guides the user through a travel plan using a virtual display device and three-dimensional images,

[0852] A system that includes this.

[0853] (Claim 2)

[0854] The system according to claim 1, characterized in that the virtual guidance means presents a travel plan using audio and three-dimensional images, and the content changes according to the user's emotions.

[0855] (Claim 3)

[0856] The system according to claim 1, characterized in that the generative model regenerates a travel plan based on the results of the user's sentiment analysis and provides the user with an optimized plan. [Explanation of Symbols]

[0857] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for users to input their personal interests and travel preferences using a device, A means by which a server receives and analyzes information obtained from the terminal and generates a travel plan using a generative model, A means of presenting a generated travel plan to the user using virtual guidance technology, A means of adjusting travel plans based on user feedback, A means of providing information related to the adjusted travel plan in conjunction with external data services, A system that includes this.

2. The system according to claim 1, wherein the virtual guidance technology explains the travel plan using audio and video.

3. The system according to claim 1, wherein the generative model regenerates a travel plan based on user feedback.

Images