system

Abstract

Provide a system. 【Solution means】 A device for inputting travel information, A device for automatically selecting tourist destinations based on the travel information, A device for forming a visit schedule for the selected tourist destinations, A device for displaying the formed visit schedule on a user terminal, A device for updating a travel plan based on the user's preferences, A device for realistically displaying tourist destination information and travel routes through a visual device, A device for analyzing voice information to receive instructions and perform language conversion, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is a problem that it is difficult for travelers to construct an optimal tourism plan when facing language and cultural barriers and information overload situations. In particular, at a place visited for the first time, it is difficult to select appropriate tourist destinations and create an efficient schedule, and it is difficult to obtain a highly satisfactory travel experience.

Means for Solving the Problems

[0005] The present invention provides means for inputting traveler information and automatically selecting tourist destinations based on the input information. Further, by generating an access schedule for the selected tourist destinations and displaying it on a user terminal, it can be easily used by travelers. Furthermore, by incorporating means for dynamically updating the travel plan according to the user's preferences, a flexible plan tailored to individual needs becomes possible.

[0006] "Travel information" refers to information provided by travelers, including their destination, interests, length of stay, and specific requests.

[0007] A "tourist destination" refers to a place or facility that travelers visit for its intended purpose, and includes historical buildings, natural landscapes, and amusement facilities.

[0008] A "visit schedule" is a timeline organized in chronological order to allow travelers to efficiently visit tourist destinations.

[0009] A "user terminal" is an electronic device carried by travelers, used for inputting tourist information, checking plans, navigation, and other similar functions.

[0010] A "travel plan" is an overall plan that includes the order and itinerary of visiting tourist attractions, optimized based on information provided by the traveler. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0012] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0013] First, the terms used in the following description will be explained.

[0014] In the following embodiments, a processor with a reference number (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Further, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the tagged RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by a processor.

[0016] In the following embodiments, the tagged storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0017] In the following embodiments, the tagged communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F controls communication between a plurality of computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when three or more matters are connected and expressed by "and / or", the same concept as "A and / or B" is applied.

[0019] [First Embodiment]

[0020] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0021] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] This invention relates to a system that provides travelers with personalized sightseeing guides and travel coordination services. Through interaction between a server, a terminal, and a user, this system handles the input of travel information, the selection of tourist destinations, the generation of visit schedules, and the display and updating of travel plans.

[0033] The user enters travel information such as their interests, planned destinations, and travel duration into the device. This information is sent to the server, which uses an AI model to select appropriate tourist destinations based on the entered information. The server refers to a database of tourist destinations to identify the places best suited to the user's interests and generates a visit schedule. The generated schedule is then presented to the user through the device.

[0034] The server provides an efficient travel plan by optimizing travel routes between tourist destinations. For example, if a user visits Tokyo, the server selects tourist destinations such as "Tokyo Tower" and "Akihabara anime shops" and generates the optimal order of visits.

[0035] The device visually organizes and displays information to make it easy for users to review their plans. Furthermore, the device supports changes to conditions based on user preferences and real-time schedule updates during the trip.

[0036] Users can check their travel plans on their devices and make changes on the spot. Furthermore, they can use the device's navigation and translation functions during their trip to navigate smoothly and communicate with locals.

[0037] This invention enhances the overall travel experience by providing dynamic and flexible travel plans that enable travelers to have a satisfying travel experience.

[0038] The following describes the processing flow.

[0039] Step 1:

[0040] The user enters travel information into the terminal. This includes the travel destination, areas of interest, length of stay, and specific requests.

[0041] Step 2:

[0042] The terminal packages the user's input information and sends it to the server. This information is necessary for processing in the next step.

[0043] Step 3:

[0044] The server analyzes the travel information it receives. An AI model is applied to extract tourist destinations, restaurants, and events from the database based on the user's interests.

[0045] Step 4:

[0046] The server generates a visit schedule that takes into account travel between tourist destinations based on the extracted information. By creating a plan that visits tourist destinations in the optimal order, it improves the efficiency of the trip.

[0047] Step 5:

[0048] The server sends the generated visit schedule and related information to the terminal. The formatted data is provided in a way that is easy for the user to understand.

[0049] Step 6:

[0050] The terminal analyzes data received from the server and displays it in the user interface. It presents visually organized information, helping users easily understand the plan.

[0051] Step 7:

[0052] The user reviews the proposed plan. They make any necessary changes to the plan and finalize the schedule.

[0053] Step 8:

[0054] When traveling, users utilize their devices to access real-time navigation and translation features, enabling smoother travel between tourist destinations and easier communication with locals.

[0055] Step 9:

[0056] After their trip, users enter feedback about their experience into their device. This information is collected on a server and used to improve the AI ​​model.

[0057] (Example 1)

[0058] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0059] Modern travelers need to combine data from diverse sources to design their own travel routes that suit their interests, which requires considerable time and effort. They also face challenges in responding to sudden changes in plans during their trip and communicating in different language environments.

[0060] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0061] In this invention, the server includes means for acquiring travel information, means for automatically selecting tourist destination candidates based on the travel information using a generative model, and means for creating a visit schedule based on the selected tourist destination candidates. This enables travelers to automatically generate efficient and personalized travel plans, and to respond to immediate schedule changes and cross-cultural communication.

[0062] "Travel information" refers to data about destinations, itineraries, activities of interest, and tourist attractions that travelers need when planning their trips.

[0063] A "generative model" is an algorithm that derives the optimal result based on input data, and in this invention, it is a model that includes artificial intelligence technology used for selecting tourist destinations.

[0064] A "potential tourist destination" is a list of tourist spots that travelers can visit, selected based on travel information.

[0065] A "visiting schedule" is a travel plan that shows the order in which selected tourist destinations will be visited and the time allocation for each.

[0066] A "user device" refers to a device used by travelers to input and receive information, and includes electronic devices such as smartphones and tablets.

[0067] "Means for modifying travel plans" refers to systems or processes for restructuring and updating travel plans in response to user change requests.

[0068] This invention is a system for automating travel planning and optimizing it to user needs. The system operates using terminals, servers, and generative AI models.

[0069] Terminal-based operation and information gathering

[0070] Users can input travel information using devices such as smartphones and tablets. This information includes travel destinations, available dates, activities of interest, and places to visit.

[0071] Server-based information processing and analysis

[0072] Travel information sent from the device is aggregated on a server. The server uses a cloud-based generative AI model to analyze the travel information and form a list of the most appropriate tourist destinations. The server retrieves detailed data about tourist destinations from a database and creates a visit schedule tailored to the user's interests and priorities.

[0073] Specifically, the server utilizes a generative model to generate tourist destination selection prompts based on input data. For example, it might use a prompt like, "The user plans to visit Tokyo and is interested in historical sites and anime. Please suggest recommended tourist destinations and create a schedule with an efficient order of visits."

[0074] Providing information to users and interaction

[0075] As a concrete example, if a user visits Tokyo, the system will present a plan on the user's device that includes visiting "tourist attractions" and "anime-related spots." The device will display information tailored to the user's needs and allow for real-time plan updates, supporting the user in enjoying their trip comfortably.

[0076] The device not only displays travel plans but also includes a translation function to overcome language barriers during travel. This allows users to enjoy their trip while smoothly obtaining local information.

[0077] In this way, the system can provide users with a dynamic, flexible, and personalized travel experience.

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] The user enters travel information using a terminal.

[0081] Specifically, you will enter information such as the city you want to visit, your travel dates, and categories of interest (for example, history, food, entertainment). The entered information will be sent from your device to the server in text format.

[0082] Step 2:

[0083] The server analyzes the received travel information and generates a prompt message.

[0084] The server generates prompt statements suitable for the AI ​​model based on the input data. In doing so, it references similar historical data in the database to support effective analysis. For example, it might generate a prompt statement such as, "The user plans to visit Tokyo and is interested in history and anime." This prompt statement then becomes the input for the next step.

[0085] Step 3:

[0086] The server uses a generative AI model to select potential tourist destinations based on the prompt message.

[0087] The generative AI model analyzes the prompt text, evaluates the tourist destination information in the database, and generates a list of tourist destinations that match the user's interests. This list includes data about each tourist destination (e.g., popularity, rating, required time, etc.).

[0088] Step 4:

[0089] The server generates a visit schedule.

[0090] The program takes a selected list of tourist destinations as input and calculates the optimal order of visits. It applies a route optimization algorithm that considers the traveler's distance and time efficiency. The output is a structured itinerary of visits.

[0091] Step 5:

[0092] The terminal displays the visit schedule to the user.

[0093] The visit schedule is displayed visually through the user interface. Users can check the relative locations on a map or view the schedule in a calendar format.

[0094] Step 6:

[0095] Users review the presented travel plan and request changes as needed.

[0096] Users use their devices to input feedback and change requests regarding their travel plans. These requests are then sent back to the server, and the travel plans are updated.

[0097] Step 7:

[0098] During their trip, users utilize the device's navigation function.

[0099] The device provides real-time navigation information based on the visit schedule. It also utilizes translation functions to help users understand local information and facilitate communication.

[0100] This entire process allows travelers to efficiently enjoy a personalized travel experience.

[0101] (Application Example 1)

[0102] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0103] There is a growing need to provide personalized travel plans that cater to the diverse interests and needs of travelers, and to support smooth travel and communication by providing real-time visual and auditory information during their trips. However, conventional systems have been unable to adequately meet these needs, making it difficult to improve the travel experience.

[0104] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0105] In this invention, the server includes a device for inputting travel information, a device for automatically selecting tourist destinations based on the travel information, and a device for forming a visit schedule for the selected tourist destinations. This enables the provision of travel plans optimized to the user's interests and the visual and auditory presentation of information during the trip.

[0106] A "travel information input device" is a device used to collect information such as a traveler's interests and preferences, accommodation, and itinerary, and input it into a system.

[0107] A "device that automatically selects tourist destinations" is a device that automatically selects appropriate tourist destinations using an algorithm based on the travel information entered.

[0108] A "device for creating a visit schedule" is a system component that generates a schedule for efficiently visiting selected tourist destinations.

[0109] A "device for displaying on the user's terminal" is a device that visually organizes the generated visit schedule and tourist destination information and presents it to the user through a user interface.

[0110] A "device that updates travel plans based on user preferences" is a system that dynamically modifies existing travel plans in response to user feedback and new preferences.

[0111] A "device that realistically displays tourist information and travel routes through a visual device" refers to a technology that uses smart glasses or similar devices to present tourist information and navigation as augmented reality.

[0112] A "device that analyzes voice information, receives instructions, and performs language conversion" is a system component that understands user instructions using voice recognition technology and performs translation processing for multilingual support.

[0113] The system for realizing this invention mainly consists of a server, a terminal, and user interaction. The server includes a device for inputting travel information, a device for automatically selecting tourist destinations, and a device for generating a visit itinerary. Specifically, the server uses generative AI models such as TENSORFLOW® or PyTorch to analyze travel information sent by the user and select the optimal tourist destination.

[0114] The terminal is equipped with a visual device and a voice analysis device, and visually displays the generated visit schedule to the user. The visual device, such as smart glasses, plays a role in extending the visual information displayed to the user in the real environment and can accept voice commands. For voice analysis, Google® Speech-to-Text and Google Translate API are used to analyze the user's voice instructions and perform language conversion.

[0115] Users can input travel information and create travel plans that include tourist destinations tailored to their interests by operating the terminal. To assist with travel, the system provides route information using GPS and compass functions and implements real-time navigation based on the user's current location.

[0116] For example, if a user plans a trip to Tokyo and gives a voice command saying, "Take me to Tokyo Tower," the smart glasses will display the optimal route from the user's current location to Tokyo Tower. Along the way, users can obtain information about points of interest simply by looking at them, and local languages ​​are translated into other languages, enabling natural communication with locals.

[0117] An example of a prompt message could be: "If the user's interest is related to anime, please select a recommended tourist destination."

[0118] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0119] Step 1:

[0120] Users input travel information through the terminal's interface. This information includes areas of interest, planned destinations, and travel itinerary. This information is then transmitted from the terminal to the server as digital data.

[0121] Step 2:

[0122] The server analyzes the received travel information and uses a generative AI model (for example, a model based on TensorFlow or PyTorch) to select tourist destinations that are suitable for the user's interests. Based on the travel information received as input, it calculates the degree of fit for the candidate tourist destinations stored in the database and outputs a list of the most suitable tourist destinations.

[0123] Step 3:

[0124] The server generates a travel itinerary based on the selected tourist destinations. Based on the input tourist destination information, it creates an optimal schedule considering the available visiting times and travel times for each location. The generated schedule contributes to the user's efficient travel planning.

[0125] Step 4:

[0126] The terminal visually displays the visit schedule and tourist destination information retrieved from the server to the user. This information is then displayed as an overlay on the real world through a visual device (e.g., smart glasses). This allows the user to intuitively grasp the tourist destination information in relation to their actual location.

[0127] Step 5:

[0128] Users can check and modify their schedules using eye-tracking or voice commands. The device understands the commands using voice analysis technology (such as Google Speech-to-Text) and sends them to the server as new input data. This allows the system to re-select tourist destinations based on the new criteria and update the schedule.

[0129] Step 6:

[0130] The device uses GPS and compass functions to provide real-time navigation information to assist with on-site travel. Taking the current location as input, it outputs the optimal route to the destination, helping users travel smoothly on foot or using public transportation.

[0131] Step 7:

[0132] When a user needs translation assistance while in a foreign country, the device uses the Google Translate API to translate the input into the appropriate language. It translates the input voice and text data and presents the output to the user, supporting smooth communication.

[0133] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0134] This invention relates to a system that provides personalized tourist guides and travel coordination services to travelers. In particular, by combining it with an emotion engine that recognizes the user's emotions, more sophisticated customization becomes possible.

[0135] The user enters travel information into the device, including their destination, interests, and length of stay. The device sends this information to the server, but before that, it has a function to collect emotional data through the user's facial expressions and tone of voice. The emotion engine analyzes this data to identify the user's current emotional state.

[0136] The server considers emotional data along with the received travel information and uses an AI model to optimally select tourist destinations and experiences. If the user is in a relaxed mood, quiet, nature-based tourist destinations are likely to be included in the plan. On the other hand, if the user is stressed, relaxing leisure activities and activities will be suggested.

[0137] The server generates a visit schedule that reflects emotional data and sends it to the terminal. The terminal visualizes this information and displays it to the user. For example, if the user is feeling overwhelmed in Tokyo, the server might suggest relaxing options such as "a picnic at Shinjuku Gyoen."

[0138] Throughout the trip, the user's emotions are continuously monitored, and an emotion engine feeds this data back to the server in real time. Based on this information, the server re-selects the visit schedule and sightseeing destinations if necessary. If the user starts to feel tired, adjustments are made, such as easing the schedule and increasing relaxation time at the hotel.

[0139] This invention is a system that, in this way, utilizes user emotional data to provide travelers with the most suitable plans, thereby enabling a richer travel experience.

[0140] The following describes the processing flow.

[0141] Step 1:

[0142] The user enters travel information into the device. This information includes the travel destination, interests, length of stay, and specific requests.

[0143] Step 2:

[0144] The device uses facial recognition cameras and voice analysis functions to detect the user's current emotional state. This allows it to collect data that identifies the user's stress level and level of enjoyment.

[0145] Step 3:

[0146] The device sends collected travel information and sentiment data to the server. This data transmission initiates processing on the server side.

[0147] Step 4:

[0148] The server uses an AI model to analyze travel information and sentiment data. This allows it to select the most suitable tourist destinations and activities for the user's current situation.

[0149] Step 5:

[0150] The server generates a visit schedule that takes the user's emotional state into account. For example, if the user is seeking relaxation, quiet tourist spots and relaxation facilities will be added to the plan.

[0151] Step 6:

[0152] The server generates a schedule and sends it to the terminal. The terminal visually organizes this information and presents it to the user in an easy-to-understand format.

[0153] Step 7:

[0154] Users review the proposed plan from their device and make changes or approvals as needed, ensuring flexibility in their travel plans.

[0155] Step 8:

[0156] During the trip, the device periodically monitors the user's emotional state and feeds that data back to the server. Based on this, the server dynamically adjusts the visit schedule.

[0157] Step 9:

[0158] If user stress is detected through the emotion engine, it's possible to ease the pre-set schedule or suggest new relaxation options.

[0159] (Example 2)

[0160] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0161] Traditional travel planning systems were unable to respond to individual user emotions and real-time changes, making it difficult to provide travelers with the optimal sightseeing experience. Furthermore, the inability to flexibly adjust schedules based on emotions meant that stress during travel could not be reduced.

[0162] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0163] In this invention, the server includes means for automatically selecting tourist destinations based on travel-related information and emotional data, means for generating emotional data by analyzing the user's facial expressions and voice data, and means for monitoring the user's emotional state in real time in order to realistically adjust the itinerary. This enables a personalized travel experience based on the user's current emotional state and flexible schedule management that reduces stress.

[0164] "Travel-related information" refers to information necessary for planning a trip, such as a traveler's destination, interests, and length of stay.

[0165] "Emotional data" refers to information about the user's current mental state and emotions, analyzed from their facial expressions and voice.

[0166] A "tourist spot" is a place of interest or attraction that is selected for travelers to visit.

[0167] A "travel itinerary" refers to a schedule that includes the order and timing of visits to each tourist spot a traveler will be visiting.

[0168] "User terminal" refers to a digital device used by a user to input and receive travel plans.

[0169] "Monitoring in real time" means continuously and immediately evaluating and recording the user's emotional state.

[0170] A "translation function" is a feature that supports language conversion between different languages.

[0171] This system combines multiple technological elements to provide travelers with personalized travel plans.

[0172] The user enters information about their travel destination, interests, and length of stay into the device. The device uses its built-in camera and microphone to record the user's facial expressions and voice tone, and collects emotional data based on this. Common digital devices are used for this data collection.

[0173] The emotion engine uses Microsoft® Azure® Face API and IBM Watson® Tone Analyzer to analyze user facial expressions and voice data and generate emotion data. This emotion data is incorporated as an important parameter in the user's travel planning.

[0174] The server uses a generative AI model to select tourist destinations based on travel-related information and sentiment data received from the device. This AI model utilizes common platforms such as Google Cloud AI. Based on the selected tourist destinations, the server creates a travel itinerary and sends it to the device.

[0175] The terminal visualizes the visit itinerary and displays it in an easy-to-understand format for the user. The visualized information is provided to the user using geographic information services such as the Google Maps API.

[0176] For example, if the system determines that the user is seeking relaxation during their trip to Tokyo, it might suggest the option of "a picnic at Shinjuku Gyoen National Garden." An example of a prompt used in this case would be, "Please suggest relaxing experiences for the user during their trip to Tokyo."

[0177] This system monitors users' emotions in real time during their trip and adjusts the itinerary as needed. This process makes it possible to provide users with the best possible travel experience.

[0178] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0179] Step 1:

[0180] The user enters information about their travel destination, interests, and length of stay into the terminal. The terminal formats this information as travel-related data. Specifically, it collects the text data selected or entered by the user in the input fields and generates a dataset to send to the server. This dataset is the information that forms the basis for processing in the next step.

[0181] Step 2:

[0182] The device uses its built-in camera and microphone to record the user's facial expressions and voice tone. This data serves as input for analysis by the emotion engine. The device collects the recorded data in real time using the camera and microphone sensors, and then standardizes the data format. This process provides the foundational data for generating emotion data.

[0183] Step 3:

[0184] The device sends the facial and voice data acquired in Step 2 to the emotion engine. The emotion engine analyzes this data and generates the user's emotion data. Specifically, it inputs the data into Microsoft Azure's Face API or IBM Watson's Tone Analyzer, and outputs the emotional state as quantitative data based on the analysis results. The obtained emotion data is used to customize the travel plan in the next step.

[0185] Step 4:

[0186] The server combines travel-related information and sentiment data received from the terminal to select tourist destinations. Here, the server uses a generative AI model to generate prompt sentences, which are used as input. For example, it might generate a prompt sentence such as "Suggest a relaxing experience for the user." Based on this, the AI ​​model outputs a list of selected tourist destinations.

[0187] Step 5:

[0188] The server creates a travel itinerary based on the information of the selected tourist spots. Specifically, it optimizes the order in which to visit each selected tourist spot and compiles it into an actual schedule. This itinerary is output as a plan that takes into account the user's convenience, adjusting the time of day and travel routes.

[0189] Step 6:

[0190] The server sends the generated itinerary to the device. The device receives this information and uses a method to present it to the user visually. In particular, by using the Google Maps API or similar to mark the visited locations on a map and visually displaying the schedule, the user can easily understand their plans.

[0191] Step 7:

[0192] Throughout the trip, the device continuously monitors the user's facial expressions and voice, updating emotional data in real time. This allows the server to adjust the itinerary as needed. For example, if the user is tired, the next part of the itinerary can be changed to a less strenuous one, incorporating breaks.

[0193] (Application Example 2)

[0194] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0195] Traditional travel planning systems have struggled to customize plans to reflect travelers' emotions and preferences, failing to optimize comfort and enjoyment during their trips. Furthermore, the lack of real-time adjustments to the visiting schedule based on the traveler's current emotional state meant there was room for improvement in the traveler's experience. In response, there is a need for flexible planning that takes travelers' emotional states into account.

[0196] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0197] In this invention, the server includes means for inputting travel information, means for updating the travel plan based on the user's emotional data, and means for collecting and analyzing emotional data in real time. This makes it possible to provide personalized travel plans that take into account the traveler's emotional state.

[0198] "Travel information" refers to information such as the destination a traveler wants to visit, the activities they are interested in, and the length of their stay.

[0199] A "tourist destination" refers to famous places and tourist spots that travelers can visit and experience.

[0200] A "visit schedule" refers to a plan that outlines the dates, times, and order in which to visit selected tourist destinations.

[0201] A "display terminal" refers to an electronic device used to provide users with generated visit schedules and travel plans.

[0202] "Emotional data" refers to the analysis results that show the user's current emotional state, and includes numerical information collected from facial expressions and tone of voice.

[0203] A "user interface" refers to the means by which a user inputs operational instructions into a system or visually confirms their emotional preferences.

[0204] A "travel route" refers to the optimal path for moving between different tourist destinations.

[0205] "Voice translation services" refer to technologies that translate speech to facilitate communication between different languages.

[0206] The system for implementing this invention combines data analysis utilizing an emotion engine with AI-driven plan generation to provide travelers with a personalized travel experience.

[0207] System Configuration

[0208] The server has an input mechanism for receiving travel information entered by travelers. Users enter their destination, interests, and length of stay using electronic devices such as smartphones and tablets. Before the input information is sent to the server, emotional data is collected on the device using facial recognition modules (e.g., Amazon Rekognition) and voice analysis modules.

[0209] The server is equipped with an emotion engine to analyze collected emotional data and identify the user's current emotional state. Based on this information and travel information, a generative AI model (e.g., Google Cloud AI) is used to automatically select tourist destinations and construct an optimal visit schedule. Furthermore, the generated schedule is visually presented to the user through a user interface.

[0210] During travel, the device monitors the user's emotional data in real time and feeds it back to the server as needed. The server then adjusts the schedule based on this data and updates the information through the user interface. In this way, the user's travel experience is always kept optimal.

[0211] Specific example

[0212] For example, if a user is feeling stressed while visiting a busy city, the server might suggest a quiet and relaxing tourist destination, such as "a stroll in a nearby nature park." It could also provide further relaxation by playing healing music tailored to the user's emotional state from their smart device.

[0213] Examples of generated AI prompts

[0214] "Please generate a recommended sightseeing plan for users whose current emotional state is relaxed. Include quiet tourist spots and relaxing activities."

[0215] This system aims to create new, emotionally engaging travel experiences for travelers, helping them to create pleasant memories.

[0216] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0217] Step 1:

[0218] The device receives travel information from the user, such as destination, interests, and length of stay, as input. In addition, it uses the camera and microphone to acquire emotional data from facial expressions and voice tone, and uses this data as input. The data is preprocessed, and the user's emotional state is quantified using a facial recognition API. This becomes the input data to the server.

[0219] Step 2:

[0220] The server acquires travel information and sentiment data transmitted from the terminal. Based on the acquired data, it uses a generative AI model to automatically select tourist destinations. It takes travel information and sentiment data as input and generates a list of tourist destinations as output. The AI ​​model executes a process to plan tourist destinations that match the traveler's preferences based on their emotions.

[0221] Step 3:

[0222] The server constructs a visit schedule based on a list of tourist destinations. This schedule includes the order and date / time of visits to the destinations and uses an optimization algorithm to minimize travel time. It takes a list of tourist destinations as input and generates an optimized visit schedule as output.

[0223] Step 4:

[0224] The server sends the generated visit schedule to the terminal. The terminal displays this schedule visually to the user using a user interface. The user can review the schedule and make additional inputs if necessary. Inputs include a schedule confirmation action, and output is a visualized travel plan.

[0225] Step 5:

[0226] The device continuously monitors the user's emotional data throughout the trip. This data is sent to a server in real time, which adjusts the visit schedule as needed. Real-time emotional data is used as input, and an adjusted schedule is generated as output. For example, if fatigue is detected, a new schedule including rest activities is suggested.

[0227] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0228] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0229] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0230] [Second Embodiment]

[0231] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0232] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0233] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0234] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0235] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0236] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0237] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0238] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0239] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0240] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0241] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0242] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0243] This invention relates to a system that provides travelers with personalized sightseeing guides and travel coordination services. Through interaction between a server, a terminal, and a user, this system handles the input of travel information, the selection of tourist destinations, the generation of visit schedules, and the display and updating of travel plans.

[0244] The user enters travel information such as their interests, planned destinations, and travel duration into the device. This information is sent to the server, which uses an AI model to select appropriate tourist destinations based on the entered information. The server refers to a database of tourist destinations to identify the places best suited to the user's interests and generates a visit schedule. The generated schedule is then presented to the user through the device.

[0245] The server provides an efficient travel plan by optimizing travel routes between tourist destinations. For example, if a user visits Tokyo, the server selects tourist destinations such as "Tokyo Tower" and "Akihabara anime shops" and generates the optimal order of visits.

[0246] The device visually organizes and displays information to make it easy for users to review their plans. Furthermore, the device supports changes to conditions based on user preferences and real-time schedule updates during the trip.

[0247] Users can check their travel plans on their devices and make changes on the spot. Furthermore, they can use the device's navigation and translation functions during their trip to navigate smoothly and communicate with locals.

[0248] This invention enhances the overall travel experience by providing dynamic and flexible travel plans that enable travelers to have a satisfying travel experience.

[0249] The following describes the processing flow.

[0250] Step 1:

[0251] The user enters travel information into the terminal. This includes the travel destination, areas of interest, length of stay, and specific requests.

[0252] Step 2:

[0253] The terminal packages the user's input information and sends it to the server. This information is necessary for processing in the next step.

[0254] Step 3:

[0255] The server analyzes the travel information it receives. An AI model is applied to extract tourist destinations, restaurants, and events from the database based on the user's interests.

[0256] Step 4:

[0257] The server generates a visit schedule that takes into account travel between tourist destinations based on the extracted information. By creating a plan that visits tourist destinations in the optimal order, it improves the efficiency of the trip.

[0258] Step 5:

[0259] The server sends the generated visit schedule and related information to the terminal. The formatted data is provided in a way that is easy for the user to understand.

[0260] Step 6:

[0261] The terminal analyzes data received from the server and displays it in the user interface. It presents visually organized information, helping users easily understand the plan.

[0262] Step 7:

[0263] The user reviews the proposed plan. They make any necessary changes to the plan and finalize the schedule.

[0264] Step 8:

[0265] When traveling, users utilize their devices to access real-time navigation and translation features, enabling smoother travel between tourist destinations and easier communication with locals.

[0266] Step 9:

[0267] After their trip, users enter feedback about their experience into their device. This information is collected on a server and used to improve the AI ​​model.

[0268] (Example 1)

[0269] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0270] Modern travelers need to combine data from diverse sources to design their own travel routes that suit their interests, which requires considerable time and effort. They also face challenges in responding to sudden changes in plans during their trip and communicating in different language environments.

[0271] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0272] In this invention, the server includes means for acquiring travel information, means for automatically selecting tourist destination candidates based on the travel information using a generative model, and means for creating a visit schedule based on the selected tourist destination candidates. This enables travelers to automatically generate efficient and personalized travel plans, and to respond to immediate schedule changes and cross-cultural communication.

[0273] "Travel information" refers to data about destinations, itineraries, activities of interest, and tourist attractions that travelers need when planning their trips.

[0274] A "generative model" is an algorithm that derives the optimal result based on input data, and in this invention, it is a model that includes artificial intelligence technology used for selecting tourist destinations.

[0275] A "potential tourist destination" is a list of tourist spots that travelers can visit, selected based on travel information.

[0276] A "visiting schedule" is a travel plan that shows the order in which selected tourist destinations will be visited and the time allocation for each.

[0277] A "user device" refers to a device used by travelers to input and receive information, and includes electronic devices such as smartphones and tablets.

[0278] "Means for modifying travel plans" refers to systems or processes for restructuring and updating travel plans in response to user change requests.

[0279] This invention is a system for automating travel planning and optimizing it to user needs. The system operates using terminals, servers, and generative AI models.

[0280] Terminal-based operation and information gathering

[0281] Users can input travel information using devices such as smartphones and tablets. This information includes travel destinations, available dates, activities of interest, and places to visit.

[0282] Server-based information processing and analysis

[0283] Travel information sent from the device is aggregated on a server. The server uses a cloud-based generative AI model to analyze the travel information and form a list of the most appropriate tourist destinations. The server retrieves detailed data about tourist destinations from a database and creates a visit schedule tailored to the user's interests and priorities.

[0284] Specifically, the server utilizes a generation model to generate a tourist destination selection prompt based on the input data. For example, a prompt such as "The user plans to visit Tokyo and is interested in historical places and anime. Please propose recommended tourist destinations and create a schedule in an efficient visiting order." is used.

[0285] Providing information to and interacting with the user

[0286] As a specific example, when the user visits Tokyo, the system presents a plan for "famous tourist attractions" and "anime-related spots" to the user's terminal. The terminal enables information display according to the user's convenience and real-time plan updates, and supports the user to enjoy the trip comfortably.

[0287] The terminal not only displays the plan but also has a translation function to overcome the language barrier during the trip. Thereby, the user can enjoy the trip while smoothly obtaining local information.

[0288] In this way, the system can provide a dynamic and flexible personalized travel experience to the user.

[0289] The flow of the specific process in Example 1 will be described using FIG. 11.

[0290] Step 1:

[0291] The user inputs travel information using the terminal.

[0292] Specifically, information such as the city to visit, travel schedule, categories of interest (e.g., history, food, entertainment) is input. The input information is transmitted from the terminal to the server in text format.

[0293] Step 2:

[0294] The server analyzes the received travel information and generates a prompt sentence.

[0295] The server generates prompt statements suitable for the AI ​​model based on the input data. In doing so, it references similar historical data in the database to support effective analysis. For example, it might generate a prompt statement such as, "The user plans to visit Tokyo and is interested in history and anime." This prompt statement then becomes the input for the next step.

[0296] Step 3:

[0297] The server uses a generative AI model to select potential tourist destinations based on the prompt message.

[0298] The generative AI model analyzes the prompt text, evaluates the tourist destination information in the database, and generates a list of tourist destinations that match the user's interests. This list includes data about each tourist destination (e.g., popularity, rating, required time, etc.).

[0299] Step 4:

[0300] The server generates a visit schedule.

[0301] The program takes a selected list of tourist destinations as input and calculates the optimal order of visits. It applies a route optimization algorithm that considers the traveler's distance and time efficiency. The output is a structured itinerary of visits.

[0302] Step 5:

[0303] The terminal displays the visit schedule to the user.

[0304] The visit schedule is displayed visually through the user interface. Users can check the relative locations on a map or view the schedule in a calendar format.

[0305] Step 6:

[0306] The user checks the presented travel plan and makes a change request if necessary.

[0307] The user uses the terminal to input feedback or change requests for the travel plan. This request is sent to the server again, and the travel plan is updated.

[0308] Step 7:

[0309] During the trip, the user uses the navigation function of the terminal.

[0310] The terminal provides real-time navigation information based on the visit schedule. It also utilizes the translation function to understand local information and assist with communication.

[0311] Through this series of processes, travelers can efficiently enjoy a personalized travel experience.

[0312] (Application Example 1)

[0313] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0314] There is a need to provide a personalized travel plan according to the diverse interests and needs of travelers, and to obtain real-time visual and auditory information during the trip to assist with smooth movement and communication. However, conventional systems have not been able to fully meet these needs, making it difficult to improve the travel experience.

[0315] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0316] In this invention, the server includes a device for inputting travel information, a device for automatically selecting tourist destinations based on the travel information, and a device for forming a visit schedule for the selected tourist destinations. This enables the provision of travel plans optimized to the user's interests and the visual and auditory presentation of information during the trip.

[0317] A "travel information input device" is a device used to collect information such as a traveler's interests and preferences, accommodation, and itinerary, and input it into a system.

[0318] A "device that automatically selects tourist destinations" is a device that automatically selects appropriate tourist destinations using an algorithm based on the travel information entered.

[0319] A "device for creating a visit schedule" is a system component that generates a schedule for efficiently visiting selected tourist destinations.

[0320] A "device for displaying on the user's terminal" is a device that visually organizes the generated visit schedule and tourist destination information and presents it to the user through a user interface.

[0321] A "device that updates travel plans based on user preferences" is a system that dynamically modifies existing travel plans in response to user feedback and new preferences.

[0322] A "device that realistically displays tourist information and travel routes through a visual device" refers to a technology that uses smart glasses or similar devices to present tourist information and navigation as augmented reality.

[0323] A "device that analyzes voice information, receives instructions, and performs language conversion" is a system component that understands user instructions using voice recognition technology and performs translation processing for multilingual support.

[0324] The system for realizing this invention mainly consists of a server, a terminal, and user interaction. The server includes a device for inputting travel information, a device for automatically selecting tourist destinations, and a device for generating a visit itinerary. Specifically, the server uses generative AI models such as TensorFlow or PyTorch to analyze travel information sent by the user and select the optimal tourist destination.

[0325] The terminal is equipped with a visual device and a voice analysis device, and visually displays the generated visit schedule to the user. The visual device, such as smart glasses, plays a role in extending the visual information displayed to the user in the real environment and can accept voice commands. For voice analysis, Google Speech-to-Text and Google Translate APIs are used to analyze the user's voice instructions and perform language conversion.

[0326] Users can input travel information and create travel plans that include tourist destinations tailored to their interests by operating the terminal. To assist with travel, the system provides route information using GPS and compass functions and implements real-time navigation based on the user's current location.

[0327] For example, if a user plans a trip to Tokyo and gives a voice command saying, "Take me to Tokyo Tower," the smart glasses will display the optimal route from the user's current location to Tokyo Tower. Along the way, users can obtain information about points of interest simply by looking at them, and local languages ​​are translated into other languages, enabling natural communication with locals.

[0328] An example of a prompt message could be: "If the user's interest is related to anime, please select a recommended tourist destination."

[0329] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0330] Step 1:

[0331] Users input travel information through the terminal's interface. This information includes areas of interest, planned destinations, and travel itinerary. This information is then transmitted from the terminal to the server as digital data.

[0332] Step 2:

[0333] The server analyzes the received travel information and uses a generative AI model (for example, a model based on TensorFlow or PyTorch) to select tourist destinations that are suitable for the user's interests. Based on the travel information received as input, it calculates the degree of fit for the candidate tourist destinations stored in the database and outputs a list of the most suitable tourist destinations.

[0334] Step 3:

[0335] The server generates a travel itinerary based on the selected tourist destinations. Based on the input tourist destination information, it creates an optimal schedule considering the available visiting times and travel times for each location. The generated schedule contributes to the user's efficient travel planning.

[0336] Step 4:

[0337] The terminal visually displays the visit schedule and tourist destination information retrieved from the server to the user. This information is then displayed as an overlay on the real world through a visual device (e.g., smart glasses). This allows the user to intuitively grasp the tourist destination information in relation to their actual location.

[0338] Step 5:

[0339] Users can check and modify their schedules using eye-tracking or voice commands. The device understands the commands using voice analysis technology (such as Google Speech-to-Text) and sends them to the server as new input data. This allows the system to re-select tourist destinations based on the new criteria and update the schedule.

[0340] Step 6:

[0341] The device uses GPS and compass functions to provide real-time navigation information to assist with on-site travel. Taking the current location as input, it outputs the optimal route to the destination, helping users travel smoothly on foot or using public transportation.

[0342] Step 7:

[0343] When a user needs translation assistance while in a foreign country, the device uses the Google Translate API to translate the input into the appropriate language. It translates the input voice and text data and presents the output to the user, supporting smooth communication.

[0344] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0345] This invention relates to a system that provides personalized tourist guides and travel coordination services to travelers. In particular, by combining it with an emotion engine that recognizes the user's emotions, more sophisticated customization becomes possible.

[0346] The user enters travel information into the device, including their destination, interests, and length of stay. The device sends this information to the server, but before that, it has a function to collect emotional data through the user's facial expressions and tone of voice. The emotion engine analyzes this data to identify the user's current emotional state.

[0347] The server considers emotional data along with the received travel information and uses an AI model to optimally select tourist destinations and experiences. If the user is in a relaxed mood, quiet, nature-based tourist destinations are likely to be included in the plan. On the other hand, if the user is stressed, relaxing leisure activities and activities will be suggested.

[0348] The server generates a visit schedule that reflects emotional data and sends it to the terminal. The terminal visualizes this information and displays it to the user. For example, if the user is feeling overwhelmed in Tokyo, the server might suggest relaxing options such as "a picnic at Shinjuku Gyoen."

[0349] Throughout the trip, the user's emotions are continuously monitored, and an emotion engine feeds this data back to the server in real time. Based on this information, the server re-selects the visit schedule and sightseeing destinations if necessary. If the user starts to feel tired, adjustments are made, such as easing the schedule and increasing relaxation time at the hotel.

[0350] This invention is a system that, in this way, utilizes user emotional data to provide travelers with the most suitable plans, thereby enabling a richer travel experience.

[0351] The following describes the processing flow.

[0352] Step 1:

[0353] The user enters travel information into the device. This information includes the travel destination, interests, length of stay, and specific requests.

[0354] Step 2:

[0355] The device uses facial recognition cameras and voice analysis functions to detect the user's current emotional state. This allows it to collect data that identifies the user's stress level and level of enjoyment.

[0356] Step 3:

[0357] The device sends collected travel information and sentiment data to the server. This data transmission initiates processing on the server side.

[0358] Step 4:

[0359] The server uses an AI model to analyze travel information and sentiment data. This allows it to select the most suitable tourist destinations and activities for the user's current situation.

[0360] Step 5:

[0361] The server generates a visit schedule that takes the user's emotional state into account. For example, if the user is seeking relaxation, quiet tourist spots and relaxation facilities will be added to the plan.

[0362] Step 6:

[0363] The server generates a schedule and sends it to the terminal. The terminal visually organizes this information and presents it to the user in an easy-to-understand format.

[0364] Step 7:

[0365] Users review the proposed plan from their device and make changes or approvals as needed, ensuring flexibility in their travel plans.

[0366] Step 8:

[0367] During the trip, the device periodically monitors the user's emotional state and feeds that data back to the server. Based on this, the server dynamically adjusts the visit schedule.

[0368] Step 9:

[0369] If user stress is detected through the emotion engine, it's possible to ease the pre-set schedule or suggest new relaxation options.

[0370] (Example 2)

[0371] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0372] Traditional travel planning systems were unable to respond to individual user emotions and real-time changes, making it difficult to provide travelers with the optimal sightseeing experience. Furthermore, the inability to flexibly adjust schedules based on emotions meant that stress during travel could not be reduced.

[0373] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0374] In this invention, the server includes means for automatically selecting tourist destinations based on travel-related information and emotional data, means for generating emotional data by analyzing the user's facial expressions and voice data, and means for monitoring the user's emotional state in real time in order to realistically adjust the itinerary. This enables a personalized travel experience based on the user's current emotional state and flexible schedule management that reduces stress.

[0375] "Travel-related information" refers to information necessary for planning a trip, such as a traveler's destination, interests, and length of stay.

[0376] "Emotional data" refers to information about the user's current mental state and emotions, analyzed from their facial expressions and voice.

[0377] A "tourist spot" is a place of interest or attraction that is selected for travelers to visit.

[0378] A "travel itinerary" refers to a schedule that includes the order and timing of visits to each tourist spot a traveler will be visiting.

[0379] "User terminal" refers to a digital device used by a user to input and receive travel plans.

[0380] "Monitoring in real time" means continuously and immediately evaluating and recording the user's emotional state.

[0381] A "translation function" is a feature that supports language conversion between different languages.

[0382] This system combines multiple technological elements to provide travelers with personalized travel plans.

[0383] The user enters information about their travel destination, interests, and length of stay into the device. The device uses its built-in camera and microphone to record the user's facial expressions and voice tone, and collects emotional data based on this. Common digital devices are used for this data collection.

[0384] The emotion engine uses Microsoft Azure's Face API and IBM Watson's Tone Analyzer to analyze the user's facial expressions and voice data and generate emotion data. This emotion data is then incorporated as an important parameter in the user's travel planning.

[0385] The server uses a generative AI model to select tourist destinations based on travel-related information and sentiment data received from the device. This AI model utilizes common platforms such as Google Cloud AI. Based on the selected tourist destinations, the server creates a travel itinerary and sends it to the device.

[0386] The terminal visualizes the visit itinerary and displays it in an easy-to-understand format for the user. The visualized information is provided to the user using geographic information services such as the Google Maps API.

[0387] For example, if the system determines that the user is seeking relaxation during their trip to Tokyo, it might suggest the option of "a picnic at Shinjuku Gyoen National Garden." An example of a prompt used in this case would be, "Please suggest relaxing experiences for the user during their trip to Tokyo."

[0388] This system monitors users' emotions in real time during their trip and adjusts the itinerary as needed. This process makes it possible to provide users with the best possible travel experience.

[0389] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0390] Step 1:

[0391] The user enters information about their travel destination, interests, and length of stay into the terminal. The terminal formats this information as travel-related data. Specifically, it collects the text data selected or entered by the user in the input fields and generates a dataset to send to the server. This dataset is the information that forms the basis for processing in the next step.

[0392] Step 2:

[0393] The device uses its built-in camera and microphone to record the user's facial expressions and voice tone. This data serves as input for analysis by the emotion engine. The device collects the recorded data in real time using the camera and microphone sensors, and then standardizes the data format. This process provides the foundational data for generating emotion data.

[0394] Step 3:

[0395] The device sends the facial and voice data acquired in Step 2 to the emotion engine. The emotion engine analyzes this data and generates the user's emotion data. Specifically, it inputs the data into Microsoft Azure's Face API or IBM Watson's Tone Analyzer, and outputs the emotional state as quantitative data based on the analysis results. The obtained emotion data is used to customize the travel plan in the next step.

[0396] Step 4:

[0397] The server combines travel-related information and sentiment data received from the terminal to select tourist destinations. Here, the server uses a generative AI model to generate prompt sentences, which are used as input. For example, it might generate a prompt sentence such as "Suggest a relaxing experience for the user." Based on this, the AI ​​model outputs a list of selected tourist destinations.

[0398] Step 5:

[0399] The server creates a travel itinerary based on the information of the selected tourist spots. Specifically, it optimizes the order in which to visit each selected tourist spot and compiles it into an actual schedule. This itinerary is output as a plan that takes into account the user's convenience, adjusting the time of day and travel routes.

[0400] Step 6:

[0401] The server sends the generated itinerary to the device. The device receives this information and uses a method to present it to the user visually. In particular, by using the Google Maps API or similar to mark the visited locations on a map and visually displaying the schedule, the user can easily understand their plans.

[0402] Step 7:

[0403] Throughout the trip, the device continuously monitors the user's facial expressions and voice, updating emotional data in real time. This allows the server to adjust the itinerary as needed. For example, if the user is tired, the next part of the itinerary can be changed to a less strenuous one, incorporating breaks.

[0404] (Application Example 2)

[0405] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0406] Traditional travel planning systems have struggled to customize plans to reflect travelers' emotions and preferences, failing to optimize comfort and enjoyment during their trips. Furthermore, the lack of real-time adjustments to the visiting schedule based on the traveler's current emotional state meant there was room for improvement in the traveler's experience. In response, there is a need for flexible planning that takes travelers' emotional states into account.

[0407] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0408] In this invention, the server includes means for inputting travel information, means for updating the travel plan based on the user's emotional data, and means for collecting and analyzing emotional data in real time. This makes it possible to provide personalized travel plans that take into account the traveler's emotional state.

[0409] "Travel information" refers to information such as the destination a traveler wants to visit, the activities they are interested in, and the length of their stay.

[0410] A "tourist destination" refers to famous places and tourist spots that travelers can visit and experience.

[0411] A "visit schedule" refers to a plan that outlines the dates, times, and order in which to visit selected tourist destinations.

[0412] A "display terminal" refers to an electronic device used to provide users with generated visit schedules and travel plans.

[0413] "Emotional data" refers to the analysis results that show the user's current emotional state, and includes numerical information collected from facial expressions and tone of voice.

[0414] A "user interface" refers to the means by which a user inputs operational instructions into a system or visually confirms their emotional preferences.

[0415] A "travel route" refers to the optimal path for moving between different tourist destinations.

[0416] "Voice translation services" refer to technologies that translate speech to facilitate communication between different languages.

[0417] The system for implementing this invention combines data analysis utilizing an emotion engine with AI-driven plan generation to provide travelers with a personalized travel experience.

[0418] System Configuration

[0419] The server has an input mechanism for receiving travel information entered by travelers. Users enter their destination, interests, and length of stay using electronic devices such as smartphones and tablets. Before the input information is sent to the server, emotional data is collected on the device using facial recognition modules (e.g., Amazon Rekognition) and voice analysis modules.

[0420] The server is equipped with an emotion engine to analyze collected emotional data and identify the user's current emotional state. Based on this information and travel information, a generative AI model (e.g., Google Cloud AI) is used to automatically select tourist destinations and construct an optimal visit schedule. Furthermore, the generated schedule is visually presented to the user through a user interface.

[0421] During travel, the device monitors the user's emotional data in real time and feeds it back to the server as needed. The server then adjusts the schedule based on this data and updates the information through the user interface. In this way, the user's travel experience is always kept optimal.

[0422] Specific example

[0423] For example, if a user is feeling stressed while visiting a busy city, the server might suggest a quiet and relaxing tourist destination, such as "a stroll in a nearby nature park." It could also provide further relaxation by playing healing music tailored to the user's emotional state from their smart device.

[0424] Examples of generated AI prompts

[0425] "Please generate a recommended sightseeing plan for users whose current emotional state is relaxed. Include quiet tourist spots and relaxing activities."

[0426] This system aims to create new, emotionally engaging travel experiences for travelers, helping them to create pleasant memories.

[0427] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0428] Step 1:

[0429] The device receives travel information from the user, such as destination, interests, and length of stay, as input. In addition, it uses the camera and microphone to acquire emotional data from facial expressions and voice tone, and uses this data as input. The data is preprocessed, and the user's emotional state is quantified using a facial recognition API. This becomes the input data to the server.

[0430] Step 2:

[0431] The server acquires travel information and sentiment data transmitted from the terminal. Based on the acquired data, it uses a generative AI model to automatically select tourist destinations. It takes travel information and sentiment data as input and generates a list of tourist destinations as output. The AI ​​model executes a process to plan tourist destinations that match the traveler's preferences based on their emotions.

[0432] Step 3:

[0433] The server constructs a visit schedule based on a list of tourist destinations. This schedule includes the order and date / time of visits to the destinations and uses an optimization algorithm to minimize travel time. It takes a list of tourist destinations as input and generates an optimized visit schedule as output.

[0434] Step 4:

[0435] The server sends the generated visit schedule to the terminal. The terminal displays this schedule visually to the user using a user interface. The user can review the schedule and make additional inputs if necessary. Inputs include a schedule confirmation action, and output is a visualized travel plan.

[0436] Step 5:

[0437] The device continuously monitors the user's emotional data throughout the trip. This data is sent to a server in real time, which adjusts the visit schedule as needed. Real-time emotional data is used as input, and an adjusted schedule is generated as output. For example, if fatigue is detected, a new schedule including rest activities is suggested.

[0438] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0439] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0440] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0441] [Third Embodiment]

[0442] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0443] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0444] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0445] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0446] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0447] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0448] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0449] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0450] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0451] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0452] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0453] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0454] This invention relates to a system that provides travelers with personalized sightseeing guides and travel coordination services. Through interaction between a server, a terminal, and a user, this system handles the input of travel information, the selection of tourist destinations, the generation of visit schedules, and the display and updating of travel plans.

[0455] The user enters travel information such as their interests, planned destinations, and travel duration into the device. This information is sent to the server, which uses an AI model to select appropriate tourist destinations based on the entered information. The server refers to a database of tourist destinations to identify the places best suited to the user's interests and generates a visit schedule. The generated schedule is then presented to the user through the device.

[0456] The server provides an efficient travel plan by optimizing travel routes between tourist destinations. For example, if a user visits Tokyo, the server selects tourist destinations such as "Tokyo Tower" and "Akihabara anime shops" and generates the optimal order of visits.

[0457] The device visually organizes and displays information to make it easy for users to review their plans. Furthermore, the device supports changes to conditions based on user preferences and real-time schedule updates during the trip.

[0458] Users can check their travel plans on their devices and make changes on the spot. Furthermore, they can use the device's navigation and translation functions during their trip to navigate smoothly and communicate with locals.

[0459] This invention enhances the overall travel experience by providing dynamic and flexible travel plans that enable travelers to have a satisfying travel experience.

[0460] The following describes the processing flow.

[0461] Step 1:

[0462] The user enters travel information into the terminal. This includes the travel destination, areas of interest, length of stay, and specific requests.

[0463] Step 2:

[0464] The terminal packages the user's input information and sends it to the server. This information is necessary for processing in the next step.

[0465] Step 3:

[0466] The server analyzes the travel information it receives. An AI model is applied to extract tourist destinations, restaurants, and events from the database based on the user's interests.

[0467] Step 4:

[0468] The server generates a visit schedule that takes into account travel between tourist destinations based on the extracted information. By creating a plan that visits tourist destinations in the optimal order, it improves the efficiency of the trip.

[0469] Step 5:

[0470] The server sends the generated visit schedule and related information to the terminal. The formatted data is provided in a way that is easy for the user to understand.

[0471] Step 6:

[0472] The terminal analyzes data received from the server and displays it in the user interface. It presents visually organized information, helping users easily understand the plan.

[0473] Step 7:

[0474] The user reviews the proposed plan. They make any necessary changes to the plan and finalize the schedule.

[0475] Step 8:

[0476] When traveling, users utilize their devices to access real-time navigation and translation features, enabling smoother travel between tourist destinations and easier communication with locals.

[0477] Step 9:

[0478] After their trip, users enter feedback about their experience into their device. This information is collected on a server and used to improve the AI ​​model.

[0479] (Example 1)

[0480] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0481] Modern travelers need to combine data from diverse sources to design their own travel routes that suit their interests, which requires considerable time and effort. They also face challenges in responding to sudden changes in plans during their trip and communicating in different language environments.

[0482] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0483] In this invention, the server includes means for acquiring travel information, means for automatically selecting tourist destination candidates based on the travel information using a generative model, and means for creating a visit schedule based on the selected tourist destination candidates. This enables travelers to automatically generate efficient and personalized travel plans, and to respond to immediate schedule changes and cross-cultural communication.

[0484] "Travel information" refers to data about destinations, itineraries, activities of interest, and tourist attractions that travelers need when planning their trips.

[0485] A "generative model" is an algorithm that derives the optimal result based on input data, and in this invention, it is a model that includes artificial intelligence technology used for selecting tourist destinations.

[0486] A "potential tourist destination" is a list of tourist spots that travelers can visit, selected based on travel information.

[0487] A "visiting schedule" is a travel plan that shows the order in which selected tourist destinations will be visited and the time allocation for each.

[0488] A "user device" refers to a device used by travelers to input and receive information, and includes electronic devices such as smartphones and tablets.

[0489] "Means for modifying travel plans" refers to systems or processes for restructuring and updating travel plans in response to user change requests.

[0490] This invention is a system for automating travel planning and optimizing it to user needs. The system operates using terminals, servers, and generative AI models.

[0491] Terminal-based operation and information gathering

[0492] Users can input travel information using devices such as smartphones and tablets. This information includes travel destinations, available dates, activities of interest, and places to visit.

[0493] Server-based information processing and analysis

[0494] Travel information sent from the device is aggregated on a server. The server uses a cloud-based generative AI model to analyze the travel information and form a list of the most appropriate tourist destinations. The server retrieves detailed data about tourist destinations from a database and creates a visit schedule tailored to the user's interests and priorities.

[0495] Specifically, the server utilizes a generative model to generate tourist destination selection prompts based on input data. For example, it might use a prompt like, "The user plans to visit Tokyo and is interested in historical sites and anime. Please suggest recommended tourist destinations and create a schedule with an efficient order of visits."

[0496] Providing information to users and interaction

[0497] As a concrete example, if a user visits Tokyo, the system will present a plan on the user's device that includes visiting "tourist attractions" and "anime-related spots." The device will display information tailored to the user's needs and allow for real-time plan updates, supporting the user in enjoying their trip comfortably.

[0498] The device not only displays travel plans but also includes a translation function to overcome language barriers during travel. This allows users to enjoy their trip while smoothly obtaining local information.

[0499] In this way, the system can provide users with a dynamic, flexible, and personalized travel experience.

[0500] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0501] Step 1:

[0502] The user enters travel information using a terminal.

[0503] Specifically, you will enter information such as the city you want to visit, your travel dates, and categories of interest (for example, history, food, entertainment). The entered information will be sent from your device to the server in text format.

[0504] Step 2:

[0505] The server analyzes the received travel information and generates a prompt message.

[0506] The server generates prompt statements suitable for the AI ​​model based on the input data. In doing so, it references similar historical data in the database to support effective analysis. For example, it might generate a prompt statement such as, "The user plans to visit Tokyo and is interested in history and anime." This prompt statement then becomes the input for the next step.

[0507] Step 3:

[0508] The server uses a generative AI model to select potential tourist destinations based on the prompt message.

[0509] The generative AI model analyzes the prompt text, evaluates the tourist destination information in the database, and generates a list of tourist destinations that match the user's interests. This list includes data about each tourist destination (e.g., popularity, rating, required time, etc.).

[0510] Step 4:

[0511] The server generates a visit schedule.

[0512] The program takes a selected list of tourist destinations as input and calculates the optimal order of visits. It applies a route optimization algorithm that considers the traveler's distance and time efficiency. The output is a structured itinerary of visits.

[0513] Step 5:

[0514] The terminal displays the visit schedule to the user.

[0515] The visit schedule is displayed visually through the user interface. Users can check the relative locations on a map or view the schedule in a calendar format.

[0516] Step 6:

[0517] Users review the presented travel plan and request changes as needed.

[0518] Users use their devices to input feedback and change requests regarding their travel plans. These requests are then sent back to the server, and the travel plans are updated.

[0519] Step 7:

[0520] During their trip, users utilize the device's navigation function.

[0521] The device provides real-time navigation information based on the visit schedule. It also utilizes translation functions to help users understand local information and facilitate communication.

[0522] This entire process allows travelers to efficiently enjoy a personalized travel experience.

[0523] (Application Example 1)

[0524] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0525] There is a growing need to provide personalized travel plans that cater to the diverse interests and needs of travelers, and to support smooth travel and communication by providing real-time visual and auditory information during their trips. However, conventional systems have been unable to adequately meet these needs, making it difficult to improve the travel experience.

[0526] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0527] In this invention, the server includes a device for inputting travel information, a device for automatically selecting tourist destinations based on the travel information, and a device for forming a visit schedule for the selected tourist destinations. This enables the provision of travel plans optimized to the user's interests and the visual and auditory presentation of information during the trip.

[0528] A "travel information input device" is a device used to collect information such as a traveler's interests and preferences, accommodation, and itinerary, and input it into a system.

[0529] A "device that automatically selects tourist destinations" is a device that automatically selects appropriate tourist destinations using an algorithm based on the travel information entered.

[0530] A "device for creating a visit schedule" is a system component that generates a schedule for efficiently visiting selected tourist destinations.

[0531] A "device for displaying on the user's terminal" is a device that visually organizes the generated visit schedule and tourist destination information and presents it to the user through a user interface.

[0532] A "device that updates travel plans based on user preferences" is a system that dynamically modifies existing travel plans in response to user feedback and new preferences.

[0533] A "device that realistically displays tourist information and travel routes through a visual device" refers to a technology that uses smart glasses or similar devices to present tourist information and navigation as augmented reality.

[0534] A "device that analyzes voice information, receives instructions, and performs language conversion" is a system component that understands user instructions using voice recognition technology and performs translation processing for multilingual support.

[0535] The system for realizing this invention mainly consists of a server, a terminal, and user interaction. The server includes a device for inputting travel information, a device for automatically selecting tourist destinations, and a device for generating a visit itinerary. Specifically, the server uses generative AI models such as TensorFlow or PyTorch to analyze travel information sent by the user and select the optimal tourist destination.

[0536] The terminal is equipped with a visual device and a voice analysis device, and visually displays the generated visit schedule to the user. The visual device, such as smart glasses, plays a role in extending the visual information displayed to the user in the real environment and can accept voice commands. For voice analysis, Google Speech-to-Text and Google Translate APIs are used to analyze the user's voice instructions and perform language conversion.

[0537] Users can input travel information and create travel plans that include tourist destinations tailored to their interests by operating the terminal. To assist with travel, the system provides route information using GPS and compass functions and implements real-time navigation based on the user's current location.

[0538] For example, if a user plans a trip to Tokyo and gives a voice command saying, "Take me to Tokyo Tower," the smart glasses will display the optimal route from the user's current location to Tokyo Tower. Along the way, users can obtain information about points of interest simply by looking at them, and local languages ​​are translated into other languages, enabling natural communication with locals.

[0539] An example of a prompt message could be: "If the user's interest is related to anime, please select a recommended tourist destination."

[0540] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0541] Step 1:

[0542] Users input travel information through the terminal's interface. This information includes areas of interest, planned destinations, and travel itinerary. This information is then transmitted from the terminal to the server as digital data.

[0543] Step 2:

[0544] The server analyzes the received travel information and uses a generative AI model (for example, a model based on TensorFlow or PyTorch) to select tourist destinations that are suitable for the user's interests. Based on the travel information received as input, it calculates the degree of fit for the candidate tourist destinations stored in the database and outputs a list of the most suitable tourist destinations.

[0545] Step 3:

[0546] The server generates a travel itinerary based on the selected tourist destinations. Based on the input tourist destination information, it creates an optimal schedule considering the available visiting times and travel times for each location. The generated schedule contributes to the user's efficient travel planning.

[0547] Step 4:

[0548] The terminal visually displays the visit schedule and tourist destination information retrieved from the server to the user. This information is then displayed as an overlay on the real world through a visual device (e.g., smart glasses). This allows the user to intuitively grasp the tourist destination information in relation to their actual location.

[0549] Step 5:

[0550] Users can check and modify their schedules using eye-tracking or voice commands. The device understands the commands using voice analysis technology (such as Google Speech-to-Text) and sends them to the server as new input data. This allows the system to re-select tourist destinations based on the new criteria and update the schedule.

[0551] Step 6:

[0552] The device uses GPS and compass functions to provide real-time navigation information to assist with on-site travel. Taking the current location as input, it outputs the optimal route to the destination, helping users travel smoothly on foot or using public transportation.

[0553] Step 7:

[0554] When a user needs translation assistance while in a foreign country, the device uses the Google Translate API to translate the input into the appropriate language. It translates the input voice and text data and presents the output to the user, supporting smooth communication.

[0555] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0556] This invention relates to a system that provides personalized tourist guides and travel coordination services to travelers. In particular, by combining it with an emotion engine that recognizes the user's emotions, more sophisticated customization becomes possible.

[0557] The user enters travel information into the device, including their destination, interests, and length of stay. The device sends this information to the server, but before that, it has a function to collect emotional data through the user's facial expressions and tone of voice. The emotion engine analyzes this data to identify the user's current emotional state.

[0558] The server considers emotional data along with the received travel information and uses an AI model to optimally select tourist destinations and experiences. If the user is in a relaxed mood, quiet, nature-based tourist destinations are likely to be included in the plan. On the other hand, if the user is stressed, relaxing leisure activities and activities will be suggested.

[0559] The server generates a visit schedule that reflects emotional data and sends it to the terminal. The terminal visualizes this information and displays it to the user. For example, if the user is feeling overwhelmed in Tokyo, the server might suggest relaxing options such as "a picnic at Shinjuku Gyoen."

[0560] Throughout the trip, the user's emotions are continuously monitored, and an emotion engine feeds this data back to the server in real time. Based on this information, the server re-selects the visit schedule and sightseeing destinations if necessary. If the user starts to feel tired, adjustments are made, such as easing the schedule and increasing relaxation time at the hotel.

[0561] This invention is a system that, in this way, utilizes user emotional data to provide travelers with the most suitable plans, thereby enabling a richer travel experience.

[0562] The following describes the processing flow.

[0563] Step 1:

[0564] The user enters travel information into the device. This information includes the travel destination, interests, length of stay, and specific requests.

[0565] Step 2:

[0566] The device uses facial recognition cameras and voice analysis functions to detect the user's current emotional state. This allows it to collect data that identifies the user's stress level and level of enjoyment.

[0567] Step 3:

[0568] The device sends collected travel information and sentiment data to the server. This data transmission initiates processing on the server side.

[0569] Step 4:

[0570] The server uses an AI model to analyze travel information and sentiment data. This allows it to select the most suitable tourist destinations and activities for the user's current situation.

[0571] Step 5:

[0572] The server generates a visit schedule that takes the user's emotional state into account. For example, if the user is seeking relaxation, quiet tourist spots and relaxation facilities will be added to the plan.

[0573] Step 6:

[0574] The server generates a schedule and sends it to the terminal. The terminal visually organizes this information and presents it to the user in an easy-to-understand format.

[0575] Step 7:

[0576] Users review the proposed plan from their device and make changes or approvals as needed, ensuring flexibility in their travel plans.

[0577] Step 8:

[0578] During the trip, the device periodically monitors the user's emotional state and feeds that data back to the server. Based on this, the server dynamically adjusts the visit schedule.

[0579] Step 9:

[0580] If user stress is detected through the emotion engine, it's possible to ease the pre-set schedule or suggest new relaxation options.

[0581] (Example 2)

[0582] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0583] Traditional travel planning systems were unable to respond to individual user emotions and real-time changes, making it difficult to provide travelers with the optimal sightseeing experience. Furthermore, the inability to flexibly adjust schedules based on emotions meant that stress during travel could not be reduced.

[0584] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0585] In this invention, the server includes means for automatically selecting tourist destinations based on travel-related information and emotional data, means for generating emotional data by analyzing the user's facial expressions and voice data, and means for monitoring the user's emotional state in real time in order to realistically adjust the itinerary. This enables a personalized travel experience based on the user's current emotional state and flexible schedule management that reduces stress.

[0586] "Travel-related information" refers to information necessary for planning a trip, such as a traveler's destination, interests, and length of stay.

[0587] "Emotional data" refers to information about the user's current mental state and emotions, analyzed from their facial expressions and voice.

[0588] A "tourist spot" is a place of interest or attraction that is selected for travelers to visit.

[0589] A "travel itinerary" refers to a schedule that includes the order and timing of visits to each tourist spot a traveler will be visiting.

[0590] "User terminal" refers to a digital device used by a user to input and receive travel plans.

[0591] "Monitoring in real time" means continuously and immediately evaluating and recording the user's emotional state.

[0592] A "translation function" is a feature that supports language conversion between different languages.

[0593] This system combines multiple technological elements to provide travelers with personalized travel plans.

[0594] The user enters information about their travel destination, interests, and length of stay into the device. The device uses its built-in camera and microphone to record the user's facial expressions and voice tone, and collects emotional data based on this. Common digital devices are used for this data collection.

[0595] The emotion engine uses Microsoft Azure's Face API and IBM Watson's Tone Analyzer to analyze the user's facial expressions and voice data and generate emotion data. This emotion data is then incorporated as an important parameter in the user's travel planning.

[0596] The server uses a generative AI model to select tourist destinations based on travel-related information and sentiment data received from the device. This AI model utilizes common platforms such as Google Cloud AI. Based on the selected tourist destinations, the server creates a travel itinerary and sends it to the device.

[0597] The terminal visualizes the visit itinerary and displays it in an easy-to-understand format for the user. The visualized information is provided to the user using geographic information services such as the Google Maps API.

[0598] For example, if the system determines that the user is seeking relaxation during their trip to Tokyo, it might suggest the option of "a picnic at Shinjuku Gyoen National Garden." An example of a prompt used in this case would be, "Please suggest relaxing experiences for the user during their trip to Tokyo."

[0599] This system monitors users' emotions in real time during their trip and adjusts the itinerary as needed. This process makes it possible to provide users with the best possible travel experience.

[0600] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0601] Step 1:

[0602] The user enters information about their travel destination, interests, and length of stay into the terminal. The terminal formats this information as travel-related data. Specifically, it collects the text data selected or entered by the user in the input fields and generates a dataset to send to the server. This dataset is the information that forms the basis for processing in the next step.

[0603] Step 2:

[0604] The device uses its built-in camera and microphone to record the user's facial expressions and voice tone. This data serves as input for analysis by the emotion engine. The device collects the recorded data in real time using the camera and microphone sensors, and then standardizes the data format. This process provides the foundational data for generating emotion data.

[0605] Step 3:

[0606] The device sends the facial and voice data acquired in Step 2 to the emotion engine. The emotion engine analyzes this data and generates the user's emotion data. Specifically, it inputs the data into Microsoft Azure's Face API or IBM Watson's Tone Analyzer, and outputs the emotional state as quantitative data based on the analysis results. The obtained emotion data is used to customize the travel plan in the next step.

[0607] Step 4:

[0608] The server combines travel-related information and sentiment data received from the terminal to select tourist destinations. Here, the server uses a generative AI model to generate prompt sentences, which are used as input. For example, it might generate a prompt sentence such as "Suggest a relaxing experience for the user." Based on this, the AI ​​model outputs a list of selected tourist destinations.

[0609] Step 5:

[0610] The server creates a travel itinerary based on the information of the selected tourist spots. Specifically, it optimizes the order in which to visit each selected tourist spot and compiles it into an actual schedule. This itinerary is output as a plan that takes into account the user's convenience, adjusting the time of day and travel routes.

[0611] Step 6:

[0612] The server sends the generated itinerary to the device. The device receives this information and uses a method to present it to the user visually. In particular, by using the Google Maps API or similar to mark the visited locations on a map and visually displaying the schedule, the user can easily understand their plans.

[0613] Step 7:

[0614] Throughout the trip, the device continuously monitors the user's facial expressions and voice, updating emotional data in real time. This allows the server to adjust the itinerary as needed. For example, if the user is tired, the next part of the itinerary can be changed to a less strenuous one, incorporating breaks.

[0615] (Application Example 2)

[0616] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0617] Traditional travel planning systems have struggled to customize plans to reflect travelers' emotions and preferences, failing to optimize comfort and enjoyment during their trips. Furthermore, the lack of real-time adjustments to the visiting schedule based on the traveler's current emotional state meant there was room for improvement in the traveler's experience. In response, there is a need for flexible planning that takes travelers' emotional states into account.

[0618] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0619] In this invention, the server includes means for inputting travel information, means for updating the travel plan based on the user's emotional data, and means for collecting and analyzing emotional data in real time. This makes it possible to provide personalized travel plans that take into account the traveler's emotional state.

[0620] "Travel information" refers to information such as the destination a traveler wants to visit, the activities they are interested in, and the length of their stay.

[0621] A "tourist destination" refers to famous places and tourist spots that travelers can visit and experience.

[0622] A "visit schedule" refers to a plan that outlines the dates, times, and order in which to visit selected tourist destinations.

[0623] A "display terminal" refers to an electronic device used to provide users with generated visit schedules and travel plans.

[0624] "Emotional data" refers to the analysis results that show the user's current emotional state, and includes numerical information collected from facial expressions and tone of voice.

[0625] A "user interface" refers to the means by which a user inputs operational instructions into a system or visually confirms their emotional preferences.

[0626] A "travel route" refers to the optimal path for moving between different tourist destinations.

[0627] "Voice translation services" refer to technologies that translate speech to facilitate communication between different languages.

[0628] The system for implementing this invention combines data analysis utilizing an emotion engine with AI-driven plan generation to provide travelers with a personalized travel experience.

[0629] System Configuration

[0630] The server has an input mechanism for receiving travel information entered by travelers. Users enter their destination, interests, and length of stay using electronic devices such as smartphones and tablets. Before the input information is sent to the server, emotional data is collected on the device using facial recognition modules (e.g., Amazon Rekognition) and voice analysis modules.

[0631] The server is equipped with an emotion engine to analyze collected emotional data and identify the user's current emotional state. Based on this information and travel information, a generative AI model (e.g., Google Cloud AI) is used to automatically select tourist destinations and construct an optimal visit schedule. Furthermore, the generated schedule is visually presented to the user through a user interface.

[0632] During travel, the device monitors the user's emotional data in real time and feeds it back to the server as needed. The server then adjusts the schedule based on this data and updates the information through the user interface. In this way, the user's travel experience is always kept optimal.

[0633] Specific example

[0634] For example, if a user is feeling stressed while visiting a busy city, the server might suggest a quiet and relaxing tourist destination, such as "a stroll in a nearby nature park." It could also provide further relaxation by playing healing music tailored to the user's emotional state from their smart device.

[0635] Examples of generated AI prompts

[0636] "Please generate a recommended sightseeing plan for users whose current emotional state is relaxed. Include quiet tourist spots and relaxing activities."

[0637] This system aims to create new, emotionally engaging travel experiences for travelers, helping them to create pleasant memories.

[0638] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0639] Step 1:

[0640] The device receives travel information from the user, such as destination, interests, and length of stay, as input. In addition, it uses the camera and microphone to acquire emotional data from facial expressions and voice tone, and uses this data as input. The data is preprocessed, and the user's emotional state is quantified using a facial recognition API. This becomes the input data to the server.

[0641] Step 2:

[0642] The server acquires travel information and sentiment data transmitted from the terminal. Based on the acquired data, it uses a generative AI model to automatically select tourist destinations. It takes travel information and sentiment data as input and generates a list of tourist destinations as output. The AI ​​model executes a process to plan tourist destinations that match the traveler's preferences based on their emotions.

[0643] Step 3:

[0644] The server constructs a visit schedule based on a list of tourist destinations. This schedule includes the order and date / time of visits to the destinations and uses an optimization algorithm to minimize travel time. It takes a list of tourist destinations as input and generates an optimized visit schedule as output.

[0645] Step 4:

[0646] The server sends the generated visit schedule to the terminal. The terminal displays this schedule visually to the user using a user interface. The user can review the schedule and make additional inputs if necessary. Inputs include a schedule confirmation action, and output is a visualized travel plan.

[0647] Step 5:

[0648] The device continuously monitors the user's emotional data throughout the trip. This data is sent to a server in real time, which adjusts the visit schedule as needed. Real-time emotional data is used as input, and an adjusted schedule is generated as output. For example, if fatigue is detected, a new schedule including rest activities is suggested.

[0649] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0650] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0651] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0652] [Fourth Embodiment]

[0653] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0654] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0655] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0656] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0657] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0658] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0659] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0660] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0661] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0662] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0663] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0664] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0665] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0666] This invention relates to a system that provides travelers with personalized sightseeing guides and travel coordination services. Through interaction between a server, a terminal, and a user, this system handles the input of travel information, the selection of tourist destinations, the generation of visit schedules, and the display and updating of travel plans.

[0667] The user enters travel information such as their interests, planned destinations, and travel duration into the device. This information is sent to the server, which uses an AI model to select appropriate tourist destinations based on the entered information. The server refers to a database of tourist destinations to identify the places best suited to the user's interests and generates a visit schedule. The generated schedule is then presented to the user through the device.

[0668] The server provides an efficient travel plan by optimizing travel routes between tourist destinations. For example, if a user visits Tokyo, the server selects tourist destinations such as "Tokyo Tower" and "Akihabara anime shops" and generates the optimal order of visits.

[0669] The device visually organizes and displays information to make it easy for users to review their plans. Furthermore, the device supports changes to conditions based on user preferences and real-time schedule updates during the trip.

[0670] Users can check their travel plans on their devices and make changes on the spot. Furthermore, they can use the device's navigation and translation functions during their trip to navigate smoothly and communicate with locals.

[0671] This invention enhances the overall travel experience by providing dynamic and flexible travel plans that enable travelers to have a satisfying travel experience.

[0672] The following describes the processing flow.

[0673] Step 1:

[0674] The user enters travel information into the terminal. This includes the travel destination, areas of interest, length of stay, and specific requests.

[0675] Step 2:

[0676] The terminal packages the user's input information and sends it to the server. This information is necessary for processing in the next step.

[0677] Step 3:

[0678] The server analyzes the travel information it receives. An AI model is applied to extract tourist destinations, restaurants, and events from the database based on the user's interests.

[0679] Step 4:

[0680] The server generates a visit schedule that takes into account travel between tourist destinations based on the extracted information. By creating a plan that visits tourist destinations in the optimal order, it improves the efficiency of the trip.

[0681] Step 5:

[0682] The server sends the generated visit schedule and related information to the terminal. The formatted data is provided in a way that is easy for the user to understand.

[0683] Step 6:

[0684] The terminal analyzes data received from the server and displays it in the user interface. It presents visually organized information, helping users easily understand the plan.

[0685] Step 7:

[0686] The user reviews the proposed plan. They make any necessary changes to the plan and finalize the schedule.

[0687] Step 8:

[0688] When traveling, users utilize their devices to access real-time navigation and translation features, enabling smoother travel between tourist destinations and easier communication with locals.

[0689] Step 9:

[0690] After their trip, users enter feedback about their experience into their device. This information is collected on a server and used to improve the AI ​​model.

[0691] (Example 1)

[0692] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0693] Modern travelers need to combine data from diverse sources to design their own travel routes that suit their interests, which requires considerable time and effort. They also face challenges in responding to sudden changes in plans during their trip and communicating in different language environments.

[0694] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0695] In this invention, the server includes means for acquiring travel information, means for automatically selecting tourist destination candidates based on the travel information using a generative model, and means for creating a visit schedule based on the selected tourist destination candidates. This enables travelers to automatically generate efficient and personalized travel plans, and to respond to immediate schedule changes and cross-cultural communication.

[0696] "Travel information" refers to data about destinations, itineraries, activities of interest, and tourist attractions that travelers need when planning their trips.

[0697] A "generative model" is an algorithm that derives the optimal result based on input data, and in this invention, it is a model that includes artificial intelligence technology used for selecting tourist destinations.

[0698] A "potential tourist destination" is a list of tourist spots that travelers can visit, selected based on travel information.

[0699] A "visiting schedule" is a travel plan that shows the order in which selected tourist destinations will be visited and the time allocation for each.

[0700] A "user device" refers to a device used by travelers to input and receive information, and includes electronic devices such as smartphones and tablets.

[0701] "Means for modifying travel plans" refers to systems or processes for restructuring and updating travel plans in response to user change requests.

[0702] This invention is a system for automating travel planning and optimizing it to user needs. The system operates using terminals, servers, and generative AI models.

[0703] Terminal-based operation and information gathering

[0704] Users can input travel information using devices such as smartphones and tablets. This information includes travel destinations, available dates, activities of interest, and places to visit.

[0705] Server-based information processing and analysis

[0706] Travel information sent from the device is aggregated on a server. The server uses a cloud-based generative AI model to analyze the travel information and form a list of the most appropriate tourist destinations. The server retrieves detailed data about tourist destinations from a database and creates a visit schedule tailored to the user's interests and priorities.

[0707] Specifically, the server utilizes a generative model to generate tourist destination selection prompts based on input data. For example, it might use a prompt like, "The user plans to visit Tokyo and is interested in historical sites and anime. Please suggest recommended tourist destinations and create a schedule with an efficient order of visits."

[0708] Providing information to users and interaction

[0709] As a concrete example, if a user visits Tokyo, the system will present a plan on the user's device that includes visiting "tourist attractions" and "anime-related spots." The device will display information tailored to the user's needs and allow for real-time plan updates, supporting the user in enjoying their trip comfortably.

[0710] The device not only displays travel plans but also includes a translation function to overcome language barriers during travel. This allows users to enjoy their trip while smoothly obtaining local information.

[0711] In this way, the system can provide users with a dynamic, flexible, and personalized travel experience.

[0712] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0713] Step 1:

[0714] The user enters travel information using a terminal.

[0715] Specifically, you will enter information such as the city you want to visit, your travel dates, and categories of interest (for example, history, food, entertainment). The entered information will be sent from your device to the server in text format.

[0716] Step 2:

[0717] The server analyzes the received travel information and generates a prompt message.

[0718] The server generates prompt statements suitable for the AI ​​model based on the input data. In doing so, it references similar historical data in the database to support effective analysis. For example, it might generate a prompt statement such as, "The user plans to visit Tokyo and is interested in history and anime." This prompt statement then becomes the input for the next step.

[0719] Step 3:

[0720] The server uses a generative AI model to select potential tourist destinations based on the prompt message.

[0721] The generative AI model analyzes the prompt text, evaluates the tourist destination information in the database, and generates a list of tourist destinations that match the user's interests. This list includes data about each tourist destination (e.g., popularity, rating, required time, etc.).

[0722] Step 4:

[0723] The server generates a visit schedule.

[0724] The program takes a selected list of tourist destinations as input and calculates the optimal order of visits. It applies a route optimization algorithm that considers the traveler's distance and time efficiency. The output is a structured itinerary of visits.

[0725] Step 5:

[0726] The terminal displays the visit schedule to the user.

[0727] The visit schedule is displayed visually through the user interface. Users can check the relative locations on a map or view the schedule in a calendar format.

[0728] Step 6:

[0729] Users review the presented travel plan and request changes as needed.

[0730] Users use their devices to input feedback and change requests regarding their travel plans. These requests are then sent back to the server, and the travel plans are updated.

[0731] Step 7:

[0732] During their trip, users utilize the device's navigation function.

[0733] The device provides real-time navigation information based on the visit schedule. It also utilizes translation functions to help users understand local information and facilitate communication.

[0734] This entire process allows travelers to efficiently enjoy a personalized travel experience.

[0735] (Application Example 1)

[0736] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0737] There is a growing need to provide personalized travel plans that cater to the diverse interests and needs of travelers, and to support smooth travel and communication by providing real-time visual and auditory information during their trips. However, conventional systems have been unable to adequately meet these needs, making it difficult to improve the travel experience.

[0738] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0739] In this invention, the server includes a device for inputting travel information, a device for automatically selecting tourist destinations based on the travel information, and a device for forming a visit schedule for the selected tourist destinations. This enables the provision of travel plans optimized to the user's interests and the visual and auditory presentation of information during the trip.

[0740] A "travel information input device" is a device used to collect information such as a traveler's interests and preferences, accommodation, and itinerary, and input it into a system.

[0741] A "device that automatically selects tourist destinations" is a device that automatically selects appropriate tourist destinations using an algorithm based on the travel information entered.

[0742] A "device for creating a visit schedule" is a system component that generates a schedule for efficiently visiting selected tourist destinations.

[0743] A "device for displaying on the user's terminal" is a device that visually organizes the generated visit schedule and tourist destination information and presents it to the user through a user interface.

[0744] A "device that updates travel plans based on user preferences" is a system that dynamically modifies existing travel plans in response to user feedback and new preferences.

[0745] A "device that realistically displays tourist information and travel routes through a visual device" refers to a technology that uses smart glasses or similar devices to present tourist information and navigation as augmented reality.

[0746] A "device that analyzes voice information, receives instructions, and performs language conversion" is a system component that understands user instructions using voice recognition technology and performs translation processing for multilingual support.

[0747] The system for realizing this invention mainly consists of a server, a terminal, and user interaction. The server includes a device for inputting travel information, a device for automatically selecting tourist destinations, and a device for generating a visit itinerary. Specifically, the server uses generative AI models such as TensorFlow or PyTorch to analyze travel information sent by the user and select the optimal tourist destination.

[0748] The terminal is equipped with a visual device and a voice analysis device, and visually displays the generated visit schedule to the user. The visual device, such as smart glasses, plays a role in extending the visual information displayed to the user in the real environment and can accept voice commands. For voice analysis, Google Speech-to-Text and Google Translate APIs are used to analyze the user's voice instructions and perform language conversion.

[0749] Users can input travel information and create travel plans that include tourist destinations tailored to their interests by operating the terminal. To assist with travel, the system provides route information using GPS and compass functions and implements real-time navigation based on the user's current location.

[0750] For example, if a user plans a trip to Tokyo and gives a voice command saying, "Take me to Tokyo Tower," the smart glasses will display the optimal route from the user's current location to Tokyo Tower. Along the way, users can obtain information about points of interest simply by looking at them, and local languages ​​are translated into other languages, enabling natural communication with locals.

[0751] An example of a prompt message could be: "If the user's interest is related to anime, please select a recommended tourist destination."

[0752] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0753] Step 1:

[0754] Users input travel information through the terminal's interface. This information includes areas of interest, planned destinations, and travel itinerary. This information is then transmitted from the terminal to the server as digital data.

[0755] Step 2:

[0756] The server analyzes the received travel information and uses a generative AI model (for example, a model based on TensorFlow or PyTorch) to select tourist destinations that are suitable for the user's interests. Based on the travel information received as input, it calculates the degree of fit for the candidate tourist destinations stored in the database and outputs a list of the most suitable tourist destinations.

[0757] Step 3:

[0758] The server generates a travel itinerary based on the selected tourist destinations. Based on the input tourist destination information, it creates an optimal schedule considering the available visiting times and travel times for each location. The generated schedule contributes to the user's efficient travel planning.

[0759] Step 4:

[0760] The terminal visually displays the visit schedule and tourist destination information retrieved from the server to the user. This information is then displayed as an overlay on the real world through a visual device (e.g., smart glasses). This allows the user to intuitively grasp the tourist destination information in relation to their actual location.

[0761] Step 5:

[0762] Users can check and modify their schedules using eye-tracking or voice commands. The device understands the commands using voice analysis technology (such as Google Speech-to-Text) and sends them to the server as new input data. This allows the system to re-select tourist destinations based on the new criteria and update the schedule.

[0763] Step 6:

[0764] The device uses GPS and compass functions to provide real-time navigation information to assist with on-site travel. Taking the current location as input, it outputs the optimal route to the destination, helping users travel smoothly on foot or using public transportation.

[0765] Step 7:

[0766] When a user needs translation assistance while in a foreign country, the device uses the Google Translate API to translate the input into the appropriate language. It translates the input voice and text data and presents the output to the user, supporting smooth communication.

[0767] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0768] This invention relates to a system that provides personalized tourist guides and travel coordination services to travelers. In particular, by combining it with an emotion engine that recognizes the user's emotions, more sophisticated customization becomes possible.

[0769] The user enters travel information into the device, including their destination, interests, and length of stay. The device sends this information to the server, but before that, it has a function to collect emotional data through the user's facial expressions and tone of voice. The emotion engine analyzes this data to identify the user's current emotional state.

[0770] The server considers emotional data along with the received travel information and uses an AI model to optimally select tourist destinations and experiences. If the user is in a relaxed mood, quiet, nature-based tourist destinations are likely to be included in the plan. On the other hand, if the user is stressed, relaxing leisure activities and activities will be suggested.

[0771] The server generates a visit schedule that reflects emotional data and sends it to the terminal. The terminal visualizes this information and displays it to the user. For example, if the user is feeling overwhelmed in Tokyo, the server might suggest relaxing options such as "a picnic at Shinjuku Gyoen."

[0772] Throughout the trip, the user's emotions are continuously monitored, and an emotion engine feeds this data back to the server in real time. Based on this information, the server re-selects the visit schedule and sightseeing destinations if necessary. If the user starts to feel tired, adjustments are made, such as easing the schedule and increasing relaxation time at the hotel.

[0773] This invention is a system that, in this way, utilizes user emotional data to provide travelers with the most suitable plans, thereby enabling a richer travel experience.

[0774] The following describes the processing flow.

[0775] Step 1:

[0776] The user enters travel information into the device. This information includes the travel destination, interests, length of stay, and specific requests.

[0777] Step 2:

[0778] The device uses facial recognition cameras and voice analysis functions to detect the user's current emotional state. This allows it to collect data that identifies the user's stress level and level of enjoyment.

[0779] Step 3:

[0780] The device sends collected travel information and sentiment data to the server. This data transmission initiates processing on the server side.

[0781] Step 4:

[0782] The server uses an AI model to analyze travel information and sentiment data. This allows it to select the most suitable tourist destinations and activities for the user's current situation.

[0783] Step 5:

[0784] The server generates a visit schedule that takes the user's emotional state into account. For example, if the user is seeking relaxation, quiet tourist spots and relaxation facilities will be added to the plan.

[0785] Step 6:

[0786] The server generates a schedule and sends it to the terminal. The terminal visually organizes this information and presents it to the user in an easy-to-understand format.

[0787] Step 7:

[0788] Users review the proposed plan from their device and make changes or approvals as needed, ensuring flexibility in their travel plans.

[0789] Step 8:

[0790] During the trip, the device periodically monitors the user's emotional state and feeds that data back to the server. Based on this, the server dynamically adjusts the visit schedule.

[0791] Step 9:

[0792] If user stress is detected through the emotion engine, it's possible to ease the pre-set schedule or suggest new relaxation options.

[0793] (Example 2)

[0794] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0795] Traditional travel planning systems were unable to respond to individual user emotions and real-time changes, making it difficult to provide travelers with the optimal sightseeing experience. Furthermore, the inability to flexibly adjust schedules based on emotions meant that stress during travel could not be reduced.

[0796] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0797] In this invention, the server includes means for automatically selecting tourist destinations based on travel-related information and emotional data, means for generating emotional data by analyzing the user's facial expressions and voice data, and means for monitoring the user's emotional state in real time in order to realistically adjust the itinerary. This enables a personalized travel experience based on the user's current emotional state and flexible schedule management that reduces stress.

[0798] "Travel-related information" refers to information necessary for planning a trip, such as a traveler's destination, interests, and length of stay.

[0799] "Emotional data" refers to information about the user's current mental state and emotions, analyzed from their facial expressions and voice.

[0800] A "tourist spot" is a place of interest or attraction that is selected for travelers to visit.

[0801] A "travel itinerary" refers to a schedule that includes the order and timing of visits to each tourist spot a traveler will be visiting.

[0802] "User terminal" refers to a digital device used by a user to input and receive travel plans.

[0803] "Monitoring in real time" means continuously and immediately evaluating and recording the user's emotional state.

[0804] A "translation function" is a feature that supports language conversion between different languages.

[0805] This system combines multiple technological elements to provide travelers with personalized travel plans.

[0806] The user enters information about their travel destination, interests, and length of stay into the device. The device uses its built-in camera and microphone to record the user's facial expressions and voice tone, and collects emotional data based on this. Common digital devices are used for this data collection.

[0807] The emotion engine uses Microsoft Azure's Face API and IBM Watson's Tone Analyzer to analyze the user's facial expressions and voice data and generate emotion data. This emotion data is then incorporated as an important parameter in the user's travel planning.

[0808] The server uses a generative AI model to select tourist destinations based on travel-related information and sentiment data received from the device. This AI model utilizes common platforms such as Google Cloud AI. Based on the selected tourist destinations, the server creates a travel itinerary and sends it to the device.

[0809] The terminal visualizes the visit itinerary and displays it in an easy-to-understand format for the user. The visualized information is provided to the user using geographic information services such as the Google Maps API.

[0810] For example, if the system determines that the user is seeking relaxation during their trip to Tokyo, it might suggest the option of "a picnic at Shinjuku Gyoen National Garden." An example of a prompt used in this case would be, "Please suggest relaxing experiences for the user during their trip to Tokyo."

[0811] This system monitors users' emotions in real time during their trip and adjusts the itinerary as needed. This process makes it possible to provide users with the best possible travel experience.

[0812] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0813] Step 1:

[0814] The user enters information about their travel destination, interests, and length of stay into the terminal. The terminal formats this information as travel-related data. Specifically, it collects the text data selected or entered by the user in the input fields and generates a dataset to send to the server. This dataset is the information that forms the basis for processing in the next step.

[0815] Step 2:

[0816] The device uses its built-in camera and microphone to record the user's facial expressions and voice tone. This data serves as input for analysis by the emotion engine. The device collects the recorded data in real time using the camera and microphone sensors, and then standardizes the data format. This process provides the foundational data for generating emotion data.

[0817] Step 3:

[0818] The device sends the facial and voice data acquired in Step 2 to the emotion engine. The emotion engine analyzes this data and generates the user's emotion data. Specifically, it inputs the data into Microsoft Azure's Face API or IBM Watson's Tone Analyzer, and outputs the emotional state as quantitative data based on the analysis results. The obtained emotion data is used to customize the travel plan in the next step.

[0819] Step 4:

[0820] The server combines travel-related information and sentiment data received from the terminal to select tourist destinations. Here, the server uses a generative AI model to generate prompt sentences, which are used as input. For example, it might generate a prompt sentence such as "Suggest a relaxing experience for the user." Based on this, the AI ​​model outputs a list of selected tourist destinations.

[0821] Step 5:

[0822] The server creates a travel itinerary based on the information of the selected tourist spots. Specifically, it optimizes the order in which to visit each selected tourist spot and compiles it into an actual schedule. This itinerary is output as a plan that takes into account the user's convenience, adjusting the time of day and travel routes.

[0823] Step 6:

[0824] The server sends the generated itinerary to the device. The device receives this information and uses a method to present it to the user visually. In particular, by using the Google Maps API or similar to mark the visited locations on a map and visually displaying the schedule, the user can easily understand their plans.

[0825] Step 7:

[0826] Throughout the trip, the device continuously monitors the user's facial expressions and voice, updating emotional data in real time. This allows the server to adjust the itinerary as needed. For example, if the user is tired, the next part of the itinerary can be changed to a less strenuous one, incorporating breaks.

[0827] (Application Example 2)

[0828] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0829] Traditional travel planning systems have struggled to customize plans to reflect travelers' emotions and preferences, failing to optimize comfort and enjoyment during their trips. Furthermore, the lack of real-time adjustments to the visiting schedule based on the traveler's current emotional state meant there was room for improvement in the traveler's experience. In response, there is a need for flexible planning that takes travelers' emotional states into account.

[0830] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0831] In this invention, the server includes means for inputting travel information, means for updating the travel plan based on the user's emotional data, and means for collecting and analyzing emotional data in real time. This makes it possible to provide personalized travel plans that take into account the traveler's emotional state.

[0832] "Travel information" refers to information such as the destination a traveler wants to visit, the activities they are interested in, and the length of their stay.

[0833] A "tourist destination" refers to famous places and tourist spots that travelers can visit and experience.

[0834] A "visit schedule" refers to a plan that outlines the dates, times, and order in which to visit selected tourist destinations.

[0835] A "display terminal" refers to an electronic device used to provide users with generated visit schedules and travel plans.

[0836] "Emotional data" refers to the analysis results that show the user's current emotional state, and includes numerical information collected from facial expressions and tone of voice.

[0837] A "user interface" refers to the means by which a user inputs operational instructions into a system or visually confirms their emotional preferences.

[0838] A "travel route" refers to the optimal path for moving between different tourist destinations.

[0839] "Voice translation services" refer to technologies that translate speech to facilitate communication between different languages.

[0840] The system for implementing this invention combines data analysis utilizing an emotion engine with AI-driven plan generation to provide travelers with a personalized travel experience.

[0841] System Configuration

[0842] The server has an input mechanism for receiving travel information entered by travelers. Users enter their destination, interests, and length of stay using electronic devices such as smartphones and tablets. Before the input information is sent to the server, emotional data is collected on the device using facial recognition modules (e.g., Amazon Rekognition) and voice analysis modules.

[0843] The server is equipped with an emotion engine to analyze collected emotional data and identify the user's current emotional state. Based on this information and travel information, a generative AI model (e.g., Google Cloud AI) is used to automatically select tourist destinations and construct an optimal visit schedule. Furthermore, the generated schedule is visually presented to the user through a user interface.

[0844] During travel, the device monitors the user's emotional data in real time and feeds it back to the server as needed. The server then adjusts the schedule based on this data and updates the information through the user interface. In this way, the user's travel experience is always kept optimal.

[0845] Specific example

[0846] For example, if a user is feeling stressed while visiting a busy city, the server might suggest a quiet and relaxing tourist destination, such as "a stroll in a nearby nature park." It could also provide further relaxation by playing healing music tailored to the user's emotional state from their smart device.

[0847] Examples of generated AI prompts

[0848] "Please generate a recommended sightseeing plan for users whose current emotional state is relaxed. Include quiet tourist spots and relaxing activities."

[0849] This system aims to create new, emotionally engaging travel experiences for travelers, helping them to create pleasant memories.

[0850] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0851] Step 1:

[0852] The device receives travel information from the user, such as destination, interests, and length of stay, as input. In addition, it uses the camera and microphone to acquire emotional data from facial expressions and voice tone, and uses this data as input. The data is preprocessed, and the user's emotional state is quantified using a facial recognition API. This becomes the input data to the server.

[0853] Step 2:

[0854] The server acquires travel information and sentiment data transmitted from the terminal. Based on the acquired data, it uses a generative AI model to automatically select tourist destinations. It takes travel information and sentiment data as input and generates a list of tourist destinations as output. The AI ​​model executes a process to plan tourist destinations that match the traveler's preferences based on their emotions.

[0855] Step 3:

[0856] The server constructs a visit schedule based on a list of tourist destinations. This schedule includes the order and date / time of visits to the destinations and uses an optimization algorithm to minimize travel time. It takes a list of tourist destinations as input and generates an optimized visit schedule as output.

[0857] Step 4:

[0858] The server sends the generated visit schedule to the terminal. The terminal displays this schedule visually to the user using a user interface. The user can review the schedule and make additional inputs if necessary. Inputs include a schedule confirmation action, and output is a visualized travel plan.

[0859] Step 5:

[0860] The device continuously monitors the user's emotional data throughout the trip. This data is sent to a server in real time, which adjusts the visit schedule as needed. Real-time emotional data is used as input, and an adjusted schedule is generated as output. For example, if fatigue is detected, a new schedule including rest activities is suggested.

[0861] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0862] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0863] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0864] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0865] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0866] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0867] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0868] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0869] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0870] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0871] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0872] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0873] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0874] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0875] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0876] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0877] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0878] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0879] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0880] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0881] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0882] The following is further disclosed regarding the embodiments described above.

[0883] (Claim 1)

[0884] Means of entering travel information,

[0885] A means for automatically selecting tourist destinations based on the aforementioned travel information,

[0886] Means for generating a visit schedule to the aforementioned tourist destination,

[0887] A means for displaying the generated visit schedule on the user's terminal,

[0888] A means of updating travel plans based on user preferences,

[0889] A system that includes this.

[0890] (Claim 2)

[0891] The system according to claim 1, comprising means for optimizing travel routes between tourist destinations.

[0892] (Claim 3)

[0893] The system according to claim 1, comprising means for providing translation services for different languages.

[0894] "Example 1"

[0895] (Claim 1)

[0896] Means of obtaining travel information,

[0897] A means for automatically selecting tourist destination candidates based on the aforementioned travel information using a generative model,

[0898] A means of creating a visit schedule based on selected tourist destination candidates,

[0899] Means for presenting the aforementioned visit schedule to the user device,

[0900] A means to modify travel plans according to user preferences,

[0901] A system that includes this.

[0902] (Claim 2)

[0903] The system according to claim 1, comprising means for optimizing routes between tourist destinations.

[0904] (Claim 3)

[0905] The system according to claim 1, comprising means for providing translation services in different languages.

[0906] "Application Example 1"

[0907] (Claim 1)

[0908] A device for entering travel information,

[0909] A device that automatically selects tourist destinations based on the aforementioned travel information,

[0910] A device for creating a visit schedule to the selected tourist destinations,

[0911] A device that displays the formed visit schedule on the user's terminal,

[0912] A device that updates travel plans based on user preferences,

[0913] A device that realistically displays tourist information and travel routes through a visual device,

[0914] A device that analyzes voice information, receives instructions, and performs language conversion,

[0915] A system that includes this.

[0916] (Claim 2)

[0917] The system according to claim 1, comprising a device that optimizes travel routes between tourist destinations and provides guidance information based on the current location.

[0918] (Claim 3)

[0919] The system according to claim 1, comprising a device that provides a translation function for different languages ​​and allows setting changes by voice command.

[0920] "Example 2 of combining an emotion engine"

[0921] (Claim 1)

[0922] A means of entering travel-related information,

[0923] A means for automatically selecting tourist destinations based on the aforementioned travel-related information and sentiment data,

[0924] A means for generating emotion data by analyzing the user's facial expressions and voice data,

[0925] Means for generating a visit itinerary to the aforementioned tourist destinations,

[0926] A means of visually displaying the generated visit itinerary on the user's terminal,

[0927] A means of updating travel plans based on the user's emotional state,

[0928] A system that includes this.

[0929] (Claim 2)

[0930] The system according to claim 1, comprising means for monitoring the emotional state of a user in real time in order to realistically adjust the visit itinerary.

[0931] (Claim 3)

[0932] The system according to claim 1, comprising means for providing a translation function that supports various languages.

[0933] "Application example 2 when combining with an emotional engine"

[0934] (Claim 1)

[0935] Means of entering travel information,

[0936] A means for automatically selecting tourist destinations based on the aforementioned travel information,

[0937] Means for generating a visit schedule to the aforementioned tourist destination,

[0938] A means for displaying the generated visit schedule on a display terminal,

[0939] A means of updating travel plans based on user sentiment data,

[0940] A means for collecting and analyzing sentiment data in real time in order to respond to the aforementioned update,

[0941] A means of visualizing emotion-based preferences through a user interface,

[0942] A system that includes this.

[0943] (Claim 2)

[0944] The system according to claim 1, comprising means for optimizing travel routes between tourist destinations.

[0945] (Claim 3)

[0946] The system according to claim 1, comprising means for providing a voice conversion service for different languages. [Explanation of symbols]

[0947] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A device for entering travel information, A device that automatically selects tourist destinations based on the aforementioned travel information, A device for creating a visit schedule to the selected tourist destinations, A device that displays the formed visit schedule on the user's terminal, A device that updates travel plans based on user preferences, A device that realistically displays tourist information and travel routes through a visual device, A device that analyzes voice information, receives instructions, and performs language conversion, A system that includes this.

2. The system according to claim 1, comprising a device that optimizes travel routes between tourist destinations and provides guidance information based on the current location.

3. The system according to claim 1, comprising a device that provides a translation function for different languages ​​and allows setting changes by voice command.

Images