system

Abstract

We provide the system. [Solution] A means of obtaining the user's location information and preference information, A means of analyzing and suggesting potential destinations based on acquired location and preference information, A means of obtaining real-time information related to potential visit locations and optimizing visits, A method for making reservations and ticket arrangements for potential destinations all in one place, Means for providing information and cultural information in multiple languages, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As factors for a traveler to feel stressed in a foreign country, there are language barriers, lack of necessary information, and complexity of reservations and arrangements. These factors not only impair the traveler's experience but also cause waste of time and difficulty in travel planning. In addition, it is difficult to provide services according to various preferences of travelers, and it is not easy to realize an optimal travel experience for each individual. Therefore, there is a need for a system that integrally provides personalized information, multilingual support, and smooth reservation and arrangement for travelers.

Means for Solving the Problems

[0005] This invention provides means for acquiring the user's location and preference information, and has a function to analyze and suggest potential destinations based on the acquired information. It also includes means for acquiring real-time weather and traffic information to create an optimal travel plan, thereby improving the travel experience. Furthermore, it provides means for making reservations and ticket arrangements for suggested destinations in one go, reducing the procedural stress faced by travelers. In addition, it enables multilingual support, eliminating language barriers and facilitating information access in foreign countries. In this way, it significantly improves convenience for travelers and provides a personalized travel experience.

[0006] "User location information" refers to data acquired by the device that indicates the user's current geographical location.

[0007] "Preference information" refers to data that indicates a user's preferences and interests, and is obtained from past behavior, selection history, and social media activity.

[0008] "Suggested destinations" refer to tourist spots, shops, events, etc., that are recommended to be visited based on the user's preferences and location information.

[0009] "Real-time information" refers to information that shows the current situation or state, and includes time-dependent data such as weather data and traffic conditions.

[0010] "Multilingual information" refers to information provided in different languages ​​for users who speak different languages.

[0011] "Means of analysis and proposal" refers to the process and methods of processing collected data and presenting the most suitable options based on the user's interests.

[0012] "A means of handling reservations and ticket arrangements all at once" refers to a function that allows the system to automatically handle reservations and ticket purchases for destinations or events selected by the user. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] An example of an embodiment of the system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] As an embodiment of this invention, an information system is configured to function as a personal assistant for travelers. The system includes three main components: a user terminal, a central server, and an external information provision service, each working in cooperation with the others.

[0035] First, when a user launches an application, the device uses GPS to obtain the user's current geographical location. It also retrieves the user's profile, past travel history, and social media activity data, and sends this information to a central server.

[0036] The server analyzes user preferences using generative AI and machine learning algorithms based on data received from the terminal. This automatically generates personalized travel plans that are optimal for each individual user. The suggested options include tourist spots, events, restaurants, etc., and these also reflect real-time information.

[0037] Furthermore, the server uses APIs from external data providers to obtain weather information, traffic conditions, congestion levels, and user reviews in real time, and uses this information to optimize visits. For example, the provided plans are always optimized according to the current situation, including the order of visits and modes of transportation.

[0038] Furthermore, the server utilizes a multilingual natural language processing engine to provide detailed guidance and cultural information in the user's chosen language. This feature allows users to receive information with confidence, even in foreign countries, without experiencing language barriers.

[0039] Users can view the presented sightseeing plans through their devices and select destinations and events that interest them. Based on their selections, the server uses that information to execute booking and ticket purchase procedures. At this stage, the system integrates with partner systems to ensure that necessary reservations and arrangements are made quickly and reliably.

[0040] For example, a user visiting Tokyo might be suggested popular restaurants and the latest art events around Shibuya as places they can quickly visit in the evening after work. These options are displayed in real time on a map, including information on transportation delays. In this way, users can easily create a visit plan that suits their preferences.

[0041] Thus, the present invention centralizes the complex process of travel arrangements and provides an optimal and personalized travel experience.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The device detects when the user launches an application and uses GPS to obtain the user's current location. It also obtains the user's permission to collect user profile information, past travel history, social media activity, and other data.

[0045] Step 2:

[0046] The device transmits the collected location and preference data to a central server. After receiving this information, the server begins analysis by comparing it with past history and similar user behavior data stored in its database.

[0047] Step 3:

[0048] The server uses a generation AI to list potential destinations based on the user's interests from the analyzed data. This list consists of tourist attractions, events, restaurants, etc., and each option is scored according to the user's past preferences.

[0049] Step 4:

[0050] The server obtains weather information, traffic conditions, congestion levels, and user reviews in real time through external API services. Based on this latest information, it designs the optimal order of visits and modes of transportation for each destination and incorporates them into the proposed plan.

[0051] Step 5:

[0052] The terminal displays the suggestions received from the server to the user via an interface. The user views these suggestions and selects and decides on places to visit and events to participate in according to their preferences.

[0053] Step 6:

[0054] Based on user selections, the server processes relevant reservations and ticket purchases. It integrates with partner reservation platforms to quickly complete necessary reservation procedures and sends reservation confirmation notifications to the user's device.

[0055] Step 7:

[0056] The server utilizes a natural language processing engine to provide cultural descriptions of the destination and additional travel information in the user's chosen language. This information includes relevant multilingual guidelines and precautions.

[0057] Step 8:

[0058] Based on the information provided, users can enjoy their travel experience with peace of mind. Additional inquiries during the trip can also be quickly addressed through the device via the server.

[0059] In this way, the "TravelMate" system provides users with comprehensive travel support.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] One challenge travelers face when visiting new places is the difficulty in creating efficient and personalized travel plans. Traditional methods require users to search through vast amounts of information and create schedules themselves, which can lead to information overload and make choices difficult. Furthermore, it was difficult to flexibly adjust plans based on real-time information. This invention aims to solve these problems and provide users with a stress-free travel experience.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes means for collecting the user's geographical location information and preference information, means for analyzing and suggesting potential destinations using a generating AI model based on the collected geographical location information and preference information, and means for acquiring real-time information related to the potential destinations and dynamically optimizing the order of visits and means of transportation. As a result, the user can automatically receive a real-time optimized visit plan that suits their current location and interests.

[0065] "Geographic location information" refers to data that indicates the user's current physical location, and is obtained using technologies such as GPS.

[0066] "Preference information" refers to data about users' preferences and interests, generated based on travel history and social media data.

[0067] A "generative AI model" refers to an algorithm or system that uses machine learning techniques to analyze data and generate optimal suggestions for the user.

[0068] "Real-time information" refers to the latest data on current situations and conditions, including weather information, traffic conditions, and congestion levels.

[0069] "Means for providing information in multiple languages" refers to systems and processes for translating and providing information according to the language selected by the user.

[0070] A "natural language processing engine" is a technology that enables computers to understand and process human language, making it possible to provide users with information tailored to their language.

[0071] This invention is configured as an information system that provides travelers with personalized travel plans. The system mainly consists of three components: a user terminal, a central server, and an external information provision service.

[0072] First, when the travel app is launched, the device uses its built-in GPS function to obtain the user's geographical location. It also collects the user's profile information, past travel history, and social media activity data, and sends this data to a central server. This process accumulates information about the user's preferences.

[0073] The server utilizes a generated AI model based on data sent from the terminal and uses machine learning algorithms to create personalized sightseeing plans for the user. The server can use machine learning libraries such as TENSORFLOW® and PyTorch. Furthermore, the server obtains real-time information from external data providers via APIs and dynamically optimizes the visit plan based on weather information, traffic conditions, congestion levels, and other factors.

[0074] Furthermore, the server utilizes a multilingual natural language processing engine. It generates guidance and cultural information in the language selected by the user, making the information easier to understand. This enables the provision of quick and accurate information to users who speak different languages.

[0075] Users can view sightseeing plans provided through their devices, select options that interest them, and make necessary reservations and purchase tickets. This process is made possible by the server's rapid communication with the linked reservation system.

[0076] For example, when a user is visiting Tokyo, the server can use data on the user's current location and past travel preferences to suggest restaurants and art events around Shibuya suitable for dinner. The server can also use real-time data, including transportation delay information, to optimize the order of visits and help users make efficient use of their time.

[0077] An example of a prompt sentence for the generative AI model would be, "Based on the user's current location and past travel history, please suggest a suitable sightseeing plan for the evening in Tokyo."

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] When a user launches a travel app, the device uses GPS to obtain their current geographical location. This location information becomes the device's input data. The device also collects the user's profile information, past travel history, and social media activity data. This process creates a comprehensive dataset about the user, which is then sent to the server as output.

[0081] Step 2:

[0082] The server uses location information and user data received from the device as input. The server analyzes this data using a generating AI model and machine learning algorithms to understand user preferences. This analysis process identifies certain patterns and generates potential destinations that the user is likely to be interested in. The result is a personalized travel plan tailored to the user's preferences.

[0083] Step 3:

[0084] The server retrieves real-time information from external information services to further refine the generated sightseeing plans. Specifically, this includes weather information, traffic conditions, congestion levels, and user reviews. This becomes the input data, and the server uses this information to optimize the order of visits and modes of transportation to obtain the most efficient plan. This process involves real-time data retrieval and integration via APIs, and a ready-to-use sightseeing plan is output.

[0085] Step 4:

[0086] The server utilizes a multilingual natural language processing engine to generate guide and cultural information based on the user's selected language. The generated tourist plans and related information are input and provided to the user in an easily understandable format. This allows users to receive the necessary information without experiencing language barriers.

[0087] Step 5:

[0088] Users view travel plans provided through their devices. They select destinations and events of interest from the displayed plans and send their selections to the server. The server receives this information, integrates with partner booking systems, and quickly processes reservations and ticket purchases. As a result, a detailed travel plan is generated based on the user's selections.

[0089] (Application Example 1)

[0090] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0091] For travelers to enjoy sightseeing smoothly in a foreign land, it is necessary to select appropriate destinations and modes of transportation, and to have information provided that transcends language barriers. However, proposing the optimal plan for each individual traveler and autonomously providing local transportation and sightseeing guidance is difficult with conventional technology. To solve this problem, a system that takes into account the traveler's preferences and real-time situation is required.

[0092] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0093] In this invention, the server includes means for acquiring the user's location information and preference information; means for analyzing and proposing potential destinations based on the acquired location information and preference information; means for acquiring real-time information related to potential destinations and optimizing the visit; means for making reservations and ticket arrangements for potential destinations in a single process; means for providing guidance information and cultural information in multiple languages; and means for an autonomous guide robot accompanying the traveler to guide them through facilities and tourist attractions. This enables personalized sightseeing experiences for individual travelers and facilitates guidance that transcends language barriers.

[0094] "User location information" refers to data that indicates the user's current geographical location.

[0095] "Preference information" refers to information that reflects a user's personal preferences and interests.

[0096] "Potential destinations" are potential tourist spots and facilities that the user might visit.

[0097] "Real-time information" refers to information that instantly reflects the current situation, including data on weather, traffic, and congestion.

[0098] "Means of providing information and cultural information in multiple languages" refers to technologies that provide users with tourist guides and information about the local cultural background in multiple languages.

[0099] An "autonomous guide robot" is a robot that accompanies travelers and provides guidance while moving autonomously.

[0100] This system is comprised of three main components: a user terminal, a central server, and an autonomous guide robot. First, the terminal uses a GPS module to obtain the user's current location, collects the user's profile, past travel history, and activity data, and transmits it to the server. The server utilizes this data and a generative AI model to analyze and generate optimal destination suggestions for each individual user. Real-time information is obtained from external information services via APIs to optimize visits. This includes suggestions for visit order and transportation methods based on weather, traffic, and congestion. Furthermore, a multilingual natural language processing engine is used to provide detailed guidance and cultural information in the user's chosen language.

[0101] Furthermore, autonomous guide robots accompany users and provide directions to suggested destinations. The robots communicate with a central server and provide real-time tourist information in conjunction with the user's smartphone or smart glasses. For example, when a user visiting Tokyo heads to a tourist spot in Shibuya, the robot can suggest the optimal route based on weather and public transportation information, and explain the history and highlights of each spot.

[0102] For example, if a user enters a prompt such as, "Let's go to Shibuya next. Please tell me the best way to get there. Also, please tell me some local tips," the system will process the relevant data and, through a guide robot, will be able to provide the best destination, mode of transportation, and local tips.

[0103] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0104] Step 1:

[0105] The device uses a GPS module to obtain the user's current geographical location. Input is GPS data, and output is location information. The obtained location information is sent to the server. The device also simultaneously collects and sends the user's profile data, past travel history, and activity data to the server.

[0106] Step 2:

[0107] The server analyzes the user's location information, profile data, past travel history, and activity data received from the terminal. Inputs are user information and historical data, while output is user preference information. A generative AI model is used to infer user preferences and generate personalized, optimal destination suggestions.

[0108] Step 3:

[0109] The server obtains real-time information from external information providers via APIs. Input is API requests, and output is real-time information. This includes data on weather, traffic, and congestion. Based on the acquired real-time information, the server optimizes the order of visits and modes of transportation.

[0110] Step 4:

[0111] The server uses a multilingual natural language processing engine to generate guidance and cultural information in the language selected by the user. Input is text data and language selection, and output is multilingual guidance and cultural information.

[0112] Step 5:

[0113] The user reviews suggested destinations via their terminal. The input is a list of destinations, and the output is the user's selection. Based on the user's selection, the server automatically makes reservations and arranges tickets for the suggested destinations. Specifically, it communicates with partner systems to secure the necessary reservations and arrangements.

[0114] Step 6:

[0115] The autonomous guide robot connects to the user's smartphone or smart glasses and begins guiding them to suggested destinations. Input is destination data and real-time information, and output is optimal route guidance. The robot navigates the designated route and provides real-time explanations and local trivia at each spot.

[0116] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0117] This invention is an information system for realizing highly personalized travel assistance that meets the individual needs of travelers. One embodiment of this invention includes an application installed on the user's terminal, a central server, and an emotion engine.

[0118] Upon system startup, the device uses GPS to detect the user's current location and, with the user's permission, collects profile information, past travel history, and social media data. This data is sent to a central server and analyzed in real time.

[0119] The server uses the received data to generate potential destinations based on the user's preferences and past behavior. Here, the emotion engine plays a crucial role. The emotion engine uses voice analysis and facial recognition technology to detect the user's emotional state and then suggests appropriate destinations based on this. For example, if the user is feeling tired, the emotion engine will prioritize selecting places and activities that promote relaxation.

[0120] Furthermore, the server retrieves weather, traffic, and congestion information in real time through APIs from external data providers to optimize visit plans. The optimized plan is dynamically changed according to the user's emotional state, providing the most comfortable experience for the user.

[0121] Users can view sightseeing plans and event information provided by the server through their devices and select destinations based on their interests. The system automatically makes reservations and ticket arrangements based on these selections and notifies the user of the results.

[0122] For example, if the emotion engine detects that a user visiting Kyoto is feeling stressed, the system can suggest calm gardens or relaxation facilities, and even book a quiet cafe. By providing optimal destinations tailored to the user's emotions in this way, travelers can have a more satisfying experience.

[0123] The system as a whole possesses advanced analysis and suggestion capabilities linked to an emotion engine, and enables information provision that transcends language barriers through multilingual support. In this way, the present invention can improve the quality of travel support and significantly increase user satisfaction.

[0124] The following describes the processing flow.

[0125] Step 1:

[0126] When a user launches an application, the device uses GPS to obtain its current location. It also obtains permission from the user to use voice input and the camera, and collects voice and image data for the emotion engine to analyze emotions.

[0127] Step 2:

[0128] The device transmits acquired location information, voice data, image data, and other data such as the user's profile, past travel history, and social media activity to a central server via the internet.

[0129] Step 3:

[0130] The server analyzes received location and preference data and uses generative AI to generate suggested destinations based on the user's preferences. It also uses an emotion engine to identify the user's emotional state from their voice and image data.

[0131] Step 4:

[0132] Based on the analysis results from the emotion engine, the server prioritizes potential destinations to match the user's current emotions. Specifically, if the user is feeling stressed, it prioritizes relaxing places, and if they are excited, it suggests active events.

[0133] Step 5:

[0134] The server uses external APIs to obtain real-time weather, traffic, and congestion information. This information is used to determine the optimal order of visits and mode of transportation for already proposed destinations.

[0135] Step 6:

[0136] The device displays suggestions received from the server to the user. These suggestions include destinations, optimal modes of transportation, and weather-related notes. The user can then use this information to select destinations and activities.

[0137] Step 7:

[0138] Based on the destinations and events selected by the user, the server quickly makes relevant reservations and ticket arrangements. It works with partner platforms to notify the user of the arrangement details.

[0139] Step 8:

[0140] The server, based on the user's selection, utilizes multilingual natural language processing to provide information about selected destinations and cultural aspects in the user's preferred language. This information includes cultural background and local guidelines.

[0141] This series of processes allows users to create travel plans optimized to their emotional state and preferences, resulting in a more satisfying travel experience.

[0142] (Example 2)

[0143] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0144] Modern travelers demand personalized travel plans based on diverse preferences and emotional states, but traditional systems have struggled to accurately reflect these in real time. Furthermore, the lack of multilingual information and inadequate responses to unexpected changes in circumstances during destination selection and booking makes it difficult to provide travelers with highly satisfying travel experiences.

[0145] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0146] In this invention, the server includes means for analyzing the user's emotional state, means for optimizing suggested destinations based on the analyzed emotional state, and means for acquiring real-time information related to potential destinations and optimizing the visit. This enables the automatic and dynamic generation of travel plans tailored to the user's preferences and emotional state, the suggestion of appropriate destinations, and the provision of information in multiple languages.

[0147] "User" refers to an individual who uses an information system to optimize their travel plan.

[0148] "Location information" refers to data about a user's physical location, obtained using GPS or other location-based technologies.

[0149] "Preference information" refers to data collected based on users' interests, preferences, and past behavioral history.

[0150] "Emotional state" refers to the user's psychological and emotional state, and is measured through voice analysis and facial recognition.

[0151] "Real-time information" refers to the latest weather information, transportation status, congestion information, etc., related to potential destinations.

[0152] "Proposal method" refers to a technical method or mechanism for selecting and presenting the most suitable destination based on acquired data.

[0153] "Optimization" refers to the process of improving a travel plan to best suit the user's preferences and circumstances.

[0154] "Multilingual information provision" refers to the function of providing information in the corresponding language to users who speak different languages.

[0155] This information system consists of an application installed on the user's terminal, a central server, and an emotion engine, all designed to provide users with personalized travel plans. Specific software includes software libraries for voice analysis and facial recognition, as well as a database management system. Hardware includes terminals equipped with GPS modules, cameras, and microphones, along with a cloud-based server.

[0156] When the system starts up, the device uses GPS to determine the user's current location. Within the limits permitted by the user, the device also collects profile information, past travel history, and relevant data from social networks. This data is transmitted to a central server and analyzed in real time.

[0157] The server first generates potential destinations based on the user's preferences and past behavior history. This process uses machine learning to analyze the user's past preferences. Next, the emotion engine analyzes voice data and images captured by the camera to evaluate the user's current emotional state. Based on this information, the server updates the list of potential destinations and optimizes the suggestions.

[0158] Furthermore, the server obtains the latest weather information, traffic conditions, congestion levels, and other data through APIs from external data providers. This allows for the generation of optimal travel plans tailored to the user's current situation. Users can then view the suggested sightseeing plans and event information via their devices and make choices based on their own interests.

[0159] For example, if the emotion engine detects that a user is experiencing stress while visiting Kyoto, the system can suggest tranquil gardens and relaxation facilities, and even allow the user to book a relaxing cafe. In this way, users can experience destinations tailored to their emotional state, resulting in a highly satisfying trip.

[0160] An example of a prompt is, "Tell me some recommended relaxing places in Kyoto." Based on this prompt, the system can use a generative AI model to provide information tailored to the user.

[0161] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0162] Step 1:

[0163] The terminal uses GPS to obtain the user's current location when the system starts up. It receives GPS signals as input and obtains location information as output. This helps determine the user's physical location and prepares the system to determine the specific service area.

[0164] Step 2:

[0165] With the user's permission, the device collects profile information, past travel history, and social network data. It uses user approval and user ID as input and forms a personalized dataset as output. This data identifies the user's preferences and past tastes.

[0166] Step 3:

[0167] The terminal encrypts the collected data and sends it to a central server. It uses the data collected in the previous step as input and performs secure data communication as output. This process ensures data confidentiality during transmission to the server.

[0168] Step 4:

[0169] The server analyzes the received location information and user data to generate potential destinations. It uses data obtained from the device as input and outputs a list of destinations generated by a machine learning model. This list is calculated based on the user's past behavior patterns and preferences.

[0170] Step 5:

[0171] The server uses an emotion engine that performs voice analysis and facial recognition to analyze the user's emotional state. It uses voice and image data as input and obtains the detected emotional state as output. This emotional data is then used to suggest the next destination.

[0172] Step 6:

[0173] The server obtains real-time information via an external API and optimizes visit plans. It receives weather, traffic, and congestion information as input and generates optimized visit plans as output. These plans are dynamically updated based on the latest conditions.

[0174] Step 7:

[0175] The user accesses an optimization plan provided by the server via their device and selects destinations of interest. The user receives the optimization plan as input and confirms a list of selected destinations as output. Actions are then determined based on the user's selections.

[0176] Step 8:

[0177] The server automatically makes reservations and arranges tickets for destinations based on the user's selections and sends notifications. It receives a list of the user's selections as input and sends reservation confirmations and ticket information to the user's terminal as output. This process allows the user to prepare for their trip with minimal effort.

[0178] (Application Example 2)

[0179] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0180] In cities visited by travelers, there is a growing need to improve travel experience satisfaction by dynamically suggesting optimal destinations based on individual emotional states and preferences. However, conventional systems merely provide pre-planned itineraries and lack the ability to consider users' real-time emotions. Therefore, providing more personalized travel support is a challenge.

[0181] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0182] In this invention, the server includes means for acquiring the user's location information and preference information, means for analyzing and proposing potential destinations based on the acquired location information and preference information, and means for dynamically selecting an appropriate destination by analyzing the user's emotional state. This makes it possible to propose highly personalized destinations based on the user's emotional state and preferences.

[0183] A "user" refers to an individual or group that receives travel assistance through the information system.

[0184] "Location information" is data that indicates the geographical location where the user is currently located.

[0185] "Preference information" refers to data about users' interests and preferences.

[0186] "Potential destinations" refer to destinations and activities that should be considered in the user's travel plans.

[0187] "Real-time information" refers to the latest information regarding the current situation and conditions.

[0188] "Reservation" refers to the act of making or securing an arrangement for a specific time or place in advance.

[0189] "Procedures" refers to the collective term for the necessary preparations and applications required before a user visits a destination.

[0190] "Multilingual" refers to the ability to provide or respond to information in multiple different languages.

[0191] "Emotional state" refers to the psychological or emotional condition that a user is experiencing at a particular time.

[0192] "Dynamic selection" refers to a process of making flexible choices in response to changes in circumstances and conditions.

[0193] The system implementing this invention is an information system comprising a user terminal, a central server, and an engine for sentiment analysis. The terminal uses a GPS, camera, and microphone to collect the user's location information and sentiment data. As a result, location information, voice, and image data are acquired in real time.

[0194] The device utilizes an emotion analysis engine to analyze the user's emotional state based on image and audio data. This analysis employs algorithms that infer emotions from facial expressions and voice tone. The analysis results are sent to a server, which allows the server to understand the user's emotional state.

[0195] The server analyzes potential destinations based on received location, preference, and emotional state information. The server obtains real-time information such as weather and traffic data via APIs from external data providers to optimize the plan. Simultaneously, it dynamically selects activities and destinations that the user prefers.

[0196] Ultimately, the server provides users with optimized visit plans and automatically handles reservations and entry procedures. Furthermore, the information provided is multilingual, enabling users to access travel information regardless of language barriers.

[0197] For example, if a user visiting Kyoto enters into the app via their device, "I'm feeling a bit tired, so I'm looking for a place to relax," the sentiment analysis engine will process this request, and the server will suggest quiet gardens or relaxation facilities to the user. By utilizing a generative AI model and prompting the user with their desired destination in real time, a more personalized experience is provided.

[0198] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0199] Step 1:

[0200] The device uses GPS to obtain the user's current location information. This location information consists of latitude and longitude data obtained directly from the device's built-in GPS function, and this becomes the input for the next processing step.

[0201] Step 2:

[0202] The device uses its camera and microphone to collect the user's facial expressions and voice. This data is used as input for emotion analysis and is temporarily stored on the device.

[0203] Step 3:

[0204] The device sends facial expression and voice data to an emotion analysis engine to analyze the user's emotional state. Here, image and voice pattern recognition algorithms are used to infer emotions, and the results are output as data.

[0205] Step 4:

[0206] The device transmits sentiment analysis results and location information to a central server. This data serves as crucial input for determining the user's preferences and potential destinations.

[0207] Step 5:

[0208] The server uses a generative AI model to create prompt messages based on the received data and selects the optimal destination. Weather and traffic information is obtained from an external data provider's API, and an algorithm is used to analyze it and dynamically generate a list of destinations.

[0209] Step 6:

[0210] The server generates the optimal visit plan for the user based on a list of potential destinations. This plan includes reservation information and entry procedures, and is optimized in real time.

[0211] Step 7:

[0212] The server sends the final visit plan to the terminal, which then displays this plan to the user. The terminal uses its multilingual capabilities to provide the information in the user's chosen language.

[0213] Step 8:

[0214] Based on the proposed visit plan, the user selects specific actions. If necessary, the user can generate new prompts and input them back into the system to receive even more personalized suggestions.

[0215] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0216] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0217] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0218] [Second Embodiment]

[0219] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0220] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0221] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0222] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0223] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0224] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0225] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0226] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0227] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0228] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0229] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0230] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0231] As an embodiment of this invention, an information system is configured to function as a personal assistant for travelers. The system includes three main components: a user terminal, a central server, and an external information provision service, each working in cooperation with the others.

[0232] First, when a user launches an application, the device uses GPS to obtain the user's current geographical location. It also retrieves the user's profile, past travel history, and social media activity data, and sends this information to a central server.

[0233] The server analyzes user preferences using generative AI and machine learning algorithms based on data received from the terminal. This automatically generates personalized travel plans that are optimal for each individual user. The suggested options include tourist spots, events, restaurants, etc., and these also reflect real-time information.

[0234] Furthermore, the server uses APIs from external data providers to obtain weather information, traffic conditions, congestion levels, and user reviews in real time, and uses this information to optimize visits. For example, the provided plans are always optimized according to the current situation, including the order of visits and modes of transportation.

[0235] Furthermore, the server utilizes a multilingual natural language processing engine to provide detailed guidance and cultural information in the user's chosen language. This feature allows users to receive information with confidence, even in foreign countries, without experiencing language barriers.

[0236] Users can view the presented sightseeing plans through their devices and select destinations and events that interest them. Based on their selections, the server uses that information to execute booking and ticket purchase procedures. At this stage, the system integrates with partner systems to ensure that necessary reservations and arrangements are made quickly and reliably.

[0237] For example, a user visiting Tokyo might be suggested popular restaurants and the latest art events around Shibuya as places they can quickly visit in the evening after work. These options are displayed in real time on a map, including information on transportation delays. In this way, users can easily create a visit plan that suits their preferences.

[0238] Thus, the present invention centralizes the complex process of travel arrangements and provides an optimal and personalized travel experience.

[0239] The following describes the processing flow.

[0240] Step 1:

[0241] The device detects when the user launches an application and uses GPS to obtain the user's current location. It also obtains the user's permission to collect user profile information, past travel history, social media activity, and other data.

[0242] Step 2:

[0243] The device transmits the collected location and preference data to a central server. After receiving this information, the server begins analysis by comparing it with past history and similar user behavior data stored in its database.

[0244] Step 3:

[0245] The server uses a generation AI to list potential destinations based on the user's interests from the analyzed data. This list consists of tourist attractions, events, restaurants, etc., and each option is scored according to the user's past preferences.

[0246] Step 4:

[0247] The server obtains weather information, traffic conditions, congestion levels, and user reviews in real time through external API services. Based on this latest information, it designs the optimal order of visits and modes of transportation for each destination and incorporates them into the proposed plan.

[0248] Step 5:

[0249] The terminal displays the suggestions received from the server to the user via an interface. The user views these suggestions and selects and decides on places to visit and events to participate in according to their preferences.

[0250] Step 6:

[0251] Based on user selections, the server processes relevant reservations and ticket purchases. It integrates with partner reservation platforms to quickly complete necessary reservation procedures and sends reservation confirmation notifications to the user's device.

[0252] Step 7:

[0253] The server utilizes a natural language processing engine to provide cultural descriptions of the destination and additional travel information in the user's chosen language. This information includes relevant multilingual guidelines and precautions.

[0254] Step 8:

[0255] Based on the information provided, users can enjoy their travel experience with peace of mind. Additional inquiries during the trip can also be quickly addressed through the device via the server.

[0256] In this way, the "TravelMate" system provides users with comprehensive travel support.

[0257] (Example 1)

[0258] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0259] One challenge travelers face when visiting new places is the difficulty in creating efficient and personalized travel plans. Traditional methods require users to search through vast amounts of information and create schedules themselves, which can lead to information overload and make choices difficult. Furthermore, it was difficult to flexibly adjust plans based on real-time information. This invention aims to solve these problems and provide users with a stress-free travel experience.

[0260] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0261] In this invention, the server includes means for collecting the user's geographical location information and preference information, means for analyzing and suggesting potential destinations using a generating AI model based on the collected geographical location information and preference information, and means for acquiring real-time information related to the potential destinations and dynamically optimizing the order of visits and means of transportation. As a result, the user can automatically receive a real-time optimized visit plan that suits their current location and interests.

[0262] "Geographic location information" refers to data that indicates the user's current physical location, and is obtained using technologies such as GPS.

[0263] "Preference information" refers to data about users' preferences and interests, generated based on travel history and social media data.

[0264] A "generative AI model" refers to an algorithm or system that uses machine learning techniques to analyze data and generate optimal suggestions for the user.

[0265] "Real-time information" refers to the latest data on current situations and conditions, including weather information, traffic conditions, and congestion levels.

[0266] "Means for providing information in multiple languages" refers to systems and processes for translating and providing information according to the language selected by the user.

[0267] A "natural language processing engine" is a technology that enables computers to understand and process human language, making it possible to provide users with information tailored to their language.

[0268] This invention is configured as an information system that provides travelers with personalized travel plans. The system mainly consists of three components: a user terminal, a central server, and an external information provision service.

[0269] First, when the travel app is launched, the device uses its built-in GPS function to obtain the user's geographical location. It also collects the user's profile information, past travel history, and social media activity data, and sends this data to a central server. This process accumulates information about the user's preferences.

[0270] The server utilizes a generated AI model based on data sent from the terminal and uses machine learning algorithms to create personalized sightseeing plans for the user. The server can use machine learning libraries such as TensorFlow and PyTorch. Furthermore, the server obtains real-time information from external data providers via APIs and dynamically optimizes the visit plan based on weather information, traffic conditions, congestion levels, and other factors.

[0271] Furthermore, the server utilizes a multilingual natural language processing engine. It generates guidance and cultural information in the language selected by the user, making the information easier to understand. This enables the provision of quick and accurate information to users who speak different languages.

[0272] Users can view sightseeing plans provided through their devices, select options that interest them, and make necessary reservations and purchase tickets. This process is made possible by the server's rapid communication with the linked reservation system.

[0273] For example, when a user is visiting Tokyo, the server can use data on the user's current location and past travel preferences to suggest restaurants and art events around Shibuya suitable for dinner. The server can also use real-time data, including transportation delay information, to optimize the order of visits and help users make efficient use of their time.

[0274] An example of a prompt sentence for the generative AI model would be, "Based on the user's current location and past travel history, please suggest a suitable sightseeing plan for the evening in Tokyo."

[0275] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0276] Step 1:

[0277] When a user launches a travel app, the device uses GPS to obtain their current geographical location. This location information becomes the device's input data. The device also collects the user's profile information, past travel history, and social media activity data. This process creates a comprehensive dataset about the user, which is then sent to the server as output.

[0278] Step 2:

[0279] The server uses location information and user data received from the device as input. The server analyzes this data using a generating AI model and machine learning algorithms to understand user preferences. This analysis process identifies certain patterns and generates potential destinations that the user is likely to be interested in. The result is a personalized travel plan tailored to the user's preferences.

[0280] Step 3:

[0281] The server retrieves real-time information from external information services to further refine the generated sightseeing plans. Specifically, this includes weather information, traffic conditions, congestion levels, and user reviews. This becomes the input data, and the server uses this information to optimize the order of visits and modes of transportation to obtain the most efficient plan. This process involves real-time data retrieval and integration via APIs, and a ready-to-use sightseeing plan is output.

[0282] Step 4:

[0283] The server utilizes a multilingual natural language processing engine to generate guidance information and cultural information based on the language selected by the user. Here, the generated tourism plans and related information are input, and provided to the user in an easy-to-understand format as output. This enables the user to receive the necessary information without feeling the language barrier.

[0284] Step 5:

[0285] The user browses the tourism plan provided through the terminal. The user selects the places to visit or events of interest from the displayed plans, and transmits the selected information as input to the server. The server receives this and coordinates with the partnered reservation system to quickly make reservations and purchase tickets. As a result, a specific travel plan is output based on the user's selection.

[0286] (Application Example 1)

[0287] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0288] In order for a traveler to smoothly enjoy tourism in a foreign country, it is necessary to select appropriate places to visit and means of transportation, and provide information beyond the language barrier. However, it has been difficult with conventional technologies to propose an optimal plan for each individual traveler and to autonomously conduct local travel and tourism guidance. To solve this problem, a system that takes into account the traveler's preferences and real-time situations is required.

[0289] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following respective means.

[0290] In this invention, the server includes means for acquiring the user's location information and preference information; means for analyzing and proposing potential destinations based on the acquired location information and preference information; means for acquiring real-time information related to potential destinations and optimizing the visit; means for making reservations and ticket arrangements for potential destinations in a single process; means for providing guidance information and cultural information in multiple languages; and means for an autonomous guide robot accompanying the traveler to guide them through facilities and tourist attractions. This enables personalized sightseeing experiences for individual travelers and facilitates guidance that transcends language barriers.

[0291] "User location information" refers to data that indicates the user's current geographical location.

[0292] "Preference information" refers to information that reflects a user's personal preferences and interests.

[0293] "Potential destinations" are potential tourist spots and facilities that the user might visit.

[0294] "Real-time information" refers to information that instantly reflects the current situation, including data on weather, traffic, and congestion.

[0295] "Means of providing information and cultural information in multiple languages" refers to technologies that provide users with tourist guides and information about the local cultural background in multiple languages.

[0296] An "autonomous guide robot" is a robot that accompanies travelers and provides guidance while moving autonomously.

[0297] This system is comprised of three main components: a user terminal, a central server, and an autonomous guide robot. First, the terminal uses a GPS module to obtain the user's current location, collects the user's profile, past travel history, and activity data, and transmits it to the server. The server utilizes this data and a generative AI model to analyze and generate optimal destination suggestions for each individual user. Real-time information is obtained from external information services via APIs to optimize visits. This includes suggestions for visit order and transportation methods based on weather, traffic, and congestion. Furthermore, a multilingual natural language processing engine is used to provide detailed guidance and cultural information in the user's chosen language.

[0298] Furthermore, autonomous guide robots accompany users and provide directions to suggested destinations. The robots communicate with a central server and provide real-time tourist information in conjunction with the user's smartphone or smart glasses. For example, when a user visiting Tokyo heads to a tourist spot in Shibuya, the robot can suggest the optimal route based on weather and public transportation information, and explain the history and highlights of each spot.

[0299] For example, if a user enters a prompt such as, "Let's go to Shibuya next. Please tell me the best way to get there. Also, please tell me some local tips," the system will process the relevant data and, through a guide robot, will be able to provide the best destination, mode of transportation, and local tips.

[0300] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0301] Step 1:

[0302] The device uses a GPS module to obtain the user's current geographical location. Input is GPS data, and output is location information. The obtained location information is sent to the server. The device also simultaneously collects and sends the user's profile data, past travel history, and activity data to the server.

[0303] Step 2:

[0304] The server analyzes the user's location information, profile data, past travel history, and activity data received from the terminal. The input is user information and history data, and the output is the user's preference information. Use a generative AI model to infer the user's preferences and generate optimal visit destination candidates individually.

[0305] Step 3:

[0306] The server obtains real-time information through the API from an external information provider service. The input is the API request, and the output is the real-time information. This includes data on weather, traffic, and congestion status. Based on the obtained real-time information, optimize the order of visits and the means of transportation.

[0307] Step 4:

[0308] The server uses a natural language processing engine that supports multiple languages to generate guidance information and cultural information in the language selected by the user. The input is text data and language selection, and the output is multilingual guidance and cultural information.

[0309] Step 5:

[0310] The user checks the proposed visit destination candidates through the terminal. The input is the visit candidate list, and the output is the user's selection. Based on the user's selection, the server automatically makes reservations and arranges tickets for the visit destination candidates. As a specific operation, communicate with the partner system to secure the necessary reservations and arrangements.

[0311] Step 6:

[0312] The autonomous guide robot connects to the user's smartphone or smart glasses and begins guiding them to suggested destinations. Input is destination data and real-time information, and output is optimal route guidance. The robot navigates the designated route and provides real-time explanations and local trivia at each spot.

[0313] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0314] This invention is an information system for realizing highly personalized travel assistance that meets the individual needs of travelers. One embodiment of this invention includes an application installed on the user's terminal, a central server, and an emotion engine.

[0315] Upon system startup, the device uses GPS to detect the user's current location and, with the user's permission, collects profile information, past travel history, and social media data. This data is sent to a central server and analyzed in real time.

[0316] The server uses the received data to generate potential destinations based on the user's preferences and past behavior. Here, the emotion engine plays a crucial role. The emotion engine uses voice analysis and facial recognition technology to detect the user's emotional state and then suggests appropriate destinations based on this. For example, if the user is feeling tired, the emotion engine will prioritize selecting places and activities that promote relaxation.

[0317] Furthermore, the server retrieves weather, traffic, and congestion information in real time through APIs from external data providers to optimize visit plans. The optimized plan is dynamically changed according to the user's emotional state, providing the most comfortable experience for the user.

[0318] Users can view sightseeing plans and event information provided by the server through their devices and select destinations based on their interests. The system automatically makes reservations and ticket arrangements based on these selections and notifies the user of the results.

[0319] For example, if the emotion engine detects that a user visiting Kyoto is feeling stressed, the system can suggest calm gardens or relaxation facilities, and even book a quiet cafe. By providing optimal destinations tailored to the user's emotions in this way, travelers can have a more satisfying experience.

[0320] The system as a whole possesses advanced analysis and suggestion capabilities linked to an emotion engine, and enables information provision that transcends language barriers through multilingual support. In this way, the present invention can improve the quality of travel support and significantly increase user satisfaction.

[0321] The following describes the processing flow.

[0322] Step 1:

[0323] When a user launches an application, the device uses GPS to obtain its current location. It also obtains permission from the user to use voice input and the camera, and collects voice and image data for the emotion engine to analyze emotions.

[0324] Step 2:

[0325] The device transmits acquired location information, voice data, image data, and other data such as the user's profile, past travel history, and social media activity to a central server via the internet.

[0326] Step 3:

[0327] The server analyzes received location and preference data and uses generative AI to generate suggested destinations based on the user's preferences. It also uses an emotion engine to identify the user's emotional state from their voice and image data.

[0328] Step 4:

[0329] Based on the analysis results from the emotion engine, the server prioritizes potential destinations to match the user's current emotions. Specifically, if the user is feeling stressed, it prioritizes relaxing places, and if they are excited, it suggests active events.

[0330] Step 5:

[0331] The server uses external APIs to obtain real-time weather, traffic, and congestion information. This information is used to determine the optimal order of visits and mode of transportation for already proposed destinations.

[0332] Step 6:

[0333] The device displays suggestions received from the server to the user. These suggestions include destinations, optimal modes of transportation, and weather-related notes. The user can then use this information to select destinations and activities.

[0334] Step 7:

[0335] Based on the destinations and events selected by the user, the server quickly makes relevant reservations and ticket arrangements. It works with partner platforms to notify the user of the arrangement details.

[0336] Step 8:

[0337] The server, based on the user's selection, utilizes multilingual natural language processing to provide information about selected destinations and cultural aspects in the user's preferred language. This information includes cultural background and local guidelines.

[0338] This series of processes allows users to create travel plans optimized to their emotional state and preferences, resulting in a more satisfying travel experience.

[0339] (Example 2)

[0340] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0341] Modern travelers demand personalized travel plans based on diverse preferences and emotional states, but traditional systems have struggled to accurately reflect these in real time. Furthermore, the lack of multilingual information and inadequate responses to unexpected changes in circumstances during destination selection and booking makes it difficult to provide travelers with highly satisfying travel experiences.

[0342] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0343] In this invention, the server includes means for analyzing the user's emotional state, means for optimizing suggested destinations based on the analyzed emotional state, and means for acquiring real-time information related to potential destinations and optimizing the visit. This enables the automatic and dynamic generation of travel plans tailored to the user's preferences and emotional state, the suggestion of appropriate destinations, and the provision of information in multiple languages.

[0344] "User" refers to an individual who uses an information system to optimize their travel plan.

[0345] "Location information" refers to data about a user's physical location, obtained using GPS or other location-based technologies.

[0346] "Preference information" refers to data collected based on users' interests, preferences, and past behavioral history.

[0347] "Emotional state" refers to the user's psychological and emotional state, and is measured through voice analysis and facial recognition.

[0348] "Real-time information" refers to the latest weather information, transportation status, congestion information, etc., related to potential destinations.

[0349] "Proposal method" refers to a technical method or mechanism for selecting and presenting the most suitable destination based on acquired data.

[0350] "Optimization" refers to the process of improving a travel plan to best suit the user's preferences and circumstances.

[0351] "Multilingual information provision" refers to the function of providing information in the corresponding language to users who speak different languages.

[0352] This information system consists of an application installed on the user's terminal, a central server, and an emotion engine, all designed to provide users with personalized travel plans. Specific software includes software libraries for voice analysis and facial recognition, as well as a database management system. Hardware includes terminals equipped with GPS modules, cameras, and microphones, along with a cloud-based server.

[0353] When the system starts up, the device uses GPS to determine the user's current location. Within the limits permitted by the user, the device also collects profile information, past travel history, and relevant data from social networks. This data is transmitted to a central server and analyzed in real time.

[0354] The server first generates potential destinations based on the user's preferences and past behavior history. This process uses machine learning to analyze the user's past preferences. Next, the emotion engine analyzes voice data and images captured by the camera to evaluate the user's current emotional state. Based on this information, the server updates the list of potential destinations and optimizes the suggestions.

[0355] Furthermore, the server obtains the latest weather information, traffic conditions, congestion levels, and other data through APIs from external data providers. This allows for the generation of optimal travel plans tailored to the user's current situation. Users can then view the suggested sightseeing plans and event information via their devices and make choices based on their own interests.

[0356] For example, if the emotion engine detects that a user is experiencing stress while visiting Kyoto, the system can suggest tranquil gardens and relaxation facilities, and even allow the user to book a relaxing cafe. In this way, users can experience destinations tailored to their emotional state, resulting in a highly satisfying trip.

[0357] An example of a prompt is, "Tell me some recommended relaxing places in Kyoto." Based on this prompt, the system can use a generative AI model to provide information tailored to the user.

[0358] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0359] Step 1:

[0360] The terminal uses GPS to obtain the user's current location when the system starts up. It receives GPS signals as input and obtains location information as output. This helps determine the user's physical location and prepares the system to determine the specific service area.

[0361] Step 2:

[0362] With the user's permission, the device collects profile information, past travel history, and social network data. It uses user approval and user ID as input and forms a personalized dataset as output. This data identifies the user's preferences and past tastes.

[0363] Step 3:

[0364] The terminal encrypts the collected data and sends it to a central server. It uses the data collected in the previous step as input and performs secure data communication as output. This process ensures data confidentiality during transmission to the server.

[0365] Step 4:

[0366] The server analyzes the received location information and user data to generate potential destinations. It uses data obtained from the device as input and outputs a list of destinations generated by a machine learning model. This list is calculated based on the user's past behavior patterns and preferences.

[0367] Step 5:

[0368] The server uses an emotion engine that performs voice analysis and facial recognition to analyze the user's emotional state. It uses voice and image data as input and obtains the detected emotional state as output. This emotional data is then used to suggest the next destination.

[0369] Step 6:

[0370] The server obtains real-time information via an external API and optimizes visit plans. It receives weather, traffic, and congestion information as input and generates optimized visit plans as output. These plans are dynamically updated based on the latest conditions.

[0371] Step 7:

[0372] The user accesses an optimization plan provided by the server via their device and selects destinations of interest. The user receives the optimization plan as input and confirms a list of selected destinations as output. Actions are then determined based on the user's selections.

[0373] Step 8:

[0374] The server automatically makes reservations and arranges tickets for destinations based on the user's selections and sends notifications. It receives a list of the user's selections as input and sends reservation confirmations and ticket information to the user's terminal as output. This process allows the user to prepare for their trip with minimal effort.

[0375] (Application Example 2)

[0376] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0377] In cities visited by travelers, there is a growing need to improve travel experience satisfaction by dynamically suggesting optimal destinations based on individual emotional states and preferences. However, conventional systems merely provide pre-planned itineraries and lack the ability to consider users' real-time emotions. Therefore, providing more personalized travel support is a challenge.

[0378] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0379] In this invention, the server includes means for acquiring the user's location information and preference information, means for analyzing and proposing potential destinations based on the acquired location information and preference information, and means for dynamically selecting an appropriate destination by analyzing the user's emotional state. This makes it possible to propose highly personalized destinations based on the user's emotional state and preferences.

[0380] A "user" refers to an individual or group that receives travel assistance through the information system.

[0381] "Location information" is data that indicates the geographical location where the user is currently located.

[0382] "Preference information" refers to data about users' interests and preferences.

[0383] "Potential destinations" refer to destinations and activities that should be considered in the user's travel plans.

[0384] "Real-time information" refers to the latest information regarding the current situation and conditions.

[0385] "Reservation" refers to the act of making or securing an arrangement for a specific time or place in advance.

[0386] "Procedures" refers to the collective term for the necessary preparations and applications required before a user visits a destination.

[0387] "Multilingual" refers to the ability to provide or respond to information in multiple different languages.

[0388] "Emotional state" refers to the psychological or emotional condition that a user is experiencing at a particular time.

[0389] "Dynamic selection" refers to a process of making flexible choices in response to changes in circumstances and conditions.

[0390] The system implementing this invention is an information system comprising a user terminal, a central server, and an engine for sentiment analysis. The terminal uses a GPS, camera, and microphone to collect the user's location information and sentiment data. As a result, location information, voice, and image data are acquired in real time.

[0391] The device utilizes an emotion analysis engine to analyze the user's emotional state based on image and audio data. This analysis employs algorithms that infer emotions from facial expressions and voice tone. The analysis results are sent to a server, which allows the server to understand the user's emotional state.

[0392] The server analyzes potential destinations based on received location, preference, and emotional state information. The server obtains real-time information such as weather and traffic data via APIs from external data providers to optimize the plan. Simultaneously, it dynamically selects activities and destinations that the user prefers.

[0393] Ultimately, the server provides users with optimized visit plans and automatically handles reservations and entry procedures. Furthermore, the information provided is multilingual, enabling users to access travel information regardless of language barriers.

[0394] For example, if a user visiting Kyoto enters into the app via their device, "I'm feeling a bit tired, so I'm looking for a place to relax," the sentiment analysis engine will process this request, and the server will suggest quiet gardens or relaxation facilities to the user. By utilizing a generative AI model and prompting the user with their desired destination in real time, a more personalized experience is provided.

[0395] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0396] Step 1:

[0397] The device uses GPS to obtain the user's current location information. This location information consists of latitude and longitude data obtained directly from the device's built-in GPS function, and this becomes the input for the next processing step.

[0398] Step 2:

[0399] The device uses its camera and microphone to collect the user's facial expressions and voice. This data is used as input for emotion analysis and is temporarily stored on the device.

[0400] Step 3:

[0401] The device sends facial expression and voice data to an emotion analysis engine to analyze the user's emotional state. Here, image and voice pattern recognition algorithms are used to infer emotions, and the results are output as data.

[0402] Step 4:

[0403] The device transmits sentiment analysis results and location information to a central server. This data serves as crucial input for determining the user's preferences and potential destinations.

[0404] Step 5:

[0405] The server uses a generative AI model to create prompt messages based on the received data and selects the optimal destination. Weather and traffic information is obtained from an external data provider's API, and an algorithm is used to analyze it and dynamically generate a list of destinations.

[0406] Step 6:

[0407] The server generates the optimal visit plan for the user based on a list of potential destinations. This plan includes reservation information and entry procedures, and is optimized in real time.

[0408] Step 7:

[0409] The server sends the final visit plan to the terminal, which then displays this plan to the user. The terminal uses its multilingual capabilities to provide the information in the user's chosen language.

[0410] Step 8:

[0411] Based on the proposed visit plan, the user selects specific actions. If necessary, the user can generate new prompts and input them back into the system to receive even more personalized suggestions.

[0412] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0413] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0414] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0415] [Third Embodiment]

[0416] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0417] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0418] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0419] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0420] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0421] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0422] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0423] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0424] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0425] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0426] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0427] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0428] As an embodiment of this invention, an information system is configured to function as a personal assistant for travelers. The system includes three main components: a user terminal, a central server, and an external information provision service, each working in cooperation with the others.

[0429] First, when a user launches an application, the device uses GPS to obtain the user's current geographical location. It also retrieves the user's profile, past travel history, and social media activity data, and sends this information to a central server.

[0430] The server analyzes user preferences using generative AI and machine learning algorithms based on data received from the terminal. This automatically generates personalized travel plans that are optimal for each individual user. The suggested options include tourist spots, events, restaurants, etc., and these also reflect real-time information.

[0431] Furthermore, the server uses APIs from external data providers to obtain weather information, traffic conditions, congestion levels, and user reviews in real time, and uses this information to optimize visits. For example, the provided plans are always optimized according to the current situation, including the order of visits and modes of transportation.

[0432] Furthermore, the server utilizes a multilingual natural language processing engine to provide detailed guidance and cultural information in the user's chosen language. This feature allows users to receive information with confidence, even in foreign countries, without experiencing language barriers.

[0433] Users can view the presented sightseeing plans through their devices and select destinations and events that interest them. Based on their selections, the server uses that information to execute booking and ticket purchase procedures. At this stage, the system integrates with partner systems to ensure that necessary reservations and arrangements are made quickly and reliably.

[0434] For example, a user visiting Tokyo might be suggested popular restaurants and the latest art events around Shibuya as places they can quickly visit in the evening after work. These options are displayed in real time on a map, including information on transportation delays. In this way, users can easily create a visit plan that suits their preferences.

[0435] Thus, the present invention centralizes the complex process of travel arrangements and provides an optimal and personalized travel experience.

[0436] The following describes the processing flow.

[0437] Step 1:

[0438] The device detects when the user launches an application and uses GPS to obtain the user's current location. It also obtains the user's permission to collect user profile information, past travel history, social media activity, and other data.

[0439] Step 2:

[0440] The device transmits the collected location and preference data to a central server. After receiving this information, the server begins analysis by comparing it with past history and similar user behavior data stored in its database.

[0441] Step 3:

[0442] The server uses a generation AI to list potential destinations based on the user's interests from the analyzed data. This list consists of tourist attractions, events, restaurants, etc., and each option is scored according to the user's past preferences.

[0443] Step 4:

[0444] The server obtains weather information, traffic conditions, congestion levels, and user reviews in real time through external API services. Based on this latest information, it designs the optimal order of visits and modes of transportation for each destination and incorporates them into the proposed plan.

[0445] Step 5:

[0446] The terminal displays the suggestions received from the server to the user via an interface. The user views these suggestions and selects and decides on places to visit and events to participate in according to their preferences.

[0447] Step 6:

[0448] Based on user selections, the server processes relevant reservations and ticket purchases. It integrates with partner reservation platforms to quickly complete necessary reservation procedures and sends reservation confirmation notifications to the user's device.

[0449] Step 7:

[0450] The server utilizes a natural language processing engine to provide cultural descriptions of the destination and additional travel information in the user's chosen language. This information includes relevant multilingual guidelines and precautions.

[0451] Step 8:

[0452] Based on the information provided, users can enjoy their travel experience with peace of mind. Additional inquiries during the trip can also be quickly addressed through the device via the server.

[0453] In this way, the "TravelMate" system provides users with comprehensive travel support.

[0454] (Example 1)

[0455] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0456] One challenge travelers face when visiting new places is the difficulty in creating efficient and personalized travel plans. Traditional methods require users to search through vast amounts of information and create schedules themselves, which can lead to information overload and make choices difficult. Furthermore, it was difficult to flexibly adjust plans based on real-time information. This invention aims to solve these problems and provide users with a stress-free travel experience.

[0457] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0458] In this invention, the server includes means for collecting the user's geographical location information and preference information, means for analyzing and suggesting potential destinations using a generating AI model based on the collected geographical location information and preference information, and means for acquiring real-time information related to the potential destinations and dynamically optimizing the order of visits and means of transportation. As a result, the user can automatically receive a real-time optimized visit plan that suits their current location and interests.

[0459] "Geographic location information" refers to data that indicates the user's current physical location, and is obtained using technologies such as GPS.

[0460] "Preference information" refers to data about users' preferences and interests, generated based on travel history and social media data.

[0461] A "generative AI model" refers to an algorithm or system that uses machine learning techniques to analyze data and generate optimal suggestions for the user.

[0462] "Real-time information" refers to the latest data on current situations and conditions, including weather information, traffic conditions, and congestion levels.

[0463] "Means for providing information in multiple languages" refers to systems and processes for translating and providing information according to the language selected by the user.

[0464] A "natural language processing engine" is a technology that enables computers to understand and process human language, making it possible to provide users with information tailored to their language.

[0465] This invention is configured as an information system that provides travelers with personalized travel plans. The system mainly consists of three components: a user terminal, a central server, and an external information provision service.

[0466] First, when the travel app is launched, the device uses its built-in GPS function to obtain the user's geographical location. It also collects the user's profile information, past travel history, and social media activity data, and sends this data to a central server. This process accumulates information about the user's preferences.

[0467] The server utilizes a generated AI model based on data sent from the terminal and uses machine learning algorithms to create personalized sightseeing plans for the user. The server can use machine learning libraries such as TensorFlow and PyTorch. Furthermore, the server obtains real-time information from external data providers via APIs and dynamically optimizes the visit plan based on weather information, traffic conditions, congestion levels, and other factors.

[0468] Furthermore, the server utilizes a multilingual natural language processing engine. It generates guidance and cultural information in the language selected by the user, making the information easier to understand. This enables the provision of quick and accurate information to users who speak different languages.

[0469] Users can view sightseeing plans provided through their devices, select options that interest them, and make necessary reservations and purchase tickets. This process is made possible by the server's rapid communication with the linked reservation system.

[0470] For example, when a user is visiting Tokyo, the server can use data on the user's current location and past travel preferences to suggest restaurants and art events around Shibuya suitable for dinner. The server can also use real-time data, including transportation delay information, to optimize the order of visits and help users make efficient use of their time.

[0471] An example of a prompt sentence for the generative AI model would be, "Based on the user's current location and past travel history, please suggest a suitable sightseeing plan for the evening in Tokyo."

[0472] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0473] Step 1:

[0474] When a user launches a travel app, the device uses GPS to obtain their current geographical location. This location information becomes the device's input data. The device also collects the user's profile information, past travel history, and social media activity data. This process creates a comprehensive dataset about the user, which is then sent to the server as output.

[0475] Step 2:

[0476] The server uses location information and user data received from the device as input. The server analyzes this data using a generating AI model and machine learning algorithms to understand user preferences. This analysis process identifies certain patterns and generates potential destinations that the user is likely to be interested in. The result is a personalized travel plan tailored to the user's preferences.

[0477] Step 3:

[0478] The server retrieves real-time information from external information services to further refine the generated sightseeing plans. Specifically, this includes weather information, traffic conditions, congestion levels, and user reviews. This becomes the input data, and the server uses this information to optimize the order of visits and modes of transportation to obtain the most efficient plan. This process involves real-time data retrieval and integration via APIs, and a ready-to-use sightseeing plan is output.

[0479] Step 4:

[0480] The server utilizes a multilingual natural language processing engine to generate guide and cultural information based on the user's selected language. The generated tourist plans and related information are input and provided to the user in an easily understandable format. This allows users to receive the necessary information without experiencing language barriers.

[0481] Step 5:

[0482] Users view travel plans provided through their devices. They select destinations and events of interest from the displayed plans and send their selections to the server. The server receives this information, integrates with partner booking systems, and quickly processes reservations and ticket purchases. As a result, a detailed travel plan is generated based on the user's selections.

[0483] (Application Example 1)

[0484] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0485] For travelers to enjoy sightseeing smoothly in a foreign land, it is necessary to select appropriate destinations and modes of transportation, and to have information provided that transcends language barriers. However, proposing the optimal plan for each individual traveler and autonomously providing local transportation and sightseeing guidance is difficult with conventional technology. To solve this problem, a system that takes into account the traveler's preferences and real-time situation is required.

[0486] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0487] In this invention, the server includes means for acquiring the user's location information and preference information; means for analyzing and proposing potential destinations based on the acquired location information and preference information; means for acquiring real-time information related to potential destinations and optimizing the visit; means for making reservations and ticket arrangements for potential destinations in a single process; means for providing guidance information and cultural information in multiple languages; and means for an autonomous guide robot accompanying the traveler to guide them through facilities and tourist attractions. This enables personalized sightseeing experiences for individual travelers and facilitates guidance that transcends language barriers.

[0488] "User location information" refers to data that indicates the user's current geographical location.

[0489] "Preference information" refers to information that reflects a user's personal preferences and interests.

[0490] "Potential destinations" are potential tourist spots and facilities that the user might visit.

[0491] "Real-time information" refers to information that instantly reflects the current situation, including data on weather, traffic, and congestion.

[0492] "Means of providing information and cultural information in multiple languages" refers to technologies that provide users with tourist guides and information about the local cultural background in multiple languages.

[0493] An "autonomous guide robot" is a robot that accompanies travelers and provides guidance while moving autonomously.

[0494] This system is comprised of three main components: a user terminal, a central server, and an autonomous guide robot. First, the terminal uses a GPS module to obtain the user's current location, collects the user's profile, past travel history, and activity data, and transmits it to the server. The server utilizes this data and a generative AI model to analyze and generate optimal destination suggestions for each individual user. Real-time information is obtained from external information services via APIs to optimize visits. This includes suggestions for visit order and transportation methods based on weather, traffic, and congestion. Furthermore, a multilingual natural language processing engine is used to provide detailed guidance and cultural information in the user's chosen language.

[0495] Furthermore, autonomous guide robots accompany users and provide directions to suggested destinations. The robots communicate with a central server and provide real-time tourist information in conjunction with the user's smartphone or smart glasses. For example, when a user visiting Tokyo heads to a tourist spot in Shibuya, the robot can suggest the optimal route based on weather and public transportation information, and explain the history and highlights of each spot.

[0496] For example, if a user enters a prompt such as, "Let's go to Shibuya next. Please tell me the best way to get there. Also, please tell me some local tips," the system will process the relevant data and, through a guide robot, will be able to provide the best destination, mode of transportation, and local tips.

[0497] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0498] Step 1:

[0499] The device uses a GPS module to obtain the user's current geographical location. Input is GPS data, and output is location information. The obtained location information is sent to the server. The device also simultaneously collects and sends the user's profile data, past travel history, and activity data to the server.

[0500] Step 2:

[0501] The server analyzes the user's location information, profile data, past travel history, and activity data received from the terminal. Inputs are user information and historical data, while output is user preference information. A generative AI model is used to infer user preferences and generate personalized, optimal destination suggestions.

[0502] Step 3:

[0503] The server obtains real-time information from external information providers via APIs. Input is API requests, and output is real-time information. This includes data on weather, traffic, and congestion. Based on the acquired real-time information, the server optimizes the order of visits and modes of transportation.

[0504] Step 4:

[0505] The server uses a multilingual natural language processing engine to generate guidance and cultural information in the language selected by the user. Input is text data and language selection, and output is multilingual guidance and cultural information.

[0506] Step 5:

[0507] The user reviews suggested destinations via their terminal. The input is a list of destinations, and the output is the user's selection. Based on the user's selection, the server automatically makes reservations and arranges tickets for the suggested destinations. Specifically, it communicates with partner systems to secure the necessary reservations and arrangements.

[0508] Step 6:

[0509] The autonomous guide robot connects to the user's smartphone or smart glasses and begins guiding them to suggested destinations. Input is destination data and real-time information, and output is optimal route guidance. The robot navigates the designated route and provides real-time explanations and local trivia at each spot.

[0510] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0511] This invention is an information system for realizing highly personalized travel assistance that meets the individual needs of travelers. One embodiment of this invention includes an application installed on the user's terminal, a central server, and an emotion engine.

[0512] Upon system startup, the device uses GPS to detect the user's current location and, with the user's permission, collects profile information, past travel history, and social media data. This data is sent to a central server and analyzed in real time.

[0513] The server uses the received data to generate potential destinations based on the user's preferences and past behavior. Here, the emotion engine plays a crucial role. The emotion engine uses voice analysis and facial recognition technology to detect the user's emotional state and then suggests appropriate destinations based on this. For example, if the user is feeling tired, the emotion engine will prioritize selecting places and activities that promote relaxation.

[0514] Furthermore, the server retrieves weather, traffic, and congestion information in real time through APIs from external data providers to optimize visit plans. The optimized plan is dynamically changed according to the user's emotional state, providing the most comfortable experience for the user.

[0515] Users can view sightseeing plans and event information provided by the server through their devices and select destinations based on their interests. The system automatically makes reservations and ticket arrangements based on these selections and notifies the user of the results.

[0516] For example, if the emotion engine detects that a user visiting Kyoto is feeling stressed, the system can suggest calm gardens or relaxation facilities, and even book a quiet cafe. By providing optimal destinations tailored to the user's emotions in this way, travelers can have a more satisfying experience.

[0517] The system as a whole possesses advanced analysis and suggestion capabilities linked to an emotion engine, and enables information provision that transcends language barriers through multilingual support. In this way, the present invention can improve the quality of travel support and significantly increase user satisfaction.

[0518] The following describes the processing flow.

[0519] Step 1:

[0520] When a user launches an application, the device uses GPS to obtain its current location. It also obtains permission from the user to use voice input and the camera, and collects voice and image data for the emotion engine to analyze emotions.

[0521] Step 2:

[0522] The device transmits acquired location information, voice data, image data, and other data such as the user's profile, past travel history, and social media activity to a central server via the internet.

[0523] Step 3:

[0524] The server analyzes received location and preference data and uses generative AI to generate suggested destinations based on the user's preferences. It also uses an emotion engine to identify the user's emotional state from their voice and image data.

[0525] Step 4:

[0526] Based on the analysis results from the emotion engine, the server prioritizes potential destinations to match the user's current emotions. Specifically, if the user is feeling stressed, it prioritizes relaxing places, and if they are excited, it suggests active events.

[0527] Step 5:

[0528] The server uses external APIs to obtain real-time weather, traffic, and congestion information. This information is used to determine the optimal order of visits and mode of transportation for already proposed destinations.

[0529] Step 6:

[0530] The device displays suggestions received from the server to the user. These suggestions include destinations, optimal modes of transportation, and weather-related notes. The user can then use this information to select destinations and activities.

[0531] Step 7:

[0532] Based on the destinations and events selected by the user, the server quickly makes relevant reservations and ticket arrangements. It works with partner platforms to notify the user of the arrangement details.

[0533] Step 8:

[0534] The server, based on the user's selection, utilizes multilingual natural language processing to provide information about selected destinations and cultural aspects in the user's preferred language. This information includes cultural background and local guidelines.

[0535] This series of processes allows users to create travel plans optimized to their emotional state and preferences, resulting in a more satisfying travel experience.

[0536] (Example 2)

[0537] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0538] Modern travelers demand personalized travel plans based on diverse preferences and emotional states, but traditional systems have struggled to accurately reflect these in real time. Furthermore, the lack of multilingual information and inadequate responses to unexpected changes in circumstances during destination selection and booking makes it difficult to provide travelers with highly satisfying travel experiences.

[0539] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0540] In this invention, the server includes means for analyzing the user's emotional state, means for optimizing suggested destinations based on the analyzed emotional state, and means for acquiring real-time information related to potential destinations and optimizing the visit. This enables the automatic and dynamic generation of travel plans tailored to the user's preferences and emotional state, the suggestion of appropriate destinations, and the provision of information in multiple languages.

[0541] "User" refers to an individual who uses an information system to optimize their travel plan.

[0542] "Location information" refers to data about a user's physical location, obtained using GPS or other location-based technologies.

[0543] "Preference information" refers to data collected based on users' interests, preferences, and past behavioral history.

[0544] "Emotional state" refers to the user's psychological and emotional state, and is measured through voice analysis and facial recognition.

[0545] "Real-time information" refers to the latest weather information, transportation status, congestion information, etc., related to potential destinations.

[0546] "Proposal method" refers to a technical method or mechanism for selecting and presenting the most suitable destination based on acquired data.

[0547] "Optimization" refers to the process of improving a travel plan to best suit the user's preferences and circumstances.

[0548] "Multilingual information provision" refers to the function of providing information in the corresponding language to users who speak different languages.

[0549] This information system consists of an application installed on the user's terminal, a central server, and an emotion engine, all designed to provide users with personalized travel plans. Specific software includes software libraries for voice analysis and facial recognition, as well as a database management system. Hardware includes terminals equipped with GPS modules, cameras, and microphones, along with a cloud-based server.

[0550] When the system starts up, the device uses GPS to determine the user's current location. Within the limits permitted by the user, the device also collects profile information, past travel history, and relevant data from social networks. This data is transmitted to a central server and analyzed in real time.

[0551] The server first generates potential destinations based on the user's preferences and past behavior history. This process uses machine learning to analyze the user's past preferences. Next, the emotion engine analyzes voice data and images captured by the camera to evaluate the user's current emotional state. Based on this information, the server updates the list of potential destinations and optimizes the suggestions.

[0552] Furthermore, the server obtains the latest weather information, traffic conditions, congestion levels, and other data through APIs from external data providers. This allows for the generation of optimal travel plans tailored to the user's current situation. Users can then view the suggested sightseeing plans and event information via their devices and make choices based on their own interests.

[0553] For example, if the emotion engine detects that a user is experiencing stress while visiting Kyoto, the system can suggest tranquil gardens and relaxation facilities, and even allow the user to book a relaxing cafe. In this way, users can experience destinations tailored to their emotional state, resulting in a highly satisfying trip.

[0554] An example of a prompt is, "Tell me some recommended relaxing places in Kyoto." Based on this prompt, the system can use a generative AI model to provide information tailored to the user.

[0555] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0556] Step 1:

[0557] The terminal uses GPS to obtain the user's current location when the system starts up. It receives GPS signals as input and obtains location information as output. This helps determine the user's physical location and prepares the system to determine the specific service area.

[0558] Step 2:

[0559] With the user's permission, the device collects profile information, past travel history, and social network data. It uses user approval and user ID as input and forms a personalized dataset as output. This data identifies the user's preferences and past tastes.

[0560] Step 3:

[0561] The terminal encrypts the collected data and sends it to a central server. It uses the data collected in the previous step as input and performs secure data communication as output. This process ensures data confidentiality during transmission to the server.

[0562] Step 4:

[0563] The server analyzes the received location information and user data to generate potential destinations. It uses data obtained from the device as input and outputs a list of destinations generated by a machine learning model. This list is calculated based on the user's past behavior patterns and preferences.

[0564] Step 5:

[0565] The server uses an emotion engine that performs voice analysis and facial recognition to analyze the user's emotional state. It uses voice and image data as input and obtains the detected emotional state as output. This emotional data is then used to suggest the next destination.

[0566] Step 6:

[0567] The server obtains real-time information via an external API and optimizes visit plans. It receives weather, traffic, and congestion information as input and generates optimized visit plans as output. These plans are dynamically updated based on the latest conditions.

[0568] Step 7:

[0569] The user accesses an optimization plan provided by the server via their device and selects destinations of interest. The user receives the optimization plan as input and confirms a list of selected destinations as output. Actions are then determined based on the user's selections.

[0570] Step 8:

[0571] The server automatically makes reservations and arranges tickets for destinations based on the user's selections and sends notifications. It receives a list of the user's selections as input and sends reservation confirmations and ticket information to the user's terminal as output. This process allows the user to prepare for their trip with minimal effort.

[0572] (Application Example 2)

[0573] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0574] In cities visited by travelers, there is a growing need to improve travel experience satisfaction by dynamically suggesting optimal destinations based on individual emotional states and preferences. However, conventional systems merely provide pre-planned itineraries and lack the ability to consider users' real-time emotions. Therefore, providing more personalized travel support is a challenge.

[0575] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0576] In this invention, the server includes means for acquiring the user's location information and preference information, means for analyzing and proposing potential destinations based on the acquired location information and preference information, and means for dynamically selecting an appropriate destination by analyzing the user's emotional state. This makes it possible to propose highly personalized destinations based on the user's emotional state and preferences.

[0577] A "user" refers to an individual or group that receives travel assistance through the information system.

[0578] "Location information" is data that indicates the geographical location where the user is currently located.

[0579] "Preference information" refers to data about users' interests and preferences.

[0580] "Potential destinations" refer to destinations and activities that should be considered in the user's travel plans.

[0581] "Real-time information" refers to the latest information regarding the current situation and conditions.

[0582] "Reservation" refers to the act of making or securing an arrangement for a specific time or place in advance.

[0583] "Procedures" refers to the collective term for the necessary preparations and applications required before a user visits a destination.

[0584] "Multilingual" refers to the ability to provide or respond to information in multiple different languages.

[0585] "Emotional state" refers to the psychological or emotional condition that a user is experiencing at a particular time.

[0586] "Dynamic selection" refers to a process of making flexible choices in response to changes in circumstances and conditions.

[0587] The system implementing this invention is an information system comprising a user terminal, a central server, and an engine for sentiment analysis. The terminal uses a GPS, camera, and microphone to collect the user's location information and sentiment data. As a result, location information, voice, and image data are acquired in real time.

[0588] The device utilizes an emotion analysis engine to analyze the user's emotional state based on image and audio data. This analysis employs algorithms that infer emotions from facial expressions and voice tone. The analysis results are sent to a server, which allows the server to understand the user's emotional state.

[0589] The server analyzes potential destinations based on received location, preference, and emotional state information. The server obtains real-time information such as weather and traffic data via APIs from external data providers to optimize the plan. Simultaneously, it dynamically selects activities and destinations that the user prefers.

[0590] Ultimately, the server provides users with optimized visit plans and automatically handles reservations and entry procedures. Furthermore, the information provided is multilingual, enabling users to access travel information regardless of language barriers.

[0591] For example, if a user visiting Kyoto enters into the app via their device, "I'm feeling a bit tired, so I'm looking for a place to relax," the sentiment analysis engine will process this request, and the server will suggest quiet gardens or relaxation facilities to the user. By utilizing a generative AI model and prompting the user with their desired destination in real time, a more personalized experience is provided.

[0592] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0593] Step 1:

[0594] The device uses GPS to obtain the user's current location information. This location information consists of latitude and longitude data obtained directly from the device's built-in GPS function, and this becomes the input for the next processing step.

[0595] Step 2:

[0596] The device uses its camera and microphone to collect the user's facial expressions and voice. This data is used as input for emotion analysis and is temporarily stored on the device.

[0597] Step 3:

[0598] The device sends facial expression and voice data to an emotion analysis engine to analyze the user's emotional state. Here, image and voice pattern recognition algorithms are used to infer emotions, and the results are output as data.

[0599] Step 4:

[0600] The device transmits sentiment analysis results and location information to a central server. This data serves as crucial input for determining the user's preferences and potential destinations.

[0601] Step 5:

[0602] The server uses a generative AI model to create prompt messages based on the received data and selects the optimal destination. Weather and traffic information is obtained from an external data provider's API, and an algorithm is used to analyze it and dynamically generate a list of destinations.

[0603] Step 6:

[0604] The server generates the optimal visit plan for the user based on a list of potential destinations. This plan includes reservation information and entry procedures, and is optimized in real time.

[0605] Step 7:

[0606] The server sends the final visit plan to the terminal, which then displays this plan to the user. The terminal uses its multilingual capabilities to provide the information in the user's chosen language.

[0607] Step 8:

[0608] Based on the proposed visit plan, the user selects specific actions. If necessary, the user can generate new prompts and input them back into the system to receive even more personalized suggestions.

[0609] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0610] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0611] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0612] [Fourth Embodiment]

[0613] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0614] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0615] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0616] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0617] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0618] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0619] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0620] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0621] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0622] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0623] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0624] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0625] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0626] As an embodiment of this invention, an information system is configured to function as a personal assistant for travelers. The system includes three main components: a user terminal, a central server, and an external information provision service, each working in cooperation with the others.

[0627] First, when a user launches an application, the device uses GPS to obtain the user's current geographical location. It also retrieves the user's profile, past travel history, and social media activity data, and sends this information to a central server.

[0628] The server analyzes user preferences using generative AI and machine learning algorithms based on data received from the terminal. This automatically generates personalized travel plans that are optimal for each individual user. The suggested options include tourist spots, events, restaurants, etc., and these also reflect real-time information.

[0629] Furthermore, the server uses APIs from external data providers to obtain weather information, traffic conditions, congestion levels, and user reviews in real time, and uses this information to optimize visits. For example, the provided plans are always optimized according to the current situation, including the order of visits and modes of transportation.

[0630] Furthermore, the server utilizes a multilingual natural language processing engine to provide detailed guidance and cultural information in the user's chosen language. This feature allows users to receive information with confidence, even in foreign countries, without experiencing language barriers.

[0631] Users can view the presented sightseeing plans through their devices and select destinations and events that interest them. Based on their selections, the server uses that information to execute booking and ticket purchase procedures. At this stage, the system integrates with partner systems to ensure that necessary reservations and arrangements are made quickly and reliably.

[0632] For example, a user visiting Tokyo might be suggested popular restaurants and the latest art events around Shibuya as places they can quickly visit in the evening after work. These options are displayed in real time on a map, including information on transportation delays. In this way, users can easily create a visit plan that suits their preferences.

[0633] Thus, the present invention centralizes the complex process of travel arrangements and provides an optimal and personalized travel experience.

[0634] The following describes the processing flow.

[0635] Step 1:

[0636] The device detects when the user launches an application and uses GPS to obtain the user's current location. It also obtains the user's permission to collect user profile information, past travel history, social media activity, and other data.

[0637] Step 2:

[0638] The device transmits the collected location and preference data to a central server. After receiving this information, the server begins analysis by comparing it with past history and similar user behavior data stored in its database.

[0639] Step 3:

[0640] The server uses a generation AI to list potential destinations based on the user's interests from the analyzed data. This list consists of tourist attractions, events, restaurants, etc., and each option is scored according to the user's past preferences.

[0641] Step 4:

[0642] The server obtains weather information, traffic conditions, congestion levels, and user reviews in real time through external API services. Based on this latest information, it designs the optimal order of visits and modes of transportation for each destination and incorporates them into the proposed plan.

[0643] Step 5:

[0644] The terminal displays the suggestions received from the server to the user via an interface. The user views these suggestions and selects and decides on places to visit and events to participate in according to their preferences.

[0645] Step 6:

[0646] Based on user selections, the server processes relevant reservations and ticket purchases. It integrates with partner reservation platforms to quickly complete necessary reservation procedures and sends reservation confirmation notifications to the user's device.

[0647] Step 7:

[0648] The server utilizes a natural language processing engine to provide cultural descriptions of the destination and additional travel information in the user's chosen language. This information includes relevant multilingual guidelines and precautions.

[0649] Step 8:

[0650] Based on the information provided, users can enjoy their travel experience with peace of mind. Additional inquiries during the trip can also be quickly addressed through the device via the server.

[0651] In this way, the "TravelMate" system provides users with comprehensive travel support.

[0652] (Example 1)

[0653] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0654] One challenge travelers face when visiting new places is the difficulty in creating efficient and personalized travel plans. Traditional methods require users to search through vast amounts of information and create schedules themselves, which can lead to information overload and make choices difficult. Furthermore, it was difficult to flexibly adjust plans based on real-time information. This invention aims to solve these problems and provide users with a stress-free travel experience.

[0655] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0656] In this invention, the server includes means for collecting the user's geographical location information and preference information, means for analyzing and suggesting potential destinations using a generating AI model based on the collected geographical location information and preference information, and means for acquiring real-time information related to the potential destinations and dynamically optimizing the order of visits and means of transportation. As a result, the user can automatically receive a real-time optimized visit plan that suits their current location and interests.

[0657] "Geographic location information" refers to data that indicates the user's current physical location, and is obtained using technologies such as GPS.

[0658] "Preference information" refers to data about users' preferences and interests, generated based on travel history and social media data.

[0659] A "generative AI model" refers to an algorithm or system that uses machine learning techniques to analyze data and generate optimal suggestions for the user.

[0660] "Real-time information" refers to the latest data on current situations and conditions, including weather information, traffic conditions, and congestion levels.

[0661] "Means for providing information in multiple languages" refers to systems and processes for translating and providing information according to the language selected by the user.

[0662] A "natural language processing engine" is a technology that enables computers to understand and process human language, making it possible to provide users with information tailored to their language.

[0663] This invention is configured as an information system that provides travelers with personalized travel plans. The system mainly consists of three components: a user terminal, a central server, and an external information provision service.

[0664] First, when the travel app is launched, the device uses its built-in GPS function to obtain the user's geographical location. It also collects the user's profile information, past travel history, and social media activity data, and sends this data to a central server. This process accumulates information about the user's preferences.

[0665] The server utilizes a generated AI model based on data sent from the terminal and uses machine learning algorithms to create personalized sightseeing plans for the user. The server can use machine learning libraries such as TensorFlow and PyTorch. Furthermore, the server obtains real-time information from external data providers via APIs and dynamically optimizes the visit plan based on weather information, traffic conditions, congestion levels, and other factors.

[0666] Furthermore, the server utilizes a multilingual natural language processing engine. It generates guidance and cultural information in the language selected by the user, making the information easier to understand. This enables the provision of quick and accurate information to users who speak different languages.

[0667] Users can view sightseeing plans provided through their devices, select options that interest them, and make necessary reservations and purchase tickets. This process is made possible by the server's rapid communication with the linked reservation system.

[0668] For example, when a user is visiting Tokyo, the server can use data on the user's current location and past travel preferences to suggest restaurants and art events around Shibuya suitable for dinner. The server can also use real-time data, including transportation delay information, to optimize the order of visits and help users make efficient use of their time.

[0669] An example of a prompt sentence for the generative AI model would be, "Based on the user's current location and past travel history, please suggest a suitable sightseeing plan for the evening in Tokyo."

[0670] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0671] Step 1:

[0672] When a user launches a travel app, the device uses GPS to obtain their current geographical location. This location information becomes the device's input data. The device also collects the user's profile information, past travel history, and social media activity data. This process creates a comprehensive dataset about the user, which is then sent to the server as output.

[0673] Step 2:

[0674] The server uses location information and user data received from the device as input. The server analyzes this data using a generating AI model and machine learning algorithms to understand user preferences. This analysis process identifies certain patterns and generates potential destinations that the user is likely to be interested in. The result is a personalized travel plan tailored to the user's preferences.

[0675] Step 3:

[0676] The server retrieves real-time information from external information services to further refine the generated sightseeing plans. Specifically, this includes weather information, traffic conditions, congestion levels, and user reviews. This becomes the input data, and the server uses this information to optimize the order of visits and modes of transportation to obtain the most efficient plan. This process involves real-time data retrieval and integration via APIs, and a ready-to-use sightseeing plan is output.

[0677] Step 4:

[0678] The server utilizes a multilingual natural language processing engine to generate guide and cultural information based on the user's selected language. The generated tourist plans and related information are input and provided to the user in an easily understandable format. This allows users to receive the necessary information without experiencing language barriers.

[0679] Step 5:

[0680] Users view travel plans provided through their devices. They select destinations and events of interest from the displayed plans and send their selections to the server. The server receives this information, integrates with partner booking systems, and quickly processes reservations and ticket purchases. As a result, a detailed travel plan is generated based on the user's selections.

[0681] (Application Example 1)

[0682] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0683] For travelers to enjoy sightseeing smoothly in a foreign land, it is necessary to select appropriate destinations and modes of transportation, and to have information provided that transcends language barriers. However, proposing the optimal plan for each individual traveler and autonomously providing local transportation and sightseeing guidance is difficult with conventional technology. To solve this problem, a system that takes into account the traveler's preferences and real-time situation is required.

[0684] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0685] In this invention, the server includes means for acquiring the user's location information and preference information; means for analyzing and proposing potential destinations based on the acquired location information and preference information; means for acquiring real-time information related to potential destinations and optimizing the visit; means for making reservations and ticket arrangements for potential destinations in a single process; means for providing guidance information and cultural information in multiple languages; and means for an autonomous guide robot accompanying the traveler to guide them through facilities and tourist attractions. This enables personalized sightseeing experiences for individual travelers and facilitates guidance that transcends language barriers.

[0686] "User location information" refers to data that indicates the user's current geographical location.

[0687] "Preference information" refers to information that reflects a user's personal preferences and interests.

[0688] "Potential destinations" are potential tourist spots and facilities that the user might visit.

[0689] "Real-time information" refers to information that instantly reflects the current situation, including data on weather, traffic, and congestion.

[0690] "Means of providing information and cultural information in multiple languages" refers to technologies that provide users with tourist guides and information about the local cultural background in multiple languages.

[0691] An "autonomous guide robot" is a robot that accompanies travelers and provides guidance while moving autonomously.

[0692] This system is comprised of three main components: a user terminal, a central server, and an autonomous guide robot. First, the terminal uses a GPS module to obtain the user's current location, collects the user's profile, past travel history, and activity data, and transmits it to the server. The server utilizes this data and a generative AI model to analyze and generate optimal destination suggestions for each individual user. Real-time information is obtained from external information services via APIs to optimize visits. This includes suggestions for visit order and transportation methods based on weather, traffic, and congestion. Furthermore, a multilingual natural language processing engine is used to provide detailed guidance and cultural information in the user's chosen language.

[0693] Furthermore, autonomous guide robots accompany users and provide directions to suggested destinations. The robots communicate with a central server and provide real-time tourist information in conjunction with the user's smartphone or smart glasses. For example, when a user visiting Tokyo heads to a tourist spot in Shibuya, the robot can suggest the optimal route based on weather and public transportation information, and explain the history and highlights of each spot.

[0694] For example, if a user enters a prompt such as, "Let's go to Shibuya next. Please tell me the best way to get there. Also, please tell me some local tips," the system will process the relevant data and, through a guide robot, will be able to provide the best destination, mode of transportation, and local tips.

[0695] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0696] Step 1:

[0697] The device uses a GPS module to obtain the user's current geographical location. Input is GPS data, and output is location information. The obtained location information is sent to the server. The device also simultaneously collects and sends the user's profile data, past travel history, and activity data to the server.

[0698] Step 2:

[0699] The server analyzes the user's location information, profile data, past travel history, and activity data received from the terminal. Inputs are user information and historical data, while output is user preference information. A generative AI model is used to infer user preferences and generate personalized, optimal destination suggestions.

[0700] Step 3:

[0701] The server obtains real-time information from external information providers via APIs. Input is API requests, and output is real-time information. This includes data on weather, traffic, and congestion. Based on the acquired real-time information, the server optimizes the order of visits and modes of transportation.

[0702] Step 4:

[0703] The server uses a multilingual natural language processing engine to generate guidance and cultural information in the language selected by the user. Input is text data and language selection, and output is multilingual guidance and cultural information.

[0704] Step 5:

[0705] The user reviews suggested destinations via their terminal. The input is a list of destinations, and the output is the user's selection. Based on the user's selection, the server automatically makes reservations and arranges tickets for the suggested destinations. Specifically, it communicates with partner systems to secure the necessary reservations and arrangements.

[0706] Step 6:

[0707] The autonomous guide robot connects to the user's smartphone or smart glasses and begins guiding them to suggested destinations. Input is destination data and real-time information, and output is optimal route guidance. The robot navigates the designated route and provides real-time explanations and local trivia at each spot.

[0708] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0709] This invention is an information system for realizing highly personalized travel assistance that meets the individual needs of travelers. One embodiment of this invention includes an application installed on the user's terminal, a central server, and an emotion engine.

[0710] Upon system startup, the device uses GPS to detect the user's current location and, with the user's permission, collects profile information, past travel history, and social media data. This data is sent to a central server and analyzed in real time.

[0711] The server uses the received data to generate potential destinations based on the user's preferences and past behavior. Here, the emotion engine plays a crucial role. The emotion engine uses voice analysis and facial recognition technology to detect the user's emotional state and then suggests appropriate destinations based on this. For example, if the user is feeling tired, the emotion engine will prioritize selecting places and activities that promote relaxation.

[0712] Furthermore, the server retrieves weather, traffic, and congestion information in real time through APIs from external data providers to optimize visit plans. The optimized plan is dynamically changed according to the user's emotional state, providing the most comfortable experience for the user.

[0713] Users can view sightseeing plans and event information provided by the server through their devices and select destinations based on their interests. The system automatically makes reservations and ticket arrangements based on these selections and notifies the user of the results.

[0714] For example, if the emotion engine detects that a user visiting Kyoto is feeling stressed, the system can suggest calm gardens or relaxation facilities, and even book a quiet cafe. By providing optimal destinations tailored to the user's emotions in this way, travelers can have a more satisfying experience.

[0715] The system as a whole possesses advanced analysis and suggestion capabilities linked to an emotion engine, and enables information provision that transcends language barriers through multilingual support. In this way, the present invention can improve the quality of travel support and significantly increase user satisfaction.

[0716] The following describes the processing flow.

[0717] Step 1:

[0718] When a user launches an application, the device uses GPS to obtain its current location. It also obtains permission from the user to use voice input and the camera, and collects voice and image data for the emotion engine to analyze emotions.

[0719] Step 2:

[0720] The device transmits acquired location information, voice data, image data, and other data such as the user's profile, past travel history, and social media activity to a central server via the internet.

[0721] Step 3:

[0722] The server analyzes received location and preference data and uses generative AI to generate suggested destinations based on the user's preferences. It also uses an emotion engine to identify the user's emotional state from their voice and image data.

[0723] Step 4:

[0724] Based on the analysis results from the emotion engine, the server prioritizes potential destinations to match the user's current emotions. Specifically, if the user is feeling stressed, it prioritizes relaxing places, and if they are excited, it suggests active events.

[0725] Step 5:

[0726] The server uses external APIs to obtain real-time weather, traffic, and congestion information. This information is used to determine the optimal order of visits and mode of transportation for already proposed destinations.

[0727] Step 6:

[0728] The device displays suggestions received from the server to the user. These suggestions include destinations, optimal modes of transportation, and weather-related notes. The user can then use this information to select destinations and activities.

[0729] Step 7:

[0730] Based on the destinations and events selected by the user, the server quickly makes relevant reservations and ticket arrangements. It works with partner platforms to notify the user of the arrangement details.

[0731] Step 8:

[0732] The server, based on the user's selection, utilizes multilingual natural language processing to provide information about selected destinations and cultural aspects in the user's preferred language. This information includes cultural background and local guidelines.

[0733] This series of processes allows users to create travel plans optimized to their emotional state and preferences, resulting in a more satisfying travel experience.

[0734] (Example 2)

[0735] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0736] Modern travelers demand personalized travel plans based on diverse preferences and emotional states, but traditional systems have struggled to accurately reflect these in real time. Furthermore, the lack of multilingual information and inadequate responses to unexpected changes in circumstances during destination selection and booking makes it difficult to provide travelers with highly satisfying travel experiences.

[0737] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0738] In this invention, the server includes means for analyzing the user's emotional state, means for optimizing suggested destinations based on the analyzed emotional state, and means for acquiring real-time information related to potential destinations and optimizing the visit. This enables the automatic and dynamic generation of travel plans tailored to the user's preferences and emotional state, the suggestion of appropriate destinations, and the provision of information in multiple languages.

[0739] "User" refers to an individual who uses an information system to optimize their travel plan.

[0740] "Location information" refers to data about a user's physical location, obtained using GPS or other location-based technologies.

[0741] "Preference information" refers to data collected based on users' interests, preferences, and past behavioral history.

[0742] "Emotional state" refers to the user's psychological and emotional state, and is measured through voice analysis and facial recognition.

[0743] "Real-time information" refers to the latest weather information, transportation status, congestion information, etc., related to potential destinations.

[0744] "Proposal method" refers to a technical method or mechanism for selecting and presenting the most suitable destination based on acquired data.

[0745] "Optimization" refers to the process of improving a travel plan to best suit the user's preferences and circumstances.

[0746] "Multilingual information provision" refers to the function of providing information in the corresponding language to users who speak different languages.

[0747] This information system consists of an application installed on the user's terminal, a central server, and an emotion engine, all designed to provide users with personalized travel plans. Specific software includes software libraries for voice analysis and facial recognition, as well as a database management system. Hardware includes terminals equipped with GPS modules, cameras, and microphones, along with a cloud-based server.

[0748] When the system starts up, the device uses GPS to determine the user's current location. Within the limits permitted by the user, the device also collects profile information, past travel history, and relevant data from social networks. This data is transmitted to a central server and analyzed in real time.

[0749] The server first generates potential destinations based on the user's preferences and past behavior history. This process uses machine learning to analyze the user's past preferences. Next, the emotion engine analyzes voice data and images captured by the camera to evaluate the user's current emotional state. Based on this information, the server updates the list of potential destinations and optimizes the suggestions.

[0750] Furthermore, the server obtains the latest weather information, traffic conditions, congestion levels, and other data through APIs from external data providers. This allows for the generation of optimal travel plans tailored to the user's current situation. Users can then view the suggested sightseeing plans and event information via their devices and make choices based on their own interests.

[0751] For example, if the emotion engine detects that a user is experiencing stress while visiting Kyoto, the system can suggest tranquil gardens and relaxation facilities, and even allow the user to book a relaxing cafe. In this way, users can experience destinations tailored to their emotional state, resulting in a highly satisfying trip.

[0752] An example of a prompt is, "Tell me some recommended relaxing places in Kyoto." Based on this prompt, the system can use a generative AI model to provide information tailored to the user.

[0753] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0754] Step 1:

[0755] The terminal uses GPS to obtain the user's current location when the system starts up. It receives GPS signals as input and obtains location information as output. This helps determine the user's physical location and prepares the system to determine the specific service area.

[0756] Step 2:

[0757] With the user's permission, the device collects profile information, past travel history, and social network data. It uses user approval and user ID as input and forms a personalized dataset as output. This data identifies the user's preferences and past tastes.

[0758] Step 3:

[0759] The terminal encrypts the collected data and sends it to a central server. It uses the data collected in the previous step as input and performs secure data communication as output. This process ensures data confidentiality during transmission to the server.

[0760] Step 4:

[0761] The server analyzes the received location information and user data to generate potential destinations. It uses data obtained from the device as input and outputs a list of destinations generated by a machine learning model. This list is calculated based on the user's past behavior patterns and preferences.

[0762] Step 5:

[0763] The server uses an emotion engine that performs voice analysis and facial recognition to analyze the user's emotional state. It uses voice and image data as input and obtains the detected emotional state as output. This emotional data is then used to suggest the next destination.

[0764] Step 6:

[0765] The server obtains real-time information via an external API and optimizes visit plans. It receives weather, traffic, and congestion information as input and generates optimized visit plans as output. These plans are dynamically updated based on the latest conditions.

[0766] Step 7:

[0767] The user accesses an optimization plan provided by the server via their device and selects destinations of interest. The user receives the optimization plan as input and confirms a list of selected destinations as output. Actions are then determined based on the user's selections.

[0768] Step 8:

[0769] The server automatically makes reservations and arranges tickets for destinations based on the user's selections and sends notifications. It receives a list of the user's selections as input and sends reservation confirmations and ticket information to the user's terminal as output. This process allows the user to prepare for their trip with minimal effort.

[0770] (Application Example 2)

[0771] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0772] In cities visited by travelers, there is a growing need to improve travel experience satisfaction by dynamically suggesting optimal destinations based on individual emotional states and preferences. However, conventional systems merely provide pre-planned itineraries and lack the ability to consider users' real-time emotions. Therefore, providing more personalized travel support is a challenge.

[0773] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0774] In this invention, the server includes means for acquiring the user's location information and preference information, means for analyzing and proposing potential destinations based on the acquired location information and preference information, and means for dynamically selecting an appropriate destination by analyzing the user's emotional state. This makes it possible to propose highly personalized destinations based on the user's emotional state and preferences.

[0775] A "user" refers to an individual or group that receives travel assistance through the information system.

[0776] "Location information" is data that indicates the geographical location where the user is currently located.

[0777] "Preference information" refers to data about users' interests and preferences.

[0778] "Potential destinations" refer to destinations and activities that should be considered in the user's travel plans.

[0779] "Real-time information" refers to the latest information regarding the current situation and conditions.

[0780] "Reservation" refers to the act of making or securing an arrangement for a specific time or place in advance.

[0781] "Procedures" refers to the collective term for the necessary preparations and applications required before a user visits a destination.

[0782] "Multilingual" refers to the ability to provide or respond to information in multiple different languages.

[0783] "Emotional state" refers to the psychological or emotional condition that a user is experiencing at a particular time.

[0784] "Dynamic selection" refers to a process of making flexible choices in response to changes in circumstances and conditions.

[0785] The system implementing this invention is an information system comprising a user terminal, a central server, and an engine for sentiment analysis. The terminal uses a GPS, camera, and microphone to collect the user's location information and sentiment data. As a result, location information, voice, and image data are acquired in real time.

[0786] The device utilizes an emotion analysis engine to analyze the user's emotional state based on image and audio data. This analysis employs algorithms that infer emotions from facial expressions and voice tone. The analysis results are sent to a server, which allows the server to understand the user's emotional state.

[0787] The server analyzes potential destinations based on received location, preference, and emotional state information. The server obtains real-time information such as weather and traffic data via APIs from external data providers to optimize the plan. Simultaneously, it dynamically selects activities and destinations that the user prefers.

[0788] Ultimately, the server provides users with optimized visit plans and automatically handles reservations and entry procedures. Furthermore, the information provided is multilingual, enabling users to access travel information regardless of language barriers.

[0789] For example, if a user visiting Kyoto enters into the app via their device, "I'm feeling a bit tired, so I'm looking for a place to relax," the sentiment analysis engine will process this request, and the server will suggest quiet gardens or relaxation facilities to the user. By utilizing a generative AI model and prompting the user with their desired destination in real time, a more personalized experience is provided.

[0790] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0791] Step 1:

[0792] The device uses GPS to obtain the user's current location information. This location information consists of latitude and longitude data obtained directly from the device's built-in GPS function, and this becomes the input for the next processing step.

[0793] Step 2:

[0794] The device uses its camera and microphone to collect the user's facial expressions and voice. This data is used as input for emotion analysis and is temporarily stored on the device.

[0795] Step 3:

[0796] The device sends facial expression and voice data to an emotion analysis engine to analyze the user's emotional state. Here, image and voice pattern recognition algorithms are used to infer emotions, and the results are output as data.

[0797] Step 4:

[0798] The device transmits sentiment analysis results and location information to a central server. This data serves as crucial input for determining the user's preferences and potential destinations.

[0799] Step 5:

[0800] The server uses a generative AI model to create prompt messages based on the received data and selects the optimal destination. Weather and traffic information is obtained from an external data provider's API, and an algorithm is used to analyze it and dynamically generate a list of destinations.

[0801] Step 6:

[0802] The server generates the optimal visit plan for the user based on a list of potential destinations. This plan includes reservation information and entry procedures, and is optimized in real time.

[0803] Step 7:

[0804] The server sends the final visit plan to the terminal, which then displays this plan to the user. The terminal uses its multilingual capabilities to provide the information in the user's chosen language.

[0805] Step 8:

[0806] Based on the proposed visit plan, the user selects specific actions. If necessary, the user can generate new prompts and input them back into the system to receive even more personalized suggestions.

[0807] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0808] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0809] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0810] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0811] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0812] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0813] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0814] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0815] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0816] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0817] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0818] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0819] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0820] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0821] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0822] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0823] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0824] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0825] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0826] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0827] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0828] The following is further disclosed regarding the embodiments described above.

[0829] (Claim 1)

[0830] A means of obtaining the user's location information and preference information,

[0831] A means of analyzing and suggesting potential destinations based on acquired location and preference information,

[0832] A means of obtaining real-time information related to potential visit locations and optimizing visits,

[0833] A method for making reservations and ticket arrangements for potential destinations all in one place,

[0834] Means for providing information and cultural information in multiple languages,

[0835] A system that includes this.

[0836] (Claim 2)

[0837] The system according to claim 1, characterized in that real-time information related to the aforementioned potential destinations includes weather information and traffic information.

[0838] (Claim 3)

[0839] The system according to claim 1, characterized in that the suggestion means generates suggestions using past travel history and user activity data on social media.

[0840] "Example 1"

[0841] (Claim 1)

[0842] A means for collecting users' geographical location information and preference information,

[0843] A means for analyzing and proposing potential destinations using a generation AI model based on collected geographic location information and preference information,

[0844] A means to obtain real-time information related to potential destinations and dynamically optimize the order of visits and means of transportation,

[0845] A means of making reservations for potential destinations and purchasing tickets all at once through a linked system,

[0846] A means for providing guidance and cultural information in multiple languages ​​using a natural language processing engine,

[0847] A system that includes this.

[0848] (Claim 2)

[0849] The system according to claim 1, characterized in that real-time information related to the aforementioned potential destinations includes weather information, traffic information, and congestion information.

[0850] (Claim 3)

[0851] The system according to claim 1, characterized in that the suggestion means generates suggestions using past travel history and user activity data on social media.

[0852] "Application Example 1"

[0853] (Claim 1)

[0854] A means of obtaining the user's location information and preference information,

[0855] A means of analyzing and suggesting potential destinations based on acquired location and preference information,

[0856] A means of obtaining real-time information related to potential visit locations and optimizing visits,

[0857] A method for making reservations and ticket arrangements for potential destinations all in one place,

[0858] Means for providing information and cultural information in multiple languages,

[0859] A means of guiding travelers through facilities and tourist attractions using autonomous guide robots,

[0860] A system that includes this.

[0861] (Claim 2)

[0862] The system according to claim 1, characterized in that real-time information related to the aforementioned potential destinations includes weather information and traffic information.

[0863] (Claim 3)

[0864] The system according to claim 1, characterized in that the suggestion means generates suggestions using past travel history and user activity data on social media.

[0865] "Example 2 of combining an emotion engine"

[0866] (Claim 1)

[0867] A means of obtaining the user's location information and preference information,

[0868] A means of analyzing and suggesting potential destinations based on acquired location and preference information,

[0869] A means of analyzing the emotional state of users,

[0870] A means to optimize suggestions for visiting locations based on analyzed emotional states,

[0871] A means of obtaining real-time information related to potential visit locations and optimizing visits,

[0872] A method for making reservations and arranging admission tickets for potential destinations all at once,

[0873] Means for providing information and cultural information in multiple languages,

[0874] A system that includes this.

[0875] (Claim 2)

[0876] The system according to claim 1, characterized in that real-time information related to the aforementioned potential destinations includes weather information, transportation status, and congestion information.

[0877] (Claim 3)

[0878] The system according to claim 1, characterized in that the proposed means generates proposals using voice analysis and facial recognition technology, utilizing past travel history and user activity data on social networks.

[0879] "Application example 2 when combining with an emotional engine"

[0880] (Claim 1)

[0881] A means of obtaining the user's location information and preference information,

[0882] A means of analyzing and suggesting potential destinations based on acquired location and preference information,

[0883] A means of obtaining real-time information related to potential visit locations and optimizing visits,

[0884] A method for making reservations and handling procedures for potential destinations all at once,

[0885] Means for providing information and cultural information in multiple languages,

[0886] A means of dynamically selecting appropriate destinations by analyzing the user's emotional state,

[0887] A system that includes this.

[0888] (Claim 2)

[0889] The system according to claim 1, characterized in that real-time information related to the aforementioned potential destinations includes environmental information and movement information.

[0890] (Claim 3)

[0891] The system according to claim 1, characterized in that the proposal means generates proposals using past activity history and user data on network media. [Explanation of Symbols]

[0892] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of obtaining the user's location information and preference information, A means of analyzing and suggesting potential destinations based on acquired location and preference information, A means of obtaining real-time information related to potential visit locations and optimizing visits, A method for making reservations and ticket arrangements for potential destinations all in one place, Means for providing information and cultural information in multiple languages, A system that includes this.

2. The system according to claim 1, characterized in that real-time information related to the aforementioned potential destinations includes weather information and traffic information.

3. The system according to claim 1, characterized in that the suggestion means generates suggestions using past travel history and user activity data on social media.

Images