system

Abstract

We provide the system. [Solution] A data storage means for recording the user's basic information, A means of obtaining external information based on temporary and basic information received from the user, A means for analyzing acquired external information using a generative model to create an optimized travel route and action plan, Information display means for displaying the created route and plan to the user, A means of integrating and presenting local information and event information that users are interested in, A system that includes this.

Description

Technical Field

[0004] , ,

[0005] , , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In conventional mobile and tourism plan providing systems, it has been difficult to sufficiently consider the individual wishes and current situations of users, and there has been a demand for quickly responding to changes in traffic conditions and facility congestion. In addition, there is a lack of visually understandable plan presentation, and improving the user experience has become an issue.

Means for Solving the Problems

[0005] This invention provides a system that generates optimized travel routes and itinerary plans in real time using a generation model, based on pre-registered information from the user, collected requested information and external data entered each time, and generated by the user. This system includes a plan presentation means using a visually easy-to-understand image generation model, and furthermore, it significantly improves user convenience by constantly monitoring changes in traffic conditions and weather information and regenerating new routes and plans as needed.

[0006] A "database for storing user information" is a system memory device that stores information such as the user's address, favorite places, and preferred mode of transportation, which is entered during initial setup, for use in subsequent processes.

[0007] "Information entered each time" refers to specific, variable information that users input each time, such as destinations, desired order of visits, budget, and number of people.

[0008] "Means for collecting external data" refers to functions or devices for acquiring information from external sources such as traffic conditions, weather forecasts, and facility congestion in real time.

[0009] A "generative model" is an algorithm or system used to derive optimized travel routes and itinerary plans based on given data.

[0010] An "optimized travel route and itinerary plan" is the result of designing an efficient and comfortable travel route and visiting schedule based on the user's wishes and circumstances.

[0011] "Information display means" refers to a display or application interface used to communicate generated travel routes and itinerary plans to users.

[0012] An "image generation model" is an algorithm or software used to generate maps or graphics in order to make information display visually easier to understand.

[0013] "Means for monitoring real-time operational and weather information" refers to sensors and software functions that constantly detect the latest operational status of transportation systems and changes in weather, and utilize them within the system as needed. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when combined with an emotion engine.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a processor with a reference number (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a RAM (Random Access Memory) with a reference number is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a storage with a reference number is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is a system for providing users with optimal travel routes and sightseeing plans by integrating user information from prior to use and information gathered and analyzed from external sources. Specific embodiments of this system are described below.

[0036] First, the user accesses the application through their device and enters their personal information. This information includes their address, favorite places to visit, preferred mode of transportation, health information such as allergies, and items they need to bring with them when traveling. This information is stored in a database by the server so that it does not need to be re-entered each time the service is used.

[0037] Next, when planning a trip or travel, users input information such as destination, order of visits, budget, and number of people into their device each time. This information is sent to a server and collected along with other external data (for example, traffic conditions, weather forecasts, facility congestion, etc.).

[0038] The server feeds the collected information into a generative model to generate optimized travel routes and sightseeing plans. This generative model takes multiple variables into account and optimizes the routes and plans to match the user's desired conditions. The plans are then visually presented to the user via an information display system. This presentation uses maps and graphic formats based on an image generative model, allowing the user to grasp the details of the routes and plans at a glance.

[0039] Furthermore, the server monitors traffic conditions and weather changes in real time, and regenerates routes and plans based on the new conditions if any changes occur. This functionality allows users to flexibly adapt to fluctuating external environments.

[0040] As an example of how this system works, consider a scenario where a user plans a weekend trip to a tourist destination. The user inputs destinations such as a zoo, restaurant, and souvenir shop into the application, and specifies their budget and the number of people. Based on these factors, the server calculates a plan that takes into account transportation schedules, weather forecasts, and the operating status of the destinations, and generates a visual route map. This allows the user to enjoy sightseeing with the most efficient and comfortable schedule.

[0041] Thus, the present invention provides an excellent mobility and tourism support system that improves the user experience and can quickly respond to changing circumstances.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] Users enter their personal information through their device. This information includes their address, favorite spots, preferred mode of transportation, and allergy information, which are all included in the application's initial setup screen.

[0045] Step 2:

[0046] The terminal sends the pre-entered information to the server. At this time, the information is encrypted using a secure communication protocol and transmitted safely.

[0047] Step 3:

[0048] The server stores the received preliminary information in a database. This stored information will be used in future plan generation to improve convenience.

[0049] Step 4:

[0050] When users plan trips or travel, they input information such as destination, order of visits, budget, and number of people into their device each time. These input fields are specific to the current plan.

[0051] Step 5:

[0052] The device transmits information to the server in real time as it moves. This includes the user's current location, which is used to generate accurate plans.

[0053] Step 6:

[0054] The server collects information as it goes along, along with external data from external sources such as traffic conditions, weather forecasts, and congestion levels at visited facilities.

[0055] Step 7:

[0056] The server uses a generative model to generate optimized travel routes and itinerary plans based on the collected data. The generative model utilizes AI algorithms to efficiently process multiple variables.

[0057] Step 8:

[0058] The server visually represents the generated plan using an image generation model and converts it into a format that is easy for the user to understand, such as maps and graphics.

[0059] Step 9:

[0060] The server sends visualized routes and plans to the terminal, and the user uses this information to confirm their travel and sightseeing schedule.

[0061] Step 10:

[0062] The server monitors traffic and weather information in real time. If there are any changes, it regenerates routes and plans based on the new information and immediately sends updated information to the terminal.

[0063] This process allows users to quickly obtain the optimal plan based on the information they enter in advance and each time.

[0064] (Example 1)

[0065] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0066] In modern travel and transportation, users need to constantly adapt to changing external conditions, such as traffic congestion, weather changes, and facility occupancy. However, many current systems cannot respond to these changes in real time, making it difficult to provide users with the best possible plan. Furthermore, customization to individual user preferences and conditions is insufficient, and improvements in usability are needed.

[0067] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0068] In this invention, the server includes an information recording medium for storing user attribute information, means for acquiring surrounding information based on information and attribute information received from the user each time, means for inputting the acquired surrounding information into a generating AI model to create an optimized travel route and sightseeing plan, and means for updating the surrounding information in real time and regenerating the route and plan in accordance with any changes. This enables users to quickly adapt to changes in the environment and enjoy optimal travel and sightseeing plans that suit their individual preferences and conditions.

[0069] "User attribute information" refers to information unique to each individual user, such as address, preferred locations, and health considerations, which is necessary for creating individual travel and sightseeing plans.

[0070] An "information recording medium" is a medium for storing information in digital or analog format, and includes databases and cloud storage.

[0071] "On-demand information" refers to information provided by users on the spot when planning a specific trip or journey, and includes variable conditions such as destination, order of visits, budget, and number of people.

[0072] "Surrounding information" refers to dynamic information obtained from external sources, such as traffic conditions, weather information, and facility congestion levels—information that changes in real time.

[0073] A "generative AI model" is a type of artificial intelligence technology that analyzes input data and automatically creates optimal travel routes and sightseeing plans for users.

[0074] "Travel route" refers to information that shows the path a user takes to travel from their starting point to their destination, and includes efficient and safe routes.

[0075] A "tourism plan" is a travel or sightseeing schedule created based on the user's interests and circumstances, and includes elements such as destinations, schedule, and costs.

[0076] A "display device" is a device that provides visual information to a user, and includes smartphones, tablets, and computer displays.

[0077] "Image generation technology" refers to techniques for visually displaying planned routes and schedules in an easy-to-understand manner, and includes computer graphics and map generation methods.

[0078] This system utilizes advanced information processing technology to optimize travel and sightseeing plans. Based on user information, both pre-trip and on-the-go, it collects external data and uses a generative AI model to provide optimal travel routes and sightseeing plans. Furthermore, by adapting to real-time changes in traffic and weather information, it can always provide users with the most up-to-date and optimal plans.

[0079] Users access this system through devices such as smartphones and PCs. The device provides an interface for users to input pre-entered attribute information such as addresses, preferred locations, and allergies. Furthermore, users can input information such as travel destinations, order of visits, budget, and number of people as needed.

[0080] The server aggregates this information, stores it in a database, and automatically collects external information such as transportation status, weather forecasts, and congestion levels at tourist facilities based on that information. The collected information is fed into a generative AI model, which generates optimized travel routes and sightseeing plans that take multiple variables into account.

[0081] The generated travel routes and sightseeing plans are visually represented through information display devices. Image generation technology on the terminal allows users to intuitively view maps and detailed route information. This enables users to plan and execute their trips efficiently and comfortably.

[0082] In particular, as an example of a prompt, it is possible to input a specific request into the generating AI model, such as "The user will visit a zoo, a restaurant, and a souvenir shop, and create the optimal route based on the budget and number of people." With such input, the system will instantly output a plan that takes various factors into consideration.

[0083] This system enhances user convenience while offering the flexibility to respond immediately to changing conditions, providing a new level of convenience in travel planning.

[0084] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0085] Step 1:

[0086] The user launches the application on their device and enters attribute information such as address, preferred tourist destinations, and desired mode of transportation. This information is temporarily stored on the device. The information entered at this stage includes information previously registered by the user and information that changes each time a trip is taken. The output is all the input information aggregated on the device.

[0087] Step 2:

[0088] The terminal sends the entered information to the server. The server saves the received attribute information and other information to the database. This saving saves the server the trouble of re-entering information the next time it is used. The specific actions performed in this step are data format conversion and writing to the database.

[0089] Step 3:

[0090] The server collects surrounding information from external information provision systems, such as traffic conditions, weather forecasts, and congestion levels of planned destinations, based on the stored information. Inputs are user information obtained from the database and external information via APIs, while output is the collected surrounding information data. Specifically, data is acquired through API communication over the internet.

[0091] Step 4:

[0092] The server inputs the collected information into a generating AI model. Based on the prompt, this model generates optimized travel routes and sightseeing plans that consider multiple variables. The output is an optimized route or plan tailored to the user. The specific data processing here involves analyzing multiple variables and calculating the optimal choice using a predictive model.

[0093] Step 5:

[0094] The server visualizes the generated routes and plans and sends the information to the terminal. This visualization uses image generation technology and includes the process of visually formatting map data and detailed route information. The output is graphical data that can be displayed on the terminal.

[0095] Step 6:

[0096] The terminal presents the user with data received from the server. The user can use the terminal's display to review and adjust the generated plan. The output here is information presented through a user-friendly interface, specifically including touch-based actions for reviewing and reconfiguring the plan.

[0097] Step 7:

[0098] The server monitors traffic and weather information in real time, and if there are any changes, it restarts the AI ​​model based on the new conditions and updates the plan. The input is the latest external information, and the output is the latest travel route and sightseeing plan. In this step, the process of periodically acquiring information and recalculating is carried out as concrete actions.

[0099] (Application Example 1)

[0100] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0101] In today's urban environments, providing optimal routes and plans that meet the diverse needs of users during travel and sightseeing is becoming increasingly complex. This is due to the numerous factors that must be considered, including traffic congestion, weather changes, and individual user preferences and health information. Traditional methods have struggled to comprehensively process this information and provide the best recommendations in real time. Therefore, there is a need for new systems that can solve these problems and enable users to travel more comfortably and efficiently within cities and enjoy sightseeing.

[0102] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0103] In this invention, the server includes data storage means for recording the user's basic information, means for acquiring external information based on the temporary and basic information received from the user, and means for analyzing the acquired external information using a generative model to create an optimized travel route and action plan. This makes it possible to provide the optimal route and plan that integrates the user's preferences with real-time surrounding information.

[0104] "Basic user information" refers to information about the user's address, preferences, and health status.

[0105] "Temporary information" refers to information such as destination, budget, and number of people that users enter each time they travel or move around.

[0106] "External information" refers to data related to users' prior and immediate information, such as traffic conditions, weather forecasts, and facility operating status.

[0107] "Data storage means" refers to databases and storage systems used to store basic user information.

[0108] "Means of acquiring external information" refers to the means by which a server collects necessary information from external data sources based on user usage information.

[0109] "Analysis using generative models" refers to the process of using AI models to generate optimal travel routes and action plans in order to analyze collected data.

[0110] "Travel routes and itinerary plans" refer to route guidance and schedules of destinations designed to enable users to travel efficiently and enjoyably within a city and to sightsee.

[0111] "Information presentation means" refers to interfaces and devices that clearly communicate the generated travel route and action plan to the user.

[0112] This invention is a system for optimizing users' travel and sightseeing experiences. The system mainly consists of a server, a user's terminal, and an information presentation means. A specific embodiment of this system is described below.

[0113] The server first stores the user's basic information in a database. This basic information includes address, preferences, and health information. Based on the temporary information received from the user and this basic information, the server retrieves external information. This process utilizes real-time data such as traffic conditions and weather forecasts.

[0114] The acquired external information is fed into an AI-powered generative model. This generative model is built using programming languages ​​such as Python and utilizes machine learning frameworks like Tensorflow® and PyTorch. This model performs multivariate analysis to generate optimal travel routes and action plans.

[0115] The generated routes and plans are presented to the user through an application installed on their device. The information presentation methods are developed using API frameworks such as Flask and FastAPI, and are available on smartphones. The visualized route maps are provided by an image generation model, ensuring an intuitive and quickly understandable format for the user.

[0116] As a concrete example, consider a scenario where a tourist visits a city on a holiday. The user enters a zoo, restaurant, and museum as planned destinations into their device, and specifies their preferred mode of transportation and budget. Based on this information, the server considers traffic information, weather forecasts, and the congestion levels of tourist spots to generate the optimal order of visits and a detailed schedule. The user receives this information in real time via their smartphone, allowing them to respond flexibly even if their plans change.

[0117] An example of a prompt message might be, "I would like to visit the zoo and a restaurant this afternoon. Please tell me the best route and consider the weather." Such features allow users to have a comfortable and efficient urban experience.

[0118] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0119] Step 1:

[0120] The user enters basic information through a terminal. This information includes address, preferences, and health information. This information is stored in a database through a data storage system. The information entered here is the user's basic information, and the database organizes and stores it for each user.

[0121] Step 2:

[0122] Users enter temporary information on their device each time they travel or move around. This temporary information includes destination, budget, and number of people. This information is sent to the server and combined with the basic information stored in the previous step. This allows the server to identify the necessary external information.

[0123] Step 3:

[0124] The server retrieves external information such as traffic and weather based on the received temporary and basic information. It uses external APIs to collect real-time traffic and weather data and feeds this into a generative model for data analysis.

[0125] Step 4:

[0126] The server analyzes the collected external information using a generative AI model and converts the input information into the optimal travel route and action plan. This generative model considers multiple variables and selects the best option from among several routes and plans.

[0127] Step 5:

[0128] The travel route and action plan generated from the server are presented to the user through the terminal's information display mechanism. The terminal uses an image generation model to display the route and plan in a visually easy-to-understand format, allowing the user to grasp the entire plan at a glance.

[0129] Step 6:

[0130] The user selects a plan from the presented options and begins their journey. If events such as changes in traffic conditions or weather occur along the way, the server retrieves external information again, recalculates the optimal route, and sends it to the user's device. This allows the user to flexibly adapt to unexpected changes.

[0131] Step 7:

[0132] Throughout the process, users can request more detailed information or additional steps at any point by utilizing prompts tailored to their plan and route. By entering prompts such as "How long does it take to walk from my current location to the nearest station?", the system quickly generates a response.

[0133] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0134] This invention is a system for optimizing travel and sightseeing plans into a more comfortable and personalized experience by incorporating an emotion engine that recognizes the user's emotions. Specific embodiments are described below.

[0135] First, the user accesses the application through their device and enters preliminary information. This information includes the user's basic hobbies and preferences, mode of transportation, allergy information, etc., and is stored in a database on the server. Based on this preliminary information, personalized planning becomes possible.

[0136] When planning a trip or journey, users input information such as destination, order of visits, budget, and number of people into their device. The device sends this information to a server, which then collects information from external sources, such as traffic conditions, weather forecasts, and facility congestion levels. This enables the generation of plans based on the latest and most optimal information.

[0137] A key feature here is that the application implements an emotion engine. The server receives the user's emotional state and feedback obtained from the user's device via the emotion engine. This emotion engine can analyze the user's current emotions through facial recognition technology and voice analysis.

[0138] Based on data from the emotion engine, the server generates optimized travel routes and itinerary plans tailored to the user's preferences. This optimization process includes adjustments to reduce user stress and anxiety, allowing for changes in the order of destinations and the inclusion of additional relaxation points.

[0139] The generated plan is presented to the user in a visually easy-to-understand format using information display methods. For example, it can be displayed on a map or graphic on the device, utilizing an image generation model.

[0140] Furthermore, the server continuously monitors real-time operational and weather information, immediately recalculating routes and plans if any changes occur, and providing updated information to the terminal. This ensures that users always receive the latest information that reflects new circumstances.

[0141] For example, if a user wants to visit a zoo and a theme park on the weekend, the emotional engine starts from their morning state and predicts the order in which they should visit to maximize their satisfaction. For instance, it prioritizes rides when the user is calm and inserts restful events when they are tired, thereby maximizing the user's experience. In this way, the system of the present invention provides an innovative and personalized travel and sightseeing experience by adapting to the emotional state of each individual user.

[0142] The following describes the processing flow.

[0143] Step 1:

[0144] Users access the application via their device and enter preliminary information such as address, hobbies, health information, and mode of transportation as profile information. The entered information is used as basic data necessary for generating personalized plans.

[0145] Step 2:

[0146] The terminal sends all entered pre-information to the server. The information is encrypted before transmission and securely stored in a database.

[0147] Step 3:

[0148] When planning a trip or travel, users register information such as destination, order of visits, budget, and number of people on their device each time. This enables real-time planning.

[0149] Step 4:

[0150] The device sends information to the server each time. Simultaneously, it starts analyzing the user's emotional state using an emotion engine to obtain the user's emotional state at that moment.

[0151] Step 5:

[0152] The server receives information as it goes, while simultaneously collecting external data such as traffic conditions, weather forecasts, and facility congestion levels. This makes it possible to generate realistic plans that take all these factors into account.

[0153] Step 6:

[0154] The emotion engine analyzes the user's emotions from their facial expressions and voice, and sends the results to the server. This is done to make adjustments based on the user's preferences and state.

[0155] Step 7:

[0156] The server considers the received emotional data and generates optimized travel routes and plans using a generative model. At this stage, the plan is adjusted according to the user's emotional state.

[0157] Step 8:

[0158] The server visually represents the generated route and plan and transmits it to the terminal via a display device. By using an image generation model, intuitive plan displays using maps and graphics are possible.

[0159] Step 9:

[0160] Users review the plan presented on their device and enter any additional requests or feedback as needed. Further adjustments to the plan may then be made based on this feedback.

[0161] Step 10:

[0162] During your trip, the server monitors real-time operational and weather information, and if any changes are detected, it immediately regenerates the plan and notifies your device of the latest information. This feature ensures that users always have the most up-to-date and optimal plan.

[0163] This process allows users to obtain personalized travel and sightseeing plans that also take their emotions into consideration.

[0164] (Example 2)

[0165] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0166] In recent years, there has been a growing demand for personalized travel and sightseeing experiences, but adjusting plans based on users' emotional states is not easy. Traditional systems are inadequate in considering temporary changes in users' emotions, and improvements in usability are needed. Furthermore, there is a challenge in handling dynamic plan changes based on real-time information.

[0167] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0168] In this invention, the server includes a storage means for recording the user's prior information, a means for acquiring external information based on the information received from the user each time and the prior information, a means for analyzing the acquired external information using a generation model and generating an optimized travel route and itinerary plan, and an emotion analysis means for analyzing the user's emotional state and adjusting the itinerary plan based on the emotional state. This enables flexible planning according to the user's current emotional state, allowing for a better provision of a personalized experience.

[0169] "Users" refer to individuals or groups who use the system and are the entities responsible for planning travel or sightseeing.

[0170] "Prior information" refers to data such as basic personal information, hobbies, preferences, means of transportation, and allergy information that users provide before using the system.

[0171] "On-demand information" refers to information that users provide as needed when traveling or moving around, such as destination, order of visits, budget, and number of people.

[0172] "External information" refers to real-time data obtained from external sources, such as traffic conditions, weather forecasts, and facility congestion levels, which are collected by the system.

[0173] A "generative model" is a computer program that analyzes external information to formulate optimized travel routes and itinerary plans.

[0174] "Travel route" refers to the path or route used by a user when traveling.

[0175] "Itinerary plan" refers to the overall schedule, including the places and events that the user will visit.

[0176] "Emotional analysis methods" refer to technologies that analyze a user's emotional state from their facial expressions and voice, and use that information to adjust the plan.

[0177] "Display means" refers to a device or technology for visually presenting the generated routes and plans.

[0178] The system according to the present invention aims to highly personalize and optimize the user's travel and mobility experience in real time. Specific embodiments thereof are described below.

[0179] First, the user accesses the application using their device and enters preliminary information such as hobbies, preferences, mode of transportation, and allergy information. This information helps in creating an initial plan tailored to the user's preferences.

[0180] The information entered by the terminal is sent to the server, which stores it in a database. This information is then used to generate customized plans for each user.

[0181] When planning a trip or travel, users input information such as destination, order of visits, budget, and number of people via their device. The device then sends this information to a server.

[0182] The server collects external information from external sources, such as traffic conditions, weather forecasts, and facility congestion levels. This allows for planning that reflects the latest conditions.

[0183] One of the features of this system is its emotion analysis method, which analyzes the user's emotional state from their device using facial recognition technology and voice analysis.

[0184] The server analyzes the collected emotional data and external information using a generating AI model to create travel routes and itinerary plans based on the user's emotional state. This process incorporates plans that, for example, provide active activities when the user's emotions are stable, and rest when they are tired.

[0185] The generated plan is visualized by an image generation model and presented to the user on the device. This allows the user to intuitively understand and follow the plan.

[0186] Furthermore, the server monitors operational and weather information in real time and recalculates the plan as needed. This ensures that users can have the optimal experience even in fluctuating environments.

[0187] For example, if a user plans a visit to a zoo and a theme park, sentiment analysis can provide a plan that includes active rides in the morning and relaxing rest stops in the afternoon.

[0188] An example of a prompt message for a generative AI model would be, "Adjust the travel plan according to the user's emotions to provide a highly satisfying experience."

[0189] As described above, this system can take the user's emotional state into consideration and provide the optimal travel route and itinerary plan in real time.

[0190] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0191] Step 1:

[0192] Users access the application through their device and enter preliminary information. This information includes hobbies, preferences, modes of transportation, and allergy information. The device sends this information to the server. As a result, the server stores the user's basic information and provides data that helps create personalized plans.

[0193] Step 2:

[0194] When users plan trips or travel, they input information such as destination, order of visits, budget, and number of people into their device each time. The device sends this input information to the server. Based on this information, the server retrieves external information and collects basic data for plan generation.

[0195] Step 3:

[0196] Based on the collected data, the server gathers information from external sources, such as traffic conditions, weather forecasts, and facility congestion levels. The server analyzes the acquired data and prepares a plan to provide users with the best possible choices in real time.

[0197] Step 4:

[0198] The server uses emotion analysis tools to analyze the user's emotional state based on voice data and facial images obtained from the user's device. The results of this analysis quantify the user's emotional state and are output as data that influences the planning process.

[0199] Step 5:

[0200] The server uses a generative AI model to integrate user prior information, on-the-spot information, external information, and emotional data to generate optimized travel routes and itinerary plans. The output plans include activity and rest plans that take emotional states into account, aiming to improve usability.

[0201] Step 6:

[0202] The generated plan is visually represented on the device and provided to the user. Using an image generation model, maps and schedules are displayed graphically, allowing users to review the plan in an intuitive and easy-to-understand manner.

[0203] Step 7:

[0204] The server continuously monitors operational and weather information and recalculates the plan as needed. If there are any changes, the server sends the updated plan to the terminal, allowing users to always act based on the latest information.

[0205] (Application Example 2)

[0206] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0207] This invention aims to enable individual citizens and tourists to enjoy more comfortable and stress-free travel and sightseeing experiences in urban environments. Conventional systems do not take emotional states into consideration and only provide uniform information, making it difficult to propose flexible plans tailored to individual needs. Therefore, it is necessary to construct a system that takes users' emotions into account and provides dynamically optimized travel routes and itinerary suggestions.

[0208] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0209] In this invention, the server includes a data storage device for storing user information in advance, means for collecting external information based on information received from the user each time and the prior information, means for analyzing the collected external information using an analysis model and generating optimized travel routes and itinerary plans, means for recognizing the user's emotional state and dynamically adjusting the travel routes and itinerary plans according to the emotional state, and an information display method for presenting the generated routes and plans to the user. This makes it possible to provide a highly personalized and comfortable travel and sightseeing experience that takes the user's emotional state into consideration.

[0210] A "data storage device" is a device for storing users' prior information and has the function of accessing and providing information as needed.

[0211] "External information" refers to data necessary for generating travel routes and itineraries, including information such as traffic conditions, weather, and facility congestion levels.

[0212] An "analysis model" is an algorithm or computational method used to analyze collected external information and generate optimized travel routes and itineraries.

[0213] A "travel route" is a design that shows the optimal path a user takes to travel from one point to their destination.

[0214] A "planned itinerary" is a plan for a user to visit a specific destination, and includes the order of visits and the time allocation.

[0215] "Emotional state" refers to the user's psychological state, and includes conditions such as stress, anxiety, and excitement.

[0216] "Dynamic adjustment" means automatically changing the travel route and itinerary in response to changes in the user's emotional state or external information.

[0217] "Information display method" refers to a means of visually presenting the generated travel route and itinerary to the user, and includes maps and graphic interfaces.

[0218] The system for realizing this invention is configured as follows: The system uses the user's smartphone as a terminal, and a dedicated application is installed on that terminal. This application stores the user's prior information in a data storage device and collects external information through a data collection means based on the information entered by the user each time. The external information includes traffic conditions, weather information, and the level of congestion at facilities.

[0219] The server analyzes external information obtained from data collection methods using an analytical model. This analytical model utilizes machine learning frameworks such as TensorFlow and PyTorch to generate optimized travel routes and itineraries for the user. Furthermore, the terminal recognizes the user's emotional state via its camera and microphone and transmits this information to the server. The server takes this emotional state into account and dynamically adjusts the generated route and itinerary.

[0220] The generated travel routes and itineraries are presented to the user through an information display method. Map services such as Google® Maps and Mapbox are used for information display, providing a visually intuitive interface.

[0221] As a concrete example, if a family is planning a visit to a theme park and museum in a city, we propose a plan that prioritizes fun attractions that cater to the emotionally excited children, while also incorporating rest stops at appropriate times for the stressed-out parents.

[0222] An example of a prompt message could include specific usage requirements such as, "Based on the user's current emotional state and external information, generate and visually present the optimal travel route and sightseeing plan in real time." In this way, the objective of the present invention is to provide a personalized and comfortable travel and sightseeing experience that meets the needs of individual users.

[0223] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0224] Step 1:

[0225] The user enters preliminary information using a terminal. This information includes hobbies, preferences, modes of transportation, and allergy information. This information is stored in a data storage device and used as basic data for the server to generate the optimal plan for the user.

[0226] Step 2:

[0227] The user inputs information such as the destinations they wish to visit, the order of visits, and their budget into a terminal each time. The terminal sends this information to a server, which uses external information gathering methods to collect real-time data such as traffic conditions, weather information, and congestion levels. The server then uses this input information to collect external information and uses it as input for data analysis.

[0228] Step 3:

[0229] The server analyzes collected external information based on an analytical model. During this process, a generative AI model is used to generate optimized travel routes and itinerary suggestions. The analysis also incorporates user information, both prior and ongoing, to determine the most efficient and user-friendly plan.

[0230] Step 4:

[0231] The user's device uses its camera and microphone to recognize their emotional state in real time and transmit it to the server. The server receives this emotional data and dynamically adjusts the travel route and itinerary based on the analysis results. For example, if the user is feeling stressed, a relaxing spot will be incorporated into the plan.

[0232] Step 5:

[0233] The travel route and itinerary generated on the server are sent to the terminal and presented to the user using an information display method. Here, a clear and visually represented plan is provided to the user using map services such as Google Maps or Mapbox, allowing the user to travel by following the directions.

[0234] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0235] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0236] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0237] [Second Embodiment]

[0238] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0239] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0240] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0241] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0242] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0243] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0244] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0245] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0246] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0247] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0248] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0249] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0250] This invention is a system for providing users with optimal travel routes and sightseeing plans by integrating user information from prior to use and information gathered and analyzed from external sources. Specific embodiments of this system are described below.

[0251] First, the user accesses the application through their device and enters their personal information. This information includes their address, favorite places to visit, preferred mode of transportation, health information such as allergies, and items they need to bring with them when traveling. This information is stored in a database by the server so that it does not need to be re-entered each time the service is used.

[0252] Next, when planning a trip or travel, users input information such as destination, order of visits, budget, and number of people into their device each time. This information is sent to a server and collected along with other external data (for example, traffic conditions, weather forecasts, facility congestion, etc.).

[0253] The server feeds the collected information into a generative model to generate optimized travel routes and sightseeing plans. This generative model takes multiple variables into account and optimizes the routes and plans to match the user's desired conditions. The plans are then visually presented to the user via an information display system. This presentation uses maps and graphic formats based on an image generative model, allowing the user to grasp the details of the routes and plans at a glance.

[0254] Furthermore, the server monitors traffic conditions and weather changes in real time, and regenerates routes and plans based on the new conditions if any changes occur. This functionality allows users to flexibly adapt to fluctuating external environments.

[0255] As an example of how this system works, consider a scenario where a user plans a weekend trip to a tourist destination. The user inputs destinations such as a zoo, restaurant, and souvenir shop into the application, and specifies their budget and the number of people. Based on these factors, the server calculates a plan that takes into account transportation schedules, weather forecasts, and the operating status of the destinations, and generates a visual route map. This allows the user to enjoy sightseeing with the most efficient and comfortable schedule.

[0256] Thus, the present invention provides an excellent mobility and tourism support system that improves the user experience and can quickly respond to changing circumstances.

[0257] The following describes the processing flow.

[0258] Step 1:

[0259] Users enter their personal information through their device. This information includes their address, favorite spots, preferred mode of transportation, and allergy information, which are all included in the application's initial setup screen.

[0260] Step 2:

[0261] The terminal sends the pre-entered information to the server. At this time, the information is encrypted using a secure communication protocol and transmitted safely.

[0262] Step 3:

[0263] The server stores the received preliminary information in a database. This stored information will be used in future plan generation to improve convenience.

[0264] Step 4:

[0265] When users plan trips or travel, they input information such as destination, order of visits, budget, and number of people into their device each time. These input fields are specific to the current plan.

[0266] Step 5:

[0267] The device transmits information to the server in real time as it moves. This includes the user's current location, which is used to generate accurate plans.

[0268] Step 6:

[0269] The server collects information as it goes along, along with external data from external sources such as traffic conditions, weather forecasts, and congestion levels at visited facilities.

[0270] Step 7:

[0271] The server uses a generative model to generate optimized travel routes and itinerary plans based on the collected data. The generative model utilizes AI algorithms to efficiently process multiple variables.

[0272] Step 8:

[0273] The server visually represents the generated plan using an image generation model and converts it into a format that is easy for the user to understand, such as maps and graphics.

[0274] Step 9:

[0275] The server sends visualized routes and plans to the terminal, and the user uses this information to confirm their travel and sightseeing schedule.

[0276] Step 10:

[0277] The server monitors traffic and weather information in real time. If there are any changes, it regenerates routes and plans based on the new information and immediately sends updated information to the terminal.

[0278] This process allows users to quickly obtain the optimal plan based on the information they enter in advance and each time.

[0279] (Example 1)

[0280] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0281] In modern travel and transportation, users need to constantly adapt to changing external conditions, such as traffic congestion, weather changes, and facility occupancy. However, many current systems cannot respond to these changes in real time, making it difficult to provide users with the best possible plan. Furthermore, customization to individual user preferences and conditions is insufficient, and improvements in usability are needed.

[0282] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0283] In this invention, the server includes an information recording medium for storing user attribute information, means for obtaining surrounding information based on the information received from the user each time and the attribute information, means for inputting the obtained surrounding information into a generated AI model to create an optimized travel route and tourism plan, and means for updating the surrounding information in real time and regenerating the route and plan according to changes. Thereby, it becomes possible for the user to quickly adapt to environmental changes and enjoy an optimal travel and tourism plan that suits their individual preferences and conditions.

[0284] The "user attribute information" is personal information related to the user, and refers to information necessary for creating individual travel and tourism plans, such as address, preferred base, health precautions, etc.

[0285] The "information recording medium" is a medium for storing information in digital or analog form, and includes databases, cloud storage, etc.

[0286] The "information each time" is the information provided by the user on the spot when planning a specific travel or trip, and refers to variable conditions such as destination, order of visits, budget, number of people, etc.

[0287] The "surrounding information" is dynamic information obtained from the outside, and refers to information that changes in real time, such as traffic operation status, weather information, facility congestion, etc.

[0288] The "generated AI model" is part of artificial intelligence technology for analyzing the input data and automatically creating an optimal travel route and tourism plan for the user.

[0289] The "travel route" is information indicating the path when the user moves from the departure point to the destination, and includes an efficient and safe route.

[0290] The "tourism plan" is a travel and tourism schedule organized based on the user's interests and conditions, and includes elements such as visit destinations, schedules, costs, etc.

[0291] A "display device" is a device that provides visual information to a user, and includes smartphones, tablets, and computer displays.

[0292] "Image generation technology" refers to techniques for visually displaying planned routes and schedules in an easy-to-understand manner, and includes computer graphics and map generation methods.

[0293] This system utilizes advanced information processing technology to optimize travel and sightseeing plans. Based on user information, both pre-trip and on-the-go, it collects external data and uses a generative AI model to provide optimal travel routes and sightseeing plans. Furthermore, by adapting to real-time changes in traffic and weather information, it can always provide users with the most up-to-date and optimal plans.

[0294] Users access this system through devices such as smartphones and PCs. The device provides an interface for users to input pre-entered attribute information such as addresses, preferred locations, and allergies. Furthermore, users can input information such as travel destinations, order of visits, budget, and number of people as needed.

[0295] The server aggregates this information, stores it in a database, and automatically collects external information such as transportation status, weather forecasts, and congestion levels at tourist facilities based on that information. The collected information is fed into a generative AI model, which generates optimized travel routes and sightseeing plans that take multiple variables into account.

[0296] The generated travel routes and sightseeing plans are visually represented through information display devices. Image generation technology on the terminal allows users to intuitively view maps and detailed route information. This enables users to plan and execute their trips efficiently and comfortably.

[0297] In particular, as an example of a prompt, it is possible to input a specific request into the generating AI model, such as "The user will visit a zoo, a restaurant, and a souvenir shop, and create the optimal route based on the budget and number of people." With such input, the system will instantly output a plan that takes various factors into consideration.

[0298] This system enhances user convenience while offering the flexibility to respond immediately to changing conditions, providing a new level of convenience in travel planning.

[0299] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0300] Step 1:

[0301] The user launches the application on their device and enters attribute information such as address, preferred tourist destinations, and desired mode of transportation. This information is temporarily stored on the device. The information entered at this stage includes information previously registered by the user and information that changes each time a trip is taken. The output is all the input information aggregated on the device.

[0302] Step 2:

[0303] The terminal sends the entered information to the server. The server saves the received attribute information and other information to the database. This saving saves the server the trouble of re-entering information the next time it is used. The specific actions performed in this step are data format conversion and writing to the database.

[0304] Step 3:

[0305] The server collects surrounding information from external information provision systems, such as traffic conditions, weather forecasts, and congestion levels of planned destinations, based on the stored information. Inputs are user information obtained from the database and external information via APIs, while output is the collected surrounding information data. Specifically, data is acquired through API communication over the internet.

[0306] Step 4:

[0307] The server inputs the collected information into the generated AI model. Based on the prompt text, this model generates an optimized travel route and tourism plan considering multiple variables. The output is an optimized route and plan suitable for the user. The specific data processing here is to analyze multiple variables and calculate the optimal options by the prediction model.

[0308] Step 5:

[0309] The server visualizes the generated route and plan and transmits the information to the terminal. Image generation technology is used for this visualization, which includes the operation of visually formatting map data and detailed route information. The output is graphical data that can be displayed on the terminal side.

[0310] Step 6:

[0311] The terminal presents the data received from the server to the user. The user can use the display of the terminal to view and adjust the generated plan. The output here is the presentation of information through a highly convenient user interface. Specifically, it includes the operations of viewing and resetting the plan by touch operation.

[0312] Step 7:

[0313] The server monitors traffic information, weather information, etc. in real time. If there are changes, it starts the generated AI model again based on the new conditions and updates the plan. The input is the latest external information, and the output is the latest travel route and tourism plan. In this step, the periodic execution of information acquisition and the process of recalculation are carried out as specific operations.

[0314] (Application Example 1)

[0315] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0316] In today's urban environments, providing optimal routes and plans that meet the diverse needs of users during travel and sightseeing is becoming increasingly complex. This is due to the numerous factors that must be considered, including traffic congestion, weather changes, and individual user preferences and health information. Traditional methods have struggled to comprehensively process this information and provide the best recommendations in real time. Therefore, there is a need for new systems that can solve these problems and enable users to travel more comfortably and efficiently within cities and enjoy sightseeing.

[0317] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0318] In this invention, the server includes data storage means for recording the user's basic information, means for acquiring external information based on the temporary and basic information received from the user, and means for analyzing the acquired external information using a generative model to create an optimized travel route and action plan. This makes it possible to provide the optimal route and plan that integrates the user's preferences with real-time surrounding information.

[0319] "Basic user information" refers to information about the user's address, preferences, and health status.

[0320] "Temporary information" refers to information such as destination, budget, and number of people that users enter each time they travel or move around.

[0321] "External information" refers to data related to users' prior and immediate information, such as traffic conditions, weather forecasts, and facility operating status.

[0322] "Data storage means" refers to databases and storage systems used to store basic user information.

[0323] "Means of acquiring external information" refers to the means by which a server collects necessary information from external data sources based on user usage information.

[0324] "Analysis using generative models" refers to the process of using AI models to generate optimal travel routes and action plans in order to analyze collected data.

[0325] "Travel routes and itinerary plans" refer to route guidance and schedules of destinations designed to enable users to travel efficiently and enjoyably within a city and to sightsee.

[0326] "Information presentation means" refers to interfaces and devices that clearly communicate the generated travel route and action plan to the user.

[0327] This invention is a system for optimizing users' travel and sightseeing experiences. The system mainly consists of a server, a user's terminal, and an information presentation means. A specific embodiment of this system is described below.

[0328] The server first stores the user's basic information in a database. This basic information includes address, preferences, and health information. Based on the temporary information received from the user and this basic information, the server retrieves external information. This process utilizes real-time data such as traffic conditions and weather forecasts.

[0329] The acquired external information is fed into an AI-powered generative model. This generative model is built using programming languages ​​such as Python and utilizes machine learning frameworks like TensorFlow and PyTorch. This model performs multivariate analysis to generate optimal travel routes and action plans.

[0330] The generated routes and plans are presented to the user through an application installed on their device. The information presentation methods are developed using API frameworks such as Flask and FastAPI, and are available on smartphones. The visualized route maps are provided by an image generation model, ensuring an intuitive and quickly understandable format for the user.

[0331] As a concrete example, consider a scenario where a tourist visits a city on a holiday. The user enters a zoo, restaurant, and museum as planned destinations into their device, and specifies their preferred mode of transportation and budget. Based on this information, the server considers traffic information, weather forecasts, and the congestion levels of tourist spots to generate the optimal order of visits and a detailed schedule. The user receives this information in real time via their smartphone, allowing them to respond flexibly even if their plans change.

[0332] An example of a prompt message might be, "I would like to visit the zoo and a restaurant this afternoon. Please tell me the best route and consider the weather." Such features allow users to have a comfortable and efficient urban experience.

[0333] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0334] Step 1:

[0335] The user enters basic information through a terminal. This information includes address, preferences, and health information. This information is stored in a database through a data storage system. The information entered here is the user's basic information, and the database organizes and stores it for each user.

[0336] Step 2:

[0337] Users enter temporary information on their device each time they travel or move around. This temporary information includes destination, budget, and number of people. This information is sent to the server and combined with the basic information stored in the previous step. This allows the server to identify the necessary external information.

[0338] Step 3:

[0339] The server retrieves external information such as traffic and weather based on the received temporary and basic information. It uses external APIs to collect real-time traffic and weather data and feeds this into a generative model for data analysis.

[0340] Step 4:

[0341] The server analyzes the collected external information using a generative AI model and converts the input information into the optimal travel route and action plan. This generative model considers multiple variables and selects the best option from among several routes and plans.

[0342] Step 5:

[0343] The travel route and action plan generated from the server are presented to the user through the terminal's information display mechanism. The terminal uses an image generation model to display the route and plan in a visually easy-to-understand format, allowing the user to grasp the entire plan at a glance.

[0344] Step 6:

[0345] The user selects a plan from the presented options and begins their journey. If events such as changes in traffic conditions or weather occur along the way, the server retrieves external information again, recalculates the optimal route, and sends it to the user's device. This allows the user to flexibly adapt to unexpected changes.

[0346] Step 7:

[0347] Throughout the process, users can request more detailed information or additional steps at any point by utilizing prompts tailored to their plan and route. By entering prompts such as "How long does it take to walk from my current location to the nearest station?", the system quickly generates a response.

[0348] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0349] This invention is a system for optimizing travel and sightseeing plans into a more comfortable and personalized experience by incorporating an emotion engine that recognizes the user's emotions. Specific embodiments are described below.

[0350] First, the user accesses the application through their device and enters preliminary information. This information includes the user's basic hobbies and preferences, mode of transportation, allergy information, etc., and is stored in a database on the server. Based on this preliminary information, personalized planning becomes possible.

[0351] When planning a trip or journey, users input information such as destination, order of visits, budget, and number of people into their device. The device sends this information to a server, which then collects information from external sources, such as traffic conditions, weather forecasts, and facility congestion levels. This enables the generation of plans based on the latest and most optimal information.

[0352] A key feature here is that the application implements an emotion engine. The server receives the user's emotional state and feedback obtained from the user's device via the emotion engine. This emotion engine can analyze the user's current emotions through facial recognition technology and voice analysis.

[0353] Based on data from the emotion engine, the server generates optimized travel routes and itinerary plans tailored to the user's preferences. This optimization process includes adjustments to reduce user stress and anxiety, allowing for changes in the order of destinations and the inclusion of additional relaxation points.

[0354] The generated plan is presented to the user in a visually easy-to-understand format using information display methods. For example, it can be displayed on a map or graphic on the device, utilizing an image generation model.

[0355] Furthermore, the server continuously monitors real-time operational and weather information, immediately recalculating routes and plans if any changes occur, and providing updated information to the terminal. This ensures that users always receive the latest information that reflects new circumstances.

[0356] For example, if a user wants to visit a zoo and a theme park on the weekend, the emotional engine starts from their morning state and predicts the order in which they should visit to maximize their satisfaction. For instance, it prioritizes rides when the user is calm and inserts restful events when they are tired, thereby maximizing the user's experience. In this way, the system of the present invention provides an innovative and personalized travel and sightseeing experience by adapting to the emotional state of each individual user.

[0357] The following describes the processing flow.

[0358] Step 1:

[0359] Users access the application via their device and enter preliminary information such as address, hobbies, health information, and mode of transportation as profile information. The entered information is used as basic data necessary for generating personalized plans.

[0360] Step 2:

[0361] The terminal sends all entered pre-information to the server. The information is encrypted before transmission and securely stored in a database.

[0362] Step 3:

[0363] When planning a trip or travel, users register information such as destination, order of visits, budget, and number of people on their device each time. This enables real-time planning.

[0364] Step 4:

[0365] The device sends information to the server each time. Simultaneously, it starts analyzing the user's emotional state using an emotion engine to obtain the user's emotional state at that moment.

[0366] Step 5:

[0367] The server receives information as it goes, while simultaneously collecting external data such as traffic conditions, weather forecasts, and facility congestion levels. This makes it possible to generate realistic plans that take all these factors into account.

[0368] Step 6:

[0369] The emotion engine analyzes the user's emotions from their facial expressions and voice, and sends the results to the server. This is done to make adjustments based on the user's preferences and state.

[0370] Step 7:

[0371] The server considers the received emotional data and generates optimized travel routes and plans using a generative model. At this stage, the plan is adjusted according to the user's emotional state.

[0372] Step 8:

[0373] The server visually represents the generated route and plan and transmits it to the terminal via a display device. By using an image generation model, intuitive plan displays using maps and graphics are possible.

[0374] Step 9:

[0375] Users review the plan presented on their device and enter any additional requests or feedback as needed. Further adjustments to the plan may then be made based on this feedback.

[0376] Step 10:

[0377] During your trip, the server monitors real-time operational and weather information, and if any changes are detected, it immediately regenerates the plan and notifies your device of the latest information. This feature ensures that users always have the most up-to-date and optimal plan.

[0378] This process allows users to obtain personalized travel and sightseeing plans that also take their emotions into consideration.

[0379] (Example 2)

[0380] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0381] In recent years, there has been a growing demand for personalized travel and sightseeing experiences, but adjusting plans based on users' emotional states is not easy. Traditional systems are inadequate in considering temporary changes in users' emotions, and improvements in usability are needed. Furthermore, there is a challenge in handling dynamic plan changes based on real-time information.

[0382] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0383] In this invention, the server includes a storage means for recording the user's prior information, a means for acquiring external information based on the information received from the user each time and the prior information, a means for analyzing the acquired external information using a generation model and generating an optimized travel route and itinerary plan, and an emotion analysis means for analyzing the user's emotional state and adjusting the itinerary plan based on the emotional state. This enables flexible planning according to the user's current emotional state, allowing for a better provision of a personalized experience.

[0384] "Users" refer to individuals or groups who use the system and are the entities responsible for planning travel or sightseeing.

[0385] "Prior information" refers to data such as basic personal information, hobbies, preferences, means of transportation, and allergy information that users provide before using the system.

[0386] "On-demand information" refers to information that users provide as needed when traveling or moving around, such as destination, order of visits, budget, and number of people.

[0387] "External information" refers to real-time data obtained from external sources, such as traffic conditions, weather forecasts, and facility congestion levels, which are collected by the system.

[0388] A "generative model" is a computer program that analyzes external information to formulate optimized travel routes and itinerary plans.

[0389] "Travel route" refers to the path or route used by a user when traveling.

[0390] "Itinerary plan" refers to the overall schedule, including the places and events that the user will visit.

[0391] "Emotional analysis methods" refer to technologies that analyze a user's emotional state from their facial expressions and voice, and use that information to adjust the plan.

[0392] "Display means" refers to a device or technology for visually presenting the generated routes and plans.

[0393] The system according to the present invention aims to highly personalize and optimize the user's travel and mobility experience in real time. Specific embodiments thereof are described below.

[0394] First, the user accesses the application using their device and enters preliminary information such as hobbies, preferences, mode of transportation, and allergy information. This information helps in creating an initial plan tailored to the user's preferences.

[0395] The information entered by the terminal is sent to the server, which stores it in a database. This information is then used to generate customized plans for each user.

[0396] When planning a trip or travel, users input information such as destination, order of visits, budget, and number of people via their device. The device then sends this information to a server.

[0397] The server collects external information from external sources, such as traffic conditions, weather forecasts, and facility congestion levels. This allows for planning that reflects the latest conditions.

[0398] One of the features of this system is its emotion analysis method, which analyzes the user's emotional state from their device using facial recognition technology and voice analysis.

[0399] The server analyzes the collected emotional data and external information using a generating AI model to create travel routes and itinerary plans based on the user's emotional state. This process incorporates plans that, for example, provide active activities when the user's emotions are stable, and rest when they are tired.

[0400] The generated plan is visualized by an image generation model and presented to the user on the device. This allows the user to intuitively understand and follow the plan.

[0401] Furthermore, the server monitors operational and weather information in real time and recalculates the plan as needed. This ensures that users can have the optimal experience even in fluctuating environments.

[0402] For example, if a user plans a visit to a zoo and a theme park, sentiment analysis can provide a plan that includes active rides in the morning and relaxing rest stops in the afternoon.

[0403] An example of a prompt message for a generative AI model would be, "Adjust the travel plan according to the user's emotions to provide a highly satisfying experience."

[0404] As described above, this system can take the user's emotional state into consideration and provide the optimal travel route and itinerary plan in real time.

[0405] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0406] Step 1:

[0407] Users access the application through their device and enter preliminary information. This information includes hobbies, preferences, modes of transportation, and allergy information. The device sends this information to the server. As a result, the server stores the user's basic information and provides data that helps create personalized plans.

[0408] Step 2:

[0409] When users plan trips or travel, they input information such as destination, order of visits, budget, and number of people into their device each time. The device sends this input information to the server. Based on this information, the server retrieves external information and collects basic data for plan generation.

[0410] Step 3:

[0411] Based on the collected data, the server gathers information from external sources, such as traffic conditions, weather forecasts, and facility congestion levels. The server analyzes the acquired data and prepares a plan to provide users with the best possible choices in real time.

[0412] Step 4:

[0413] The server uses emotion analysis tools to analyze the user's emotional state based on voice data and facial images obtained from the user's device. The results of this analysis quantify the user's emotional state and are output as data that influences the planning process.

[0414] Step 5:

[0415] The server uses a generative AI model to integrate user prior information, on-the-spot information, external information, and emotional data to generate optimized travel routes and itinerary plans. The output plans include activity and rest plans that take emotional states into account, aiming to improve usability.

[0416] Step 6:

[0417] The generated plan is visually represented on the device and provided to the user. Using an image generation model, maps and schedules are displayed graphically, allowing users to review the plan in an intuitive and easy-to-understand manner.

[0418] Step 7:

[0419] The server continuously monitors operational and weather information and recalculates the plan as needed. If there are any changes, the server sends the updated plan to the terminal, allowing users to always act based on the latest information.

[0420] (Application Example 2)

[0421] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0422] This invention aims to enable individual citizens and tourists to enjoy more comfortable and stress-free travel and sightseeing experiences in urban environments. Conventional systems do not take emotional states into consideration and only provide uniform information, making it difficult to propose flexible plans tailored to individual needs. Therefore, it is necessary to construct a system that takes users' emotions into account and provides dynamically optimized travel routes and itinerary suggestions.

[0423] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0424] In this invention, the server includes a data storage device for storing user information in advance, means for collecting external information based on information received from the user each time and the prior information, means for analyzing the collected external information using an analysis model and generating optimized travel routes and itinerary plans, means for recognizing the user's emotional state and dynamically adjusting the travel routes and itinerary plans according to the emotional state, and an information display method for presenting the generated routes and plans to the user. This makes it possible to provide a highly personalized and comfortable travel and sightseeing experience that takes the user's emotional state into consideration.

[0425] A "data storage device" is a device for storing users' prior information and has the function of accessing and providing information as needed.

[0426] "External information" refers to data necessary for generating travel routes and itineraries, including information such as traffic conditions, weather, and facility congestion levels.

[0427] An "analysis model" is an algorithm or computational method used to analyze collected external information and generate optimized travel routes and itineraries.

[0428] A "travel route" is a design that shows the optimal path a user takes to travel from one point to their destination.

[0429] A "planned itinerary" is a plan for a user to visit a specific destination, and includes the order of visits and the time allocation.

[0430] "Emotional state" refers to the user's psychological state, and includes conditions such as stress, anxiety, and excitement.

[0431] "Dynamic adjustment" means automatically changing the travel route and itinerary in response to changes in the user's emotional state or external information.

[0432] "Information display method" refers to a means of visually presenting the generated travel route and itinerary to the user, and includes maps and graphic interfaces.

[0433] The system for realizing this invention is configured as follows: The system uses the user's smartphone as a terminal, and a dedicated application is installed on that terminal. This application stores the user's prior information in a data storage device and collects external information through a data collection means based on the information entered by the user each time. The external information includes traffic conditions, weather information, and the level of congestion at facilities.

[0434] The server analyzes external information obtained from data collection methods using an analytical model. This analytical model utilizes machine learning frameworks such as TensorFlow and PyTorch to generate optimized travel routes and itineraries for the user. Furthermore, the terminal recognizes the user's emotional state via its camera and microphone and transmits this information to the server. The server takes this emotional state into account and dynamically adjusts the generated route and itinerary.

[0435] The generated travel routes and itineraries are presented to the user through an information display method. Map services such as Google Maps and Mapbox are used for information display, providing a visually intuitive interface.

[0436] As a concrete example, if a family is planning a visit to a theme park and museum in a city, we propose a plan that prioritizes fun attractions that cater to the emotionally excited children, while also incorporating rest stops at appropriate times for the stressed-out parents.

[0437] An example of a prompt message could include specific usage requirements such as, "Based on the user's current emotional state and external information, generate and visually present the optimal travel route and sightseeing plan in real time." In this way, the objective of the present invention is to provide a personalized and comfortable travel and sightseeing experience that meets the needs of individual users.

[0438] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0439] Step 1:

[0440] The user enters preliminary information using a terminal. This information includes hobbies, preferences, modes of transportation, and allergy information. This information is stored in a data storage device and used as basic data for the server to generate the optimal plan for the user.

[0441] Step 2:

[0442] The user inputs information such as the destinations they wish to visit, the order of visits, and their budget into a terminal each time. The terminal sends this information to a server, which uses external information gathering methods to collect real-time data such as traffic conditions, weather information, and congestion levels. The server then uses this input information to collect external information and uses it as input for data analysis.

[0443] Step 3:

[0444] The server analyzes collected external information based on an analytical model. During this process, a generative AI model is used to generate optimized travel routes and itinerary suggestions. The analysis also incorporates user information, both prior and ongoing, to determine the most efficient and user-friendly plan.

[0445] Step 4:

[0446] The user's device uses its camera and microphone to recognize their emotional state in real time and transmit it to the server. The server receives this emotional data and dynamically adjusts the travel route and itinerary based on the analysis results. For example, if the user is feeling stressed, a relaxing spot will be incorporated into the plan.

[0447] Step 5:

[0448] The travel route and itinerary generated on the server are sent to the terminal and presented to the user using an information display method. Here, a clear and visually represented plan is provided to the user using map services such as Google Maps or Mapbox, allowing the user to travel by following the directions.

[0449] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0450] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0451] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0452] [Third Embodiment]

[0453] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0454] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0455] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0456] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0457] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0458] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0459] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0460] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0461] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0462] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0463] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0464] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0465] This invention is a system for providing users with optimal travel routes and sightseeing plans by integrating user information from prior to use and information gathered and analyzed from external sources. Specific embodiments of this system are described below.

[0466] First, the user accesses the application through their device and enters their personal information. This information includes their address, favorite places to visit, preferred mode of transportation, health information such as allergies, and items they need to bring with them when traveling. This information is stored in a database by the server so that it does not need to be re-entered each time the service is used.

[0467] Next, when planning a trip or travel, users input information such as destination, order of visits, budget, and number of people into their device each time. This information is sent to a server and collected along with other external data (for example, traffic conditions, weather forecasts, facility congestion, etc.).

[0468] The server feeds the collected information into a generative model to generate optimized travel routes and sightseeing plans. This generative model takes multiple variables into account and optimizes the routes and plans to match the user's desired conditions. The plans are then visually presented to the user via an information display system. This presentation uses maps and graphic formats based on an image generative model, allowing the user to grasp the details of the routes and plans at a glance.

[0469] Furthermore, the server monitors traffic conditions and weather changes in real time, and regenerates routes and plans based on the new conditions if any changes occur. This functionality allows users to flexibly adapt to fluctuating external environments.

[0470] As an example of how this system works, consider a scenario where a user plans a weekend trip to a tourist destination. The user inputs destinations such as a zoo, restaurant, and souvenir shop into the application, and specifies their budget and the number of people. Based on these factors, the server calculates a plan that takes into account transportation schedules, weather forecasts, and the operating status of the destinations, and generates a visual route map. This allows the user to enjoy sightseeing with the most efficient and comfortable schedule.

[0471] Thus, the present invention provides an excellent mobility and tourism support system that improves the user experience and can quickly respond to changing circumstances.

[0472] The following describes the processing flow.

[0473] Step 1:

[0474] Users enter their personal information through their device. This information includes their address, favorite spots, preferred mode of transportation, and allergy information, which are all included in the application's initial setup screen.

[0475] Step 2:

[0476] The terminal sends the pre-entered information to the server. At this time, the information is encrypted using a secure communication protocol and transmitted safely.

[0477] Step 3:

[0478] The server stores the received preliminary information in a database. This stored information will be used in future plan generation to improve convenience.

[0479] Step 4:

[0480] When users plan trips or travel, they input information such as destination, order of visits, budget, and number of people into their device each time. These input fields are specific to the current plan.

[0481] Step 5:

[0482] The device transmits information to the server in real time as it moves. This includes the user's current location, which is used to generate accurate plans.

[0483] Step 6:

[0484] The server collects information as it goes along, along with external data from external sources such as traffic conditions, weather forecasts, and congestion levels at visited facilities.

[0485] Step 7:

[0486] The server uses a generative model to generate optimized travel routes and itinerary plans based on the collected data. The generative model utilizes AI algorithms to efficiently process multiple variables.

[0487] Step 8:

[0488] The server visually represents the generated plan using an image generation model and converts it into a format that is easy for the user to understand, such as maps and graphics.

[0489] Step 9:

[0490] The server sends visualized routes and plans to the terminal, and the user uses this information to confirm their travel and sightseeing schedule.

[0491] Step 10:

[0492] The server monitors traffic and weather information in real time. If there are any changes, it regenerates routes and plans based on the new information and immediately sends updated information to the terminal.

[0493] This process allows users to quickly obtain the optimal plan based on the information they enter in advance and each time.

[0494] (Example 1)

[0495] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0496] In modern travel and transportation, users need to constantly adapt to changing external conditions, such as traffic congestion, weather changes, and facility occupancy. However, many current systems cannot respond to these changes in real time, making it difficult to provide users with the best possible plan. Furthermore, customization to individual user preferences and conditions is insufficient, and improvements in usability are needed.

[0497] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0498] In this invention, the server includes an information recording medium for storing user attribute information, means for acquiring surrounding information based on information and attribute information received from the user each time, means for inputting the acquired surrounding information into a generating AI model to create an optimized travel route and sightseeing plan, and means for updating the surrounding information in real time and regenerating the route and plan in accordance with any changes. This enables users to quickly adapt to changes in the environment and enjoy optimal travel and sightseeing plans that suit their individual preferences and conditions.

[0499] "User attribute information" refers to information unique to each individual user, such as address, preferred locations, and health considerations, which is necessary for creating individual travel and sightseeing plans.

[0500] An "information recording medium" is a medium for storing information in digital or analog format, and includes databases and cloud storage.

[0501] "On-demand information" refers to information provided by users on the spot when planning a specific trip or journey, and includes variable conditions such as destination, order of visits, budget, and number of people.

[0502] "Surrounding information" refers to dynamic information obtained from external sources, such as traffic conditions, weather information, and facility congestion levels—information that changes in real time.

[0503] A "generative AI model" is a type of artificial intelligence technology that analyzes input data and automatically creates optimal travel routes and sightseeing plans for users.

[0504] "Travel route" refers to information that shows the path a user takes to travel from their starting point to their destination, and includes efficient and safe routes.

[0505] A "tourism plan" is a travel or sightseeing schedule created based on the user's interests and circumstances, and includes elements such as destinations, schedule, and costs.

[0506] A "display device" is a device that provides visual information to a user, and includes smartphones, tablets, and computer displays.

[0507] "Image generation technology" refers to techniques for visually displaying planned routes and schedules in an easy-to-understand manner, and includes computer graphics and map generation methods.

[0508] This system utilizes advanced information processing technology to optimize travel and sightseeing plans. Based on user information, both pre-trip and on-the-go, it collects external data and uses a generative AI model to provide optimal travel routes and sightseeing plans. Furthermore, by adapting to real-time changes in traffic and weather information, it can always provide users with the most up-to-date and optimal plans.

[0509] Users access this system through devices such as smartphones and PCs. The device provides an interface for users to input pre-entered attribute information such as addresses, preferred locations, and allergies. Furthermore, users can input information such as travel destinations, order of visits, budget, and number of people as needed.

[0510] The server aggregates this information, stores it in a database, and automatically collects external information such as transportation status, weather forecasts, and congestion levels at tourist facilities based on that information. The collected information is fed into a generative AI model, which generates optimized travel routes and sightseeing plans that take multiple variables into account.

[0511] The generated travel routes and sightseeing plans are visually represented through information display devices. Image generation technology on the terminal allows users to intuitively view maps and detailed route information. This enables users to plan and execute their trips efficiently and comfortably.

[0512] In particular, as an example of a prompt, it is possible to input a specific request into the generating AI model, such as "The user will visit a zoo, a restaurant, and a souvenir shop, and create the optimal route based on the budget and number of people." With such input, the system will instantly output a plan that takes various factors into consideration.

[0513] This system enhances user convenience while offering the flexibility to respond immediately to changing conditions, providing a new level of convenience in travel planning.

[0514] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0515] Step 1:

[0516] The user launches the application on their device and enters attribute information such as address, preferred tourist destinations, and desired mode of transportation. This information is temporarily stored on the device. The information entered at this stage includes information previously registered by the user and information that changes each time a trip is taken. The output is all the input information aggregated on the device.

[0517] Step 2:

[0518] The terminal sends the entered information to the server. The server saves the received attribute information and other information to the database. This saving saves the server the trouble of re-entering information the next time it is used. The specific actions performed in this step are data format conversion and writing to the database.

[0519] Step 3:

[0520] The server collects surrounding information from external information provision systems, such as traffic conditions, weather forecasts, and congestion levels of planned destinations, based on the stored information. Inputs are user information obtained from the database and external information via APIs, while output is the collected surrounding information data. Specifically, data is acquired through API communication over the internet.

[0521] Step 4:

[0522] The server inputs the collected information into a generating AI model. Based on the prompt, this model generates optimized travel routes and sightseeing plans that consider multiple variables. The output is an optimized route or plan tailored to the user. The specific data processing here involves analyzing multiple variables and calculating the optimal choice using a predictive model.

[0523] Step 5:

[0524] The server visualizes the generated routes and plans and sends the information to the terminal. This visualization uses image generation technology and includes the process of visually formatting map data and detailed route information. The output is graphical data that can be displayed on the terminal.

[0525] Step 6:

[0526] The terminal presents the user with data received from the server. The user can use the terminal's display to review and adjust the generated plan. The output here is information presented through a user-friendly interface, specifically including touch-based actions for reviewing and reconfiguring the plan.

[0527] Step 7:

[0528] The server monitors traffic and weather information in real time, and if there are any changes, it restarts the AI ​​model based on the new conditions and updates the plan. The input is the latest external information, and the output is the latest travel route and sightseeing plan. In this step, the process of periodically acquiring information and recalculating is carried out as concrete actions.

[0529] (Application Example 1)

[0530] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0531] In today's urban environments, providing optimal routes and plans that meet the diverse needs of users during travel and sightseeing is becoming increasingly complex. This is due to the numerous factors that must be considered, including traffic congestion, weather changes, and individual user preferences and health information. Traditional methods have struggled to comprehensively process this information and provide the best recommendations in real time. Therefore, there is a need for new systems that can solve these problems and enable users to travel more comfortably and efficiently within cities and enjoy sightseeing.

[0532] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0533] In this invention, the server includes data storage means for recording the user's basic information, means for acquiring external information based on the temporary and basic information received from the user, and means for analyzing the acquired external information using a generative model to create an optimized travel route and action plan. This makes it possible to provide the optimal route and plan that integrates the user's preferences with real-time surrounding information.

[0534] "Basic user information" refers to information about the user's address, preferences, and health status.

[0535] "Temporary information" refers to information such as destination, budget, and number of people that users enter each time they travel or move around.

[0536] "External information" refers to data related to users' prior and immediate information, such as traffic conditions, weather forecasts, and facility operating status.

[0537] "Data storage means" refers to databases and storage systems used to store basic user information.

[0538] "Means of acquiring external information" refers to the means by which a server collects necessary information from external data sources based on user usage information.

[0539] "Analysis using generative models" refers to the process of using AI models to generate optimal travel routes and action plans in order to analyze collected data.

[0540] "Travel routes and itinerary plans" refer to route guidance and schedules of destinations designed to enable users to travel efficiently and enjoyably within a city and to sightsee.

[0541] "Information presentation means" refers to interfaces and devices that clearly communicate the generated travel route and action plan to the user.

[0542] This invention is a system for optimizing users' travel and sightseeing experiences. The system mainly consists of a server, a user's terminal, and an information presentation means. A specific embodiment of this system is described below.

[0543] The server first stores the user's basic information in a database. This basic information includes address, preferences, and health information. Based on the temporary information received from the user and this basic information, the server retrieves external information. This process utilizes real-time data such as traffic conditions and weather forecasts.

[0544] The acquired external information is fed into an AI-powered generative model. This generative model is built using programming languages ​​such as Python and utilizes machine learning frameworks like TensorFlow and PyTorch. This model performs multivariate analysis to generate optimal travel routes and action plans.

[0545] The generated routes and plans are presented to the user through an application installed on their device. The information presentation methods are developed using API frameworks such as Flask and FastAPI, and are available on smartphones. The visualized route maps are provided by an image generation model, ensuring an intuitive and quickly understandable format for the user.

[0546] As a concrete example, consider a scenario where a tourist visits a city on a holiday. The user enters a zoo, restaurant, and museum as planned destinations into their device, and specifies their preferred mode of transportation and budget. Based on this information, the server considers traffic information, weather forecasts, and the congestion levels of tourist spots to generate the optimal order of visits and a detailed schedule. The user receives this information in real time via their smartphone, allowing them to respond flexibly even if their plans change.

[0547] An example of a prompt message might be, "I would like to visit the zoo and a restaurant this afternoon. Please tell me the best route and consider the weather." Such features allow users to have a comfortable and efficient urban experience.

[0548] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0549] Step 1:

[0550] The user enters basic information through a terminal. This information includes address, preferences, and health information. This information is stored in a database through a data storage system. The information entered here is the user's basic information, and the database organizes and stores it for each user.

[0551] Step 2:

[0552] Users enter temporary information on their device each time they travel or move around. This temporary information includes destination, budget, and number of people. This information is sent to the server and combined with the basic information stored in the previous step. This allows the server to identify the necessary external information.

[0553] Step 3:

[0554] The server retrieves external information such as traffic and weather based on the received temporary and basic information. It uses external APIs to collect real-time traffic and weather data and feeds this into a generative model for data analysis.

[0555] Step 4:

[0556] The server analyzes the collected external information using a generative AI model and converts the input information into the optimal travel route and action plan. This generative model considers multiple variables and selects the best option from among several routes and plans.

[0557] Step 5:

[0558] The travel route and action plan generated from the server are presented to the user through the terminal's information display mechanism. The terminal uses an image generation model to display the route and plan in a visually easy-to-understand format, allowing the user to grasp the entire plan at a glance.

[0559] Step 6:

[0560] The user selects a plan from the presented options and begins their journey. If events such as changes in traffic conditions or weather occur along the way, the server retrieves external information again, recalculates the optimal route, and sends it to the user's device. This allows the user to flexibly adapt to unexpected changes.

[0561] Step 7:

[0562] Throughout the process, users can request more detailed information or additional steps at any point by utilizing prompts tailored to their plan and route. By entering prompts such as "How long does it take to walk from my current location to the nearest station?", the system quickly generates a response.

[0563] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0564] This invention is a system for optimizing travel and sightseeing plans into a more comfortable and personalized experience by incorporating an emotion engine that recognizes the user's emotions. Specific embodiments are described below.

[0565] First, the user accesses the application through their device and enters preliminary information. This information includes the user's basic hobbies and preferences, mode of transportation, allergy information, etc., and is stored in a database on the server. Based on this preliminary information, personalized planning becomes possible.

[0566] When planning a trip or journey, users input information such as destination, order of visits, budget, and number of people into their device. The device sends this information to a server, which then collects information from external sources, such as traffic conditions, weather forecasts, and facility congestion levels. This enables the generation of plans based on the latest and most optimal information.

[0567] A key feature here is that the application implements an emotion engine. The server receives the user's emotional state and feedback obtained from the user's device via the emotion engine. This emotion engine can analyze the user's current emotions through facial recognition technology and voice analysis.

[0568] Based on data from the emotion engine, the server generates optimized travel routes and itinerary plans tailored to the user's preferences. This optimization process includes adjustments to reduce user stress and anxiety, allowing for changes in the order of destinations and the inclusion of additional relaxation points.

[0569] The generated plan is presented to the user in a visually easy-to-understand format using information display methods. For example, it can be displayed on a map or graphic on the device, utilizing an image generation model.

[0570] Furthermore, the server continuously monitors real-time operational and weather information, immediately recalculating routes and plans if any changes occur, and providing updated information to the terminal. This ensures that users always receive the latest information that reflects new circumstances.

[0571] For example, if a user wants to visit a zoo and a theme park on the weekend, the emotional engine starts from their morning state and predicts the order in which they should visit to maximize their satisfaction. For instance, it prioritizes rides when the user is calm and inserts restful events when they are tired, thereby maximizing the user's experience. In this way, the system of the present invention provides an innovative and personalized travel and sightseeing experience by adapting to the emotional state of each individual user.

[0572] The following describes the processing flow.

[0573] Step 1:

[0574] Users access the application via their device and enter preliminary information such as address, hobbies, health information, and mode of transportation as profile information. The entered information is used as basic data necessary for generating personalized plans.

[0575] Step 2:

[0576] The terminal sends all entered pre-information to the server. The information is encrypted before transmission and securely stored in a database.

[0577] Step 3:

[0578] When planning a trip or travel, users register information such as destination, order of visits, budget, and number of people on their device each time. This enables real-time planning.

[0579] Step 4:

[0580] The device sends information to the server each time. Simultaneously, it starts analyzing the user's emotional state using an emotion engine to obtain the user's emotional state at that moment.

[0581] Step 5:

[0582] The server receives information as it goes, while simultaneously collecting external data such as traffic conditions, weather forecasts, and facility congestion levels. This makes it possible to generate realistic plans that take all these factors into account.

[0583] Step 6:

[0584] The emotion engine analyzes the user's emotions from their facial expressions and voice, and sends the results to the server. This is done to make adjustments based on the user's preferences and state.

[0585] Step 7:

[0586] The server considers the received emotional data and generates optimized travel routes and plans using a generative model. At this stage, the plan is adjusted according to the user's emotional state.

[0587] Step 8:

[0588] The server visually represents the generated route and plan and transmits it to the terminal via a display device. By using an image generation model, intuitive plan displays using maps and graphics are possible.

[0589] Step 9:

[0590] Users review the plan presented on their device and enter any additional requests or feedback as needed. Further adjustments to the plan may then be made based on this feedback.

[0591] Step 10:

[0592] During your trip, the server monitors real-time operational and weather information, and if any changes are detected, it immediately regenerates the plan and notifies your device of the latest information. This feature ensures that users always have the most up-to-date and optimal plan.

[0593] This process allows users to obtain personalized travel and sightseeing plans that also take their emotions into consideration.

[0594] (Example 2)

[0595] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0596] In recent years, there has been a growing demand for personalized travel and sightseeing experiences, but adjusting plans based on users' emotional states is not easy. Traditional systems are inadequate in considering temporary changes in users' emotions, and improvements in usability are needed. Furthermore, there is a challenge in handling dynamic plan changes based on real-time information.

[0597] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0598] In this invention, the server includes a storage means for recording the user's prior information, a means for acquiring external information based on the information received from the user each time and the prior information, a means for analyzing the acquired external information using a generation model and generating an optimized travel route and itinerary plan, and an emotion analysis means for analyzing the user's emotional state and adjusting the itinerary plan based on the emotional state. This enables flexible planning according to the user's current emotional state, allowing for a better provision of a personalized experience.

[0599] "Users" refer to individuals or groups who use the system and are the entities responsible for planning travel or sightseeing.

[0600] "Prior information" refers to data such as basic personal information, hobbies, preferences, means of transportation, and allergy information that users provide before using the system.

[0601] "On-demand information" refers to information that users provide as needed when traveling or moving around, such as destination, order of visits, budget, and number of people.

[0602] "External information" refers to real-time data obtained from external sources, such as traffic conditions, weather forecasts, and facility congestion levels, which are collected by the system.

[0603] A "generative model" is a computer program that analyzes external information to formulate optimized travel routes and itinerary plans.

[0604] "Travel route" refers to the path or route used by a user when traveling.

[0605] "Itinerary plan" refers to the overall schedule, including the places and events that the user will visit.

[0606] "Emotional analysis methods" refer to technologies that analyze a user's emotional state from their facial expressions and voice, and use that information to adjust the plan.

[0607] "Display means" refers to a device or technology for visually presenting the generated routes and plans.

[0608] The system according to the present invention aims to highly personalize and optimize the user's travel and mobility experience in real time. Specific embodiments thereof are described below.

[0609] First, the user accesses the application using their device and enters preliminary information such as hobbies, preferences, mode of transportation, and allergy information. This information helps in creating an initial plan tailored to the user's preferences.

[0610] The information entered by the terminal is sent to the server, which stores it in a database. This information is then used to generate customized plans for each user.

[0611] When planning a trip or travel, users input information such as destination, order of visits, budget, and number of people via their device. The device then sends this information to a server.

[0612] The server collects external information from external sources, such as traffic conditions, weather forecasts, and facility congestion levels. This allows for planning that reflects the latest conditions.

[0613] One of the features of this system is its emotion analysis method, which analyzes the user's emotional state from their device using facial recognition technology and voice analysis.

[0614] The server analyzes the collected emotional data and external information using a generating AI model to create travel routes and itinerary plans based on the user's emotional state. This process incorporates plans that, for example, provide active activities when the user's emotions are stable, and rest when they are tired.

[0615] The generated plan is visualized by an image generation model and presented to the user on the device. This allows the user to intuitively understand and follow the plan.

[0616] Furthermore, the server monitors operational and weather information in real time and recalculates the plan as needed. This ensures that users can have the optimal experience even in fluctuating environments.

[0617] For example, if a user plans a visit to a zoo and a theme park, sentiment analysis can provide a plan that includes active rides in the morning and relaxing rest stops in the afternoon.

[0618] An example of a prompt message for a generative AI model would be, "Adjust the travel plan according to the user's emotions to provide a highly satisfying experience."

[0619] As described above, this system can take the user's emotional state into consideration and provide the optimal travel route and itinerary plan in real time.

[0620] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0621] Step 1:

[0622] Users access the application through their device and enter preliminary information. This information includes hobbies, preferences, modes of transportation, and allergy information. The device sends this information to the server. As a result, the server stores the user's basic information and provides data that helps create personalized plans.

[0623] Step 2:

[0624] When users plan trips or travel, they input information such as destination, order of visits, budget, and number of people into their device each time. The device sends this input information to the server. Based on this information, the server retrieves external information and collects basic data for plan generation.

[0625] Step 3:

[0626] Based on the collected data, the server gathers information from external sources, such as traffic conditions, weather forecasts, and facility congestion levels. The server analyzes the acquired data and prepares a plan to provide users with the best possible choices in real time.

[0627] Step 4:

[0628] The server uses emotion analysis tools to analyze the user's emotional state based on voice data and facial images obtained from the user's device. The results of this analysis quantify the user's emotional state and are output as data that influences the planning process.

[0629] Step 5:

[0630] The server uses a generative AI model to integrate user prior information, on-the-spot information, external information, and emotional data to generate optimized travel routes and itinerary plans. The output plans include activity and rest plans that take emotional states into account, aiming to improve usability.

[0631] Step 6:

[0632] The generated plan is visually represented on the device and provided to the user. Using an image generation model, maps and schedules are displayed graphically, allowing users to review the plan in an intuitive and easy-to-understand manner.

[0633] Step 7:

[0634] The server continuously monitors operational and weather information and recalculates the plan as needed. If there are any changes, the server sends the updated plan to the terminal, allowing users to always act based on the latest information.

[0635] (Application Example 2)

[0636] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0637] This invention aims to enable individual citizens and tourists to enjoy more comfortable and stress-free travel and sightseeing experiences in urban environments. Conventional systems do not take emotional states into consideration and only provide uniform information, making it difficult to propose flexible plans tailored to individual needs. Therefore, it is necessary to construct a system that takes users' emotions into account and provides dynamically optimized travel routes and itinerary suggestions.

[0638] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0639] In this invention, the server includes a data storage device for storing user information in advance, means for collecting external information based on information received from the user each time and the prior information, means for analyzing the collected external information using an analysis model and generating optimized travel routes and itinerary plans, means for recognizing the user's emotional state and dynamically adjusting the travel routes and itinerary plans according to the emotional state, and an information display method for presenting the generated routes and plans to the user. This makes it possible to provide a highly personalized and comfortable travel and sightseeing experience that takes the user's emotional state into consideration.

[0640] A "data storage device" is a device for storing users' prior information and has the function of accessing and providing information as needed.

[0641] "External information" refers to data necessary for generating travel routes and itineraries, including information such as traffic conditions, weather, and facility congestion levels.

[0642] An "analysis model" is an algorithm or computational method used to analyze collected external information and generate optimized travel routes and itineraries.

[0643] A "travel route" is a design that shows the optimal path a user takes to travel from one point to their destination.

[0644] A "planned itinerary" is a plan for a user to visit a specific destination, and includes the order of visits and the time allocation.

[0645] "Emotional state" refers to the user's psychological state, and includes conditions such as stress, anxiety, and excitement.

[0646] "Dynamic adjustment" means automatically changing the travel route and itinerary in response to changes in the user's emotional state or external information.

[0647] "Information display method" refers to a means of visually presenting the generated travel route and itinerary to the user, and includes maps and graphic interfaces.

[0648] The system for realizing this invention is configured as follows: The system uses the user's smartphone as a terminal, and a dedicated application is installed on that terminal. This application stores the user's prior information in a data storage device and collects external information through a data collection means based on the information entered by the user each time. The external information includes traffic conditions, weather information, and the level of congestion at facilities.

[0649] The server analyzes external information obtained from data collection methods using an analytical model. This analytical model utilizes machine learning frameworks such as TensorFlow and PyTorch to generate optimized travel routes and itineraries for the user. Furthermore, the terminal recognizes the user's emotional state via its camera and microphone and transmits this information to the server. The server takes this emotional state into account and dynamically adjusts the generated route and itinerary.

[0650] The generated travel routes and itineraries are presented to the user through an information display method. Map services such as Google Maps and Mapbox are used for information display, providing a visually intuitive interface.

[0651] As a concrete example, if a family is planning a visit to a theme park and museum in a city, we propose a plan that prioritizes fun attractions that cater to the emotionally excited children, while also incorporating rest stops at appropriate times for the stressed-out parents.

[0652] An example of a prompt message could include specific usage requirements such as, "Based on the user's current emotional state and external information, generate and visually present the optimal travel route and sightseeing plan in real time." In this way, the objective of the present invention is to provide a personalized and comfortable travel and sightseeing experience that meets the needs of individual users.

[0653] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0654] Step 1:

[0655] The user enters preliminary information using a terminal. This information includes hobbies, preferences, modes of transportation, and allergy information. This information is stored in a data storage device and used as basic data for the server to generate the optimal plan for the user.

[0656] Step 2:

[0657] The user inputs information such as the destinations they wish to visit, the order of visits, and their budget into a terminal each time. The terminal sends this information to a server, which uses external information gathering methods to collect real-time data such as traffic conditions, weather information, and congestion levels. The server then uses this input information to collect external information and uses it as input for data analysis.

[0658] Step 3:

[0659] The server analyzes collected external information based on an analytical model. During this process, a generative AI model is used to generate optimized travel routes and itinerary suggestions. The analysis also incorporates user information, both prior and ongoing, to determine the most efficient and user-friendly plan.

[0660] Step 4:

[0661] The user's device uses its camera and microphone to recognize their emotional state in real time and transmit it to the server. The server receives this emotional data and dynamically adjusts the travel route and itinerary based on the analysis results. For example, if the user is feeling stressed, a relaxing spot will be incorporated into the plan.

[0662] Step 5:

[0663] The travel route and itinerary generated on the server are sent to the terminal and presented to the user using an information display method. Here, a clear and visually represented plan is provided to the user using map services such as Google Maps or Mapbox, allowing the user to travel by following the directions.

[0664] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0665] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0666] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0667] [Fourth Embodiment]

[0668] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0669] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0670] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0671] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0672] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0673] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0674] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0675] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0676] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0677] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0678] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0679] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0680] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0681] This invention is a system for providing users with optimal travel routes and sightseeing plans by integrating user information from prior to use and information gathered and analyzed from external sources. Specific embodiments of this system are described below.

[0682] First, the user accesses the application through their device and enters their personal information. This information includes their address, favorite places to visit, preferred mode of transportation, health information such as allergies, and items they need to bring with them when traveling. This information is stored in a database by the server so that it does not need to be re-entered each time the service is used.

[0683] Next, when planning a trip or travel, users input information such as destination, order of visits, budget, and number of people into their device each time. This information is sent to a server and collected along with other external data (for example, traffic conditions, weather forecasts, facility congestion, etc.).

[0684] The server feeds the collected information into a generative model to generate optimized travel routes and sightseeing plans. This generative model takes multiple variables into account and optimizes the routes and plans to match the user's desired conditions. The plans are then visually presented to the user via an information display system. This presentation uses maps and graphic formats based on an image generative model, allowing the user to grasp the details of the routes and plans at a glance.

[0685] Furthermore, the server monitors traffic conditions and weather changes in real time, and regenerates routes and plans based on the new conditions if any changes occur. This functionality allows users to flexibly adapt to fluctuating external environments.

[0686] As an example of how this system works, consider a scenario where a user plans a weekend trip to a tourist destination. The user inputs destinations such as a zoo, restaurant, and souvenir shop into the application, and specifies their budget and the number of people. Based on these factors, the server calculates a plan that takes into account transportation schedules, weather forecasts, and the operating status of the destinations, and generates a visual route map. This allows the user to enjoy sightseeing with the most efficient and comfortable schedule.

[0687] Thus, the present invention provides an excellent mobility and tourism support system that improves the user experience and can quickly respond to changing circumstances.

[0688] The following describes the processing flow.

[0689] Step 1:

[0690] Users enter their personal information through their device. This information includes their address, favorite spots, preferred mode of transportation, and allergy information, which are all included in the application's initial setup screen.

[0691] Step 2:

[0692] The terminal sends the pre-entered information to the server. At this time, the information is encrypted using a secure communication protocol and transmitted safely.

[0693] Step 3:

[0694] The server stores the received preliminary information in a database. This stored information will be used in future plan generation to improve convenience.

[0695] Step 4:

[0696] When users plan trips or travel, they input information such as destination, order of visits, budget, and number of people into their device each time. These input fields are specific to the current plan.

[0697] Step 5:

[0698] The device transmits information to the server in real time as it moves. This includes the user's current location, which is used to generate accurate plans.

[0699] Step 6:

[0700] The server collects information as it goes along, along with external data from external sources such as traffic conditions, weather forecasts, and congestion levels at visited facilities.

[0701] Step 7:

[0702] The server uses a generative model to generate optimized travel routes and itinerary plans based on the collected data. The generative model utilizes AI algorithms to efficiently process multiple variables.

[0703] Step 8:

[0704] The server visually represents the generated plan using an image generation model and converts it into a format that is easy for the user to understand, such as maps and graphics.

[0705] Step 9:

[0706] The server sends visualized routes and plans to the terminal, and the user uses this information to confirm their travel and sightseeing schedule.

[0707] Step 10:

[0708] The server monitors traffic and weather information in real time. If there are any changes, it regenerates routes and plans based on the new information and immediately sends updated information to the terminal.

[0709] This process allows users to quickly obtain the optimal plan based on the information they enter in advance and each time.

[0710] (Example 1)

[0711] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0712] In modern travel and transportation, users need to constantly adapt to changing external conditions, such as traffic congestion, weather changes, and facility occupancy. However, many current systems cannot respond to these changes in real time, making it difficult to provide users with the best possible plan. Furthermore, customization to individual user preferences and conditions is insufficient, and improvements in usability are needed.

[0713] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0714] In this invention, the server includes an information recording medium for storing user attribute information, means for acquiring surrounding information based on information and attribute information received from the user each time, means for inputting the acquired surrounding information into a generating AI model to create an optimized travel route and sightseeing plan, and means for updating the surrounding information in real time and regenerating the route and plan in accordance with any changes. This enables users to quickly adapt to changes in the environment and enjoy optimal travel and sightseeing plans that suit their individual preferences and conditions.

[0715] "User attribute information" refers to information unique to each individual user, such as address, preferred locations, and health considerations, which is necessary for creating individual travel and sightseeing plans.

[0716] An "information recording medium" is a medium for storing information in digital or analog format, and includes databases and cloud storage.

[0717] "On-demand information" refers to information provided by users on the spot when planning a specific trip or journey, and includes variable conditions such as destination, order of visits, budget, and number of people.

[0718] "Surrounding information" refers to dynamic information obtained from external sources, such as traffic conditions, weather information, and facility congestion levels—information that changes in real time.

[0719] A "generative AI model" is a type of artificial intelligence technology that analyzes input data and automatically creates optimal travel routes and sightseeing plans for users.

[0720] "Travel route" refers to information that shows the path a user takes to travel from their starting point to their destination, and includes efficient and safe routes.

[0721] A "tourism plan" is a travel or sightseeing schedule created based on the user's interests and circumstances, and includes elements such as destinations, schedule, and costs.

[0722] A "display device" is a device that provides visual information to a user, and includes smartphones, tablets, and computer displays.

[0723] "Image generation technology" refers to techniques for visually displaying planned routes and schedules in an easy-to-understand manner, and includes computer graphics and map generation methods.

[0724] This system utilizes advanced information processing technology to optimize travel and sightseeing plans. Based on user information, both pre-trip and on-the-go, it collects external data and uses a generative AI model to provide optimal travel routes and sightseeing plans. Furthermore, by adapting to real-time changes in traffic and weather information, it can always provide users with the most up-to-date and optimal plans.

[0725] Users access this system through devices such as smartphones and PCs. The device provides an interface for users to input pre-entered attribute information such as addresses, preferred locations, and allergies. Furthermore, users can input information such as travel destinations, order of visits, budget, and number of people as needed.

[0726] The server aggregates this information, stores it in a database, and automatically collects external information such as transportation status, weather forecasts, and congestion levels at tourist facilities based on that information. The collected information is fed into a generative AI model, which generates optimized travel routes and sightseeing plans that take multiple variables into account.

[0727] The generated travel routes and sightseeing plans are visually represented through information display devices. Image generation technology on the terminal allows users to intuitively view maps and detailed route information. This enables users to plan and execute their trips efficiently and comfortably.

[0728] In particular, as an example of a prompt, it is possible to input a specific request into the generating AI model, such as "The user will visit a zoo, a restaurant, and a souvenir shop, and create the optimal route based on the budget and number of people." With such input, the system will instantly output a plan that takes various factors into consideration.

[0729] This system enhances user convenience while offering the flexibility to respond immediately to changing conditions, providing a new level of convenience in travel planning.

[0730] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0731] Step 1:

[0732] The user launches the application on their device and enters attribute information such as address, preferred tourist destinations, and desired mode of transportation. This information is temporarily stored on the device. The information entered at this stage includes information previously registered by the user and information that changes each time a trip is taken. The output is all the input information aggregated on the device.

[0733] Step 2:

[0734] The terminal sends the entered information to the server. The server saves the received attribute information and other information to the database. This saving saves the server the trouble of re-entering information the next time it is used. The specific actions performed in this step are data format conversion and writing to the database.

[0735] Step 3:

[0736] The server collects surrounding information from external information provision systems, such as traffic conditions, weather forecasts, and congestion levels of planned destinations, based on the stored information. Inputs are user information obtained from the database and external information via APIs, while output is the collected surrounding information data. Specifically, data is acquired through API communication over the internet.

[0737] Step 4:

[0738] The server inputs the collected information into a generating AI model. Based on the prompt, this model generates optimized travel routes and sightseeing plans that consider multiple variables. The output is an optimized route or plan tailored to the user. The specific data processing here involves analyzing multiple variables and calculating the optimal choice using a predictive model.

[0739] Step 5:

[0740] The server visualizes the generated routes and plans and sends the information to the terminal. This visualization uses image generation technology and includes the process of visually formatting map data and detailed route information. The output is graphical data that can be displayed on the terminal.

[0741] Step 6:

[0742] The terminal presents the user with data received from the server. The user can use the terminal's display to review and adjust the generated plan. The output here is information presented through a user-friendly interface, specifically including touch-based actions for reviewing and reconfiguring the plan.

[0743] Step 7:

[0744] The server monitors traffic and weather information in real time, and if there are any changes, it restarts the AI ​​model based on the new conditions and updates the plan. The input is the latest external information, and the output is the latest travel route and sightseeing plan. In this step, the process of periodically acquiring information and recalculating is carried out as concrete actions.

[0745] (Application Example 1)

[0746] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0747] In today's urban environments, providing optimal routes and plans that meet the diverse needs of users during travel and sightseeing is becoming increasingly complex. This is due to the numerous factors that must be considered, including traffic congestion, weather changes, and individual user preferences and health information. Traditional methods have struggled to comprehensively process this information and provide the best recommendations in real time. Therefore, there is a need for new systems that can solve these problems and enable users to travel more comfortably and efficiently within cities and enjoy sightseeing.

[0748] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0749] In this invention, the server includes data storage means for recording the user's basic information, means for acquiring external information based on the temporary and basic information received from the user, and means for analyzing the acquired external information using a generative model to create an optimized travel route and action plan. This makes it possible to provide the optimal route and plan that integrates the user's preferences with real-time surrounding information.

[0750] "Basic user information" refers to information about the user's address, preferences, and health status.

[0751] "Temporary information" refers to information such as destination, budget, and number of people that users enter each time they travel or move around.

[0752] "External information" refers to data related to users' prior and immediate information, such as traffic conditions, weather forecasts, and facility operating status.

[0753] "Data storage means" refers to databases and storage systems used to store basic user information.

[0754] "Means of acquiring external information" refers to the means by which a server collects necessary information from external data sources based on user usage information.

[0755] "Analysis using generative models" refers to the process of using AI models to generate optimal travel routes and action plans in order to analyze collected data.

[0756] "Travel routes and itinerary plans" refer to route guidance and schedules of destinations designed to enable users to travel efficiently and enjoyably within a city and to sightsee.

[0757] "Information presentation means" refers to interfaces and devices that clearly communicate the generated travel route and action plan to the user.

[0758] This invention is a system for optimizing users' travel and sightseeing experiences. The system mainly consists of a server, a user's terminal, and an information presentation means. A specific embodiment of this system is described below.

[0759] The server first stores the user's basic information in a database. This basic information includes address, preferences, and health information. Based on the temporary information received from the user and this basic information, the server retrieves external information. This process utilizes real-time data such as traffic conditions and weather forecasts.

[0760] The acquired external information is fed into an AI-powered generative model. This generative model is built using programming languages ​​such as Python and utilizes machine learning frameworks like TensorFlow and PyTorch. This model performs multivariate analysis to generate optimal travel routes and action plans.

[0761] The generated routes and plans are presented to the user through an application installed on their device. The information presentation methods are developed using API frameworks such as Flask and FastAPI, and are available on smartphones. The visualized route maps are provided by an image generation model, ensuring an intuitive and quickly understandable format for the user.

[0762] As a concrete example, consider a scenario where a tourist visits a city on a holiday. The user enters a zoo, restaurant, and museum as planned destinations into their device, and specifies their preferred mode of transportation and budget. Based on this information, the server considers traffic information, weather forecasts, and the congestion levels of tourist spots to generate the optimal order of visits and a detailed schedule. The user receives this information in real time via their smartphone, allowing them to respond flexibly even if their plans change.

[0763] An example of a prompt message might be, "I would like to visit the zoo and a restaurant this afternoon. Please tell me the best route and consider the weather." Such features allow users to have a comfortable and efficient urban experience.

[0764] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0765] Step 1:

[0766] The user enters basic information through a terminal. This information includes address, preferences, and health information. This information is stored in a database through a data storage system. The information entered here is the user's basic information, and the database organizes and stores it for each user.

[0767] Step 2:

[0768] Users enter temporary information on their device each time they travel or move around. This temporary information includes destination, budget, and number of people. This information is sent to the server and combined with the basic information stored in the previous step. This allows the server to identify the necessary external information.

[0769] Step 3:

[0770] The server retrieves external information such as traffic and weather based on the received temporary and basic information. It uses external APIs to collect real-time traffic and weather data and feeds this into a generative model for data analysis.

[0771] Step 4:

[0772] The server analyzes the collected external information using a generative AI model and converts the input information into the optimal travel route and action plan. This generative model considers multiple variables and selects the best option from among several routes and plans.

[0773] Step 5:

[0774] The travel route and action plan generated from the server are presented to the user through the terminal's information display mechanism. The terminal uses an image generation model to display the route and plan in a visually easy-to-understand format, allowing the user to grasp the entire plan at a glance.

[0775] Step 6:

[0776] The user selects a plan from the presented options and begins their journey. If events such as changes in traffic conditions or weather occur along the way, the server retrieves external information again, recalculates the optimal route, and sends it to the user's device. This allows the user to flexibly adapt to unexpected changes.

[0777] Step 7:

[0778] Throughout the process, users can request more detailed information or additional steps at any point by utilizing prompts tailored to their plan and route. By entering prompts such as "How long does it take to walk from my current location to the nearest station?", the system quickly generates a response.

[0779] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0780] This invention is a system for optimizing travel and sightseeing plans into a more comfortable and personalized experience by incorporating an emotion engine that recognizes the user's emotions. Specific embodiments are described below.

[0781] First, the user accesses the application through their device and enters preliminary information. This information includes the user's basic hobbies and preferences, mode of transportation, allergy information, etc., and is stored in a database on the server. Based on this preliminary information, personalized planning becomes possible.

[0782] When planning a trip or journey, users input information such as destination, order of visits, budget, and number of people into their device. The device sends this information to a server, which then collects information from external sources, such as traffic conditions, weather forecasts, and facility congestion levels. This enables the generation of plans based on the latest and most optimal information.

[0783] A key feature here is that the application implements an emotion engine. The server receives the user's emotional state and feedback obtained from the user's device via the emotion engine. This emotion engine can analyze the user's current emotions through facial recognition technology and voice analysis.

[0784] Based on data from the emotion engine, the server generates optimized travel routes and itinerary plans tailored to the user's preferences. This optimization process includes adjustments to reduce user stress and anxiety, allowing for changes in the order of destinations and the inclusion of additional relaxation points.

[0785] The generated plan is presented to the user in a visually easy-to-understand format using information display methods. For example, it can be displayed on a map or graphic on the device, utilizing an image generation model.

[0786] Furthermore, the server continuously monitors real-time operational and weather information, immediately recalculating routes and plans if any changes occur, and providing updated information to the terminal. This ensures that users always receive the latest information that reflects new circumstances.

[0787] For example, if a user wants to visit a zoo and a theme park on the weekend, the emotional engine starts from their morning state and predicts the order in which they should visit to maximize their satisfaction. For instance, it prioritizes rides when the user is calm and inserts restful events when they are tired, thereby maximizing the user's experience. In this way, the system of the present invention provides an innovative and personalized travel and sightseeing experience by adapting to the emotional state of each individual user.

[0788] The following describes the processing flow.

[0789] Step 1:

[0790] Users access the application via their device and enter preliminary information such as address, hobbies, health information, and mode of transportation as profile information. The entered information is used as basic data necessary for generating personalized plans.

[0791] Step 2:

[0792] The terminal sends all entered pre-information to the server. The information is encrypted before transmission and securely stored in a database.

[0793] Step 3:

[0794] When planning a trip or travel, users register information such as destination, order of visits, budget, and number of people on their device each time. This enables real-time planning.

[0795] Step 4:

[0796] The device sends information to the server each time. Simultaneously, it starts analyzing the user's emotional state using an emotion engine to obtain the user's emotional state at that moment.

[0797] Step 5:

[0798] The server receives information as it goes, while simultaneously collecting external data such as traffic conditions, weather forecasts, and facility congestion levels. This makes it possible to generate realistic plans that take all these factors into account.

[0799] Step 6:

[0800] The emotion engine analyzes the user's emotions from their facial expressions and voice, and sends the results to the server. This is done to make adjustments based on the user's preferences and state.

[0801] Step 7:

[0802] The server considers the received emotional data and generates optimized travel routes and plans using a generative model. At this stage, the plan is adjusted according to the user's emotional state.

[0803] Step 8:

[0804] The server visually represents the generated route and plan and transmits it to the terminal via a display device. By using an image generation model, intuitive plan displays using maps and graphics are possible.

[0805] Step 9:

[0806] Users review the plan presented on their device and enter any additional requests or feedback as needed. Further adjustments to the plan may then be made based on this feedback.

[0807] Step 10:

[0808] During your trip, the server monitors real-time operational and weather information, and if any changes are detected, it immediately regenerates the plan and notifies your device of the latest information. This feature ensures that users always have the most up-to-date and optimal plan.

[0809] This process allows users to obtain personalized travel and sightseeing plans that also take their emotions into consideration.

[0810] (Example 2)

[0811] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0812] In recent years, there has been a growing demand for personalized travel and sightseeing experiences, but adjusting plans based on users' emotional states is not easy. Traditional systems are inadequate in considering temporary changes in users' emotions, and improvements in usability are needed. Furthermore, there is a challenge in handling dynamic plan changes based on real-time information.

[0813] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0814] In this invention, the server includes a storage means for recording the user's prior information, a means for acquiring external information based on the information received from the user each time and the prior information, a means for analyzing the acquired external information using a generation model and generating an optimized travel route and itinerary plan, and an emotion analysis means for analyzing the user's emotional state and adjusting the itinerary plan based on the emotional state. This enables flexible planning according to the user's current emotional state, allowing for a better provision of a personalized experience.

[0815] "Users" refer to individuals or groups who use the system and are the entities responsible for planning travel or sightseeing.

[0816] "Prior information" refers to data such as basic personal information, hobbies, preferences, means of transportation, and allergy information that users provide before using the system.

[0817] "On-demand information" refers to information that users provide as needed when traveling or moving around, such as destination, order of visits, budget, and number of people.

[0818] "External information" refers to real-time data obtained from external sources, such as traffic conditions, weather forecasts, and facility congestion levels, which are collected by the system.

[0819] A "generative model" is a computer program that analyzes external information to formulate optimized travel routes and itinerary plans.

[0820] "Travel route" refers to the path or route used by a user when traveling.

[0821] "Itinerary plan" refers to the overall schedule, including the places and events that the user will visit.

[0822] "Emotional analysis methods" refer to technologies that analyze a user's emotional state from their facial expressions and voice, and use that information to adjust the plan.

[0823] "Display means" refers to a device or technology for visually presenting the generated routes and plans.

[0824] The system according to the present invention aims to highly personalize and optimize the user's travel and mobility experience in real time. Specific embodiments thereof are described below.

[0825] First, the user accesses the application using their device and enters preliminary information such as hobbies, preferences, mode of transportation, and allergy information. This information helps in creating an initial plan tailored to the user's preferences.

[0826] The information entered by the terminal is sent to the server, which stores it in a database. This information is then used to generate customized plans for each user.

[0827] When planning a trip or travel, users input information such as destination, order of visits, budget, and number of people via their device. The device then sends this information to a server.

[0828] The server collects external information from external sources, such as traffic conditions, weather forecasts, and facility congestion levels. This allows for planning that reflects the latest conditions.

[0829] One of the features of this system is its emotion analysis method, which analyzes the user's emotional state from their device using facial recognition technology and voice analysis.

[0830] The server analyzes the collected emotional data and external information using a generating AI model to create travel routes and itinerary plans based on the user's emotional state. This process incorporates plans that, for example, provide active activities when the user's emotions are stable, and rest when they are tired.

[0831] The generated plan is visualized by an image generation model and presented to the user on the device. This allows the user to intuitively understand and follow the plan.

[0832] Furthermore, the server monitors operational and weather information in real time and recalculates the plan as needed. This ensures that users can have the optimal experience even in fluctuating environments.

[0833] For example, if a user plans a visit to a zoo and a theme park, sentiment analysis can provide a plan that includes active rides in the morning and relaxing rest stops in the afternoon.

[0834] An example of a prompt message for a generative AI model would be, "Adjust the travel plan according to the user's emotions to provide a highly satisfying experience."

[0835] As described above, this system can take the user's emotional state into consideration and provide the optimal travel route and itinerary plan in real time.

[0836] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0837] Step 1:

[0838] Users access the application through their device and enter preliminary information. This information includes hobbies, preferences, modes of transportation, and allergy information. The device sends this information to the server. As a result, the server stores the user's basic information and provides data that helps create personalized plans.

[0839] Step 2:

[0840] When users plan trips or travel, they input information such as destination, order of visits, budget, and number of people into their device each time. The device sends this input information to the server. Based on this information, the server retrieves external information and collects basic data for plan generation.

[0841] Step 3:

[0842] Based on the collected data, the server gathers information from external sources, such as traffic conditions, weather forecasts, and facility congestion levels. The server analyzes the acquired data and prepares a plan to provide users with the best possible choices in real time.

[0843] Step 4:

[0844] The server uses emotion analysis tools to analyze the user's emotional state based on voice data and facial images obtained from the user's device. The results of this analysis quantify the user's emotional state and are output as data that influences the planning process.

[0845] Step 5:

[0846] The server uses a generative AI model to integrate user prior information, on-the-spot information, external information, and emotional data to generate optimized travel routes and itinerary plans. The output plans include activity and rest plans that take emotional states into account, aiming to improve usability.

[0847] Step 6:

[0848] The generated plan is visually represented on the device and provided to the user. Using an image generation model, maps and schedules are displayed graphically, allowing users to review the plan in an intuitive and easy-to-understand manner.

[0849] Step 7:

[0850] The server continuously monitors operational and weather information and recalculates the plan as needed. If there are any changes, the server sends the updated plan to the terminal, allowing users to always act based on the latest information.

[0851] (Application Example 2)

[0852] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0853] This invention aims to enable individual citizens and tourists to enjoy more comfortable and stress-free travel and sightseeing experiences in urban environments. Conventional systems do not take emotional states into consideration and only provide uniform information, making it difficult to propose flexible plans tailored to individual needs. Therefore, it is necessary to construct a system that takes users' emotions into account and provides dynamically optimized travel routes and itinerary suggestions.

[0854] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0855] In this invention, the server includes a data storage device for storing user information in advance, means for collecting external information based on information received from the user each time and the prior information, means for analyzing the collected external information using an analysis model and generating optimized travel routes and itinerary plans, means for recognizing the user's emotional state and dynamically adjusting the travel routes and itinerary plans according to the emotional state, and an information display method for presenting the generated routes and plans to the user. This makes it possible to provide a highly personalized and comfortable travel and sightseeing experience that takes the user's emotional state into consideration.

[0856] A "data storage device" is a device for storing users' prior information and has the function of accessing and providing information as needed.

[0857] "External information" refers to data necessary for generating travel routes and itineraries, including information such as traffic conditions, weather, and facility congestion levels.

[0858] An "analysis model" is an algorithm or computational method used to analyze collected external information and generate optimized travel routes and itineraries.

[0859] A "travel route" is a design that shows the optimal path a user takes to travel from one point to their destination.

[0860] A "planned itinerary" is a plan for a user to visit a specific destination, and includes the order of visits and the time allocation.

[0861] "Emotional state" refers to the user's psychological state, and includes conditions such as stress, anxiety, and excitement.

[0862] "Dynamic adjustment" means automatically changing the travel route and itinerary in response to changes in the user's emotional state or external information.

[0863] "Information display method" refers to a means of visually presenting the generated travel route and itinerary to the user, and includes maps and graphic interfaces.

[0864] The system for realizing this invention is configured as follows: The system uses the user's smartphone as a terminal, and a dedicated application is installed on that terminal. This application stores the user's prior information in a data storage device and collects external information through a data collection means based on the information entered by the user each time. The external information includes traffic conditions, weather information, and the level of congestion at facilities.

[0865] The server analyzes external information obtained from data collection methods using an analytical model. This analytical model utilizes machine learning frameworks such as TensorFlow and PyTorch to generate optimized travel routes and itineraries for the user. Furthermore, the terminal recognizes the user's emotional state via its camera and microphone and transmits this information to the server. The server takes this emotional state into account and dynamically adjusts the generated route and itinerary.

[0866] The generated travel routes and itineraries are presented to the user through an information display method. Map services such as Google Maps and Mapbox are used for information display, providing a visually intuitive interface.

[0867] As a concrete example, if a family is planning a visit to a theme park and museum in a city, we propose a plan that prioritizes fun attractions that cater to the emotionally excited children, while also incorporating rest stops at appropriate times for the stressed-out parents.

[0868] An example of a prompt message could include specific usage requirements such as, "Based on the user's current emotional state and external information, generate and visually present the optimal travel route and sightseeing plan in real time." In this way, the objective of the present invention is to provide a personalized and comfortable travel and sightseeing experience that meets the needs of individual users.

[0869] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0870] Step 1:

[0871] The user enters preliminary information using a terminal. This information includes hobbies, preferences, modes of transportation, and allergy information. This information is stored in a data storage device and used as basic data for the server to generate the optimal plan for the user.

[0872] Step 2:

[0873] The user inputs information such as the destinations they wish to visit, the order of visits, and their budget into a terminal each time. The terminal sends this information to a server, which uses external information gathering methods to collect real-time data such as traffic conditions, weather information, and congestion levels. The server then uses this input information to collect external information and uses it as input for data analysis.

[0874] Step 3:

[0875] The server analyzes collected external information based on an analytical model. During this process, a generative AI model is used to generate optimized travel routes and itinerary suggestions. The analysis also incorporates user information, both prior and ongoing, to determine the most efficient and user-friendly plan.

[0876] Step 4:

[0877] The user's device uses its camera and microphone to recognize their emotional state in real time and transmit it to the server. The server receives this emotional data and dynamically adjusts the travel route and itinerary based on the analysis results. For example, if the user is feeling stressed, a relaxing spot will be incorporated into the plan.

[0878] Step 5:

[0879] The travel route and itinerary generated on the server are sent to the terminal and presented to the user using an information display method. Here, a clear and visually represented plan is provided to the user using map services such as Google Maps or Mapbox, allowing the user to travel by following the directions.

[0880] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0881] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0882] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0883] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0884] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0885] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0886] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0887] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0888] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0889] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0890] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0891] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0892] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0893] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0894] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0895] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0896] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0897] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0898] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0899] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0900] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0901] The following is further disclosed regarding the embodiments described above.

[0902] (Claim 1)

[0903] A database that stores user information in advance,

[0904] A means of collecting external data based on information received from users on an ad-hoc basis and prior information,

[0905] A means for analyzing collected external data using a generative model to generate optimized travel routes and itinerary plans,

[0906] Information display means for presenting the generated route and plan to the user,

[0907] A system that includes this.

[0908] (Claim 2)

[0909] The system according to claim 1, wherein the information display means uses an image generation model for visually representing the generated route and plan.

[0910] (Claim 3)

[0911] The system according to claim 1, further comprising means for monitoring operational information and weather information in real time, and for reanalyzing the information and generating new routes and plans in response to any changes in such information.

[0912] "Example 1"

[0913] (Claim 1)

[0914] An information recording medium for storing user attribute information,

[0915] A means of acquiring surrounding information based on information and attribute information received from the user each time,

[0916] A means for inputting acquired surrounding information into a generating AI model to create optimized travel routes and sightseeing plans,

[0917] A display device for presenting the generated route and plan to the user,

[0918] A means for updating surrounding information in real time and regenerating routes and plans accordingly if changes occur,

[0919] A system that includes this.

[0920] (Claim 2)

[0921] The system according to claim 1, wherein the display device uses image generation technology for visually displaying the generated routes and plans.

[0922] (Claim 3)

[0923] The system according to claim 1, wherein the generative AI model uses prompt statements to perform optimization that takes various conditions into account.

[0924] "Application Example 1"

[0925] (Claim 1)

[0926] A data storage means for recording the user's basic information,

[0927] A means of obtaining external information based on temporary and basic information received from the user,

[0928] A means for analyzing acquired external information using a generative model to create an optimized travel route and action plan,

[0929] Information display means for displaying the created route and plan to the user,

[0930] A means of integrating and presenting local information and event information that users are interested in,

[0931] A system that includes this.

[0932] (Claim 2)

[0933] The system according to claim 1, wherein the information presentation means uses an image generation model for visually representing the created route and plan.

[0934] (Claim 3)

[0935] The system according to claim 1, further comprising means for automatically monitoring current route information and weather conditions, reanalyzing the information in response to any changes, and creating a new route and plan.

[0936] "Example 2 of combining an emotion engine"

[0937] (Claim 1)

[0938] A storage means for recording user information in advance,

[0939] A means of obtaining external information based on information received from users on an ad-hoc basis and prior information,

[0940] A means for analyzing acquired external information using a generative model and generating an optimized travel route and itinerary plan,

[0941] The system includes an emotion analysis means for analyzing the user's emotional state, and means for adjusting the itinerary based on the emotional state.

[0942] A display means for presenting the generated route and plan to the user,

[0943] A system that includes this.

[0944] (Claim 2)

[0945] The system according to claim 1, wherein the display means uses an image generation model for visually representing the generated route and plan.

[0946] (Claim 3)

[0947] The system according to claim 1, further comprising means for monitoring operational information and weather information in real time, and for reanalyzing the information and generating new routes and plans in response to any changes in such information.

[0948] "Application example 2 when combining with an emotional engine"

[0949] (Claim 1)

[0950] A data storage device that stores user information in advance,

[0951] A means of collecting external information based on information received from users on an ad-hoc basis and prior information,

[0952] A means for analyzing collected external information using an analytical model and generating optimized travel routes and itinerary plans,

[0953] Means for recognizing the user's emotional state and dynamically adjusting the travel route and itinerary according to the said emotional state,

[0954] A method for displaying information to present the generated route and proposed route to the user,

[0955] A system that includes this.

[0956] (Claim 2)

[0957] The system according to claim 1, wherein the information display method uses image generation technology for visually representing the generated routes and plans.

[0958] (Claim 3)

[0959] The system according to claim 1, further comprising means for monitoring real-time information and weather data being generated, and for reanalyzing the information if there are any changes, in order to generate new routes and plans. [Explanation of symbols]

[0960] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A data storage means for recording the user's basic information, A means of obtaining external information based on temporary and basic information received from the user, A means for analyzing acquired external information using a generative model to create an optimized travel route and action plan, Information display means for displaying the created route and plan to the user, A means of integrating and presenting local information and event information that users are interested in, A system that includes this.

2. The system according to claim 1, wherein the information presentation means uses an image generation model for visually showing the created route and plan.

3. The system according to claim 1, further comprising means for automatically monitoring current route information and weather conditions, reanalyzing the information in response to any changes, and creating a new route and plan.

Images