System

Abstract

A system is provided.SOLUTION: The system includes an input means for inputting an activity desired by a user during a trip, a data collection means for collecting data on the user's past visit history, required time, current location, weather, crowdedness, and means of transportation, a proposal means for selecting and proposing a sightseeing facility and a dining place most suitable for the user's desire based on the collected data, and a display means for presenting the proposed trip plan to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When planning a trip, it is important to consider the activities the user wants to do (sightseeing, art appreciation, gourmet experiences, etc.) and select the most suitable spots and routes based on that. However, planning a trip requires consideration of various factors (past visit history, weather, travel time, congestion, means of transportation, etc.), and doing this manually is time-consuming and labor-intensive. Furthermore, it is difficult to respond to information that changes in real time (weather, congestion, etc.), and it is also difficult to flexibly change plans during the trip. The object of the present invention is to solve these problems. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing the following means in a travel plan creation support system. First, an input means is provided for inputting activities the user wishes to engage in during their trip. Next, a data collection means is provided for collecting data on the user's past visit history, travel time, current location, weather, congestion, and mode of transportation. A suggestion means is provided for selecting and suggesting tourist facilities and dining spots that best suit the user's preferences based on the collected data. Furthermore, the system is provided with a display means for presenting the suggested travel plan to the user. The system also improves the flexibility and adaptability of the plan by adding a means for dynamically modifying the travel plan based on past visit history data, real-time updated weather and congestion information, and travel time. Furthermore, the input means has a function for analyzing the user's interests and automatically categorizing them (sightseeing, art, gourmet, etc.), enabling more accurate suggestions. These means enable users to create optimal travel plans while reducing their burden.

[0006] A "travel planning support system" is a system that provides users with optimal travel plans based on the activities they want to do during their trip.

[0007] "User" refers to a person who uses this system to make travel plans.

[0008] "Input means" refers to the means by which a user inputs the activities and interests they wish to engage in during their trip.

[0009] "Data collection means" refers to means for collecting data on a user's past visit history, travel time, current location, weather, congestion, and means of transportation.

[0010] The "proposal method" is a means for selecting and proposing the most suitable tourist facilities and dining options to users based on the collected data.

[0011] The "display means" is a means for presenting the proposed travel plan to the user.

[0012] "Past visit history data" refers to data about places and facilities that a user has visited in the past.

[0013] "Real-time weather information" means up-to-date data on current weather conditions.

[0014] "Crowding information" is data on the congestion status of tourist spots, restaurants, etc.

[0015] "Means of transportation" refers to the means of transportation (walking, car, public transportation, etc.) used by users during their trip.

[0016] "Travel time" refers to the time it takes to visit each spot and the time it takes to travel.

[0017] "Category classification" is a process that analyzes the user's entered interests and automatically classifies them into categories such as sightseeing, art, and gourmet food.

[0018] "Means for dynamically modifying travel plans" refers to means for correcting and updating travel plans in a timely manner based on data collected in real time. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention is a travel planning support system that provides an optimal travel plan based on user input. This system has an input means for inputting activities the user wants to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and transportation methods, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, and a display means for presenting the suggested travel plan to the user.

[0041] System Programming and Processing

[0042] Initialization and accepting user input

[0043] User:

[0044] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[0045] Device:

[0046] The input information is analyzed and classified into each activity category.

[0047] The analysis results are sent to the server.

[0048] Information gathering

[0049] server:

[0050] Past visit history, current location, and mode of transportation are retrieved from the database.

[0051] Use weather information APIs and congestion information APIs to obtain real-time weather and congestion information.

[0052] Obtain data on the means of travel and the travel time required.

[0053] Data Analysis and Filtering

[0054] server:

[0055] Based on the collected data, it filters out tourist spots and dining options that you have not visited before.

[0056] For each filtered spot, a priority is determined based on weather information and congestion status.

[0057] Travel and journey time calculations

[0058] server:

[0059] Calculate travel time to each spot based on transportation options such as public transportation, car, or walking.

[0060] Consider travel time between each spot and adjust your schedule so that the total time is not too long.

[0061] Plan Generation and Optimization

[0062] server:

[0063] Rankings are based on information such as the rating of each spot, time, and distance traveled.

[0064] Arrange spots in the optimal order to generate an efficient travel plan.

[0065] If necessary, it will reflect real-time data and allow you to dynamically change your plan during your trip.

[0066] View Plans

[0067] Device:

[0068] The optimum travel plan sent from the server is displayed to the user.

[0069] Detailed information about each spot (address, opening hours, ratings, expected weather, and crowd levels) is also provided.

[0070] Information is presented in the form of maps and schedules to make it easier for users to check their travel plans.

[0071] Specific operation example

[0072] 1. User Settings:

[0073] User: Enters into the device, "I want to go sightseeing, experience art, and eat local cuisine."

[0074] Terminal: Parses input and sends it to the server.

[0075] 2. Information Collection:

[0076] Server: Checks past visit history and retrieves unvisited tourist spots and dining options from the database.

[0077] Server: Get the current weather conditions using the weather information API (e.g. sunny).

[0078] Server: Obtain congestion status using the congestion information API (e.g., not crowded).

[0079] 3. Data analysis and schedule calculation:

[0080] Server: Calculates the travel method and travel time based on the filtered spots (e.g., 30 minutes from Tokyo Station to Ueno Museum).

[0081] 4. Generate the plan:

[0082] Server: Generates a plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which to visit them (e.g., Tokyo Tower in the morning, Ueno Museum in the afternoon, and a ramen restaurant in Asakusa for lunch).

[0083] 5. Present the plan:

[0084] Terminal: Displays the game plan to the user along with a map and provides detailed information.

[0085] This will enable users to easily create efficient and comprehensive travel plans, further increasing the enjoyment of their trip.The system of the present invention can reflect users' requests in real time and dynamically change plans, making it possible to provide a flexible travel experience.

[0086] The processing flow will be explained below.

[0087] Step 1:

[0088] User:

[0089] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[0090] Step 2:

[0091] Device:

[0092] The entered user activity information is analyzed and classified into categories (tourism, art, gourmet).

[0093] The analysis results are sent to the server.

[0094] Step 3:

[0095] server:

[0096] Receives the input data sent.

[0097] The user's past visit history is retrieved from the database.

[0098] Check the user's current location.

[0099] Step 4:

[0100] server:

[0101] Use the weather information API to get the weather forecast for the planned travel date.

[0102] Use the crowd information API to obtain information on the crowding status of tourist attractions and dining spots.

[0103] Step 5:

[0104] server:

[0105] It checks past visit history and lists tourist spots and dining options the user has not yet visited.

[0106] Narrow down suitable locations to visit based on weather information and crowding levels.

[0107] Step 6:

[0108] server:

[0109] Taking into account the user's mode of transportation (walking, car, public transportation), the travel time to each candidate location is calculated.

[0110] Calculate the travel time for each mode of transportation and reflect it in the schedule.

[0111] Step 7:

[0112] server:

[0113] Rankings are made based on the ratings of each spot (reviews, number of stars, etc.), and the top spots are selected.

[0114] Optimize the order of visits and generate efficient travel plans, taking into account travel time and travel time.

[0115] Step 8:

[0116] server:

[0117] An optimized itinerary is generated and sent to the device, including detailed information (address, opening hours, ratings, weather forecast, and crowding status).

[0118] Step 9:

[0119] Device:

[0120] The submitted itinerary is displayed to the user.

[0121] The plan details are presented in the form of maps and schedules.

[0122] If the user has any additional questions or changes they would like to make, they submit them to the server.

[0123] Step 10:

[0124] server:

[0125] Receive user feedback and adjust / update your itinerary again if necessary.

[0126] Send the new plan to the device and have it reload.

[0127] As a result, users can use the system to efficiently plan their trips, and real-time information updates enable a flexible travel experience.

[0128] Example 1

[0129] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0130] Conventional travel planning support systems have struggled to automatically generate efficient travel plans that match the user's preferences and interests, and to optimize them in real time. Furthermore, technology for quickly and effectively analyzing the wide variety of requests entered by users and incorporating them into travel plans has not yet been developed. This has resulted in the problem of users having to manually search and organize information and create travel plans, which is time-consuming.

[0131] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0132] In this invention, the server includes an input means for inputting activities the user wishes to engage in during their trip, a data collection means for collecting data on the user's past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots that best suit the user's preferences based on the collected data, a display means for presenting the suggested travel plan to the user, a means for analyzing the user's input and classifying it into activity categories using natural language processing, and a means for determining the priority of each spot based on the collected data and generating an efficient travel plan. This makes it possible to automatically generate an optimal travel plan based on the user's requests and dynamically optimize the plan in real time.

[0133] The "input means" refers to a device or function that allows a user to input the activities that the user wants to do during the trip.

[0134] "Data collection means" refers to devices or functions that collect data regarding a user's past visit history, travel time, current location, weather, congestion, and means of transportation.

[0135] "Proposal means" refers to devices or functions that select and suggest tourist facilities and dining options that best suit the user's needs based on collected data.

[0136] "Display means" refers to a device or function for visually presenting the proposed travel plan to the user.

[0137] "Natural language processing" is an artificial intelligence technique used to analyze user input and classify it into activity categories.

[0138] "Filtering" is the process of extracting information that meets specific conditions from a large amount of data.

[0139] The "priority determination means" is a device or function for determining the order of visits and importance of each spot based on the collected data.

[0140] The "plan generation means" refers to a device or function for automatically creating an efficient travel plan.

[0141] The present invention relates to a travel planning support system that provides an optimal travel plan based on user input. The system includes an input unit, a data collection unit, a proposal unit, a display unit, a natural language processing unit, a filtering unit, a priority determination unit, and a plan generation unit.

[0142] System Programming and Processing

[0143] The program of this system includes the following hardware and software.

[0144] Input method: Using a smartphone or tablet application, users input the activities they want to do during their trip.

[0145] Data collection method: Data such as past visit history, current location, real-time weather information, congestion information, and travel time is collected using a database on the server and external APIs (e.g., OpenWeatherMap API, Google (registered trademark) Maps API).

[0146] Natural language processing tools: Using a generative AI model, it analyzes user input and categorizes it into categories such as tourism, art, and food.

[0147] Filtering method: Based on the collected data, we filter out tourist attractions and dining spots that the user has not visited before.

[0148] Priority determination method: Determine the priority of each filtered spot based on weather information and crowding.

[0149] Plan generation method: Based on the priority, arrange tourist spots in the optimal order and generate an efficient travel plan.

[0150] Display method: The generated travel plan is displayed to the user via a smartphone or tablet application.

[0151] Program processing

[0152] Users start the system and input the activities they want to do during their trip (e.g., "sightseeing," "art," "gourmet") The application on the device analyzes the input, classifies the activities into categories using natural language processing, and sends the data to the server.

[0153] The server retrieves past visit history from the database and collects current location and real-time weather and congestion information via an external API. It also obtains the mode of transportation used by the user and the travel time required. Based on this data, filtering is performed to extract tourist spots and dining spots that have not been visited before. The server then prioritizes destinations, taking into account weather and congestion information.

[0154] The server calculates travel time to each spot based on the mode of transportation (train, bus, car, walking, etc.) and creates an efficient schedule taking into account travel time between each spot.Finally, it selects highly rated tourist spots and dining spots based on the user's preferences and generates a plan that visits them in the optimal order.

[0155] The generated itinerary is sent to the device and displayed to the user. Detailed information about the itinerary and each spot (address, opening hours, ratings, expected weather, and congestion status) is provided in map and schedule format for easy viewing by the user.

[0156] Specific examples

[0157] 1. User Settings:

[0158] User: Enters into the device, "I want to go sightseeing, experience art, and eat local cuisine."

[0159] 2. Information Collection:

[0160] Server: Checks past visit history and retrieves unvisited tourist spots and dining spots from the database. For example, if the user has visited Sensoji Temple in the past, Skytree, which has not yet been visited, will be suggested.

[0161] Server: Get the current weather conditions using the weather information API (e.g. sunny).

[0162] Server: Obtain the congestion status using the congestion information API (e.g., Ueno Museum is not crowded).

[0163] 3. Data analysis and prioritization:

[0164] Server: Based on the filtered spots, calculate the travel method and travel time using Google Maps API (e.g., 30 minutes from Tokyo Station to Ueno Museum).

[0165] 4. Generate the plan:

[0166] Server: Generates a plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which to visit them (e.g., Tokyo Tower in the morning, Ueno Museum in the afternoon, and a ramen restaurant in Asakusa for lunch).

[0167] 5. Present the plan:

[0168] Terminal: Displays the itinerary to the user along with a map and provides detailed information.

[0169] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0170] Step 1: Accepting User Input

[0171] User: The user inputs the activities they would like to do during their trip (e.g., "I want to do some sightseeing," "I want to experience art," "I want to try local cuisine") into an application on their smartphone or tablet.

[0172] Terminal: The terminal receives the input information, analyzes it, and classifies each activity into a category (sightseeing, art, gourmet, etc.). This analysis is performed using natural language processing technology. The analysis results are then sent to the server.

[0173] Input: User activity input ("I want to go sightseeing," "I want to experience art," "I want to eat local cuisine")

[0174] Output: Categorized activity information (sightseeing, art, gourmet)

[0175] Step 2: Data collection

[0176] Server: The server retrieves past visit history and current location information from the database. It also uses external APIs (e.g., OpenWeatherMap API, Google Maps API) to collect real-time weather information, congestion information, transportation methods, and travel times.

[0177] Input: Categorized activity information

[0178] Output: Past visit history, current location, weather information, congestion information, transportation method and travel time data

[0179] Step 3: Data analysis and filtering

[0180] Server: Based on the collected data, it filters tourist spots and dining spots that the user has not visited before. Specifically, it compares the user's past visit history with the user's current location information and lists unvisited spots.

[0181] Input: Past visit history, current location, weather information, congestion information, transportation method and travel time data

[0182] Output: A filtered list of places to visit and eat

[0183] Step 4: Prioritize

[0184] Server: Determines the priority of each filtered spot based on weather and congestion information. For example, it applies rules such as prioritizing indoor spots on rainy days.

[0185] Input: Filtered list of attractions and dining options, weather information, and crowd information

[0186] Output: A prioritized list of places to see and eat

[0187] Step 5: Calculate travel time and journey time

[0188] Server: Calculates travel time to each location for each mode of transportation. Specifically, it uses the Google Maps API to obtain travel times based on modes of transportation such as public transportation, car, and walking, and adjusts travel schedules between each location.

[0189] Input: A prioritized list of places to see and eat

[0190] Output: Travel time and travel time data between each location

[0191] Step 6: Generate and optimize the plan

[0192] Server: Generates an efficient travel plan by arranging each spot in the optimal order, taking into account the rating of each spot, the duration of stay, and the travel distance. If necessary, the plan can be dynamically modified to reflect real-time data.

[0193] Input: Travel time and travel time between each spot, rating of each spot, and duration of stay

[0194] Output: Optimal travel plan

[0195] Step 7: View your plan

[0196] Terminal: The optimal travel plan sent from the server is displayed to the user. For easy viewing, the travel plan and detailed information about each spot (address, opening hours, rating, forecast weather, and congestion status) are provided in map and schedule format.

[0197] Input: Best Travel Plan

[0198] Output: A travel plan presented to the user with details about each spot

[0199] (Application example 1)

[0200] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0201] Conventional travel planning support systems suggest tourist attractions and dining options for users during their trips, but lack integration with food delivery services, allowing travelers to order meals and have them delivered to designated locations. This makes it difficult for users to enjoy an optimal dining experience while traveling without any hassle. Furthermore, systems that dynamically modify travel plans using real-time information such as weather and congestion are often inadequate. As a result, these systems are unable to provide flexible travel plans for travelers, making it difficult to adapt to unexpected situations and traffic jams.

[0202] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0203] In this invention, the server includes an input means for inputting activities the user wishes to engage in during the trip, a data collection means for collecting data on the user's past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots that best suit the user's preferences based on the collected data, a display means for presenting the suggested travel plan to the user, and a means for collecting and suggesting data on food delivery services for ordering meals for travelers and having them delivered at specified locations. This makes it possible to provide a comprehensive and dynamic travel plan that includes meal arrangements during the trip.

[0204] A "travel planning support system" is a system that selects and suggests the most suitable tourist facilities and dining spots based on the activities that the user wants to do during their trip, and presents the plan.

[0205] "Input means" refers to a device or interface that allows a user to input the activities they wish to undertake during their trip.

[0206] "Data collection means" refers to means that has the function of collecting data regarding a user's past visit history, travel time, current location, weather, congestion, and means of transportation.

[0207] The "suggestion means" is a means that has the function of selecting and suggesting tourist facilities and dining spots that best suit the user's needs based on the collected data.

[0208] "Display" refers to a device or interface for visually displaying a proposed travel plan to a user.

[0209] A "food delivery service" is a platform that provides meal-related services that allow users to order and have food delivered to a location they specify.

[0210] "Weather information" refers to meteorological data that is updated in real time and is used for planning and dynamically changing travel plans.

[0211] "Crowding information" refers to data on the congestion status of planned visits and travel routes, and is updated in real time.

[0212] "Transportation" refers to the means by which users travel during their trip, such as public transportation, car, or walking.

[0213] "Past visit history" refers to data about tourist attractions and dining options that users have visited in the past.

[0214] A "dynamically changing means" is a means that has the function of changing a travel plan to reflect real-time data.

[0215] This invention provides users with efficient and optimal travel plans and comprehensive support, including meal arrangements, through a travel planning support system. The system consists of a smartphone or other device used by the user, a server for analyzing and processing data, and an interface for proposing and displaying travel plans.

[0216] System Programming and Processing

[0217] Initialization and accepting user input

[0218] User:

[0219] The user starts up the device and inputs the activities they would like to do during their trip, such as "I want to do some sightseeing," "I want to experience art," or "I want to try local cuisine."

[0220] Device:

[0221] The information entered is analyzed, and each activity is classified into a category. The analysis results are sent to the server.

[0222] Information gathering

[0223] server:

[0224] Data on past visit history, current location, and mode of transportation is retrieved from the database.

[0225] Use weather information APIs and congestion information APIs (e.g., OpenWeatherMap API and Google Maps Traffic API) to obtain real-time weather and congestion information.

[0226] Obtain available dining destinations from a database of partner food delivery services.

[0227] Data Analysis and Filtering

[0228] server:

[0229] The collected data is analyzed to filter out tourist spots and dining options that users have not visited before.

[0230] The priority of each filtered spot is determined based on weather information and congestion status.

[0231] Travel time and order timing calculations

[0232] server:

[0233] Uses Google Maps API to calculate travel time to each location.

[0234] Calculate the timing of your order so that your meal will be delivered when you arrive at your destination.

[0235] Plan Generation and Optimization

[0236] server:

[0237] The system generates a travel plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which you visit them.

[0238] The generated plan includes the order of places to visit and events, as well as when to order meals.

[0239] If necessary, itineraries can be dynamically modified to reflect real-time data.

[0240] View Plans

[0241] Device:

[0242] The optimal travel plan sent from the server is displayed to the user in the form of a map and schedule.

[0243] It also provides detailed information about each spot and restaurant (address, opening hours, ratings, expected weather, and crowd levels).

[0244] Specific example explanation

[0245] During initialization and user input reception, the user enters into the device, "I want to go sightseeing, experience art, and eat local cuisine." The device analyzes the user's input and sends the results to the server. The server collects data based on past visit history and current location, and obtains real-time data using weather information APIs and congestion information APIs.

[0246] For example, the server calculates the travel time from Tokyo Station to the Ueno Museum of Art in 30 minutes, and determines when to order sushi using a food delivery service 30 minutes before departure.The server then creates a plan that allows the meal to be delivered upon arrival at the destination, generating an efficient plan that allows the user to enjoy sightseeing and dining.

[0247] Prompt Sentence Examples

[0248] Please tell us where you would like to go and what food you would like to eat during your trip.

[0249] Example input:

[0250] Tourist attractions: Historic buildings, museums, parks

[0251] Food preferences: Sushi, ramen, Japanese sweets

[0252] As described above, this invention provides users with a fulfilling travel experience. Examples of software and APIs used include OpenWeatherMap API, Google Maps API, and UberEats API. Hardware primarily involves smartphones, tablet devices, and servers.

[0253] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0254] Step 1:

[0255] Initialization and accepting user input

[0256] Users turn on their smartphones or tablets and input the activities they want to do during their trip. Examples of input include "I want to do some sightseeing," "I want to experience art," and "I want to try local cuisine."

[0257] The device receives the input information and analyzes it using a natural language processing algorithm. This analysis classifies the input content into categories such as "tourism," "art," and "food." The analysis results are sent to the server in JSON format.

[0258] Step 2:

[0259] Information gathering

[0260] The server retrieves the user's past visit history and current location from a database based on the data received from the device, and also obtains real-time weather information from the OpenWeatherMap API and congestion information from the Google Maps Traffic API.

[0261] In addition, the server obtains a list of meal delivery locations from the API of affiliated food delivery services (e.g., Uber Eats, Demae-can, etc.).

[0262] The input data used is user preferences in JSON format, past visit history from the database, real-time information obtained from the API, and information from food delivery services.

[0263] Step 3:

[0264] Data Analysis and Filtering

[0265] The server analyzes the collected data and filters out tourist attractions and dining options that users have not visited before, using the Python pandas library to process the data.

[0266] This allows the system to extract unvisited spots based on past visit history, and prioritize suitable spots taking into account weather and crowding conditions.

[0267] It takes user preferences, real-time information, and food delivery information organized in JSON format as input, and generates a prioritized list of spots as output.

[0268] Step 4:

[0269] Travel time and order timing calculations

[0270] The server uses the Google Maps API to calculate travel times to each location and also calculates food delivery order times to ensure meals arrive at the expected arrival times.

[0271] Use Python's datetime library to schedule your visits and calculate the optimal time to visit each spot.

[0272] The inputs are the user's current location, next destination, and food delivery service data, and the output is an adjusted visit schedule and order timing.

[0273] Step 5:

[0274] Plan Generation and Optimization

[0275] The server generates an optimal travel plan by taking into account the ratings of each spot, the order of visits, travel time, and meal ordering timing. It applies machine learning models using Python's scikit-learn for optimization.

[0276] If necessary, the itinerary is dynamically changed to reflect real-time data.

[0277] It takes user preferences, real-time data, and food delivery information as inputs and generates an optimized travel plan as output.

[0278] Step 6:

[0279] View Plans

[0280] The device receives the optimal travel plan sent from the server and visually displays it to the user. It uses React.js to display the information in the form of a map and schedule.

[0281] Detailed information about each spot and restaurant (address, opening hours, ratings, expected weather, and crowd levels) is also displayed.

[0282] It takes as input travel plan data from a server and as output a visual travel plan that is displayed on the user's screen.

[0283] Through the above steps, the travel planning support system can provide users with a fulfilling travel experience.

[0284] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0285] This invention is a travel plan creation support system that provides a more sophisticated travel experience by combining an emotion engine that recognizes the user's emotions. This system comprises an input means for inputting the activities the user wishes to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means.

[0286] System Programming and Processing

[0287] Initialization and accepting user input

[0288] User:

[0289] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[0290] Device:

[0291] The entered activity details are analyzed and classified into categories (tourism, art, gourmet).

[0292] The analysis results are sent to the server.

[0293] Information gathering

[0294] server:

[0295] The transmitted input data is received, and past visit history, current location, and mode of travel are retrieved from the database.

[0296] We use weather information APIs and congestion information APIs to obtain real-time weather conditions and congestion information.

[0297] Data Analysis and Filtering

[0298] server:

[0299] Based on the collected data, it filters out tourist spots and dining options that you have not visited before.

[0300] Narrow down suitable locations to visit based on weather information and crowding levels.

[0301] Travel and journey time calculations

[0302] server:

[0303] Taking into account the user's mode of transportation (walking, car, public transportation), the travel time to each candidate location is calculated.

[0304] The travel time between each spot is taken into account to calculate the required time and reflected in the schedule.

[0305] Emotional Engine Adjustment

[0306] server:

[0307] The emotion engine means is used to identify the user's current emotional state, for example by detecting the user's emotions from facial expression analysis using a camera or voice input.

[0308] Based on the identified emotional state, the system suggests activities and places that fit the user's current emotions, for example, if the user is feeling stressed, it suggests relaxation spots.

[0309] Plan Generation and Optimization

[0310] server:

[0311] Rankings are made based on information such as the rating of each spot, time, and distance traveled, and the top spots are selected.

[0312] Generate efficient travel plans by optimizing the order of visits taking into account travel time and required time.

[0313] The output of the emotion engine is reflected to create a plan optimized for the user's emotional state.

[0314] View Plans

[0315] Device:

[0316] The optimum travel plan sent from the server is displayed to the user.

[0317] Detailed information on each spot (address, opening hours, ratings, weather forecast, and crowd status) is provided.

[0318] Information is presented in the form of maps and schedules to make it easier for users to check their travel plans.

[0319] Specific operation example

[0320] 1. User Settings:

[0321] User: Enters into the device, "I want to go sightseeing, experience art, and eat local cuisine."

[0322] Terminal: Parses input and sends it to the server.

[0323] 2. Information Collection:

[0324] Server: Checks past visit history and retrieves unvisited tourist spots and dining options from the database.

[0325] Server: Get the current weather conditions using the weather information API (e.g. sunny).

[0326] Server: Obtain congestion status using the congestion information API (e.g., not crowded).

[0327] 3. Emotion Recognition with Emotion Engine:

[0328] Server: Analyzes the user's facial expressions and voice using an emotion engine to identify their current emotional state (e.g., relaxed, excited, stressed, etc.).

[0329] Server: Based on the identified emotion, suggest quiet cafes or tranquil gardens to users who need to relax.

[0330] 4. Data analysis and plan generation:

[0331] Server: Calculates the means of transportation and travel time based on the filtered spots (e.g., 30 minutes from Tokyo Station to Ueno Museum).

[0332] Server: Generates a plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which to visit them (e.g., Tokyo Tower in the morning, Ueno Museum in the afternoon, and a ramen restaurant in Asakusa for lunch).

[0333] 5. Present the plan:

[0334] Terminal: The travel plan sent from the server is displayed to the user along with a map, and detailed information is also provided.

[0335] As described above, this system can further enrich users' travel experience by reflecting their requests and feelings in real time and providing optimal travel plans.

[0336] The processing flow will be explained below.

[0337] Step 1:

[0338] User:

[0339] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[0340] Step 2:

[0341] Device:

[0342] The entered activity details are analyzed and classified into categories (tourism, art, gourmet).

[0343] The analysis results are sent to the server.

[0344] Step 3:

[0345] server:

[0346] The transmitted input data is received, and past visit history, current location, and mode of travel are retrieved from the database.

[0347] Check the user's current location.

[0348] Step 4:

[0349] server:

[0350] Use the weather information API to get the weather forecast for the planned travel date.

[0351] Use the crowd information API to obtain information on the crowding status of tourist attractions and dining spots.

[0352] Step 5:

[0353] server:

[0354] It checks past visit history and lists tourist spots and dining options the user has not yet visited.

[0355] Narrow down suitable locations to visit based on weather information and crowding levels.

[0356] Step 6:

[0357] server:

[0358] Taking into account the user's mode of transportation (walking, car, public transportation), the travel time to each candidate location is calculated.

[0359] The travel time between each spot is taken into account to calculate the required time and reflected in the schedule.

[0360] Step 7:

[0361] server:

[0362] The emotion engine means is used to identify the user's current emotional state, for example by detecting the user's emotions from facial expression analysis using a camera or voice input.

[0363] Step 8:

[0364] server:

[0365] Based on the identified emotional state, the system suggests activities and places that fit the user's current emotions, for example, if the user is feeling stressed, it suggests relaxation spots.

[0366] Step 9:

[0367] server:

[0368] Rankings are made based on information such as the rating of each spot, time, and distance traveled, and the top spots are selected.

[0369] Generate efficient travel plans by optimizing the order of visits taking into account travel time and required time.

[0370] The output of the emotion engine is reflected to create a plan optimized for the user's emotional state.

[0371] Step 10:

[0372] server:

[0373] An optimized itinerary is generated and sent to the device, including detailed information (address, opening hours, ratings, weather forecast, and crowding status).

[0374] Step 11:

[0375] Device:

[0376] The submitted itinerary is displayed to the user.

[0377] The plan is presented in detail in the form of a map or schedule, and if the user has any additional questions or requests for changes, they are sent to the server.

[0378] Step 12:

[0379] server:

[0380] Receive user feedback and adjust / update your itinerary again if necessary.

[0381] Send the new plan to your device and display it.

[0382] As a result, users can use the system to efficiently plan their trips, and with the help of the emotion engine, they can have a flexible travel experience that suits their mood at the time.

[0383] Example 2

[0384] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0385] Conventional travel planning support systems take into account data such as a user's past visit history, current location, weather, and congestion, but are unable to propose travel plans that reflect the user's emotional state. As a result, travel plans that do not match the user's mood or emotions are often proposed, resulting in an unsatisfactory travel experience. In addition, it is difficult to flexibly change plans in response to real-time changes in the environment or the user's emotions.

[0386] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an emotion engine means for recognizing the user's emotions and adjusting the travel plan based on the emotions; a means for dynamically modifying the travel plan taking into account past visit history data, weather information and congestion information updated in real time, and travel time; and an input means having a function for analyzing the user's interests and automatically classifying them into categories (sightseeing, art, gourmet, etc.) based on the interests. This makes it possible to provide an optimal travel plan that flexibly responds to the user's emotions and real-time environmental changes.

[0387] "Input means" refers to a means by which a user inputs into the system the activities that he or she wishes to undertake during his or her trip.

[0388] "Data collection means" refers to means for collecting data on a user's past visit history, travel time, current location, weather, congestion, and means of transportation.

[0389] The "proposal method" is a method for selecting and proposing tourist facilities and dining spots that best suit the user's needs based on the collected data.

[0390] The "display means" is a means for presenting the proposed travel plan to the user.

[0391] The "emotion engine means" is a means for recognizing the user's emotions and adjusting the travel plan based on those emotions.

[0392] A "travel planning support system" is a system that allows users to input the activities they wish to do during their trip, and creates and provides the optimal travel plan based on that input.

[0393] "Past visit history data" refers to data on tourist attractions and dining locations that a user has visited in the past.

[0394] "Weather information updated in real time" refers to information that acquires and updates the current weather conditions in real time.

[0395] "Crowding information" is information that indicates the current congestion status of tourist facilities and dining destinations.

[0396] "Travel time" refers to the time required for a user to travel from their current location to their destination.

[0397] This invention provides a more sophisticated travel experience by combining an emotion engine that recognizes the user's emotions in a travel plan creation support system. This system comprises an input means for the user to input the activities they wish to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and transportation means, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means.

[0398] The user starts the system and inputs the activities they wish to do during their trip (e.g., sightseeing, art, gourmet) into the terminal. The terminal analyzes the input activities and classifies them into categories (sightseeing, art, gourmet). The analysis results are sent to the server.

[0399] The server receives the input data and retrieves information on past visit history, current location, and mode of transportation from a database. It also uses a weather information API (for example, a general name for the weather information retrieval API) and a congestion information API (for example, a general name for the congestion information retrieval API) to retrieve real-time weather and congestion conditions.

[0400] The server uses the collected data to filter out tourist attractions and dining options that the user has not visited before, and then narrows down the list of suitable places to visit based on weather information and crowding conditions.

[0401] When calculating travel time and required time, the server takes into account the user's mode of transportation (walking, car, public transportation) and calculates the travel time to each candidate location. It also calculates the required time taking into account the travel time between each spot and reflects it in the schedule.

[0402] As an emotion engine, the server uses an emotion recognition model (e.g., an emotion analysis API) to identify the user's current emotional state. Specifically, it can detect the user's emotions from facial expression analysis using a camera or voice input. Based on the identified emotional state, it will suggest activities and spots that match the user's current emotions. For example, if the user is feeling stressed, it will suggest relaxation spots.

[0403] When generating a plan, the server ranks each spot based on information such as rating, travel time, and distance, and selects the top spots. It then optimizes the order of visits, taking travel time and required time into account, to generate an efficient travel plan. The server also reflects the output of the emotion engine to create a plan optimized for the user's emotional state.

[0404] Finally, the generated optimal travel plan is sent from the server to the device. The device displays the travel plan sent from the server to the user, providing detailed information about each spot (address, opening hours, ratings, weather forecast, and congestion status). The information is presented in the form of a map or schedule to make it easier for the user to check the travel plan.

[0405] As a specific example of how it works, if you enter "I want to go sightseeing, experience art, and eat local cuisine" into the prompt text, the system will categorize your requests into categories and create and present the most suitable travel plan. In this way, the system reflects the user's desires and emotions in real time and provides the most suitable travel plan, further enriching the user's travel experience.

[0406] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0407] Step 1: Initialize and accept user input

[0408] User:

[0409] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[0410] Input: Text information that a user types into a terminal.

[0411] Output: Parsed activity.

[0412] Device:

[0413] The activities entered by the user are analyzed and classified into categories (sightseeing, art, gourmet). For example, "I want to see Tokyo Tower, I want to go to the Ueno Museum, I want to eat ramen in Asakusa."

[0414] Input: Text information entered by the user.

[0415] Data processing: Text analysis and classification into categories.

[0416] Output: Categorized activities.

[0417] The analysis results are sent to the server.

[0418] Step 2: Gather information

[0419] server:

[0420] The transmitted input data is received, and information on the user's past visit history, current location, and means of transportation from the departure point is retrieved from the database.

[0421] Input: Analysis results sent from the device.

[0422] Data processing: Retrieve data from user database.

[0423] Output: User's past visit history and current location information.

[0424] Use a weather information API (e.g., a general name for the weather information API) or a congestion information API (e.g., a general name for the congestion information API) to obtain real-time weather conditions and congestion status.

[0425] Input: Acquired past visit history and current location information.

[0426] Data processing: Obtain weather and congestion information from the API.

[0427] Output: Real-time weather and traffic conditions.

[0428] Step 3: Data analysis and filtering

[0429] server:

[0430] Based on the collected data, it filters out tourist spots and dining options that the user has not visited before.

[0431] Input: All collected data (past visit history, current location, weather information, crowd status).

[0432] Data processing: Filter out unvisited spots.

[0433] Output: A filtered list of attractions and places to eat.

[0434] Narrow down potential places to visit based on weather information and crowding levels.

[0435] Input: Filtered list with weather and crowd information.

[0436] Data processing: Narrow down spots suitable for weather and crowd conditions.

[0437] Output: A narrowed down list of potential locations.

[0438] Step 4: Calculate travel time and journey time

[0439] server:

[0440] The travel time to each candidate location is calculated taking into account the user's mode of transportation (walking, car, public transportation). For example, "It takes 30 minutes to walk from Tokyo Station to Ueno Museum of Art."

[0441] Input: User's travel information and a narrowed list of possible locations.

[0442] Data processing: Calculate travel time to each candidate location.

[0443] Output: A list of travel times to each candidate location.

[0444] The travel time between each spot is taken into account to calculate the required time and reflected in the schedule.

[0445] Input: Travel time list and candidate location list.

[0446] Data processing: Calculate travel time between each spot and reflect it in the schedule.

[0447] Output: Visit schedule with duration.

[0448] Step 5: Emotional Engine Adjustment

[0449] server:

[0450] The emotion engine (e.g., emotion analysis API) is used to identify the user's current emotional state. Specifically, the user's emotions are detected from facial expression analysis using a camera or voice input.

[0451] Input: User's facial and voice data.

[0452] Data processing: Emotional state is analyzed using an emotion engine.

[0453] Output: The user's current emotional state.

[0454] Based on the identified emotional state, the system suggests activities and places that fit the user's current emotions, for example, if the user is feeling stressed, it suggests relaxation spots.

[0455] Input: User's emotional state and schedule.

[0456] Data manipulation: Adjust your schedule based on your emotional state.

[0457] Output: An emotion-appropriate visit schedule.

[0458] Step 6: Generate and optimize the plan

[0459] server:

[0460] Rankings are made based on information such as the rating of each spot, time, and distance traveled, and the top spots are selected.

[0461] Input: Information such as rating of the spot, time, and distance traveled.

[0462] Data processing: Ranking is carried out and the top spots are selected.

[0463] Output: A list of selected top spots.

[0464] Taking into account travel time and travel time, the order of visits is optimized to generate an efficient itinerary.

[0465] Input: Top spot list, travel time, and travel time.

[0466] Data processing: Optimize visit order and generate travel plans.

[0467] Output: An efficient travel plan.

[0468] The output of the emotion engine is reflected to create a plan optimized for the user's emotional state.

[0469] Input: Emotionally appropriate visit schedule.

[0470] Data processing: Create a plan optimized for your emotional state.

[0471] Output: Optimized trip plan.

[0472] Step 7: View your plan

[0473] Device:

[0474] The optimum travel plan sent from the server is displayed to the user.

[0475] Input: The itinerary sent from the server.

[0476] Data processing: Formatting the travel plan for display.

[0477] Output: The plan that is displayed to the user.

[0478] Detailed information on each spot (address, opening hours, ratings, weather forecast, and crowd status) is provided.

[0479] Input: Spot information within the travel plan.

[0480] Data processing: Incorporating detailed information into the display.

[0481] Output: A display with detailed information.

[0482] Information is presented in the form of maps and schedules to make it easier for users to check their travel plans.

[0483] Input: Travel plan and details.

[0484] Data processing: Formatting into map or schedule format screens.

[0485] Output: Plans displayed in map and schedule format.

[0486] This allows the system to reflect users' requests and feelings in real time and provide optimal travel plans.

[0487] (Application example 2)

[0488] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0489] Conventional travel planning support systems could suggest optimal tourist attractions and dining options by taking into account the user's past visit history, weather conditions, and congestion levels, but they did not provide plans that reflected the user's current emotional state. This made it difficult to generate optimal travel plans that addressed emotions such as stress and excitement that users felt during their trip. This is needed to further enhance the travel experience and increase satisfaction.

[0490] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0491] In this invention, the server includes an input means for inputting the activities the user wishes to engage in during their trip, a data collection means for collecting data on the user's past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots that best suit the user's preferences based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means for recognizing the user's emotions in real time and making suggestions based on those emotions. This makes it possible to provide a travel plan that matches the user's current emotional state, further enriching the travel experience.

[0492] Definitions of important words

[0493] A "travel planning support system" is a system that supports the activities that users want to undertake while traveling, suggesting the most suitable tourist facilities and places to eat by taking into consideration data such as the user's past visit history, current location, weather, congestion level, and mode of transportation.

[0494] "Input means" refers to a device or interface that allows a user to input the activities and preferences they wish to undertake during their trip.

[0495] "Data collection means" refers to means for collecting information about a user's past visit history, current location, weather, congestion, and means of transportation.

[0496] The "suggestion method" is a method of selecting and suggesting the most suitable tourist facilities and dining spots to users based on the collected data.

[0497] "Display means" refers to a means for visually presenting information such as suggested travel plans, selected tourist facilities, and dining options to users.

[0498] The "emotion engine means" is a means for recognizing the user's emotions in real time and adjusting the content of suggestions based on those emotions.

[0499] This invention is a travel plan creation support system that provides a more sophisticated travel experience by combining an emotion engine that recognizes the user's emotions. This system comprises an input means for inputting the activities the user wishes to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means.

[0500] The specific configuration and processing of a system for implementing the present invention are as follows.

[0501] Technology stack used

[0502] Hardware

[0503] Smartphone

[0504] Smart Glasses

[0505] head-mounted display

[0506] software

[0507] OpenCV for Sentiment Analysis

[0508] Google Speech-to-Text API for voice analysis

[0509] A server-side program with a unique algorithm for movie suggestions

[0510] Program processing explanation

[0511] Initialization and accepting user input

[0512] The user starts the travel planning support system and inputs the activities they would like to do during their trip. For example, they might input text such as "I want to do some sightseeing, experience art, and try local cuisine" into the device. This input is done via a smartphone, smart glasses, or a head-mounted display.

[0513] The device analyzes the entered activity details, classifies them into categories (tourism, art, gourmet), and sends the analysis results to the server.

[0514] Information gathering and emotion recognition

[0515] The server receives the input data and retrieves past visit history, current location, and mode of transportation from the database. It also obtains real-time weather and congestion information using weather information APIs and congestion information APIs.

[0516] The device uses the built-in camera and microphone to capture the user's facial expressions and voice, and recognizes emotions in real time using OpenCV and Emotion APIs (e.g., Microsoft® Face API). It also analyzes voice data with the Google Speech-to-Text API to identify emotions from speaking tone.

[0517] Generate and view plans

[0518] The server selects filtered tourist spots and dining options based on the collected data, and makes optimal suggestions based on the user's current emotional state. It generates an efficient travel plan by taking into account the ratings of each spot and the order in which they should be visited. It reflects the output of the emotion engine and creates a plan optimized for the user's emotional state.

[0519] The terminal displays the optimal travel plan sent from the server to the user and provides detailed information about each spot (address, opening hours, ratings, weather forecast, and congestion status).

[0520] Specific examples

[0521] 1. Example of user input:

[0522] "I want to see the sights, experience art, and eat local cuisine."

[0523] 2. Example prompt:

[0524] What is your current mood and what genre do you want to watch?

[0525] Example: I want to relax today, so I want to watch a comedy movie.

[0526] In this way, it is possible to provide an optimal travel plan that reflects the user's emotional state in real time.The system utilizes a generative AI model to further explore the user's needs through prompt sentences, enabling the system to achieve an optimal travel experience.

[0527] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0528] Program processing steps

[0529] Step 1:

[0530] The user starts the travel planning support system and inputs the activities and requests he / she wishes to perform during the trip.

[0531] Input: Activities such as "I want to go sightseeing, experience art, and eat local cuisine"

[0532] Output: Categorization of analyzed activities (sightseeing, art, gourmet)

[0533] Specific operation: Data entered into a smartphone, smart glasses, or head-mounted display is analyzed using natural language processing and classified into categories. The analysis results are then sent to a server.

[0534] Step 2:

[0535] The server receives the input data and retrieves past visit history, current location, and mode of transportation from the database. It also uses weather information API and congestion information API to obtain real-time weather and congestion information.

[0536] Input: User input data, visit history, current location, weather API, congestion information API

[0537] Output: Obtained weather and congestion information

[0538] Specific operation: The server executes database queries to obtain past visit history and current location data, and calls weather APIs and congestion information APIs to obtain real-time data.

[0539] Step 3:

[0540] The device uses the built-in camera and microphone to capture the user's facial expressions and voice, and recognizes emotions in real time using OpenCV and the Emotion API. The voice data is also analyzed using the Google Speech-to-Text API to identify emotions from the tone of speech.

[0541] Input: User's facial expression data, voice data

[0542] Output: Perceived emotional state (e.g., relaxed, excited, stressed)

[0543] What it does: It activates the camera and microphone, performs facial expression analysis using OpenCV and the Emotion API, and analyzes audio data using the Google Speech-to-Text API to determine emotions based on the tone of the voice.

[0544] Step 4:

[0545] Based on the collected data, the server selects filtered tourist spots and dining options, making optimal suggestions based on the user's current emotional state.

[0546] Input: Filtered tourist spots, dining options, and the user's emotional state

[0547] Output: A ranked list of the best places to see and eat

[0548] What it does: It takes into account the output of the emotion engine and applies an algorithm to generate rankings of tourist attractions and dining destinations based on past visit history and current situation data.

[0549] Step 5:

[0550] The server takes into account the ratings of each spot and the order in which they should be visited, generates an efficient travel plan, and sends it to the terminal.

[0551] Input: Ranking list of spots, rating data, visit order

[0552] Output: Optimized itinerary

[0553] Specific operation: Calculates the most efficient visiting order based on the rating, travel time, and duration of stay of each spot, and generates an optimal travel plan.

[0554] Step 6:

[0555] The terminal displays the optimal travel plan sent from the server to the user and provides detailed information about each spot (address, opening hours, ratings, weather forecast, and congestion status).

[0556] Input: Optimized itinerary, detailed information for each spot

[0557] Output: The best itinerary and details displayed to the user

[0558] What it does: It visually displays the itinerary and detailed information of each spot on a map and in schedule format, making it easy for users to check.

[0559] As a result, it is possible to realize a system that provides optimal travel plans that reflect the user's emotional state in real time, further enriching the travel experience.

[0560] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0561] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0562] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0563] [Second embodiment]

[0564] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0565] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0566] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0567] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0568] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0569] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0570] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0571] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0572] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0573] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0574] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0575] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0576] The present invention is a travel planning support system that provides an optimal travel plan based on user input. This system has an input means for inputting activities the user wants to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and transportation methods, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, and a display means for presenting the suggested travel plan to the user.

[0577] System Programming and Processing

[0578] Initialization and accepting user input

[0579] User:

[0580] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[0581] Device:

[0582] The input information is analyzed and classified into each activity category.

[0583] The analysis results are sent to the server.

[0584] Information gathering

[0585] server:

[0586] Past visit history, current location, and mode of transportation are retrieved from the database.

[0587] Use weather information APIs and congestion information APIs to obtain real-time weather and congestion information.

[0588] Obtain data on the means of travel and the travel time required.

[0589] Data Analysis and Filtering

[0590] server:

[0591] Based on the collected data, it filters out tourist spots and dining options that you have not visited before.

[0592] For each filtered spot, a priority is determined based on weather information and congestion status.

[0593] Travel and journey time calculations

[0594] server:

[0595] Calculate travel time to each spot based on transportation options such as public transportation, car, or walking.

[0596] Consider travel time between each spot and adjust your schedule so that the total time is not too long.

[0597] Plan Generation and Optimization

[0598] server:

[0599] Rankings are based on information such as the rating of each spot, time, and distance traveled.

[0600] Arrange spots in the optimal order to generate an efficient travel plan.

[0601] If necessary, it will reflect real-time data and allow you to dynamically change your plan during your trip.

[0602] View Plans

[0603] Device:

[0604] The optimum travel plan sent from the server is displayed to the user.

[0605] Detailed information about each spot (address, opening hours, ratings, expected weather, and crowd levels) is also provided.

[0606] Information is presented in the form of maps and schedules to make it easier for users to check their travel plans.

[0607] Specific operation example

[0608] 1. User Settings:

[0609] User: Enters into the device, "I want to go sightseeing, experience art, and eat local cuisine."

[0610] Terminal: Parses input and sends it to the server.

[0611] 2. Information Collection:

[0612] Server: Checks past visit history and retrieves unvisited tourist spots and dining options from the database.

[0613] Server: Get the current weather conditions using the weather information API (e.g. sunny).

[0614] Server: Obtain congestion status using the congestion information API (e.g., not crowded).

[0615] 3. Data analysis and schedule calculation:

[0616] Server: Calculates the travel method and travel time based on the filtered spots (e.g., 30 minutes from Tokyo Station to Ueno Museum).

[0617] 4. Generate the plan:

[0618] Server: Generates a plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which to visit them (e.g., Tokyo Tower in the morning, Ueno Museum in the afternoon, and a ramen restaurant in Asakusa for lunch).

[0619] 5. Present the plan:

[0620] Terminal: Displays the game plan to the user along with a map and provides detailed information.

[0621] This will enable users to easily create efficient and comprehensive travel plans, further increasing the enjoyment of their trip.The system of the present invention can reflect users' requests in real time and dynamically change plans, making it possible to provide a flexible travel experience.

[0622] The processing flow will be explained below.

[0623] Step 1:

[0624] User:

[0625] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[0626] Step 2:

[0627] Device:

[0628] The entered user activity information is analyzed and classified into categories (tourism, art, gourmet).

[0629] The analysis results are sent to the server.

[0630] Step 3:

[0631] server:

[0632] Receives the input data sent.

[0633] The user's past visit history is retrieved from the database.

[0634] Check the user's current location.

[0635] Step 4:

[0636] server:

[0637] Use the weather information API to get the weather forecast for the planned travel date.

[0638] Use the crowd information API to obtain information on the crowding status of tourist attractions and dining spots.

[0639] Step 5:

[0640] server:

[0641] It checks past visit history and lists tourist spots and dining options the user has not yet visited.

[0642] Narrow down suitable locations to visit based on weather information and crowding levels.

[0643] Step 6:

[0644] server:

[0645] Taking into account the user's mode of transportation (walking, car, public transportation), the travel time to each candidate location is calculated.

[0646] Calculate the travel time for each mode of transportation and reflect it in the schedule.

[0647] Step 7:

[0648] server:

[0649] Rankings are made based on the ratings of each spot (reviews, number of stars, etc.), and the top spots are selected.

[0650] Optimize the order of visits and generate efficient travel plans, taking into account travel time and travel time.

[0651] Step 8:

[0652] server:

[0653] An optimized itinerary is generated and sent to the device, including detailed information (address, opening hours, ratings, weather forecast, and crowding status).

[0654] Step 9:

[0655] Device:

[0656] The submitted itinerary is displayed to the user.

[0657] The plan details are presented in the form of maps and schedules.

[0658] If the user has any additional questions or changes they would like to make, they submit them to the server.

[0659] Step 10:

[0660] server:

[0661] Receive user feedback and adjust / update your itinerary again if necessary.

[0662] Send the new plan to the device and have it reload.

[0663] As a result, users can use the system to efficiently plan their trips, and real-time information updates enable a flexible travel experience.

[0664] Example 1

[0665] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0666] Conventional travel planning support systems have struggled to automatically generate efficient travel plans that match the user's preferences and interests, and to optimize them in real time. Furthermore, technology for quickly and effectively analyzing the wide variety of requests entered by users and incorporating them into travel plans has not yet been developed. This has resulted in the problem of users having to manually search and organize information and create travel plans, which is time-consuming.

[0667] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0668] In this invention, the server includes an input means for inputting activities the user wishes to engage in during their trip, a data collection means for collecting data on the user's past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots that best suit the user's preferences based on the collected data, a display means for presenting the suggested travel plan to the user, a means for analyzing the user's input and classifying it into activity categories using natural language processing, and a means for determining the priority of each spot based on the collected data and generating an efficient travel plan. This makes it possible to automatically generate an optimal travel plan based on the user's requests and dynamically optimize the plan in real time.

[0669] The "input means" refers to a device or function that allows a user to input the activities that the user wants to do during the trip.

[0670] "Data collection means" refers to devices or functions that collect data regarding a user's past visit history, travel time, current location, weather, congestion, and means of transportation.

[0671] "Proposal means" refers to devices or functions that select and suggest tourist facilities and dining options that best suit the user's needs based on collected data.

[0672] "Display means" refers to a device or function for visually presenting the proposed travel plan to the user.

[0673] "Natural language processing" is an artificial intelligence technique used to analyze user input and classify it into activity categories.

[0674] "Filtering" is the process of extracting information that meets specific conditions from a large amount of data.

[0675] The "priority determination means" is a device or function for determining the order of visits and importance of each spot based on the collected data.

[0676] The "plan generation means" refers to a device or function for automatically creating an efficient travel plan.

[0677] The present invention relates to a travel planning support system that provides an optimal travel plan based on user input. The system includes an input unit, a data collection unit, a proposal unit, a display unit, a natural language processing unit, a filtering unit, a priority determination unit, and a plan generation unit.

[0678] System Programming and Processing

[0679] The program of this system includes the following hardware and software.

[0680] Input method: Using a smartphone or tablet application, users input the activities they want to do during their trip.

[0681] Data collection method: Data such as past visit history, current location, real-time weather information, congestion information, and travel time is collected using a database on the server and external APIs (e.g., OpenWeatherMap API, Google Maps API).

[0682] Natural language processing tools: Using a generative AI model, it analyzes user input and categorizes it into categories such as tourism, art, and food.

[0683] Filtering method: Based on the collected data, we filter out tourist attractions and dining spots that the user has not visited before.

[0684] Priority determination method: Determine the priority of each filtered spot based on weather information and crowding.

[0685] Plan generation method: Based on the priority, arrange tourist spots in the optimal order and generate an efficient travel plan.

[0686] Display method: The generated travel plan is displayed to the user via a smartphone or tablet application.

[0687] Program processing

[0688] Users start the system and input the activities they want to do during their trip (e.g., "sightseeing," "art," "gourmet") The application on the device analyzes the input, classifies the activities into categories using natural language processing, and sends the data to the server.

[0689] The server retrieves past visit history from the database and collects current location and real-time weather and congestion information via an external API. It also obtains the mode of transportation used by the user and the travel time required. Based on this data, filtering is performed to extract tourist spots and dining spots that have not been visited before. The server then prioritizes destinations, taking into account weather and congestion information.

[0690] The server calculates travel time to each spot based on the mode of transportation (train, bus, car, walking, etc.) and creates an efficient schedule taking into account travel time between each spot.Finally, it selects highly rated tourist spots and dining spots based on the user's preferences and generates a plan that visits them in the optimal order.

[0691] The generated itinerary is sent to the device and displayed to the user. Detailed information about the itinerary and each spot (address, opening hours, ratings, expected weather, and congestion status) is provided in map and schedule format for easy viewing by the user.

[0692] Specific examples

[0693] 1. User Settings:

[0694] User: Enters into the device, "I want to go sightseeing, experience art, and eat local cuisine."

[0695] 2. Information Collection:

[0696] Server: Checks past visit history and retrieves unvisited tourist spots and dining spots from the database. For example, if the user has visited Sensoji Temple in the past, Skytree, which has not yet been visited, will be suggested.

[0697] Server: Get the current weather conditions using the weather information API (e.g. sunny).

[0698] Server: Obtain the congestion status using the congestion information API (e.g., Ueno Museum is not crowded).

[0699] 3. Data analysis and prioritization:

[0700] Server: Based on the filtered spots, calculate the travel method and travel time using Google Maps API (e.g., 30 minutes from Tokyo Station to Ueno Museum).

[0701] 4. Generate the plan:

[0702] Server: Generates a plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which to visit them (e.g., Tokyo Tower in the morning, Ueno Museum in the afternoon, and a ramen restaurant in Asakusa for lunch).

[0703] 5. Present the plan:

[0704] Terminal: Displays the itinerary to the user along with a map and provides detailed information.

[0705] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0706] Step 1: Accepting User Input

[0707] User: The user inputs the activities they would like to do during their trip (e.g., "I want to do some sightseeing," "I want to experience art," "I want to try local cuisine") into an application on their smartphone or tablet.

[0708] Terminal: The terminal receives the input information, analyzes it, and classifies each activity into a category (sightseeing, art, gourmet, etc.). This analysis is performed using natural language processing technology. The analysis results are then sent to the server.

[0709] Input: User activity input ("I want to go sightseeing," "I want to experience art," "I want to eat local cuisine")

[0710] Output: Categorized activity information (sightseeing, art, gourmet)

[0711] Step 2: Data collection

[0712] Server: The server retrieves past visit history and current location information from the database. It also uses external APIs (e.g., OpenWeatherMap API, Google Maps API) to collect real-time weather information, congestion information, transportation methods, and travel times.

[0713] Input: Categorized activity information

[0714] Output: Past visit history, current location, weather information, congestion information, transportation method and travel time data

[0715] Step 3: Data analysis and filtering

[0716] Server: Based on the collected data, it filters tourist spots and dining spots that the user has not visited before. Specifically, it compares the user's past visit history with the user's current location information and lists unvisited spots.

[0717] Input: Past visit history, current location, weather information, congestion information, transportation method and travel time data

[0718] Output: A filtered list of places to visit and eat

[0719] Step 4: Prioritize

[0720] Server: Determines the priority of each filtered spot based on weather and congestion information. For example, it applies rules such as prioritizing indoor spots on rainy days.

[0721] Input: Filtered list of attractions and dining options, weather information, and crowd information

[0722] Output: A prioritized list of places to see and eat

[0723] Step 5: Calculate travel time and journey time

[0724] Server: Calculates travel time to each location for each mode of transportation. Specifically, it uses the Google Maps API to obtain travel times based on modes of transportation such as public transportation, car, and walking, and adjusts travel schedules between each location.

[0725] Input: A prioritized list of places to see and eat

[0726] Output: Travel time and travel time data between each location

[0727] Step 6: Generate and optimize the plan

[0728] Server: Generates an efficient travel plan by arranging each spot in the optimal order, taking into account the rating of each spot, the duration of stay, and the travel distance. If necessary, the plan can be dynamically modified to reflect real-time data.

[0729] Input: Travel time and travel time between each spot, rating of each spot, and duration of stay

[0730] Output: Optimal travel plan

[0731] Step 7: View your plan

[0732] Terminal: The optimal travel plan sent from the server is displayed to the user. For easy viewing, the travel plan and detailed information about each spot (address, opening hours, rating, forecast weather, and congestion status) are provided in map and schedule format.

[0733] Input: Best Travel Plan

[0734] Output: A travel plan presented to the user with details about each spot

[0735] (Application example 1)

[0736] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0737] Conventional travel planning support systems suggest tourist attractions and dining options for users during their trips, but lack integration with food delivery services, allowing travelers to order meals and have them delivered to designated locations. This makes it difficult for users to enjoy an optimal dining experience while traveling without any hassle. Furthermore, systems that dynamically modify travel plans using real-time information such as weather and congestion are often inadequate. As a result, these systems are unable to provide flexible travel plans for travelers, making it difficult to adapt to unexpected situations and traffic jams.

[0738] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0739] In this invention, the server includes an input means for inputting activities the user wishes to engage in during the trip, a data collection means for collecting data on the user's past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots that best suit the user's preferences based on the collected data, a display means for presenting the suggested travel plan to the user, and a means for collecting and suggesting data on food delivery services for ordering meals for travelers and having them delivered at specified locations. This makes it possible to provide a comprehensive and dynamic travel plan that includes meal arrangements during the trip.

[0740] A "travel planning support system" is a system that selects and suggests the most suitable tourist facilities and dining spots based on the activities that the user wants to do during their trip, and presents the plan.

[0741] "Input means" refers to a device or interface that allows a user to input the activities they wish to undertake during their trip.

[0742] "Data collection means" refers to means that has the function of collecting data regarding a user's past visit history, travel time, current location, weather, congestion, and means of transportation.

[0743] The "suggestion means" is a means that has the function of selecting and suggesting tourist facilities and dining spots that best suit the user's needs based on the collected data.

[0744] "Display" refers to a device or interface for visually displaying a proposed travel plan to a user.

[0745] A "food delivery service" is a platform that provides meal-related services that allow users to order and have food delivered to a location they specify.

[0746] "Weather information" refers to meteorological data that is updated in real time and is used for planning and dynamically changing travel plans.

[0747] "Crowding information" refers to data on the congestion status of planned visits and travel routes, and is updated in real time.

[0748] "Transportation" refers to the means by which users travel during their trip, such as public transportation, car, or walking.

[0749] "Past visit history" refers to data about tourist attractions and dining options that users have visited in the past.

[0750] A "dynamically changing means" is a means that has the function of changing a travel plan to reflect real-time data.

[0751] This invention provides users with efficient and optimal travel plans and comprehensive support, including meal arrangements, through a travel planning support system. The system consists of a smartphone or other device used by the user, a server for analyzing and processing data, and an interface for proposing and displaying travel plans.

[0752] System Programming and Processing

[0753] Initialization and accepting user input

[0754] User:

[0755] The user starts up the device and inputs the activities they would like to do during their trip, such as "I want to do some sightseeing," "I want to experience art," or "I want to try local cuisine."

[0756] Device:

[0757] The information entered is analyzed, and each activity is classified into a category. The analysis results are sent to the server.

[0758] Information gathering

[0759] server:

[0760] Data on past visit history, current location, and mode of transportation is retrieved from the database.

[0761] Use weather information APIs and congestion information APIs (e.g., OpenWeatherMap API and Google Maps Traffic API) to obtain real-time weather and congestion information.

[0762] Obtain available dining destinations from a database of partner food delivery services.

[0763] Data Analysis and Filtering

[0764] server:

[0765] The collected data is analyzed to filter out tourist spots and dining options that users have not visited before.

[0766] The priority of each filtered spot is determined based on weather information and congestion status.

[0767] Travel time and order timing calculations

[0768] server:

[0769] Uses Google Maps API to calculate travel time to each location.

[0770] Calculate the timing of your order so that your meal will be delivered when you arrive at your destination.

[0771] Plan Generation and Optimization

[0772] server:

[0773] The system generates a travel plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which you visit them.

[0774] The generated plan includes the order of places to visit and events, as well as when to order meals.

[0775] If necessary, itineraries can be dynamically modified to reflect real-time data.

[0776] View Plans

[0777] Device:

[0778] The optimal travel plan sent from the server is displayed to the user in the form of a map and schedule.

[0779] It also provides detailed information about each spot and restaurant (address, opening hours, ratings, expected weather, and crowd levels).

[0780] Specific example explanation

[0781] During initialization and user input reception, the user enters into the device, "I want to go sightseeing, experience art, and eat local cuisine." The device analyzes the user's input and sends the results to the server. The server collects data based on past visit history and current location, and obtains real-time data using weather information APIs and congestion information APIs.

[0782] For example, the server calculates the travel time from Tokyo Station to the Ueno Museum of Art in 30 minutes, and determines when to order sushi using a food delivery service 30 minutes before departure.The server then creates a plan that allows the meal to be delivered upon arrival at the destination, generating an efficient plan that allows the user to enjoy sightseeing and dining.

[0783] Prompt Sentence Examples

[0784] Please tell us where you would like to go and what food you would like to eat during your trip.

[0785] Example input:

[0786] Tourist attractions: Historic buildings, museums, parks

[0787] Food preferences: Sushi, ramen, Japanese sweets

[0788] As described above, this invention provides users with a fulfilling travel experience. Examples of software and APIs used include OpenWeatherMap API, Google Maps API, and UberEats API. Hardware primarily involves smartphones, tablet devices, and servers.

[0789] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0790] Step 1:

[0791] Initialization and accepting user input

[0792] Users turn on their smartphones or tablets and input the activities they want to do during their trip. Examples of input include "I want to do some sightseeing," "I want to experience art," and "I want to try local cuisine."

[0793] The device receives the input information and analyzes it using a natural language processing algorithm. This analysis classifies the input content into categories such as "tourism," "art," and "food." The analysis results are sent to the server in JSON format.

[0794] Step 2:

[0795] Information gathering

[0796] The server retrieves the user's past visit history and current location from a database based on the data received from the device, and also obtains real-time weather information from the OpenWeatherMap API and congestion information from the Google Maps Traffic API.

[0797] In addition, the server obtains a list of meal delivery locations from the API of affiliated food delivery services (e.g., Uber Eats, Demae-can, etc.).

[0798] The input data used is user preferences in JSON format, past visit history from the database, real-time information obtained from the API, and information from food delivery services.

[0799] Step 3:

[0800] Data Analysis and Filtering

[0801] The server analyzes the collected data and filters out tourist attractions and dining options that users have not visited before, using the Python pandas library to process the data.

[0802] This allows the system to extract unvisited spots based on past visit history, and prioritize suitable spots taking into account weather and crowding conditions.

[0803] It takes user preferences, real-time information, and food delivery information organized in JSON format as input, and generates a prioritized list of spots as output.

[0804] Step 4:

[0805] Travel time and order timing calculations

[0806] The server uses the Google Maps API to calculate travel times to each location and also calculates food delivery order times to ensure meals arrive at the expected arrival times.

[0807] Use Python's datetime library to schedule your visits and calculate the optimal time to visit each spot.

[0808] The inputs are the user's current location, next destination, and food delivery service data, and the output is an adjusted visit schedule and order timing.

[0809] Step 5:

[0810] Plan Generation and Optimization

[0811] The server generates an optimal travel plan by taking into account the ratings of each spot, the order of visits, travel time, and meal ordering timing. It applies machine learning models using Python's scikit-learn for optimization.

[0812] If necessary, the itinerary is dynamically changed to reflect real-time data.

[0813] It takes user preferences, real-time data, and food delivery information as inputs and generates an optimized travel plan as output.

[0814] Step 6:

[0815] View Plans

[0816] The device receives the optimal travel plan sent from the server and visually displays it to the user. It uses React.js to display the information in the form of a map and schedule.

[0817] Detailed information about each spot and restaurant (address, opening hours, ratings, expected weather, and crowd levels) is also displayed.

[0818] It takes as input travel plan data from a server and as output a visual travel plan that is displayed on the user's screen.

[0819] Through the above steps, the travel planning support system can provide users with a fulfilling travel experience.

[0820] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0821] This invention is a travel plan creation support system that provides a more sophisticated travel experience by combining an emotion engine that recognizes the user's emotions. This system comprises an input means for inputting the activities the user wishes to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means.

[0822] System Programming and Processing

[0823] Initialization and accepting user input

[0824] User:

[0825] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[0826] Device:

[0827] The entered activity details are analyzed and classified into categories (tourism, art, gourmet).

[0828] The analysis results are sent to the server.

[0829] Information gathering

[0830] server:

[0831] The transmitted input data is received, and past visit history, current location, and mode of travel are retrieved from the database.

[0832] We use weather information APIs and congestion information APIs to obtain real-time weather conditions and congestion information.

[0833] Data Analysis and Filtering

[0834] server:

[0835] Based on the collected data, it filters out tourist spots and dining options that you have not visited before.

[0836] Narrow down suitable locations to visit based on weather information and crowding levels.

[0837] Travel and journey time calculations

[0838] server:

[0839] Taking into account the user's mode of transportation (walking, car, public transportation), the travel time to each candidate location is calculated.

[0840] The travel time between each spot is taken into account to calculate the required time and reflected in the schedule.

[0841] Emotional Engine Adjustment

[0842] server:

[0843] The emotion engine means is used to identify the user's current emotional state, for example by detecting the user's emotions from facial expression analysis using a camera or voice input.

[0844] Based on the identified emotional state, the system suggests activities and places that fit the user's current emotions, for example, if the user is feeling stressed, it suggests relaxation spots.

[0845] Plan Generation and Optimization

[0846] server:

[0847] Rankings are made based on information such as the rating of each spot, time, and distance traveled, and the top spots are selected.

[0848] Generate efficient travel plans by optimizing the order of visits taking into account travel time and required time.

[0849] The output of the emotion engine is reflected to create a plan optimized for the user's emotional state.

[0850] View Plans

[0851] Device:

[0852] The optimum travel plan sent from the server is displayed to the user.

[0853] Detailed information on each spot (address, opening hours, ratings, weather forecast, and crowd status) is provided.

[0854] Information is presented in the form of maps and schedules to make it easier for users to check their travel plans.

[0855] Specific operation example

[0856] 1. User Settings:

[0857] User: Enters into the device, "I want to go sightseeing, experience art, and eat local cuisine."

[0858] Terminal: Parses input and sends it to the server.

[0859] 2. Information Collection:

[0860] Server: Checks past visit history and retrieves unvisited tourist spots and dining options from the database.

[0861] Server: Get the current weather conditions using the weather information API (e.g. sunny).

[0862] Server: Obtain congestion status using the congestion information API (e.g., not crowded).

[0863] 3. Emotion Recognition with Emotion Engine:

[0864] Server: Analyzes the user's facial expressions and voice using an emotion engine to identify their current emotional state (e.g., relaxed, excited, stressed, etc.).

[0865] Server: Based on the identified emotion, suggest quiet cafes or tranquil gardens to users who need to relax.

[0866] 4. Data analysis and plan generation:

[0867] Server: Calculates the means of transportation and travel time based on the filtered spots (e.g., 30 minutes from Tokyo Station to Ueno Museum).

[0868] Server: Generates a plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which to visit them (e.g., Tokyo Tower in the morning, Ueno Museum in the afternoon, and a ramen restaurant in Asakusa for lunch).

[0869] 5. Present the plan:

[0870] Terminal: The travel plan sent from the server is displayed to the user along with a map, and detailed information is also provided.

[0871] As described above, this system can further enrich users' travel experience by reflecting their requests and feelings in real time and providing optimal travel plans.

[0872] The processing flow will be explained below.

[0873] Step 1:

[0874] User:

[0875] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[0876] Step 2:

[0877] Device:

[0878] The entered activity details are analyzed and classified into categories (tourism, art, gourmet).

[0879] The analysis results are sent to the server.

[0880] Step 3:

[0881] server:

[0882] The transmitted input data is received, and past visit history, current location, and mode of travel are retrieved from the database.

[0883] Check the user's current location.

[0884] Step 4:

[0885] server:

[0886] Use the weather information API to get the weather forecast for the planned travel date.

[0887] Use the crowd information API to obtain information on the crowding status of tourist attractions and dining spots.

[0888] Step 5:

[0889] server:

[0890] It checks past visit history and lists tourist spots and dining options the user has not yet visited.

[0891] Narrow down suitable locations to visit based on weather information and crowding levels.

[0892] Step 6:

[0893] server:

[0894] Taking into account the user's mode of transportation (walking, car, public transportation), the travel time to each candidate location is calculated.

[0895] The travel time between each spot is taken into account to calculate the required time and reflected in the schedule.

[0896] Step 7:

[0897] server:

[0898] The emotion engine means is used to identify the user's current emotional state, for example by detecting the user's emotions from facial expression analysis using a camera or voice input.

[0899] Step 8:

[0900] server:

[0901] Based on the identified emotional state, the system suggests activities and places that fit the user's current emotions, for example, if the user is feeling stressed, it suggests relaxation spots.

[0902] Step 9:

[0903] server:

[0904] Rankings are made based on information such as the rating of each spot, time, and distance traveled, and the top spots are selected.

[0905] Generate efficient travel plans by optimizing the order of visits taking into account travel time and required time.

[0906] The output of the emotion engine is reflected to create a plan optimized for the user's emotional state.

[0907] Step 10:

[0908] server:

[0909] An optimized itinerary is generated and sent to the device, including detailed information (address, opening hours, ratings, weather forecast, and crowding status).

[0910] Step 11:

[0911] Device:

[0912] The submitted itinerary is displayed to the user.

[0913] The plan is presented in detail in the form of a map or schedule, and if the user has any additional questions or requests for changes, they are sent to the server.

[0914] Step 12:

[0915] server:

[0916] Receive user feedback and adjust / update your itinerary again if necessary.

[0917] Send the new plan to your device and display it.

[0918] As a result, users can use the system to efficiently plan their trips, and with the help of the emotion engine, they can have a flexible travel experience that suits their mood at the time.

[0919] Example 2

[0920] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0921] Conventional travel planning support systems take into account data such as a user's past visit history, current location, weather, and congestion, but are unable to propose travel plans that reflect the user's emotional state. As a result, travel plans that do not match the user's mood or emotions are often proposed, resulting in an unsatisfactory travel experience. In addition, it is difficult to flexibly change plans in response to real-time changes in the environment or the user's emotions.

[0922] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an emotion engine means for recognizing the user's emotions and adjusting the travel plan based on the emotions; a means for dynamically modifying the travel plan taking into account past visit history data, weather information and congestion information updated in real time, and travel time; and an input means having a function for analyzing the user's interests and automatically classifying them into categories (sightseeing, art, gourmet, etc.) based on the interests. This makes it possible to provide an optimal travel plan that flexibly responds to the user's emotions and real-time environmental changes.

[0923] "Input means" refers to a means by which a user inputs into the system the activities that he or she wishes to undertake during his or her trip.

[0924] "Data collection means" refers to means for collecting data on a user's past visit history, travel time, current location, weather, congestion, and means of transportation.

[0925] The "proposal method" is a method for selecting and proposing tourist facilities and dining spots that best suit the user's needs based on the collected data.

[0926] The "display means" is a means for presenting the proposed travel plan to the user.

[0927] The "emotion engine means" is a means for recognizing the user's emotions and adjusting the travel plan based on those emotions.

[0928] A "travel planning support system" is a system that allows users to input the activities they wish to do during their trip, and creates and provides the optimal travel plan based on that input.

[0929] "Past visit history data" refers to data on tourist attractions and dining locations that a user has visited in the past.

[0930] "Weather information updated in real time" refers to information that acquires and updates the current weather conditions in real time.

[0931] "Crowding information" is information that indicates the current congestion status of tourist facilities and dining destinations.

[0932] "Travel time" refers to the time required for a user to travel from their current location to their destination.

[0933] This invention provides a more sophisticated travel experience by combining an emotion engine that recognizes the user's emotions in a travel plan creation support system. This system comprises an input means for the user to input the activities they wish to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and transportation means, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means.

[0934] The user starts the system and inputs the activities they wish to do during their trip (e.g., sightseeing, art, gourmet) into the terminal. The terminal analyzes the input activities and classifies them into categories (sightseeing, art, gourmet). The analysis results are sent to the server.

[0935] The server receives the input data and retrieves information on past visit history, current location, and mode of transportation from a database. It also uses a weather information API (for example, a general name for the weather information retrieval API) and a congestion information API (for example, a general name for the congestion information retrieval API) to retrieve real-time weather and congestion conditions.

[0936] The server uses the collected data to filter out tourist attractions and dining options that the user has not visited before, and then narrows down the list of suitable places to visit based on weather information and crowding conditions.

[0937] When calculating travel time and required time, the server takes into account the user's mode of transportation (walking, car, public transportation) and calculates the travel time to each candidate location. It also calculates the required time taking into account the travel time between each spot and reflects it in the schedule.

[0938] As an emotion engine, the server uses an emotion recognition model (e.g., an emotion analysis API) to identify the user's current emotional state. Specifically, it can detect the user's emotions from facial expression analysis using a camera or voice input. Based on the identified emotional state, it will suggest activities and spots that match the user's current emotions. For example, if the user is feeling stressed, it will suggest relaxation spots.

[0939] When generating a plan, the server ranks each spot based on information such as rating, travel time, and distance, and selects the top spots. It then optimizes the order of visits, taking travel time and required time into account, to generate an efficient travel plan. The server also reflects the output of the emotion engine to create a plan optimized for the user's emotional state.

[0940] Finally, the generated optimal travel plan is sent from the server to the device. The device displays the travel plan sent from the server to the user, providing detailed information about each spot (address, opening hours, ratings, weather forecast, and congestion status). The information is presented in the form of a map or schedule to make it easier for the user to check the travel plan.

[0941] As a specific example of how it works, if you enter "I want to go sightseeing, experience art, and eat local cuisine" into the prompt text, the system will categorize your requests into categories and create and present the most suitable travel plan. In this way, the system reflects the user's desires and emotions in real time and provides the most suitable travel plan, further enriching the user's travel experience.

[0942] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0943] Step 1: Initialize and accept user input

[0944] User:

[0945] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[0946] Input: Text information that a user types into a terminal.

[0947] Output: Parsed activity.

[0948] Device:

[0949] The activities entered by the user are analyzed and classified into categories (sightseeing, art, gourmet). For example, "I want to see Tokyo Tower, I want to go to the Ueno Museum, I want to eat ramen in Asakusa."

[0950] Input: Text information entered by the user.

[0951] Data processing: Text analysis and classification into categories.

[0952] Output: Categorized activities.

[0953] The analysis results are sent to the server.

[0954] Step 2: Gather information

[0955] server:

[0956] The transmitted input data is received, and information on the user's past visit history, current location, and means of transportation from the departure point is retrieved from the database.

[0957] Input: Analysis results sent from the device.

[0958] Data processing: Retrieve data from user database.

[0959] Output: User's past visit history and current location information.

[0960] Use a weather information API (e.g., a general name for the weather information API) or a congestion information API (e.g., a general name for the congestion information API) to obtain real-time weather conditions and congestion status.

[0961] Input: Acquired past visit history and current location information.

[0962] Data processing: Obtain weather and congestion information from the API.

[0963] Output: Real-time weather and traffic conditions.

[0964] Step 3: Data analysis and filtering

[0965] server:

[0966] Based on the collected data, it filters out tourist spots and dining options that the user has not visited before.

[0967] Input: All collected data (past visit history, current location, weather information, crowd status).

[0968] Data processing: Filter out unvisited spots.

[0969] Output: A filtered list of attractions and places to eat.

[0970] Narrow down potential places to visit based on weather information and crowding levels.

[0971] Input: Filtered list with weather and crowd information.

[0972] Data processing: Narrow down spots suitable for weather and crowd conditions.

[0973] Output: A narrowed down list of potential locations.

[0974] Step 4: Calculate travel time and journey time

[0975] server:

[0976] The travel time to each candidate location is calculated taking into account the user's mode of transportation (walking, car, public transportation). For example, "It takes 30 minutes to walk from Tokyo Station to Ueno Museum of Art."

[0977] Input: User's travel information and a narrowed list of possible locations.

[0978] Data processing: Calculate travel time to each candidate location.

[0979] Output: A list of travel times to each candidate location.

[0980] The travel time between each spot is taken into account to calculate the required time and reflected in the schedule.

[0981] Input: Travel time list and candidate location list.

[0982] Data processing: Calculate travel time between each spot and reflect it in the schedule.

[0983] Output: Visit schedule with duration.

[0984] Step 5: Emotional Engine Adjustment

[0985] server:

[0986] The emotion engine (e.g., emotion analysis API) is used to identify the user's current emotional state. Specifically, the user's emotions are detected from facial expression analysis using a camera or voice input.

[0987] Input: User's facial and voice data.

[0988] Data processing: Emotional state is analyzed using an emotion engine.

[0989] Output: The user's current emotional state.

[0990] Based on the identified emotional state, the system suggests activities and places that fit the user's current emotions, for example, if the user is feeling stressed, it suggests relaxation spots.

[0991] Input: User's emotional state and schedule.

[0992] Data manipulation: Adjust your schedule based on your emotional state.

[0993] Output: An emotion-appropriate visit schedule.

[0994] Step 6: Generate and optimize the plan

[0995] server:

[0996] Rankings are made based on information such as the rating of each spot, time, and distance traveled, and the top spots are selected.

[0997] Input: Information such as rating of the spot, time, and distance traveled.

[0998] Data processing: Ranking is carried out and the top spots are selected.

[0999] Output: A list of selected top spots.

[1000] Taking into account travel time and travel time, the order of visits is optimized to generate an efficient itinerary.

[1001] Input: Top spot list, travel time, and travel time.

[1002] Data processing: Optimize visit order and generate travel plans.

[1003] Output: An efficient travel plan.

[1004] The output of the emotion engine is reflected to create a plan optimized for the user's emotional state.

[1005] Input: Emotionally appropriate visit schedule.

[1006] Data processing: Create a plan optimized for your emotional state.

[1007] Output: Optimized trip plan.

[1008] Step 7: View your plan

[1009] Device:

[1010] The optimum travel plan sent from the server is displayed to the user.

[1011] Input: The itinerary sent from the server.

[1012] Data processing: Formatting the travel plan for display.

[1013] Output: The plan that is displayed to the user.

[1014] Detailed information on each spot (address, opening hours, ratings, weather forecast, and crowd status) is provided.

[1015] Input: Spot information within the travel plan.

[1016] Data processing: Incorporating detailed information into the display.

[1017] Output: A display with detailed information.

[1018] Information is presented in the form of maps and schedules to make it easier for users to check their travel plans.

[1019] Input: Travel plan and details.

[1020] Data processing: Formatting into map or schedule format screens.

[1021] Output: Plans displayed in map and schedule format.

[1022] This allows the system to reflect users' requests and feelings in real time and provide optimal travel plans.

[1023] (Application example 2)

[1024] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1025] Conventional travel planning support systems could suggest optimal tourist attractions and dining options by taking into account the user's past visit history, weather conditions, and congestion levels, but they did not provide plans that reflected the user's current emotional state. This made it difficult to generate optimal travel plans that addressed emotions such as stress and excitement that users felt during their trip. This is needed to further enhance the travel experience and increase satisfaction.

[1026] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1027] In this invention, the server includes an input means for inputting the activities the user wishes to engage in during their trip, a data collection means for collecting data on the user's past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots that best suit the user's preferences based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means for recognizing the user's emotions in real time and making suggestions based on those emotions. This makes it possible to provide a travel plan that matches the user's current emotional state, further enriching the travel experience.

[1028] Definitions of important words

[1029] A "travel planning support system" is a system that supports the activities that users want to undertake while traveling, suggesting the most suitable tourist facilities and places to eat by taking into consideration data such as the user's past visit history, current location, weather, congestion level, and mode of transportation.

[1030] "Input means" refers to a device or interface that allows a user to input the activities and preferences they wish to undertake during their trip.

[1031] "Data collection means" refers to means for collecting information about a user's past visit history, current location, weather, congestion, and means of transportation.

[1032] The "suggestion method" is a method of selecting and suggesting the most suitable tourist facilities and dining spots to users based on the collected data.

[1033] "Display means" refers to a means for visually presenting information such as suggested travel plans, selected tourist facilities, and dining options to users.

[1034] The "emotion engine means" is a means for recognizing the user's emotions in real time and adjusting the content of suggestions based on those emotions.

[1035] This invention is a travel plan creation support system that provides a more sophisticated travel experience by combining an emotion engine that recognizes the user's emotions. This system comprises an input means for inputting the activities the user wishes to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means.

[1036] The specific configuration and processing of a system for implementing the present invention are as follows.

[1037] Technology stack used

[1038] Hardware

[1039] Smartphone

[1040] Smart Glasses

[1041] head-mounted display

[1042] software

[1043] OpenCV for Sentiment Analysis

[1044] Google Speech-to-Text API for voice analysis

[1045] A server-side program with a unique algorithm for movie suggestions

[1046] Program processing explanation

[1047] Initialization and accepting user input

[1048] The user starts the travel planning support system and inputs the activities they would like to do during their trip. For example, they might input text such as "I want to do some sightseeing, experience art, and try local cuisine" into the device. This input is done via a smartphone, smart glasses, or a head-mounted display.

[1049] The device analyzes the entered activity details, classifies them into categories (tourism, art, gourmet), and sends the analysis results to the server.

[1050] Information gathering and emotion recognition

[1051] The server receives the input data and retrieves past visit history, current location, and mode of transportation from the database. It also obtains real-time weather and congestion information using weather information APIs and congestion information APIs.

[1052] The device uses the built-in camera and microphone to capture the user's facial expressions and voice, and recognizes emotions in real time using OpenCV and Emotion APIs (e.g., Microsoft Face API). It also analyzes voice data with the Google Speech-to-Text API to identify emotions from speaking tone.

[1053] Generate and view plans

[1054] The server selects filtered tourist spots and dining options based on the collected data, and makes optimal suggestions based on the user's current emotional state. It generates an efficient travel plan by taking into account the ratings of each spot and the order in which they should be visited. It reflects the output of the emotion engine and creates a plan optimized for the user's emotional state.

[1055] The terminal displays the optimal travel plan sent from the server to the user and provides detailed information about each spot (address, opening hours, ratings, weather forecast, and congestion status).

[1056] Specific examples

[1057] 1. Example of user input:

[1058] "I want to see the sights, experience art, and eat local cuisine."

[1059] 2. Example prompt:

[1060] What is your current mood and what genre do you want to watch?

[1061] Example: I want to relax today, so I want to watch a comedy movie.

[1062] In this way, it is possible to provide an optimal travel plan that reflects the user's emotional state in real time.The system utilizes a generative AI model to further explore the user's needs through prompt sentences, enabling the system to achieve an optimal travel experience.

[1063] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1064] Program processing steps

[1065] Step 1:

[1066] The user starts the travel planning support system and inputs the activities and requests he / she wishes to perform during the trip.

[1067] Input: Activities such as "I want to go sightseeing, experience art, and eat local cuisine"

[1068] Output: Categorization of analyzed activities (sightseeing, art, gourmet)

[1069] Specific operation: Data entered into a smartphone, smart glasses, or head-mounted display is analyzed using natural language processing and classified into categories. The analysis results are then sent to a server.

[1070] Step 2:

[1071] The server receives the input data and retrieves past visit history, current location, and mode of transportation from the database. It also uses weather information API and congestion information API to obtain real-time weather and congestion information.

[1072] Input: User input data, visit history, current location, weather API, congestion information API

[1073] Output: Obtained weather and congestion information

[1074] Specific operation: The server executes database queries to obtain past visit history and current location data, and calls weather APIs and congestion information APIs to obtain real-time data.

[1075] Step 3:

[1076] The device uses the built-in camera and microphone to capture the user's facial expressions and voice, and recognizes emotions in real time using OpenCV and the Emotion API. The voice data is also analyzed using the Google Speech-to-Text API to identify emotions from the tone of speech.

[1077] Input: User's facial expression data, voice data

[1078] Output: Perceived emotional state (e.g., relaxed, excited, stressed)

[1079] What it does: It activates the camera and microphone, performs facial expression analysis using OpenCV and the Emotion API, and analyzes audio data using the Google Speech-to-Text API to determine emotions based on the tone of the voice.

[1080] Step 4:

[1081] Based on the collected data, the server selects filtered tourist spots and dining options, making optimal suggestions based on the user's current emotional state.

[1082] Input: Filtered tourist spots, dining options, and the user's emotional state

[1083] Output: A ranked list of the best places to see and eat

[1084] What it does: It takes into account the output of the emotion engine and applies an algorithm to generate rankings of tourist attractions and dining destinations based on past visit history and current situation data.

[1085] Step 5:

[1086] The server takes into account the ratings of each spot and the order in which they should be visited, generates an efficient travel plan, and sends it to the terminal.

[1087] Input: Ranking list of spots, rating data, visit order

[1088] Output: Optimized itinerary

[1089] Specific operation: Calculates the most efficient visiting order based on the rating, travel time, and duration of stay of each spot, and generates an optimal travel plan.

[1090] Step 6:

[1091] The terminal displays the optimal travel plan sent from the server to the user and provides detailed information about each spot (address, opening hours, ratings, weather forecast, and congestion status).

[1092] Input: Optimized itinerary, detailed information for each spot

[1093] Output: The best itinerary and details displayed to the user

[1094] What it does: It visually displays the itinerary and detailed information of each spot on a map and in schedule format, making it easy for users to check.

[1095] As a result, it is possible to realize a system that provides optimal travel plans that reflect the user's emotional state in real time, further enriching the travel experience.

[1096] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1097] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1098] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1099] [Third embodiment]

[1100] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1101] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1102] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1103] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1104] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1105] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1106] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1107] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1108] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1109] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1110] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1111] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1112] The present invention is a travel planning support system that provides an optimal travel plan based on user input. This system has an input means for inputting activities the user wants to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and transportation methods, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, and a display means for presenting the suggested travel plan to the user.

[1113] System Programming and Processing

[1114] Initialization and accepting user input

[1115] User:

[1116] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[1117] Device:

[1118] The input information is analyzed and classified into each activity category.

[1119] The analysis results are sent to the server.

[1120] Information gathering

[1121] server:

[1122] Past visit history, current location, and mode of transportation are retrieved from the database.

[1123] Use weather information APIs and congestion information APIs to obtain real-time weather and congestion information.

[1124] Obtain data on the means of travel and the travel time required.

[1125] Data Analysis and Filtering

[1126] server:

[1127] Based on the collected data, it filters out tourist spots and dining options that you have not visited before.

[1128] For each filtered spot, a priority is determined based on weather information and congestion status.

[1129] Travel and journey time calculations

[1130] server:

[1131] Calculate travel time to each spot based on transportation options such as public transportation, car, or walking.

[1132] Consider travel time between each spot and adjust your schedule so that the total time is not too long.

[1133] Plan Generation and Optimization

[1134] server:

[1135] Rankings are based on information such as the rating of each spot, time, and distance traveled.

[1136] Arrange spots in the optimal order to generate an efficient travel plan.

[1137] If necessary, it will reflect real-time data and allow you to dynamically change your plan during your trip.

[1138] View Plans

[1139] Device:

[1140] The optimum travel plan sent from the server is displayed to the user.

[1141] Detailed information about each spot (address, opening hours, ratings, expected weather, and crowd levels) is also provided.

[1142] Information is presented in the form of maps and schedules to make it easier for users to check their travel plans.

[1143] Specific operation example

[1144] 1. User Settings:

[1145] User: Enters into the device, "I want to go sightseeing, experience art, and eat local cuisine."

[1146] Terminal: Parses input and sends it to the server.

[1147] 2. Information Collection:

[1148] Server: Checks past visit history and retrieves unvisited tourist spots and dining options from the database.

[1149] Server: Get the current weather conditions using the weather information API (e.g. sunny).

[1150] Server: Obtain congestion status using the congestion information API (e.g., not crowded).

[1151] 3. Data analysis and schedule calculation:

[1152] Server: Calculates the travel method and travel time based on the filtered spots (e.g., 30 minutes from Tokyo Station to Ueno Museum).

[1153] 4. Generate the plan:

[1154] Server: Generates a plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which to visit them (e.g., Tokyo Tower in the morning, Ueno Museum in the afternoon, and a ramen restaurant in Asakusa for lunch).

[1155] 5. Present the plan:

[1156] Terminal: Displays the game plan to the user along with a map and provides detailed information.

[1157] This will enable users to easily create efficient and comprehensive travel plans, further increasing the enjoyment of their trip.The system of the present invention can reflect users' requests in real time and dynamically change plans, making it possible to provide a flexible travel experience.

[1158] The processing flow will be explained below.

[1159] Step 1:

[1160] User:

[1161] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[1162] Step 2:

[1163] Device:

[1164] The entered user activity information is analyzed and classified into categories (tourism, art, gourmet).

[1165] The analysis results are sent to the server.

[1166] Step 3:

[1167] server:

[1168] Receives the input data sent.

[1169] The user's past visit history is retrieved from the database.

[1170] Check the user's current location.

[1171] Step 4:

[1172] server:

[1173] Use the weather information API to get the weather forecast for the planned travel date.

[1174] Use the crowd information API to obtain information on the crowding status of tourist attractions and dining spots.

[1175] Step 5:

[1176] server:

[1177] It checks past visit history and lists tourist spots and dining options the user has not yet visited.

[1178] Narrow down suitable locations to visit based on weather information and crowding levels.

[1179] Step 6:

[1180] server:

[1181] Taking into account the user's mode of transportation (walking, car, public transportation), the travel time to each candidate location is calculated.

[1182] Calculate the travel time for each mode of transportation and reflect it in the schedule.

[1183] Step 7:

[1184] server:

[1185] Rankings are made based on the ratings of each spot (reviews, number of stars, etc.), and the top spots are selected.

[1186] Optimize the order of visits and generate efficient travel plans, taking into account travel time and travel time.

[1187] Step 8:

[1188] server:

[1189] An optimized itinerary is generated and sent to the device, including detailed information (address, opening hours, ratings, weather forecast, and crowding status).

[1190] Step 9:

[1191] Device:

[1192] The submitted itinerary is displayed to the user.

[1193] The plan details are presented in the form of maps and schedules.

[1194] If the user has any additional questions or changes they would like to make, they submit them to the server.

[1195] Step 10:

[1196] server:

[1197] Receive user feedback and adjust / update your itinerary again if necessary.

[1198] Send the new plan to the device and have it reload.

[1199] As a result, users can use the system to efficiently plan their trips, and real-time information updates enable a flexible travel experience.

[1200] Example 1

[1201] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1202] Conventional travel planning support systems have struggled to automatically generate efficient travel plans that match the user's preferences and interests, and to optimize them in real time. Furthermore, technology for quickly and effectively analyzing the wide variety of requests entered by users and incorporating them into travel plans has not yet been developed. This has resulted in the problem of users having to manually search and organize information and create travel plans, which is time-consuming.

[1203] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1204] In this invention, the server includes an input means for inputting activities the user wishes to engage in during their trip, a data collection means for collecting data on the user's past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots that best suit the user's preferences based on the collected data, a display means for presenting the suggested travel plan to the user, a means for analyzing the user's input and classifying it into activity categories using natural language processing, and a means for determining the priority of each spot based on the collected data and generating an efficient travel plan. This makes it possible to automatically generate an optimal travel plan based on the user's requests and dynamically optimize the plan in real time.

[1205] The "input means" refers to a device or function that allows a user to input the activities that the user wants to do during the trip.

[1206] "Data collection means" refers to devices or functions that collect data regarding a user's past visit history, travel time, current location, weather, congestion, and means of transportation.

[1207] "Proposal means" refers to devices or functions that select and suggest tourist facilities and dining options that best suit the user's needs based on collected data.

[1208] "Display means" refers to a device or function for visually presenting the proposed travel plan to the user.

[1209] "Natural language processing" is an artificial intelligence technique used to analyze user input and classify it into activity categories.

[1210] "Filtering" is the process of extracting information that meets specific conditions from a large amount of data.

[1211] The "priority determination means" is a device or function for determining the order of visits and importance of each spot based on the collected data.

[1212] The "plan generation means" refers to a device or function for automatically creating an efficient travel plan.

[1213] The present invention relates to a travel planning support system that provides an optimal travel plan based on user input. The system includes an input unit, a data collection unit, a proposal unit, a display unit, a natural language processing unit, a filtering unit, a priority determination unit, and a plan generation unit.

[1214] System Programming and Processing

[1215] The program of this system includes the following hardware and software.

[1216] Input method: Using a smartphone or tablet application, users input the activities they want to do during their trip.

[1217] Data collection method: Data such as past visit history, current location, real-time weather information, congestion information, and travel time is collected using a database on the server and external APIs (e.g., OpenWeatherMap API, Google Maps API).

[1218] Natural language processing tools: Using a generative AI model, it analyzes user input and categorizes it into categories such as tourism, art, and food.

[1219] Filtering method: Based on the collected data, we filter out tourist attractions and dining spots that the user has not visited before.

[1220] Priority determination method: Determine the priority of each filtered spot based on weather information and crowding.

[1221] Plan generation method: Based on the priority, arrange tourist spots in the optimal order and generate an efficient travel plan.

[1222] Display method: The generated travel plan is displayed to the user via a smartphone or tablet application.

[1223] Program processing

[1224] Users start the system and input the activities they want to do during their trip (e.g., "sightseeing," "art," "gourmet") The application on the device analyzes the input, classifies the activities into categories using natural language processing, and sends the data to the server.

[1225] The server retrieves past visit history from the database and collects current location and real-time weather and congestion information via an external API. It also obtains the mode of transportation used by the user and the travel time required. Based on this data, filtering is performed to extract tourist spots and dining spots that have not been visited before. The server then prioritizes destinations, taking into account weather and congestion information.

[1226] The server calculates travel time to each spot based on the mode of transportation (train, bus, car, walking, etc.) and creates an efficient schedule taking into account travel time between each spot.Finally, it selects highly rated tourist spots and dining spots based on the user's preferences and generates a plan that visits them in the optimal order.

[1227] The generated itinerary is sent to the device and displayed to the user. Detailed information about the itinerary and each spot (address, opening hours, ratings, expected weather, and congestion status) is provided in map and schedule format for easy viewing by the user.

[1228] Specific examples

[1229] 1. User Settings:

[1230] User: Enters into the device, "I want to go sightseeing, experience art, and eat local cuisine."

[1231] 2. Information Collection:

[1232] Server: Checks past visit history and retrieves unvisited tourist spots and dining spots from the database. For example, if the user has visited Sensoji Temple in the past, Skytree, which has not yet been visited, will be suggested.

[1233] Server: Get the current weather conditions using the weather information API (e.g. sunny).

[1234] Server: Obtain the congestion status using the congestion information API (e.g., Ueno Museum is not crowded).

[1235] 3. Data analysis and prioritization:

[1236] Server: Based on the filtered spots, calculate the travel method and travel time using Google Maps API (e.g., 30 minutes from Tokyo Station to Ueno Museum).

[1237] 4. Generate the plan:

[1238] Server: Generates a plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which to visit them (e.g., Tokyo Tower in the morning, Ueno Museum in the afternoon, and a ramen restaurant in Asakusa for lunch).

[1239] 5. Present the plan:

[1240] Terminal: Displays the itinerary to the user along with a map and provides detailed information.

[1241] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1242] Step 1: Accepting User Input

[1243] User: The user inputs the activities they would like to do during their trip (e.g., "I want to do some sightseeing," "I want to experience art," "I want to try local cuisine") into an application on their smartphone or tablet.

[1244] Terminal: The terminal receives the input information, analyzes it, and classifies each activity into a category (sightseeing, art, gourmet, etc.). This analysis is performed using natural language processing technology. The analysis results are then sent to the server.

[1245] Input: User activity input ("I want to go sightseeing," "I want to experience art," "I want to eat local cuisine")

[1246] Output: Categorized activity information (sightseeing, art, gourmet)

[1247] Step 2: Data collection

[1248] Server: The server retrieves past visit history and current location information from the database. It also uses external APIs (e.g., OpenWeatherMap API, Google Maps API) to collect real-time weather information, congestion information, transportation methods, and travel times.

[1249] Input: Categorized activity information

[1250] Output: Past visit history, current location, weather information, congestion information, transportation method and travel time data

[1251] Step 3: Data analysis and filtering

[1252] Server: Based on the collected data, it filters tourist spots and dining spots that the user has not visited before. Specifically, it compares the user's past visit history with the user's current location information and lists unvisited spots.

[1253] Input: Past visit history, current location, weather information, congestion information, transportation method and travel time data

[1254] Output: A filtered list of places to visit and eat

[1255] Step 4: Prioritize

[1256] Server: Determines the priority of each filtered spot based on weather and congestion information. For example, it applies rules such as prioritizing indoor spots on rainy days.

[1257] Input: Filtered list of attractions and dining options, weather information, and crowd information

[1258] Output: A prioritized list of places to see and eat

[1259] Step 5: Calculate travel time and journey time

[1260] Server: Calculates travel time to each location for each mode of transportation. Specifically, it uses the Google Maps API to obtain travel times based on modes of transportation such as public transportation, car, and walking, and adjusts travel schedules between each location.

[1261] Input: A prioritized list of places to see and eat

[1262] Output: Travel time and travel time data between each location

[1263] Step 6: Generate and optimize the plan

[1264] Server: Generates an efficient travel plan by arranging each spot in the optimal order, taking into account the rating of each spot, the duration of stay, and the travel distance. If necessary, the plan can be dynamically modified to reflect real-time data.

[1265] Input: Travel time and travel time between each spot, rating of each spot, and duration of stay

[1266] Output: Optimal travel plan

[1267] Step 7: View your plan

[1268] Terminal: The optimal travel plan sent from the server is displayed to the user. For easy viewing, the travel plan and detailed information about each spot (address, opening hours, rating, forecast weather, and congestion status) are provided in map and schedule format.

[1269] Input: Best Travel Plan

[1270] Output: A travel plan presented to the user with details about each spot

[1271] (Application example 1)

[1272] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1273] Conventional travel planning support systems suggest tourist attractions and dining options for users during their trips, but lack integration with food delivery services, allowing travelers to order meals and have them delivered to designated locations. This makes it difficult for users to enjoy an optimal dining experience while traveling without any hassle. Furthermore, systems that dynamically modify travel plans using real-time information such as weather and congestion are often inadequate. As a result, these systems are unable to provide flexible travel plans for travelers, making it difficult to adapt to unexpected situations and traffic jams.

[1274] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1275] In this invention, the server includes an input means for inputting activities the user wishes to engage in during the trip, a data collection means for collecting data on the user's past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots that best suit the user's preferences based on the collected data, a display means for presenting the suggested travel plan to the user, and a means for collecting and suggesting data on food delivery services for ordering meals for travelers and having them delivered at specified locations. This makes it possible to provide a comprehensive and dynamic travel plan that includes meal arrangements during the trip.

[1276] A "travel planning support system" is a system that selects and suggests the most suitable tourist facilities and dining spots based on the activities that the user wants to do during their trip, and presents the plan.

[1277] "Input means" refers to a device or interface that allows a user to input the activities they wish to undertake during their trip.

[1278] "Data collection means" refers to means that has the function of collecting data regarding a user's past visit history, travel time, current location, weather, congestion, and means of transportation.

[1279] The "suggestion means" is a means that has the function of selecting and suggesting tourist facilities and dining spots that best suit the user's needs based on the collected data.

[1280] "Display" refers to a device or interface for visually displaying a proposed travel plan to a user.

[1281] A "food delivery service" is a platform that provides meal-related services that allow users to order and have food delivered to a location they specify.

[1282] "Weather information" refers to meteorological data that is updated in real time and is used for planning and dynamically changing travel plans.

[1283] "Crowding information" refers to data on the congestion status of planned visits and travel routes, and is updated in real time.

[1284] "Transportation" refers to the means by which users travel during their trip, such as public transportation, car, or walking.

[1285] "Past visit history" refers to data about tourist attractions and dining options that users have visited in the past.

[1286] A "dynamically changing means" is a means that has the function of changing a travel plan to reflect real-time data.

[1287] This invention provides users with efficient and optimal travel plans and comprehensive support, including meal arrangements, through a travel planning support system. The system consists of a smartphone or other device used by the user, a server for analyzing and processing data, and an interface for proposing and displaying travel plans.

[1288] System Programming and Processing

[1289] Initialization and accepting user input

[1290] User:

[1291] The user starts up the device and inputs the activities they would like to do during their trip, such as "I want to do some sightseeing," "I want to experience art," or "I want to try local cuisine."

[1292] Device:

[1293] The information entered is analyzed, and each activity is classified into a category. The analysis results are sent to the server.

[1294] Information gathering

[1295] server:

[1296] Data on past visit history, current location, and mode of transportation is retrieved from the database.

[1297] Use weather information APIs and congestion information APIs (e.g., OpenWeatherMap API and Google Maps Traffic API) to obtain real-time weather and congestion information.

[1298] Obtain available dining destinations from a database of partner food delivery services.

[1299] Data Analysis and Filtering

[1300] server:

[1301] The collected data is analyzed to filter out tourist spots and dining options that users have not visited before.

[1302] The priority of each filtered spot is determined based on weather information and congestion status.

[1303] Travel time and order timing calculations

[1304] server:

[1305] Uses Google Maps API to calculate travel time to each location.

[1306] Calculate the timing of your order so that your meal will be delivered when you arrive at your destination.

[1307] Plan Generation and Optimization

[1308] server:

[1309] The system generates a travel plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which you visit them.

[1310] The generated plan includes the order of places to visit and events, as well as when to order meals.

[1311] If necessary, itineraries can be dynamically modified to reflect real-time data.

[1312] View Plans

[1313] Device:

[1314] The optimal travel plan sent from the server is displayed to the user in the form of a map and schedule.

[1315] It also provides detailed information about each spot and restaurant (address, opening hours, ratings, expected weather, and crowd levels).

[1316] Specific example explanation

[1317] During initialization and user input reception, the user enters into the device, "I want to go sightseeing, experience art, and eat local cuisine." The device analyzes the user's input and sends the results to the server. The server collects data based on past visit history and current location, and obtains real-time data using weather information APIs and congestion information APIs.

[1318] For example, the server calculates the travel time from Tokyo Station to the Ueno Museum of Art in 30 minutes, and determines when to order sushi using a food delivery service 30 minutes before departure.The server then creates a plan that allows the meal to be delivered upon arrival at the destination, generating an efficient plan that allows the user to enjoy sightseeing and dining.

[1319] Prompt Sentence Examples

[1320] Please tell us where you would like to go and what food you would like to eat during your trip.

[1321] Example input:

[1322] Tourist attractions: Historic buildings, museums, parks

[1323] Food preferences: Sushi, ramen, Japanese sweets

[1324] As described above, this invention provides users with a fulfilling travel experience. Examples of software and APIs used include OpenWeatherMap API, Google Maps API, and UberEats API. Hardware primarily involves smartphones, tablet devices, and servers.

[1325] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1326] Step 1:

[1327] Initialization and accepting user input

[1328] Users turn on their smartphones or tablets and input the activities they want to do during their trip. Examples of input include "I want to do some sightseeing," "I want to experience art," and "I want to try local cuisine."

[1329] The device receives the input information and analyzes it using a natural language processing algorithm. This analysis classifies the input content into categories such as "tourism," "art," and "food." The analysis results are sent to the server in JSON format.

[1330] Step 2:

[1331] Information gathering

[1332] The server retrieves the user's past visit history and current location from a database based on the data received from the device, and also obtains real-time weather information from the OpenWeatherMap API and congestion information from the Google Maps Traffic API.

[1333] In addition, the server obtains a list of meal delivery locations from the API of affiliated food delivery services (e.g., Uber Eats, Demae-can, etc.).

[1334] The input data used is user preferences in JSON format, past visit history from the database, real-time information obtained from the API, and information from food delivery services.

[1335] Step 3:

[1336] Data Analysis and Filtering

[1337] The server analyzes the collected data and filters out tourist attractions and dining options that users have not visited before, using the Python pandas library to process the data.

[1338] This allows the system to extract unvisited spots based on past visit history, and prioritize suitable spots taking into account weather and crowding conditions.

[1339] It takes user preferences, real-time information, and food delivery information organized in JSON format as input, and generates a prioritized list of spots as output.

[1340] Step 4:

[1341] Travel time and order timing calculations

[1342] The server uses the Google Maps API to calculate travel times to each location and also calculates food delivery order times to ensure meals arrive at the expected arrival times.

[1343] Use Python's datetime library to schedule your visits and calculate the optimal time to visit each spot.

[1344] The inputs are the user's current location, next destination, and food delivery service data, and the output is an adjusted visit schedule and order timing.

[1345] Step 5:

[1346] Plan Generation and Optimization

[1347] The server generates an optimal travel plan by taking into account the ratings of each spot, the order of visits, travel time, and meal ordering timing. It applies machine learning models using Python's scikit-learn for optimization.

[1348] If necessary, the itinerary is dynamically changed to reflect real-time data.

[1349] It takes user preferences, real-time data, and food delivery information as inputs and generates an optimized travel plan as output.

[1350] Step 6:

[1351] View Plans

[1352] The device receives the optimal travel plan sent from the server and visually displays it to the user. It uses React.js to display the information in the form of a map and schedule.

[1353] Detailed information about each spot and restaurant (address, opening hours, ratings, expected weather, and crowd levels) is also displayed.

[1354] It takes as input travel plan data from a server and as output a visual travel plan that is displayed on the user's screen.

[1355] Through the above steps, the travel planning support system can provide users with a fulfilling travel experience.

[1356] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1357] This invention is a travel plan creation support system that provides a more sophisticated travel experience by combining an emotion engine that recognizes the user's emotions. This system comprises an input means for inputting the activities the user wishes to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means.

[1358] System Programming and Processing

[1359] Initialization and accepting user input

[1360] User:

[1361] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[1362] Device:

[1363] The entered activity details are analyzed and classified into categories (tourism, art, gourmet).

[1364] The analysis results are sent to the server.

[1365] Information gathering

[1366] server:

[1367] The transmitted input data is received, and past visit history, current location, and mode of travel are retrieved from the database.

[1368] We use weather information APIs and congestion information APIs to obtain real-time weather conditions and congestion information.

[1369] Data Analysis and Filtering

[1370] server:

[1371] Based on the collected data, it filters out tourist spots and dining options that you have not visited before.

[1372] Narrow down suitable locations to visit based on weather information and crowding levels.

[1373] Travel and journey time calculations

[1374] server:

[1375] Taking into account the user's mode of transportation (walking, car, public transportation), the travel time to each candidate location is calculated.

[1376] The travel time between each spot is taken into account to calculate the required time and reflected in the schedule.

[1377] Emotional Engine Adjustment

[1378] server:

[1379] The emotion engine means is used to identify the user's current emotional state, for example by detecting the user's emotions from facial expression analysis using a camera or voice input.

[1380] Based on the identified emotional state, the system suggests activities and places that fit the user's current emotions, for example, if the user is feeling stressed, it suggests relaxation spots.

[1381] Plan Generation and Optimization

[1382] server:

[1383] Rankings are made based on information such as the rating of each spot, time, and distance traveled, and the top spots are selected.

[1384] Generate efficient travel plans by optimizing the order of visits taking into account travel time and required time.

[1385] The output of the emotion engine is reflected to create a plan optimized for the user's emotional state.

[1386] View Plans

[1387] Device:

[1388] The optimum travel plan sent from the server is displayed to the user.

[1389] Detailed information on each spot (address, opening hours, ratings, weather forecast, and crowd status) is provided.

[1390] Information is presented in the form of maps and schedules to make it easier for users to check their travel plans.

[1391] Specific operation example

[1392] 1. User Settings:

[1393] User: Enters into the device, "I want to go sightseeing, experience art, and eat local cuisine."

[1394] Terminal: Parses input and sends it to the server.

[1395] 2. Information Collection:

[1396] Server: Checks past visit history and retrieves unvisited tourist spots and dining options from the database.

[1397] Server: Get the current weather conditions using the weather information API (e.g. sunny).

[1398] Server: Obtain congestion status using the congestion information API (e.g., not crowded).

[1399] 3. Emotion Recognition with Emotion Engine:

[1400] Server: Analyzes the user's facial expressions and voice using an emotion engine to identify their current emotional state (e.g., relaxed, excited, stressed, etc.).

[1401] Server: Based on the identified emotion, suggest quiet cafes or tranquil gardens to users who need to relax.

[1402] 4. Data analysis and plan generation:

[1403] Server: Calculates the means of transportation and travel time based on the filtered spots (e.g., 30 minutes from Tokyo Station to Ueno Museum).

[1404] Server: Generates a plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which to visit them (e.g., Tokyo Tower in the morning, Ueno Museum in the afternoon, and a ramen restaurant in Asakusa for lunch).

[1405] 5. Present the plan:

[1406] Terminal: The travel plan sent from the server is displayed to the user along with a map, and detailed information is also provided.

[1407] As described above, this system can further enrich users' travel experience by reflecting their requests and feelings in real time and providing optimal travel plans.

[1408] The processing flow will be explained below.

[1409] Step 1:

[1410] User:

[1411] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[1412] Step 2:

[1413] Device:

[1414] The entered activity details are analyzed and classified into categories (tourism, art, gourmet).

[1415] The analysis results are sent to the server.

[1416] Step 3:

[1417] server:

[1418] The transmitted input data is received, and past visit history, current location, and mode of travel are retrieved from the database.

[1419] Check the user's current location.

[1420] Step 4:

[1421] server:

[1422] Use the weather information API to get the weather forecast for the planned travel date.

[1423] Use the crowd information API to obtain information on the crowding status of tourist attractions and dining spots.

[1424] Step 5:

[1425] server:

[1426] It checks past visit history and lists tourist spots and dining options the user has not yet visited.

[1427] Narrow down suitable locations to visit based on weather information and crowding levels.

[1428] Step 6:

[1429] server:

[1430] Taking into account the user's mode of transportation (walking, car, public transportation), the travel time to each candidate location is calculated.

[1431] The travel time between each spot is taken into account to calculate the required time and reflected in the schedule.

[1432] Step 7:

[1433] server:

[1434] The emotion engine means is used to identify the user's current emotional state, for example by detecting the user's emotions from facial expression analysis using a camera or voice input.

[1435] Step 8:

[1436] server:

[1437] Based on the identified emotional state, the system suggests activities and places that fit the user's current emotions, for example, if the user is feeling stressed, it suggests relaxation spots.

[1438] Step 9:

[1439] server:

[1440] Rankings are made based on information such as the rating of each spot, time, and distance traveled, and the top spots are selected.

[1441] Generate efficient travel plans by optimizing the order of visits taking into account travel time and required time.

[1442] The output of the emotion engine is reflected to create a plan optimized for the user's emotional state.

[1443] Step 10:

[1444] server:

[1445] An optimized itinerary is generated and sent to the device, including detailed information (address, opening hours, ratings, weather forecast, and crowding status).

[1446] Step 11:

[1447] Device:

[1448] The submitted itinerary is displayed to the user.

[1449] The plan is presented in detail in the form of a map or schedule, and if the user has any additional questions or requests for changes, they are sent to the server.

[1450] Step 12:

[1451] server:

[1452] Receive user feedback and adjust / update your itinerary again if necessary.

[1453] Send the new plan to your device and display it.

[1454] As a result, users can use the system to efficiently plan their trips, and with the help of the emotion engine, they can have a flexible travel experience that suits their mood at the time.

[1455] Example 2

[1456] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1457] Conventional travel planning support systems take into account data such as a user's past visit history, current location, weather, and congestion, but are unable to propose travel plans that reflect the user's emotional state. As a result, travel plans that do not match the user's mood or emotions are often proposed, resulting in an unsatisfactory travel experience. In addition, it is difficult to flexibly change plans in response to real-time changes in the environment or the user's emotions.

[1458] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an emotion engine means for recognizing the user's emotions and adjusting the travel plan based on the emotions; a means for dynamically modifying the travel plan taking into account past visit history data, weather information and congestion information updated in real time, and travel time; and an input means having a function for analyzing the user's interests and automatically classifying them into categories (sightseeing, art, gourmet, etc.) based on the interests. This makes it possible to provide an optimal travel plan that flexibly responds to the user's emotions and real-time environmental changes.

[1459] "Input means" refers to a means by which a user inputs into the system the activities that he or she wishes to undertake during his or her trip.

[1460] "Data collection means" refers to means for collecting data on a user's past visit history, travel time, current location, weather, congestion, and means of transportation.

[1461] The "proposal method" is a method for selecting and proposing tourist facilities and dining spots that best suit the user's needs based on the collected data.

[1462] The "display means" is a means for presenting the proposed travel plan to the user.

[1463] The "emotion engine means" is a means for recognizing the user's emotions and adjusting the travel plan based on those emotions.

[1464] A "travel planning support system" is a system that allows users to input the activities they wish to do during their trip, and creates and provides the optimal travel plan based on that input.

[1465] "Past visit history data" refers to data on tourist attractions and dining locations that a user has visited in the past.

[1466] "Weather information updated in real time" refers to information that acquires and updates the current weather conditions in real time.

[1467] "Crowding information" is information that indicates the current congestion status of tourist facilities and dining destinations.

[1468] "Travel time" refers to the time required for a user to travel from their current location to their destination.

[1469] This invention provides a more sophisticated travel experience by combining an emotion engine that recognizes the user's emotions in a travel plan creation support system. This system comprises an input means for the user to input the activities they wish to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and transportation means, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means.

[1470] The user starts the system and inputs the activities they wish to do during their trip (e.g., sightseeing, art, gourmet) into the terminal. The terminal analyzes the input activities and classifies them into categories (sightseeing, art, gourmet). The analysis results are sent to the server.

[1471] The server receives the input data and retrieves information on past visit history, current location, and mode of transportation from a database. It also uses a weather information API (for example, a general name for the weather information retrieval API) and a congestion information API (for example, a general name for the congestion information retrieval API) to retrieve real-time weather and congestion conditions.

[1472] The server uses the collected data to filter out tourist attractions and dining options that the user has not visited before, and then narrows down the list of suitable places to visit based on weather information and crowding conditions.

[1473] When calculating travel time and required time, the server takes into account the user's mode of transportation (walking, car, public transportation) and calculates the travel time to each candidate location. It also calculates the required time taking into account the travel time between each spot and reflects it in the schedule.

[1474] As an emotion engine, the server uses an emotion recognition model (e.g., an emotion analysis API) to identify the user's current emotional state. Specifically, it can detect the user's emotions from facial expression analysis using a camera or voice input. Based on the identified emotional state, it will suggest activities and spots that match the user's current emotions. For example, if the user is feeling stressed, it will suggest relaxation spots.

[1475] When generating a plan, the server ranks each spot based on information such as rating, travel time, and distance, and selects the top spots. It then optimizes the order of visits, taking travel time and required time into account, to generate an efficient travel plan. The server also reflects the output of the emotion engine to create a plan optimized for the user's emotional state.

[1476] Finally, the generated optimal travel plan is sent from the server to the device. The device displays the travel plan sent from the server to the user, providing detailed information about each spot (address, opening hours, ratings, weather forecast, and congestion status). The information is presented in the form of a map or schedule to make it easier for the user to check the travel plan.

[1477] As a specific example of how it works, if you enter "I want to go sightseeing, experience art, and eat local cuisine" into the prompt text, the system will categorize your requests into categories and create and present the most suitable travel plan. In this way, the system reflects the user's desires and emotions in real time and provides the most suitable travel plan, further enriching the user's travel experience.

[1478] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1479] Step 1: Initialize and accept user input

[1480] User:

[1481] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[1482] Input: Text information that a user types into a terminal.

[1483] Output: Parsed activity.

[1484] Device:

[1485] The activities entered by the user are analyzed and classified into categories (sightseeing, art, gourmet). For example, "I want to see Tokyo Tower, I want to go to the Ueno Museum, I want to eat ramen in Asakusa."

[1486] Input: Text information entered by the user.

[1487] Data processing: Text analysis and classification into categories.

[1488] Output: Categorized activities.

[1489] The analysis results are sent to the server.

[1490] Step 2: Gather information

[1491] server:

[1492] The transmitted input data is received, and information on the user's past visit history, current location, and means of transportation from the departure point is retrieved from the database.

[1493] Input: Analysis results sent from the device.

[1494] Data processing: Retrieve data from user database.

[1495] Output: User's past visit history and current location information.

[1496] Use a weather information API (e.g., a general name for the weather information API) or a congestion information API (e.g., a general name for the congestion information API) to obtain real-time weather conditions and congestion status.

[1497] Input: Acquired past visit history and current location information.

[1498] Data processing: Obtain weather and congestion information from the API.

[1499] Output: Real-time weather and traffic conditions.

[1500] Step 3: Data analysis and filtering

[1501] server:

[1502] Based on the collected data, it filters out tourist spots and dining options that the user has not visited before.

[1503] Input: All collected data (past visit history, current location, weather information, crowd status).

[1504] Data processing: Filter out unvisited spots.

[1505] Output: A filtered list of attractions and places to eat.

[1506] Narrow down potential places to visit based on weather information and crowding levels.

[1507] Input: Filtered list with weather and crowd information.

[1508] Data processing: Narrow down spots suitable for weather and crowd conditions.

[1509] Output: A narrowed down list of potential locations.

[1510] Step 4: Calculate travel time and journey time

[1511] server:

[1512] The travel time to each candidate location is calculated taking into account the user's mode of transportation (walking, car, public transportation). For example, "It takes 30 minutes to walk from Tokyo Station to Ueno Museum of Art."

[1513] Input: User's travel information and a narrowed list of possible locations.

[1514] Data processing: Calculate travel time to each candidate location.

[1515] Output: A list of travel times to each candidate location.

[1516] The travel time between each spot is taken into account to calculate the required time and reflected in the schedule.

[1517] Input: Travel time list and candidate location list.

[1518] Data processing: Calculate travel time between each spot and reflect it in the schedule.

[1519] Output: Visit schedule with duration.

[1520] Step 5: Emotional Engine Adjustment

[1521] server:

[1522] The emotion engine (e.g., emotion analysis API) is used to identify the user's current emotional state. Specifically, the user's emotions are detected from facial expression analysis using a camera or voice input.

[1523] Input: User's facial and voice data.

[1524] Data processing: Emotional state is analyzed using an emotion engine.

[1525] Output: The user's current emotional state.

[1526] Based on the identified emotional state, the system suggests activities and places that fit the user's current emotions, for example, if the user is feeling stressed, it suggests relaxation spots.

[1527] Input: User's emotional state and schedule.

[1528] Data manipulation: Adjust your schedule based on your emotional state.

[1529] Output: An emotion-appropriate visit schedule.

[1530] Step 6: Generate and optimize the plan

[1531] server:

[1532] Rankings are made based on information such as the rating of each spot, time, and distance traveled, and the top spots are selected.

[1533] Input: Information such as rating of the spot, time, and distance traveled.

[1534] Data processing: Ranking is carried out and the top spots are selected.

[1535] Output: A list of selected top spots.

[1536] Taking into account travel time and travel time, the order of visits is optimized to generate an efficient itinerary.

[1537] Input: Top spot list, travel time, and travel time.

[1538] Data processing: Optimize visit order and generate travel plans.

[1539] Output: An efficient travel plan.

[1540] The output of the emotion engine is reflected to create a plan optimized for the user's emotional state.

[1541] Input: Emotionally appropriate visit schedule.

[1542] Data processing: Create a plan optimized for your emotional state.

[1543] Output: Optimized trip plan.

[1544] Step 7: View your plan

[1545] Device:

[1546] The optimum travel plan sent from the server is displayed to the user.

[1547] Input: The itinerary sent from the server.

[1548] Data processing: Formatting the travel plan for display.

[1549] Output: The plan that is displayed to the user.

[1550] Detailed information on each spot (address, opening hours, ratings, weather forecast, and crowd status) is provided.

[1551] Input: Spot information within the travel plan.

[1552] Data processing: Incorporating detailed information into the display.

[1553] Output: A display with detailed information.

[1554] Information is presented in the form of maps and schedules to make it easier for users to check their travel plans.

[1555] Input: Travel plan and details.

[1556] Data processing: Formatting into map or schedule format screens.

[1557] Output: Plans displayed in map and schedule format.

[1558] This allows the system to reflect users' requests and feelings in real time and provide optimal travel plans.

[1559] (Application example 2)

[1560] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1561] Conventional travel planning support systems could suggest optimal tourist attractions and dining options by taking into account the user's past visit history, weather conditions, and congestion levels, but they did not provide plans that reflected the user's current emotional state. This made it difficult to generate optimal travel plans that addressed emotions such as stress and excitement that users felt during their trip. This is needed to further enhance the travel experience and increase satisfaction.

[1562] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1563] In this invention, the server includes an input means for inputting the activities the user wishes to engage in during their trip, a data collection means for collecting data on the user's past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots that best suit the user's preferences based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means for recognizing the user's emotions in real time and making suggestions based on those emotions. This makes it possible to provide a travel plan that matches the user's current emotional state, further enriching the travel experience.

[1564] Definitions of important words

[1565] A "travel planning support system" is a system that supports the activities that users want to undertake while traveling, suggesting the most suitable tourist facilities and places to eat by taking into consideration data such as the user's past visit history, current location, weather, congestion level, and mode of transportation.

[1566] "Input means" refers to a device or interface that allows a user to input the activities and preferences they wish to undertake during their trip.

[1567] "Data collection means" refers to means for collecting information about a user's past visit history, current location, weather, congestion, and means of transportation.

[1568] The "suggestion method" is a method of selecting and suggesting the most suitable tourist facilities and dining spots to users based on the collected data.

[1569] "Display means" refers to a means for visually presenting information such as suggested travel plans, selected tourist facilities, and dining options to users.

[1570] The "emotion engine means" is a means for recognizing the user's emotions in real time and adjusting the content of suggestions based on those emotions.

[1571] This invention is a travel plan creation support system that provides a more sophisticated travel experience by combining an emotion engine that recognizes the user's emotions. This system comprises an input means for inputting the activities the user wishes to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means.

[1572] The specific configuration and processing of a system for implementing the present invention are as follows.

[1573] Technology stack used

[1574] Hardware

[1575] Smartphone

[1576] Smart Glasses

[1577] head-mounted display

[1578] software

[1579] OpenCV for Sentiment Analysis

[1580] Google Speech-to-Text API for voice analysis

[1581] A server-side program with a unique algorithm for movie suggestions

[1582] Program processing explanation

[1583] Initialization and accepting user input

[1584] The user starts the travel planning support system and inputs the activities they would like to do during their trip. For example, they might input text such as "I want to do some sightseeing, experience art, and try local cuisine" into the device. This input is done via a smartphone, smart glasses, or a head-mounted display.

[1585] The device analyzes the entered activity details, classifies them into categories (tourism, art, gourmet), and sends the analysis results to the server.

[1586] Information gathering and emotion recognition

[1587] The server receives the input data and retrieves past visit history, current location, and mode of transportation from the database. It also obtains real-time weather and congestion information using weather information APIs and congestion information APIs.

[1588] The device uses the built-in camera and microphone to capture the user's facial expressions and voice, and recognizes emotions in real time using OpenCV and Emotion APIs (e.g., Microsoft Face API). It also analyzes voice data with the Google Speech-to-Text API to identify emotions from speaking tone.

[1589] Generate and view plans

[1590] The server selects filtered tourist spots and dining options based on the collected data, and makes optimal suggestions based on the user's current emotional state. It generates an efficient travel plan by taking into account the ratings of each spot and the order in which they should be visited. It reflects the output of the emotion engine and creates a plan optimized for the user's emotional state.

[1591] The terminal displays the optimal travel plan sent from the server to the user and provides detailed information about each spot (address, opening hours, ratings, weather forecast, and congestion status).

[1592] Specific examples

[1593] 1. Example of user input:

[1594] "I want to see the sights, experience art, and eat local cuisine."

[1595] 2. Example prompt:

[1596] What is your current mood and what genre do you want to watch?

[1597] Example: I want to relax today, so I want to watch a comedy movie.

[1598] In this way, it is possible to provide an optimal travel plan that reflects the user's emotional state in real time.The system utilizes a generative AI model to further explore the user's needs through prompt sentences, enabling the system to achieve an optimal travel experience.

[1599] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1600] Program processing steps

[1601] Step 1:

[1602] The user starts the travel planning support system and inputs the activities and requests he / she wishes to perform during the trip.

[1603] Input: Activities such as "I want to go sightseeing, experience art, and eat local cuisine"

[1604] Output: Categorization of analyzed activities (sightseeing, art, gourmet)

[1605] Specific operation: Data entered into a smartphone, smart glasses, or head-mounted display is analyzed using natural language processing and classified into categories. The analysis results are then sent to a server.

[1606] Step 2:

[1607] The server receives the input data and retrieves past visit history, current location, and mode of transportation from the database. It also uses weather information API and congestion information API to obtain real-time weather and congestion information.

[1608] Input: User input data, visit history, current location, weather API, congestion information API

[1609] Output: Obtained weather and congestion information

[1610] Specific operation: The server executes database queries to obtain past visit history and current location data, and calls weather APIs and congestion information APIs to obtain real-time data.

[1611] Step 3:

[1612] The device uses the built-in camera and microphone to capture the user's facial expressions and voice, and recognizes emotions in real time using OpenCV and the Emotion API. The voice data is also analyzed using the Google Speech-to-Text API to identify emotions from the tone of speech.

[1613] Input: User's facial expression data, voice data

[1614] Output: Perceived emotional state (e.g., relaxed, excited, stressed)

[1615] What it does: It activates the camera and microphone, performs facial expression analysis using OpenCV and the Emotion API, and analyzes audio data using the Google Speech-to-Text API to determine emotions based on the tone of the voice.

[1616] Step 4:

[1617] Based on the collected data, the server selects filtered tourist spots and dining options, making optimal suggestions based on the user's current emotional state.

[1618] Input: Filtered tourist spots, dining options, and the user's emotional state

[1619] Output: A ranked list of the best places to see and eat

[1620] What it does: It takes into account the output of the emotion engine and applies an algorithm to generate rankings of tourist attractions and dining destinations based on past visit history and current situation data.

[1621] Step 5:

[1622] The server takes into account the ratings of each spot and the order in which they should be visited, generates an efficient travel plan, and sends it to the terminal.

[1623] Input: Ranking list of spots, rating data, visit order

[1624] Output: Optimized itinerary

[1625] Specific operation: Calculates the most efficient visiting order based on the rating, travel time, and duration of stay of each spot, and generates an optimal travel plan.

[1626] Step 6:

[1627] The terminal displays the optimal travel plan sent from the server to the user and provides detailed information about each spot (address, opening hours, ratings, weather forecast, and congestion status).

[1628] Input: Optimized itinerary, detailed information for each spot

[1629] Output: The best itinerary and details displayed to the user

[1630] What it does: It visually displays the itinerary and detailed information of each spot on a map and in schedule format, making it easy for users to check.

[1631] As a result, it is possible to realize a system that provides optimal travel plans that reflect the user's emotional state in real time, further enriching the travel experience.

[1632] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1633] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1634] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1635] [Fourth embodiment]

[1636] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1637] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1638] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1639] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1640] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1641] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1642] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1643] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1644] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1645] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1646] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1647] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1648] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1649] The present invention is a travel planning support system that provides an optimal travel plan based on user input. This system has an input means for inputting activities the user wants to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and transportation methods, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, and a display means for presenting the suggested travel plan to the user.

[1650] System Programming and Processing

[1651] Initialization and accepting user input

[1652] User:

[1653] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[1654] Device:

[1655] The input information is analyzed and classified into each activity category.

[1656] The analysis results are sent to the server.

[1657] Information gathering

[1658] server:

[1659] Past visit history, current location, and mode of transportation are retrieved from the database.

[1660] Use weather information APIs and congestion information APIs to obtain real-time weather and congestion information.

[1661] Obtain data on the means of travel and the travel time required.

[1662] Data Analysis and Filtering

[1663] server:

[1664] Based on the collected data, it filters out tourist spots and dining options that you have not visited before.

[1665] For each filtered spot, a priority is determined based on weather information and congestion status.

[1666] Travel and journey time calculations

[1667] server:

[1668] Calculate travel time to each spot based on transportation options such as public transportation, car, or walking.

[1669] Consider travel time between each spot and adjust your schedule so that the total time is not too long.

[1670] Plan Generation and Optimization

[1671] server:

[1672] Rankings are based on information such as the rating of each spot, time, and distance traveled.

[1673] Arrange spots in the optimal order to generate an efficient travel plan.

[1674] If necessary, it will reflect real-time data and allow you to dynamically change your plan during your trip.

[1675] View Plans

[1676] Device:

[1677] The optimum travel plan sent from the server is displayed to the user.

[1678] Detailed information about each spot (address, opening hours, ratings, expected weather, and crowd levels) is also provided.

[1679] Information is presented in the form of maps and schedules to make it easier for users to check their travel plans.

[1680] Specific operation example

[1681] 1. User Settings:

[1682] User: Enters into the device, "I want to go sightseeing, experience art, and eat local cuisine."

[1683] Terminal: Parses input and sends it to the server.

[1684] 2. Information Collection:

[1685] Server: Checks past visit history and retrieves unvisited tourist spots and dining options from the database.

[1686] Server: Get the current weather conditions using the weather information API (e.g. sunny).

[1687] Server: Obtain congestion status using the congestion information API (e.g., not crowded).

[1688] 3. Data analysis and schedule calculation:

[1689] Server: Calculates the travel method and travel time based on the filtered spots (e.g., 30 minutes from Tokyo Station to Ueno Museum).

[1690] 4. Generate the plan:

[1691] Server: Generates a plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which to visit them (e.g., Tokyo Tower in the morning, Ueno Museum in the afternoon, and a ramen restaurant in Asakusa for lunch).

[1692] 5. Present the plan:

[1693] Terminal: Displays the game plan to the user along with a map and provides detailed information.

[1694] This will enable users to easily create efficient and comprehensive travel plans, further increasing the enjoyment of their trip.The system of the present invention can reflect users' requests in real time and dynamically change plans, making it possible to provide a flexible travel experience.

[1695] The processing flow will be explained below.

[1696] Step 1:

[1697] User:

[1698] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[1699] Step 2:

[1700] Device:

[1701] The entered user activity information is analyzed and classified into categories (tourism, art, gourmet).

[1702] The analysis results are sent to the server.

[1703] Step 3:

[1704] server:

[1705] Receives the input data sent.

[1706] The user's past visit history is retrieved from the database.

[1707] Check the user's current location.

[1708] Step 4:

[1709] server:

[1710] Use the weather information API to get the weather forecast for the planned travel date.

[1711] Use the crowd information API to obtain information on the crowding status of tourist attractions and dining spots.

[1712] Step 5:

[1713] server:

[1714] It checks past visit history and lists tourist spots and dining options the user has not yet visited.

[1715] Narrow down suitable locations to visit based on weather information and crowding levels.

[1716] Step 6:

[1717] server:

[1718] Taking into account the user's mode of transportation (walking, car, public transportation), the travel time to each candidate location is calculated.

[1719] Calculate the travel time for each mode of transportation and reflect it in the schedule.

[1720] Step 7:

[1721] server:

[1722] Rankings are made based on the ratings of each spot (reviews, number of stars, etc.), and the top spots are selected.

[1723] Optimize the order of visits and generate efficient travel plans, taking into account travel time and travel time.

[1724] Step 8:

[1725] server:

[1726] An optimized itinerary is generated and sent to the device, including detailed information (address, opening hours, ratings, weather forecast, and crowding status).

[1727] Step 9:

[1728] Device:

[1729] The submitted itinerary is displayed to the user.

[1730] The plan details are presented in the form of maps and schedules.

[1731] If the user has any additional questions or changes they would like to make, they submit them to the server.

[1732] Step 10:

[1733] server:

[1734] Receive user feedback and adjust / update your itinerary again if necessary.

[1735] Send the new plan to the device and have it reload.

[1736] As a result, users can use the system to efficiently plan their trips, and real-time information updates enable a flexible travel experience.

[1737] Example 1

[1738] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1739] Conventional travel planning support systems have struggled to automatically generate efficient travel plans that match the user's preferences and interests, and to optimize them in real time. Furthermore, technology for quickly and effectively analyzing the wide variety of requests entered by users and incorporating them into travel plans has not yet been developed. This has resulted in the problem of users having to manually search and organize information and create travel plans, which is time-consuming.

[1740] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1741] In this invention, the server includes an input means for inputting activities the user wishes to engage in during their trip, a data collection means for collecting data on the user's past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots that best suit the user's preferences based on the collected data, a display means for presenting the suggested travel plan to the user, a means for analyzing the user's input and classifying it into activity categories using natural language processing, and a means for determining the priority of each spot based on the collected data and generating an efficient travel plan. This makes it possible to automatically generate an optimal travel plan based on the user's requests and dynamically optimize the plan in real time.

[1742] The "input means" refers to a device or function that allows a user to input the activities that the user wants to do during the trip.

[1743] "Data collection means" refers to devices or functions that collect data regarding a user's past visit history, travel time, current location, weather, congestion, and means of transportation.

[1744] "Proposal means" refers to devices or functions that select and suggest tourist facilities and dining options that best suit the user's needs based on collected data.

[1745] "Display means" refers to a device or function for visually presenting the proposed travel plan to the user.

[1746] "Natural language processing" is an artificial intelligence technique used to analyze user input and classify it into activity categories.

[1747] "Filtering" is the process of extracting information that meets specific conditions from a large amount of data.

[1748] The "priority determination means" is a device or function for determining the order of visits and importance of each spot based on the collected data.

[1749] The "plan generation means" refers to a device or function for automatically creating an efficient travel plan.

[1750] The present invention relates to a travel planning support system that provides an optimal travel plan based on user input. The system includes an input unit, a data collection unit, a proposal unit, a display unit, a natural language processing unit, a filtering unit, a priority determination unit, and a plan generation unit.

[1751] System Programming and Processing

[1752] The program of this system includes the following hardware and software.

[1753] Input method: Using a smartphone or tablet application, users input the activities they want to do during their trip.

[1754] Data collection method: Data such as past visit history, current location, real-time weather information, congestion information, and travel time is collected using a database on the server and external APIs (e.g., OpenWeatherMap API, Google Maps API).

[1755] Natural language processing tools: Using a generative AI model, it analyzes user input and categorizes it into categories such as tourism, art, and food.

[1756] Filtering method: Based on the collected data, we filter out tourist attractions and dining spots that the user has not visited before.

[1757] Priority determination method: Determine the priority of each filtered spot based on weather information and crowding.

[1758] Plan generation method: Based on the priority, arrange tourist spots in the optimal order and generate an efficient travel plan.

[1759] Display method: The generated travel plan is displayed to the user via a smartphone or tablet application.

[1760] Program processing

[1761] Users start the system and input the activities they want to do during their trip (e.g., "sightseeing," "art," "gourmet") The application on the device analyzes the input, classifies the activities into categories using natural language processing, and sends the data to the server.

[1762] The server retrieves past visit history from the database and collects current location and real-time weather and congestion information via an external API. It also obtains the mode of transportation used by the user and the travel time required. Based on this data, filtering is performed to extract tourist spots and dining spots that have not been visited before. The server then prioritizes destinations, taking into account weather and congestion information.

[1763] The server calculates travel time to each spot based on the mode of transportation (train, bus, car, walking, etc.) and creates an efficient schedule taking into account travel time between each spot.Finally, it selects highly rated tourist spots and dining spots based on the user's preferences and generates a plan that visits them in the optimal order.

[1764] The generated itinerary is sent to the device and displayed to the user. Detailed information about the itinerary and each spot (address, opening hours, ratings, expected weather, and congestion status) is provided in map and schedule format for easy viewing by the user.

[1765] Specific examples

[1766] 1. User Settings:

[1767] User: Enters into the device, "I want to go sightseeing, experience art, and eat local cuisine."

[1768] 2. Information Collection:

[1769] Server: Checks past visit history and retrieves unvisited tourist spots and dining spots from the database. For example, if the user has visited Sensoji Temple in the past, Skytree, which has not yet been visited, will be suggested.

[1770] Server: Get the current weather conditions using the weather information API (e.g. sunny).

[1771] Server: Obtain the congestion status using the congestion information API (e.g., Ueno Museum is not crowded).

[1772] 3. Data analysis and prioritization:

[1773] Server: Based on the filtered spots, calculate the travel method and travel time using Google Maps API (e.g., 30 minutes from Tokyo Station to Ueno Museum).

[1774] 4. Generate the plan:

[1775] Server: Generates a plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which to visit them (e.g., Tokyo Tower in the morning, Ueno Museum in the afternoon, and a ramen restaurant in Asakusa for lunch).

[1776] 5. Present the plan:

[1777] Terminal: Displays the itinerary to the user along with a map and provides detailed information.

[1778] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1779] Step 1: Accepting User Input

[1780] User: The user inputs the activities they would like to do during their trip (e.g., "I want to do some sightseeing," "I want to experience art," "I want to try local cuisine") into an application on their smartphone or tablet.

[1781] Terminal: The terminal receives the input information, analyzes it, and classifies each activity into a category (sightseeing, art, gourmet, etc.). This analysis is performed using natural language processing technology. The analysis results are then sent to the server.

[1782] Input: User activity input ("I want to go sightseeing," "I want to experience art," "I want to eat local cuisine")

[1783] Output: Categorized activity information (sightseeing, art, gourmet)

[1784] Step 2: Data collection

[1785] Server: The server retrieves past visit history and current location information from the database. It also uses external APIs (e.g., OpenWeatherMap API, Google Maps API) to collect real-time weather information, congestion information, transportation methods, and travel times.

[1786] Input: Categorized activity information

[1787] Output: Past visit history, current location, weather information, congestion information, transportation method and travel time data

[1788] Step 3: Data analysis and filtering

[1789] Server: Based on the collected data, it filters tourist spots and dining spots that the user has not visited before. Specifically, it compares the user's past visit history with the user's current location information and lists unvisited spots.

[1790] Input: Past visit history, current location, weather information, congestion information, transportation method and travel time data

[1791] Output: A filtered list of places to visit and eat

[1792] Step 4: Prioritize

[1793] Server: Determines the priority of each filtered spot based on weather and congestion information. For example, it applies rules such as prioritizing indoor spots on rainy days.

[1794] Input: Filtered list of attractions and dining options, weather information, and crowd information

[1795] Output: A prioritized list of places to see and eat

[1796] Step 5: Calculate travel time and journey time

[1797] Server: Calculates travel time to each location for each mode of transportation. Specifically, it uses the Google Maps API to obtain travel times based on modes of transportation such as public transportation, car, and walking, and adjusts travel schedules between each location.

[1798] Input: A prioritized list of places to see and eat

[1799] Output: Travel time and travel time data between each location

[1800] Step 6: Generate and optimize the plan

[1801] Server: Generates an efficient travel plan by arranging each spot in the optimal order, taking into account the rating of each spot, the duration of stay, and the travel distance. If necessary, the plan can be dynamically modified to reflect real-time data.

[1802] Input: Travel time and travel time between each spot, rating of each spot, and duration of stay

[1803] Output: Optimal travel plan

[1804] Step 7: View your plan

[1805] Terminal: The optimal travel plan sent from the server is displayed to the user. For easy viewing, the travel plan and detailed information about each spot (address, opening hours, rating, forecast weather, and congestion status) are provided in map and schedule format.

[1806] Input: Best Travel Plan

[1807] Output: A travel plan presented to the user with details about each spot

[1808] (Application example 1)

[1809] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1810] Conventional travel planning support systems suggest tourist attractions and dining options for users during their trips, but lack integration with food delivery services, allowing travelers to order meals and have them delivered to designated locations. This makes it difficult for users to enjoy an optimal dining experience while traveling without any hassle. Furthermore, systems that dynamically modify travel plans using real-time information such as weather and congestion are often inadequate. As a result, these systems are unable to provide flexible travel plans for travelers, making it difficult to adapt to unexpected situations and traffic jams.

[1811] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1812] In this invention, the server includes an input means for inputting activities the user wishes to engage in during the trip, a data collection means for collecting data on the user's past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots that best suit the user's preferences based on the collected data, a display means for presenting the suggested travel plan to the user, and a means for collecting and suggesting data on food delivery services for ordering meals for travelers and having them delivered at specified locations. This makes it possible to provide a comprehensive and dynamic travel plan that includes meal arrangements during the trip.

[1813] A "travel planning support system" is a system that selects and suggests the most suitable tourist facilities and dining spots based on the activities that the user wants to do during their trip, and presents the plan.

[1814] "Input means" refers to a device or interface that allows a user to input the activities they wish to undertake during their trip.

[1815] "Data collection means" refers to means that has the function of collecting data regarding a user's past visit history, travel time, current location, weather, congestion, and means of transportation.

[1816] The "suggestion means" is a means that has the function of selecting and suggesting tourist facilities and dining spots that best suit the user's needs based on the collected data.

[1817] "Display" refers to a device or interface for visually displaying a proposed travel plan to a user.

[1818] A "food delivery service" is a platform that provides meal-related services that allow users to order and have food delivered to a location they specify.

[1819] "Weather information" refers to meteorological data that is updated in real time and is used for planning and dynamically changing travel plans.

[1820] "Crowding information" refers to data on the congestion status of planned visits and travel routes, and is updated in real time.

[1821] "Transportation" refers to the means by which users travel during their trip, such as public transportation, car, or walking.

[1822] "Past visit history" refers to data about tourist attractions and dining options that users have visited in the past.

[1823] A "dynamically changing means" is a means that has the function of changing a travel plan to reflect real-time data.

[1824] This invention provides users with efficient and optimal travel plans and comprehensive support, including meal arrangements, through a travel planning support system. The system consists of a smartphone or other device used by the user, a server for analyzing and processing data, and an interface for proposing and displaying travel plans.

[1825] System Programming and Processing

[1826] Initialization and accepting user input

[1827] User:

[1828] The user starts up the device and inputs the activities they would like to do during their trip, such as "I want to do some sightseeing," "I want to experience art," or "I want to try local cuisine."

[1829] Device:

[1830] The information entered is analyzed, and each activity is classified into a category. The analysis results are sent to the server.

[1831] Information gathering

[1832] server:

[1833] Data on past visit history, current location, and mode of transportation is retrieved from the database.

[1834] Use weather information APIs and congestion information APIs (e.g., OpenWeatherMap API and Google Maps Traffic API) to obtain real-time weather and congestion information.

[1835] Obtain available dining destinations from a database of partner food delivery services.

[1836] Data Analysis and Filtering

[1837] server:

[1838] The collected data is analyzed to filter out tourist spots and dining options that users have not visited before.

[1839] The priority of each filtered spot is determined based on weather information and congestion status.

[1840] Travel time and order timing calculations

[1841] server:

[1842] Uses Google Maps API to calculate travel time to each location.

[1843] Calculate the timing of your order so that your meal will be delivered when you arrive at your destination.

[1844] Plan Generation and Optimization

[1845] server:

[1846] The system generates a travel plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which you visit them.

[1847] The generated plan includes the order of places to visit and events, as well as when to order meals.

[1848] If necessary, itineraries can be dynamically modified to reflect real-time data.

[1849] View Plans

[1850] Device:

[1851] The optimal travel plan sent from the server is displayed to the user in the form of a map and schedule.

[1852] It also provides detailed information about each spot and restaurant (address, opening hours, ratings, expected weather, and crowd levels).

[1853] Specific example explanation

[1854] During initialization and user input reception, the user enters into the device, "I want to go sightseeing, experience art, and eat local cuisine." The device analyzes the user's input and sends the results to the server. The server collects data based on past visit history and current location, and obtains real-time data using weather information APIs and congestion information APIs.

[1855] For example, the server calculates the travel time from Tokyo Station to the Ueno Museum of Art in 30 minutes, and determines when to order sushi using a food delivery service 30 minutes before departure.The server then creates a plan that allows the meal to be delivered upon arrival at the destination, generating an efficient plan that allows the user to enjoy sightseeing and dining.

[1856] Prompt Sentence Examples

[1857] Please tell us where you would like to go and what food you would like to eat during your trip.

[1858] Example input:

[1859] Tourist attractions: Historic buildings, museums, parks

[1860] Food preferences: Sushi, ramen, Japanese sweets

[1861] As described above, this invention provides users with a fulfilling travel experience. Examples of software and APIs used include OpenWeatherMap API, Google Maps API, and UberEats API. Hardware primarily involves smartphones, tablet devices, and servers.

[1862] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1863] Step 1:

[1864] Initialization and accepting user input

[1865] Users turn on their smartphones or tablets and input the activities they want to do during their trip. Examples of input include "I want to do some sightseeing," "I want to experience art," and "I want to try local cuisine."

[1866] The device receives the input information and analyzes it using a natural language processing algorithm. This analysis classifies the input content into categories such as "tourism," "art," and "food." The analysis results are sent to the server in JSON format.

[1867] Step 2:

[1868] Information gathering

[1869] The server retrieves the user's past visit history and current location from a database based on the data received from the device, and also obtains real-time weather information from the OpenWeatherMap API and congestion information from the Google Maps Traffic API.

[1870] In addition, the server obtains a list of meal delivery locations from the API of affiliated food delivery services (e.g., Uber Eats, Demae-can, etc.).

[1871] The input data used is user preferences in JSON format, past visit history from the database, real-time information obtained from the API, and information from food delivery services.

[1872] Step 3:

[1873] Data Analysis and Filtering

[1874] The server analyzes the collected data and filters out tourist attractions and dining options that users have not visited before, using the Python pandas library to process the data.

[1875] This allows the system to extract unvisited spots based on past visit history, and prioritize suitable spots taking into account weather and crowding conditions.

[1876] It takes user preferences, real-time information, and food delivery information organized in JSON format as input, and generates a prioritized list of spots as output.

[1877] Step 4:

[1878] Travel time and order timing calculations

[1879] The server uses the Google Maps API to calculate travel times to each location and also calculates food delivery order times to ensure meals arrive at the expected arrival times.

[1880] Use Python's datetime library to schedule your visits and calculate the optimal time to visit each spot.

[1881] The inputs are the user's current location, next destination, and food delivery service data, and the output is an adjusted visit schedule and order timing.

[1882] Step 5:

[1883] Plan Generation and Optimization

[1884] The server generates an optimal travel plan by taking into account the ratings of each spot, the order of visits, travel time, and meal ordering timing. It applies machine learning models using Python's scikit-learn for optimization.

[1885] If necessary, the itinerary is dynamically changed to reflect real-time data.

[1886] It takes user preferences, real-time data, and food delivery information as inputs and generates an optimized travel plan as output.

[1887] Step 6:

[1888] View Plans

[1889] The device receives the optimal travel plan sent from the server and visually displays it to the user. It uses React.js to display the information in the form of a map and schedule.

[1890] Detailed information about each spot and restaurant (address, opening hours, ratings, expected weather, and crowd levels) is also displayed.

[1891] It takes as input travel plan data from a server and as output a visual travel plan that is displayed on the user's screen.

[1892] Through the above steps, the travel planning support system can provide users with a fulfilling travel experience.

[1893] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1894] This invention is a travel plan creation support system that provides a more sophisticated travel experience by combining an emotion engine that recognizes the user's emotions. This system comprises an input means for inputting the activities the user wishes to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means.

[1895] System Programming and Processing

[1896] Initialization and accepting user input

[1897] User:

[1898] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[1899] Device:

[1900] The entered activity details are analyzed and classified into categories (tourism, art, gourmet).

[1901] The analysis results are sent to the server.

[1902] Information gathering

[1903] server:

[1904] The transmitted input data is received, and past visit history, current location, and mode of travel are retrieved from the database.

[1905] We use weather information APIs and congestion information APIs to obtain real-time weather conditions and congestion information.

[1906] Data Analysis and Filtering

[1907] server:

[1908] Based on the collected data, it filters out tourist spots and dining options that you have not visited before.

[1909] Narrow down suitable locations to visit based on weather information and crowding levels.

[1910] Travel and journey time calculations

[1911] server:

[1912] Taking into account the user's mode of transportation (walking, car, public transportation), the travel time to each candidate location is calculated.

[1913] The travel time between each spot is taken into account to calculate the required time and reflected in the schedule.

[1914] Emotional Engine Adjustment

[1915] server:

[1916] The emotion engine means is used to identify the user's current emotional state, for example by detecting the user's emotions from facial expression analysis using a camera or voice input.

[1917] Based on the identified emotional state, the system suggests activities and places that fit the user's current emotions, for example, if the user is feeling stressed, it suggests relaxation spots.

[1918] Plan Generation and Optimization

[1919] server:

[1920] Rankings are made based on information such as the rating of each spot, time, and distance traveled, and the top spots are selected.

[1921] Generate efficient travel plans by optimizing the order of visits taking into account travel time and required time.

[1922] The output of the emotion engine is reflected to create a plan optimized for the user's emotional state.

[1923] View Plans

[1924] Device:

[1925] The optimum travel plan sent from the server is displayed to the user.

[1926] Detailed information on each spot (address, opening hours, ratings, weather forecast, and crowd status) is provided.

[1927] Information is presented in the form of maps and schedules to make it easier for users to check their travel plans.

[1928] Specific operation example

[1929] 1. User Settings:

[1930] User: Enters into the device, "I want to go sightseeing, experience art, and eat local cuisine."

[1931] Terminal: Parses input and sends it to the server.

[1932] 2. Information Collection:

[1933] Server: Checks past visit history and retrieves unvisited tourist spots and dining options from the database.

[1934] Server: Get the current weather conditions using the weather information API (e.g. sunny).

[1935] Server: Obtain congestion status using the congestion information API (e.g., not crowded).

[1936] 3. Emotion Recognition with Emotion Engine:

[1937] Server: Analyzes the user's facial expressions and voice using an emotion engine to identify their current emotional state (e.g., relaxed, excited, stressed, etc.).

[1938] Server: Based on the identified emotion, suggest quiet cafes or tranquil gardens to users who need to relax.

[1939] 4. Data analysis and plan generation:

[1940] Server: Calculates the means of transportation and travel time based on the filtered spots (e.g., 30 minutes from Tokyo Station to Ueno Museum).

[1941] Server: Generates a plan that combines the optimal route and travel time, taking into account the ratings of each spot and the order in which to visit them (e.g., Tokyo Tower in the morning, Ueno Museum in the afternoon, and a ramen restaurant in Asakusa for lunch).

[1942] 5. Present the plan:

[1943] Terminal: The travel plan sent from the server is displayed to the user along with a map, and detailed information is also provided.

[1944] As described above, this system can further enrich users' travel experience by reflecting their requests and feelings in real time and providing optimal travel plans.

[1945] The processing flow will be explained below.

[1946] Step 1:

[1947] User:

[1948] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[1949] Step 2:

[1950] Device:

[1951] The entered activity details are analyzed and classified into categories (tourism, art, gourmet).

[1952] The analysis results are sent to the server.

[1953] Step 3:

[1954] server:

[1955] The transmitted input data is received, and past visit history, current location, and mode of travel are retrieved from the database.

[1956] Check the user's current location.

[1957] Step 4:

[1958] server:

[1959] Use the weather information API to get the weather forecast for the planned travel date.

[1960] Use the crowd information API to obtain information on the crowding status of tourist attractions and dining spots.

[1961] Step 5:

[1962] server:

[1963] It checks past visit history and lists tourist spots and dining options the user has not yet visited.

[1964] Narrow down suitable locations to visit based on weather information and crowding levels.

[1965] Step 6:

[1966] server:

[1967] Taking into account the user's mode of transportation (walking, car, public transportation), the travel time to each candidate location is calculated.

[1968] The travel time between each spot is taken into account to calculate the required time and reflected in the schedule.

[1969] Step 7:

[1970] server:

[1971] The emotion engine means is used to identify the user's current emotional state, for example by detecting the user's emotions from facial expression analysis using a camera or voice input.

[1972] Step 8:

[1973] server:

[1974] Based on the identified emotional state, the system suggests activities and places that fit the user's current emotions, for example, if the user is feeling stressed, it suggests relaxation spots.

[1975] Step 9:

[1976] server:

[1977] Rankings are made based on information such as the rating of each spot, time, and distance traveled, and the top spots are selected.

[1978] Generate efficient travel plans by optimizing the order of visits taking into account travel time and required time.

[1979] The output of the emotion engine is reflected to create a plan optimized for the user's emotional state.

[1980] Step 10:

[1981] server:

[1982] An optimized itinerary is generated and sent to the device, including detailed information (address, opening hours, ratings, weather forecast, and crowding status).

[1983] Step 11:

[1984] Device:

[1985] The submitted itinerary is displayed to the user.

[1986] The plan is presented in detail in the form of a map or schedule, and if the user has any additional questions or requests for changes, they are sent to the server.

[1987] Step 12:

[1988] server:

[1989] Receive user feedback and adjust / update your itinerary again if necessary.

[1990] Send the new plan to your device and display it.

[1991] As a result, users can use the system to efficiently plan their trips, and with the help of the emotion engine, they can have a flexible travel experience that suits their mood at the time.

[1992] Example 2

[1993] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1994] Conventional travel planning support systems take into account data such as a user's past visit history, current location, weather, and congestion, but are unable to propose travel plans that reflect the user's emotional state. As a result, travel plans that do not match the user's mood or emotions are often proposed, resulting in an unsatisfactory travel experience. In addition, it is difficult to flexibly change plans in response to real-time changes in the environment or the user's emotions.

[1995] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an emotion engine means for recognizing the user's emotions and adjusting the travel plan based on the emotions; a means for dynamically modifying the travel plan taking into account past visit history data, weather information and congestion information updated in real time, and travel time; and an input means having a function for analyzing the user's interests and automatically classifying them into categories (sightseeing, art, gourmet, etc.) based on the interests. This makes it possible to provide an optimal travel plan that flexibly responds to the user's emotions and real-time environmental changes.

[1996] "Input means" refers to a means by which a user inputs into the system the activities that he or she wishes to undertake during his or her trip.

[1997] "Data collection means" refers to means for collecting data on a user's past visit history, travel time, current location, weather, congestion, and means of transportation.

[1998] The "proposal method" is a method for selecting and proposing tourist facilities and dining spots that best suit the user's needs based on the collected data.

[1999] The "display means" is a means for presenting the proposed travel plan to the user.

[2000] The "emotion engine means" is a means for recognizing the user's emotions and adjusting the travel plan based on those emotions.

[2001] A "travel planning support system" is a system that allows users to input the activities they wish to do during their trip, and creates and provides the optimal travel plan based on that input.

[2002] "Past visit history data" refers to data on tourist attractions and dining locations that a user has visited in the past.

[2003] "Weather information updated in real time" refers to information that acquires and updates the current weather conditions in real time.

[2004] "Crowding information" is information that indicates the current congestion status of tourist facilities and dining destinations.

[2005] "Travel time" refers to the time required for a user to travel from their current location to their destination.

[2006] This invention provides a more sophisticated travel experience by combining an emotion engine that recognizes the user's emotions in a travel plan creation support system. This system comprises an input means for the user to input the activities they wish to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and transportation means, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means.

[2007] The user starts the system and inputs the activities they wish to do during their trip (e.g., sightseeing, art, gourmet) into the terminal. The terminal analyzes the input activities and classifies them into categories (sightseeing, art, gourmet). The analysis results are sent to the server.

[2008] The server receives the input data and retrieves information on past visit history, current location, and mode of transportation from a database. It also uses a weather information API (for example, a general name for the weather information retrieval API) and a congestion information API (for example, a general name for the congestion information retrieval API) to retrieve real-time weather and congestion conditions.

[2009] The server uses the collected data to filter out tourist attractions and dining options that the user has not visited before, and then narrows down the list of suitable places to visit based on weather information and crowding conditions.

[2010] When calculating travel time and required time, the server takes into account the user's mode of transportation (walking, car, public transportation) and calculates the travel time to each candidate location. It also calculates the required time taking into account the travel time between each spot and reflects it in the schedule.

[2011] As an emotion engine, the server uses an emotion recognition model (e.g., an emotion analysis API) to identify the user's current emotional state. Specifically, it can detect the user's emotions from facial expression analysis using a camera or voice input. Based on the identified emotional state, it will suggest activities and spots that match the user's current emotions. For example, if the user is feeling stressed, it will suggest relaxation spots.

[2012] When generating a plan, the server ranks each spot based on information such as rating, travel time, and distance, and selects the top spots. It then optimizes the order of visits, taking travel time and required time into account, to generate an efficient travel plan. The server also reflects the output of the emotion engine to create a plan optimized for the user's emotional state.

[2013] Finally, the generated optimal travel plan is sent from the server to the device. The device displays the travel plan sent from the server to the user, providing detailed information about each spot (address, opening hours, ratings, weather forecast, and congestion status). The information is presented in the form of a map or schedule to make it easier for the user to check the travel plan.

[2014] As a specific example of how it works, if you enter "I want to go sightseeing, experience art, and eat local cuisine" into the prompt text, the system will categorize your requests into categories and create and present the most suitable travel plan. In this way, the system reflects the user's desires and emotions in real time and provides the most suitable travel plan, further enriching the user's travel experience.

[2015] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2016] Step 1: Initialize and accept user input

[2017] User:

[2018] Start the system and enter the activities you want to do during your trip (e.g. sightseeing, art, gourmet) into the terminal.

[2019] Input: Text information that a user types into a terminal.

[2020] Output: Parsed activity.

[2021] Device:

[2022] The activities entered by the user are analyzed and classified into categories (sightseeing, art, gourmet). For example, "I want to see Tokyo Tower, I want to go to the Ueno Museum, I want to eat ramen in Asakusa."

[2023] Input: Text information entered by the user.

[2024] Data processing: Text analysis and classification into categories.

[2025] Output: Categorized activities.

[2026] The analysis results are sent to the server.

[2027] Step 2: Gather information

[2028] server:

[2029] The transmitted input data is received, and information on the user's past visit history, current location, and means of transportation from the departure point is retrieved from the database.

[2030] Input: Analysis results sent from the device.

[2031] Data processing: Retrieve data from user database.

[2032] Output: User's past visit history and current location information.

[2033] Use a weather information API (e.g., a general name for the weather information API) or a congestion information API (e.g., a general name for the congestion information API) to obtain real-time weather conditions and congestion status.

[2034] Input: Acquired past visit history and current location information.

[2035] Data processing: Obtain weather and congestion information from the API.

[2036] Output: Real-time weather and traffic conditions.

[2037] Step 3: Data analysis and filtering

[2038] server:

[2039] Based on the collected data, it filters out tourist spots and dining options that the user has not visited before.

[2040] Input: All collected data (past visit history, current location, weather information, crowd status).

[2041] Data processing: Filter out unvisited spots.

[2042] Output: A filtered list of attractions and places to eat.

[2043] Narrow down potential places to visit based on weather information and crowding levels.

[2044] Input: Filtered list with weather and crowd information.

[2045] Data processing: Narrow down spots suitable for weather and crowd conditions.

[2046] Output: A narrowed down list of potential locations.

[2047] Step 4: Calculate travel time and journey time

[2048] server:

[2049] The travel time to each candidate location is calculated taking into account the user's mode of transportation (walking, car, public transportation). For example, "It takes 30 minutes to walk from Tokyo Station to Ueno Museum of Art."

[2050] Input: User's travel information and a narrowed list of possible locations.

[2051] Data processing: Calculate travel time to each candidate location.

[2052] Output: A list of travel times to each candidate location.

[2053] The travel time between each spot is taken into account to calculate the required time and reflected in the schedule.

[2054] Input: Travel time list and candidate location list.

[2055] Data processing: Calculate travel time between each spot and reflect it in the schedule.

[2056] Output: Visit schedule with duration.

[2057] Step 5: Emotional Engine Adjustment

[2058] server:

[2059] The emotion engine (e.g., emotion analysis API) is used to identify the user's current emotional state. Specifically, the user's emotions are detected from facial expression analysis using a camera or voice input.

[2060] Input: User's facial and voice data.

[2061] Data processing: Emotional state is analyzed using an emotion engine.

[2062] Output: The user's current emotional state.

[2063] Based on the identified emotional state, the system suggests activities and places that fit the user's current emotions, for example, if the user is feeling stressed, it suggests relaxation spots.

[2064] Input: User's emotional state and schedule.

[2065] Data manipulation: Adjust your schedule based on your emotional state.

[2066] Output: An emotion-appropriate visit schedule.

[2067] Step 6: Generate and optimize the plan

[2068] server:

[2069] Rankings are made based on information such as the rating of each spot, time, and distance traveled, and the top spots are selected.

[2070] Input: Information such as rating of the spot, time, and distance traveled.

[2071] Data processing: Ranking is carried out and the top spots are selected.

[2072] Output: A list of selected top spots.

[2073] Taking into account travel time and travel time, the order of visits is optimized to generate an efficient itinerary.

[2074] Input: Top spot list, travel time, and travel time.

[2075] Data processing: Optimize visit order and generate travel plans.

[2076] Output: An efficient travel plan.

[2077] The output of the emotion engine is reflected to create a plan optimized for the user's emotional state.

[2078] Input: Emotionally appropriate visit schedule.

[2079] Data processing: Create a plan optimized for your emotional state.

[2080] Output: Optimized trip plan.

[2081] Step 7: View your plan

[2082] Device:

[2083] The optimum travel plan sent from the server is displayed to the user.

[2084] Input: The itinerary sent from the server.

[2085] Data processing: Formatting the travel plan for display.

[2086] Output: The plan that is displayed to the user.

[2087] Detailed information on each spot (address, opening hours, ratings, weather forecast, and crowd status) is provided.

[2088] Input: Spot information within the travel plan.

[2089] Data processing: Incorporating detailed information into the display.

[2090] Output: A display with detailed information.

[2091] Information is presented in the form of maps and schedules to make it easier for users to check their travel plans.

[2092] Input: Travel plan and details.

[2093] Data processing: Formatting into map or schedule format screens.

[2094] Output: Plans displayed in map and schedule format.

[2095] This allows the system to reflect users' requests and feelings in real time and provide optimal travel plans.

[2096] (Application example 2)

[2097] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2098] Conventional travel planning support systems could suggest optimal tourist attractions and dining options by taking into account the user's past visit history, weather conditions, and congestion levels, but they did not provide plans that reflected the user's current emotional state. This made it difficult to generate optimal travel plans that addressed emotions such as stress and excitement that users felt during their trip. This is needed to further enhance the travel experience and increase satisfaction.

[2099] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2100] In this invention, the server includes an input means for inputting the activities the user wishes to engage in during their trip, a data collection means for collecting data on the user's past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots that best suit the user's preferences based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means for recognizing the user's emotions in real time and making suggestions based on those emotions. This makes it possible to provide a travel plan that matches the user's current emotional state, further enriching the travel experience.

[2101] Definitions of important words

[2102] A "travel planning support system" is a system that supports the activities that users want to undertake while traveling, suggesting the most suitable tourist facilities and places to eat by taking into consideration data such as the user's past visit history, current location, weather, congestion level, and mode of transportation.

[2103] "Input means" refers to a device or interface that allows a user to input the activities and preferences they wish to undertake during their trip.

[2104] "Data collection means" refers to means for collecting information about a user's past visit history, current location, weather, congestion, and means of transportation.

[2105] The "suggestion method" is a method of selecting and suggesting the most suitable tourist facilities and dining spots to users based on the collected data.

[2106] "Display means" refers to a means for visually presenting information such as suggested travel plans, selected tourist facilities, and dining options to users.

[2107] The "emotion engine means" is a means for recognizing the user's emotions in real time and adjusting the content of suggestions based on those emotions.

[2108] This invention is a travel plan creation support system that provides a more sophisticated travel experience by combining an emotion engine that recognizes the user's emotions. This system comprises an input means for inputting the activities the user wishes to do during their trip, a data collection means for collecting data on past visit history, travel time, current location, weather, congestion, and mode of transportation, a suggestion means for selecting and suggesting tourist facilities and dining spots based on the collected data, a display means for presenting the suggested travel plan to the user, and an emotion engine means.

[2109] The specific configuration and processing of a system for implementing the present invention are as follows.

[2110] Technology stack used

[2111] Hardware

[2112] Smartphone

[2113] Smart Glasses

[2114] head-mounted display

[2115] software

[2116] OpenCV for Sentiment Analysis

[2117] Google Speech-to-Text API for voice analysis

[2118] A server-side program with a unique algorithm for movie suggestions

[2119] Program processing explanation

[2120] Initialization and accepting user input

[2121] The user starts the travel planning support system and inputs the activities they would like to do during their trip. For example, they might input text such as "I want to do some sightseeing, experience art, and try local cuisine" into the device. This input is done via a smartphone, smart glasses, or a head-mounted display.

[2122] The device analyzes the entered activity details, classifies them into categories (tourism, art, gourmet), and sends the analysis results to the server.

[2123] Information gathering and emotion recognition

[2124] The server receives the input data and retrieves past visit history, current location, and mode of transportation from the database. It also obtains real-time weather and congestion information using weather information APIs and congestion information APIs.

[2125] The device uses the built-in camera and microphone to capture the user's facial expressions and voice, and recognizes emotions in real time using OpenCV and Emotion APIs (e.g., Microsoft Face API). It also analyzes voice data with the Google Speech-to-Text API to identify emotions from speaking tone.

[2126] Generate and view plans

[2127] The server selects filtered tourist spots and dining options based on the collected data, and makes optimal suggestions based on the user's current emotional state. It generates an efficient travel plan by taking into account the ratings of each spot and the order in which they should be visited. It reflects the output of the emotion engine and creates a plan optimized for the user's emotional state.

[2128] The terminal displays the optimal travel plan sent from the server to the user and provides detailed information about each spot (address, opening hours, ratings, weather forecast, and congestion status).

[2129] Specific examples

[2130] 1. Example of user input:

[2131] "I want to see the sights, experience art, and eat local cuisine."

[2132] 2. Example prompt:

[2133] What is your current mood and what genre do you want to watch?

[2134] Example: I want to relax today, so I want to watch a comedy movie.

[2135] In this way, it is possible to provide an optimal travel plan that reflects the user's emotional state in real time.The system utilizes a generative AI model to further explore the user's needs through prompt sentences, enabling the system to achieve an optimal travel experience.

[2136] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2137] Program processing steps

[2138] Step 1:

[2139] The user starts the travel planning support system and inputs the activities and requests he / she wishes to perform during the trip.

[2140] Input: Activities such as "I want to go sightseeing, experience art, and eat local cuisine"

[2141] Output: Categorization of analyzed activities (sightseeing, art, gourmet)

[2142] Specific operation: Data entered into a smartphone, smart glasses, or head-mounted display is analyzed using natural language processing and classified into categories. The analysis results are then sent to a server.

[2143] Step 2:

[2144] The server receives the input data and retrieves past visit history, current location, and mode of transportation from the database. It also uses weather information API and congestion information API to obtain real-time weather and congestion information.

[2145] Input: User input data, visit history, current location, weather API, congestion information API

[2146] Output: Obtained weather and congestion information

[2147] Specific operation: The server executes database queries to obtain past visit history and current location data, and calls weather APIs and congestion information APIs to obtain real-time data.

[2148] Step 3:

[2149] The device uses the built-in camera and microphone to capture the user's facial expressions and voice, and recognizes emotions in real time using OpenCV and the Emotion API. The voice data is also analyzed using the Google Speech-to-Text API to identify emotions from the tone of speech.

[2150] Input: User's facial expression data, voice data

[2151] Output: Perceived emotional state (e.g., relaxed, excited, stressed)

[2152] What it does: It activates the camera and microphone, performs facial expression analysis using OpenCV and the Emotion API, and analyzes audio data using the Google Speech-to-Text API to determine emotions based on the tone of the voice.

[2153] Step 4:

[2154] Based on the collected data, the server selects filtered tourist spots and dining options, making optimal suggestions based on the user's current emotional state.

[2155] Input: Filtered tourist spots, dining options, and the user's emotional state

[2156] Output: A ranked list of the best places to see and eat

[2157] What it does: It takes into account the output of the emotion engine and applies an algorithm to generate rankings of tourist attractions and dining destinations based on past visit history and current situation data.

[2158] Step 5:

[2159] The server takes into account the ratings of each spot and the order in which they should be visited, generates an efficient travel plan, and sends it to the terminal.

[2160] Input: Ranking list of spots, rating data, visit order

[2161] Output: Optimized itinerary

[2162] Specific operation: Calculates the most efficient visiting order based on the rating, travel time, and duration of stay of each spot, and generates an optimal travel plan.

[2163] Step 6:

[2164] The terminal displays the optimal travel plan sent from the server to the user and provides detailed information about each spot (address, opening hours, ratings, weather forecast, and congestion status).

[2165] Input: Optimized itinerary, detailed information for each spot

[2166] Output: The best itinerary and details displayed to the user

[2167] What it does: It visually displays the itinerary and detailed information of each spot on a map and in schedule format, making it easy for users to check.

[2168] As a result, it is possible to realize a system that provides optimal travel plans that reflect the user's emotional state in real time, further enriching the travel experience.

[2169] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2170] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2171] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2172] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2173] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2174] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2175] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2176] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2177] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2178] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2179] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2180] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2181] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2182] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2183] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2184] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2185] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2186] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2187] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2188] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2189] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2190] The following is further disclosed regarding the above embodiment.

[2191] (Claim 1)

[2192] In a travel planning support system,

[2193] an input means for inputting the activities that the user wishes to engage in during the trip;

[2194] A data collection means for collecting data on a user's past visit history, travel time, current location, weather, congestion, and means of transportation;

[2195] Based on the collected data, a proposal method is provided to select and propose tourist facilities and dining spots that best suit the user's needs.

[2196] a display means for presenting the proposed travel plan to the user;

[2197] A system including:

[2198] (Claim 2)

[2199] 2. The system according to claim 1, further comprising means for dynamically modifying the travel plan in consideration of past visit history data, weather information, congestion information updated in real time, and travel time.

[2200] (Claim 3)

[2201] 2. The system according to claim 1, wherein the input means has a function of analyzing the user's interests and automatically classifying them into categories (such as sightseeing, art, and gourmet food) based on the analysis.

[2202] "Example 1"

[2203] (Claim 1)

[2204] an input means for inputting the activities that the user wishes to engage in during the trip;

[2205] A data collection means for collecting data on a user's past visit history, travel time, current location, weather, congestion, and means of transportation;

[2206] Based on the collected data, a proposal method is provided to select and propose tourist facilities and dining spots that best suit the user's needs.

[2207] a display means for presenting the proposed travel plan to the user;

[2208] means for analyzing user input and classifying it into activity categories using natural language processing;

[2209] A means for determining the priority of each spot based on the collected data and generating an efficient travel plan;

[2210] A system including:

[2211] (Claim 2)

[2212] 2. The system according to claim 1, further...

Claims

1. In a travel planning support system, an input means for inputting the activities that the user wishes to engage in during the trip; A data collection means for collecting data on a user's past visit history, travel time, current location, weather, congestion, and means of transportation; Based on the collected data, a proposal method is provided to select and propose tourist facilities and dining spots that best suit the user's needs. a display means for presenting the proposed travel plan to the user; A system including:

2. The system according to claim 1, further comprising means for dynamically modifying the travel plan in consideration of past visit history data, weather information, congestion information updated in real time, and travel time.

3. 2. The system according to claim 1, wherein the input means has a function of analyzing the user's interests and automatically classifying the information into categories (such as sightseeing, art, and gourmet food) based on the analysis.

Images