System

Abstract

A system is provided.SOLUTION: A system for supporting travel planning, comprising: means for integrating traffic information collected from an external database and usage statistics of a navigation application; means for performing machine learning based on the statistics and generating an optimal travel route; means for automatically generating a travel plan based on the generated travel route; means for reserving an accommodation facility based on the travel plan; and means for collecting a usage history and feedback of a traveler and updating a model for generating a next travel plan.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention relates to a system that supports travel planning. Modern travelers, especially foreign visitors to Japan and domestic travelers unfamiliar with travel, face challenges in finding optimal routes due to the complexity of traffic information and a lack of information on footpaths and local trains. Furthermore, these travelers often spend a great deal of time and effort creating travel plans and booking accommodations. The present invention aims to solve these challenges and support travelers in planning and executing trips with high convenience and minimal stress. [Means for solving the problem]

[0005] The travel planning support system according to the present invention includes the following means.

[0006] First, the system includes a means for integrating traffic information collected from external databases and navigation app usage statistics. Next, it includes a means for performing machine learning based on these statistics to generate optimal travel routes. It also includes a means for automatically generating travel plans based on the generated travel routes. It also includes a means for booking accommodations based on the travel plans. It also provides a means for collecting travelers' usage history and feedback and updating the model for generating the next travel plan. In this way, a system is realized that allows travelers to seamlessly suggest optimal routes, create travel plans, and book accommodations.

[0007] A "travel planning support system" is a system designed to help travelers select travel destinations, decide on routes, create sightseeing plans, and book accommodations efficiently and effectively.

[0008] An "external database" is a database that exists on the Internet or in the cloud and collects and stores traffic information and statistical data for navigation.

[0009] "Transportation information" refers to information that travelers need when planning their travels, such as public transportation schedules, delay information, traffic congestion status, and route information.

[0010] A "navigation app" is an application that uses GPS and map information to guide users to the optimal route and provides information on traffic conditions and routes to their destination.

[0011] "Usage statistics" refers to the collection and analysis of usage data on navigation apps and other digital tools, compiled into statistical information.

[0012] "Machine learning" refers to algorithms and techniques that learn patterns from data and use them to make predictions and decisions about new data and situations.

[0013] An "optimal travel route" is a route that represents the most preferred route for travelers, taking into account factors such as travel time, cost, convenience, and the popularity of tourist destinations.

[0014] "Automatically generated" means created automatically by a system or algorithm without human intervention.

[0015] A "travel plan" is a detailed plan for a traveler that includes the route to the destination, the order in which to visit tourist spots, information on accommodation, etc.

[0016] "Reservation of accommodation" refers to the procedure for reserving a room in advance to use an accommodation facility such as a hotel or inn.

[0017] "Usage history" refers to a record of the actions a user takes while using the system, and includes data on travel destinations, places visited, routes traveled, etc.

[0018] "Feedback" refers to the evaluations and opinions provided by users after using the system, which are used to improve and personalize the system.

[0019] "Updating a model" means improving the performance of a machine learning model by reflecting new data and feedback, thereby increasing the accuracy of future predictions and suggestions. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] 1. Field of the Invention The present invention relates to a system for assisting in travel planning. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to the drawings.

[0042] Overall system configuration

[0043] This system consists of a server, a terminal, and a user. The server collects traffic information and usage statistics of navigation apps and uses this information to train a machine learning model. The terminal receives input from the user and sends it to the server. The user inputs their destination and travel conditions, and then checks and selects travel plans suggested by the server.

[0044] System Operation

[0045] 1. Collection of traffic information

[0046] The server collects traffic information from external databases and the navigation app's API, including public transport schedules and traffic congestion status.

[0047] Example: A server uses the Google Maps API to retrieve information about Shinkansen schedules and fares from Tokyo to Kyoto.

[0048] 2. Route optimization using machine learning

[0049] The server uses the collected traffic information to train a machine learning model that leverages the user's past travel data and real-time usage statistics to suggest optimal travel routes.

[0050] Example: A server analyzes past travel data and determines, based on traveler behavior patterns, that taking the Shinkansen is the quickest and most cost-effective option.

[0051] 3. Automatic generation of travel plans

[0052] The server uses the trained model to automatically generate an itinerary that best suits the user's requirements, including routes, the order in which tourist spots are visited, and recommended accommodations.

[0053] Example: A server generates a travel plan from Tokyo to Kyoto, outputting the Shinkansen travel time, tourist spots along the way, and a list of recommended hotels.

[0054] 4. Receiving and Sending User Input

[0055] The user inputs travel conditions such as the destination and departure point, desired itinerary, and favorite tourist spots through the terminal, and this input information is sent from the terminal to the server.

[0056] Example: A user uses a smartphone app to enter the following information: "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple."

[0057] 5. Check and select your travel plan

[0058] The terminal displays the travel plans sent from the server to the user, who then checks the proposed travel plans, selects the one they like, and confirms it.

[0059] Example: A user reviews suggested travel plans on their smartphone screen and selects one to visit Kiyomizu-dera Temple.

[0060] 6. Accommodation reservations

[0061] The server automatically processes the accommodation reservation based on the travel plan selected by the user, and completes the reservation in cooperation with an external accommodation reservation system.

[0062] Example: The server uses the API of a partner accommodation booking site to reserve a room at a selected hotel and obtain a reservation confirmation ID.

[0063] 7. Real-time information provision

[0064] The server provides real-time traffic and tourist information to the user while he or she is traveling, and this information is sent to the user via the terminal.

[0065] Example: While traveling, a user receives the latest traffic information and crowding status of tourist spots on their smartphone.

[0066] 8. Usage history and feedback collection

[0067] The server collects user travel history and feedback and uses it to generate future travel plans. This information is used to update the machine learning model.

[0068] Example: After completing a trip, a user enters an evaluation of the plan provided by the app, and the data is stored on the server.

[0069] In this way, the travel planning support system of the present invention can significantly improve convenience for travelers by suggesting optimal travel routes to users, automatically generating travel plans, and seamlessly booking accommodations.

[0070] The processing flow will be explained below.

[0071] Step 1:

[0072] The server collects traffic information from external databases and the navigation app's API.

[0073] Specific operation: The server uses the Google Maps API to obtain Shinkansen schedules and fare information from Tokyo to Kyoto and stores it in a database.

[0074] Step 2:

[0075] The server trains a machine learning model based on the collected traffic information.

[0076] Specific operation: The server takes in past travel data and user behavior patterns and runs an algorithm to learn the optimal travel route that minimizes travel time and costs.

[0077] Step 3:

[0078] Users use their device to input travel conditions such as travel destination, departure point, desired dates, and favorite tourist spots into the "Reserve-san" app.

[0079] Specific operation: The user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into the input form of the smartphone app and clicks the send button.

[0080] Step 4:

[0081] The terminal converts the user's input data into JSON format and sends it to the server.

[0082] Specific operation: The device packages the input data in JSON format and sends it to the server using an HTTP request.

[0083] Step 5:

[0084] The server generates an optimal travel route and sightseeing plan based on the received user's conditions.

[0085] Specific operation: The server uses a machine learning model to generate a travel plan that includes the Shinkansen route that best suits the user's requirements, tourist spots along the way, and recommended accommodations.

[0086] Step 6:

[0087] The server converts the generated travel plan into JSON format and sends it to the user's device.

[0088] Specific operation: The server encodes the generated travel plan in JSON format and sends it to the terminal as an HTTP response.

[0089] Step 7:

[0090] The terminal displays the travel plan received from the server to the user.

[0091] Specific operation: The device parses the contents of the travel plan and displays it on the screen in a format that is easy for the user to view.

[0092] Step 8:

[0093] The user checks the displayed travel plans and selects the plan they want.

[0094] Specific operation: The user checks the suggested tourist spots and accommodations on the smartphone screen and taps to select the plan they like.

[0095] Step 9:

[0096] The device converts the user's selected plan information into JSON format and sends it to the server.

[0097] Specific behavior: The device packages the selected travel plan in JSON format and sends it to the server using an HTTP request.

[0098] Step 10:

[0099] The server then processes the reservation for the accommodation based on the user's selection.

[0100] Specific operation: The server uses the information of the selected accommodation to call the API of the affiliated reservation site, completes the reservation procedure, and obtains the reservation confirmation ID.

[0101] Step 11:

[0102] The server sends information including the reservation confirmation ID to the terminal and notifies the user.

[0103] Specific operation: The server creates a response including the reservation confirmation ID and sends it to the terminal as an HTTP response. The terminal receives this and notifies the user.

[0104] Step 12:

[0105] The server allows the user to receive real-time traffic information and tourist spot updates while traveling and transmits them to the terminal.

[0106] Specific operation: The server collects and updates real-time traffic information and tourist spot information, and sends it to the user's device as a push notification.

[0107] Step 13:

[0108] The server collects the user's travel history and feedback after the trip is completed.

[0109] What happens: A user fills out a feedback form in the app and clicks the submit button. The data is sent to the server and stored in the database.

[0110] Step 14:

[0111] The server updates and improves the machine learning model based on collected travel history and feedback.

[0112] What it does: The server uses the newly collected data to retrain the machine learning algorithm and improve the accuracy of the next itinerary generation.

[0113] Example 1

[0114] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0115] Conventional travel planning systems provide limited information on transportation and tourist attractions, and lack real-time updates. Furthermore, travel plans are often generated manually, making it difficult to provide efficient and optimal plans. Furthermore, they are unable to effectively utilize traveler usage history and feedback, making it difficult to reflect this information in the generation of next travel plans.

[0116] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0117] In this invention, the server includes means for integrating traffic information collected from an external database and usage statistics of the navigation app, means for generating an optimal travel route using a machine learning model based on the integrated information, means for automatically generating a travel plan based on the generated travel route, means for collecting traveler usage history and feedback and updating the machine learning model for generating the next travel plan, and means for providing traffic information and tourist spot information in real time, thereby enabling the user to be provided with an efficient and optimal travel plan and real-time information during the trip.

[0118] An "external database" is an information source that provides transportation information and tourist spot information that can be accessed via the Internet.

[0119] A "navigation app" is a software application that provides users with travel routes and traffic conditions.

[0120] "Traffic information" refers to information related to travel, such as public transportation schedules and road congestion conditions.

[0121] "Usage statistics" refers to statistical data including user behavior data and access frequency for navigation applications.

[0122] A "machine learning model" is a mathematical model that learns patterns from data and makes predictions and classifications.

[0123] The "optimal travel route" refers to the most efficient and comfortable travel route, taking into consideration traffic conditions, past user data, costs, etc.

[0124] "Auto-generation" refers to the process of automatically generating information or plans through a program.

[0125] "Accommodation" means a place such as a hotel or lodging facility where travelers can stay.

[0126] A "reservation system" is a system that manages reservations for accommodation, transportation, etc. online.

[0127] "Usage history" refers to a record of the user's past travels and services used.

[0128] "Feedback" refers to opinions such as ratings and impressions provided by users.

[0129] "Real-time information" refers to information that is updated immediately based on the current time.

[0130] The present invention relates to a system for assisting in travel planning, and this system is composed of a server, a terminal, and a user. The following describes in detail an embodiment of the present invention.

[0131] Overall system configuration

[0132] The server collects traffic information and usage statistics from external databases and the navigation app's API, and uses this data to train the machine learning model. The device receives input from the user and sends it to the server. The user enters their destination and travel conditions, and then confirms and selects the travel plan suggested by the server. The entire system automatically generates an efficient and optimal travel plan by linking each function together.

[0133] Hardware and software used

[0134] This invention utilizes a server, a terminal, and an external API. Specifically, the server is a computer system with a high-performance CPU and large memory capacity, and uses a programming language such as Python to train machine learning models and process data. It also collects traffic information using the Google Maps API and public transportation schedule APIs. The terminals are smartphones and tablet devices, and applications running on these devices can be native iOS or Android apps.

[0135] Data processing and calculation

[0136] The server stores traffic information obtained from external databases and the navigation app's API in a database and performs data preprocessing. For example, the server uses the Google Maps API to obtain Shinkansen schedules and fare information from Tokyo to Kyoto. The server then trains a machine learning model using libraries such as scikit-learn to generate optimal travel routes. This model may include random forests or neural networks.

[0137] The server uses the trained model to automatically generate an optimal travel plan based on the user's criteria. For example, it uses past travel data to suggest a travel route from Tokyo to Kyoto and generates a plan that includes Shinkansen travel times, tourist spots along the way, and recommended accommodations. The user enters the destination, departure point, desired dates, and favorite tourist spots into the smartphone app, and sends the information from the device to the server. An example of a prompt sentence could be "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple."

[0138] Accommodation reservations

[0139] The server automatically reserves accommodations using the API of an external accommodation reservation system based on the travel plan selected by the user. For example, it uses the Booking.com API to search for hotels that meet the user's requirements and makes a reservation. Once the reservation is complete, the server obtains a reservation confirmation ID and notifies the user.

[0140] Providing real-time information

[0141] To provide real-time traffic and tourist spot information to users while traveling, the server periodically calls the navigation app's API to obtain the latest information and sends push notifications to the device, allowing users to stay up to date with the latest information during their trip.

[0142] Collection of usage history and feedback

[0143] The server collects the user's travel history and feedback and updates the machine learning model for future travel plan generation. For example, if a user enters a plan rating in the app after completing a trip, that data is stored on the server and used for the next model training.

[0144] By integrating these functions, the travel planning support system of the present invention can provide users with efficient and optimal travel plans and real-time information during travel.

[0145] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0146] Step 1:

[0147] The server collects traffic information and usage statistics from external databases and the navigation app's API. Input data includes transportation schedules and road traffic conditions. This information is collected and stored in a database. Specifically, the server calls the Google Maps API to obtain information on the schedule and traffic congestion for the specified section.

[0148] Step 2:

[0149] The server trains a machine learning model based on the collected traffic information and usage statistics. Input data includes historical travel data and real-time traffic information. It preprocesses the data and trains a model to suggest optimal travel routes. Specifically, the server preprocesses the data using the scikit-learn library, trains the model using the random forest algorithm, and saves the model.

[0150] Step 3:

[0151] The server uses a trained machine learning model to automatically generate an optimal travel plan based on the user's criteria. Input data includes the user's travel destination, departure point, desired dates, and favorite tourist spots. The model applies the criteria to generate the optimal travel route. Specifically, the server uses the trained model to organize the plan based on the user's criteria into a pandas data frame and send it to the terminal in JSON format.

[0152] Step 4:

[0153] The user inputs travel conditions such as the destination and departure point, desired dates, and tourist spots through the device. The input information is sent to the server. Specifically, the user enters the information into the smartphone app and presses the send button. The device then sends this information to the server as an API request.

[0154] Step 5:

[0155] The device displays the travel plan sent from the server to the user. The user reviews the proposed plans, selects the one they like, and confirms it. The input data includes the travel plan from the server. The travel plan selected by the user is sent to the server as output data. Specifically, the JSON data received from the server is passed to the device app, which displays the details of the travel plan in a list view. The selected information is sent back to the server when the user taps "Confirm" and "Select."

[0156] Step 6:

[0157] The server reserves accommodation based on the travel plan selected by the user. The input data includes the travel plan selected by the user. The output data includes the accommodation reservation confirmation ID. Specifically, the server calls the API of an external accommodation reservation system to complete the reservation. It then obtains the reservation confirmation ID and notifies it to the user.

[0158] Step 7:

[0159] The server provides real-time traffic information and tourist spot information to the user. Input data includes current traffic conditions and the congestion status of tourist spots. Updated information is sent to the device as output data. Specifically, the server periodically calls the navigation app's API, obtains the latest information, and sends it to the device via push notification.

[0160] Step 8:

[0161] The server collects the user's travel history and feedback and updates the machine learning model to generate future travel plans. The input data includes the user's usage history and feedback. The output data includes the updated model. Specifically, after completing a trip, the user enters a rating in the app, and the data is saved on the server. This data is used the next time the model is trained.

[0162] (Application example 1)

[0163] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0164] Conventional travel planning support systems have had difficulty providing travelers with the real-time information they need or automatically generating optimal travel plans based on their detailed preferences. Furthermore, the complicated procedures for changing travel plans and reserving accommodations make them less user-friendly for travelers.

[0165] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0166] In this invention, the server includes means for integrating traffic information collected from an external database and usage statistics of location information apps, means for performing machine learning based on the statistics to generate an optimal travel route, means for automatically generating a travel plan based on the generated travel route, means for booking accommodations based on the travel plan, means for collecting traveler usage history and opinions and updating a model for generating the next travel plan, and means for providing the latest information on the traveler's current location and travel destination in real time, thereby enabling the automatic generation of an optimal travel plan for a traveler and the provision of real-time information.

[0167] A "travel planning support system" is a system that optimizes travelers' travel plans and supports automatic generation and accommodation reservation procedures.

[0168] "Traffic information" refers to information collected from external databases, including public transportation schedules and traffic congestion status.

[0169] A "location-based app" is an application that identifies a user's geographic location and provides navigation and surrounding information to the user.

[0170] "Usage statistics" refers to statistical data based on a user's application usage history and behavioral patterns.

[0171] "Machine learning" is a technology that uses algorithms to learn data and make predictions and classifications.

[0172] "Travel route" refers to route information that indicates the means of transportation and travel route used by a traveler.

[0173] A "travel plan" is a plan that includes travel routes, the order in which tourist spots are visited, recommended accommodations, etc., generated based on conditions specified by the traveler.

[0174] "Accommodation facilities" refer to facilities such as hotels and inns where travelers can stay.

[0175] A "reservation" is a procedure for securing accommodation, transportation, etc. in advance.

[0176] "Usage history" refers to historical information about trips taken by a user in the past and services used by the user.

[0177] "Feedback" refers to user opinions and evaluations provided after using the service.

[0178] "Real-time information" refers to information that is updated hourly, such as the latest traffic conditions and the current congestion status of tourist spots.

[0179] The present invention is a system for assisting in travel planning, and its configuration and operation steps will be described in detail. First, an outline of the program required to realize this system will be created, and the processing based on this program will be described.

[0180] Overall system configuration

[0181] This system consists of a server, a terminal, and a user.

[0182] Server Roles

[0183] Collecting information from external databases: The server collects traffic information and usage statistics from external databases and location app APIs.

[0184] Data analysis with machine learning: Train machine learning models based on collected data to generate optimal travel routes.

[0185] Automatic itinerary generation: Using the trained model, we automatically generate itineraries that best fit the user's requirements.

[0186] Providing real-time information: Providing real-time updates on a traveler's current location and destination.

[0187] Collect usage history and feedback: Collect user usage history and feedback and update the model for the next itinerary generation.

[0188] Accommodation reservations: We will process your accommodation reservations based on your proposed travel plans.

[0189] Device Role

[0190] Receive user input: Enter travel destinations, departure points, desired dates, favorite attractions, and other conditions.

[0191] Display itinerary: Display the itinerary sent from the server to the user, and confirm and select the proposed itinerary.

[0192] User Roles

[0193] Entering travel conditions: Enter travel conditions through the terminal.

[0194] Choose your itinerary: Review the suggested itineraries and choose the one you like best.

[0195] Providing feedback: Provide feedback after your trip and contribute to improving the system.

[0196] Hardware and software configuration

[0197] Hardware: Smartphone (iOS / Android), server

[0198] Software: Google Maps API, HotelBooking API, Machine Learning algorithms (e.g. Logistic Regression)

[0199] Details of data processing and calculation

[0200] 1. Data collection: The server collects traffic and location information from Google Maps API and other external databases.

[0201] 2. Data analysis: Using machine learning models (e.g., Logistic Regression) to analyze users' historical data and generate optimal travel routes.

[0202] 3. Real-time information provision: Based on the traveler's current location, the server periodically calls the API to provide the latest traffic information and congestion status of tourist attractions in real time.

[0203] 4. Obtaining feedback and updating the model: After the trip, obtain feedback from the user and use it as training data for the machine learning model.

[0204] Specific examples

[0205] Travel planning

[0206] If a user enters "I want to travel to Kyoto on March 15th," the system will prompt "Please enter the dates and places of interest." If a user enters "I want to visit Kiyomizu-dera Temple and Kinkaku-ji Temple," the system will generate and suggest the optimal itinerary.

[0207] Prompt sentences for generative AI models (examples)

[0208] A user has entered that they would like to visit Kyoto on March 15th. Their tourist attractions of interest are Kiyomizu-dera Temple and Kinkaku-ji Temple. Please suggest the best travel route and accommodation. Please create a detailed plan, including tourist attractions along the way.

[0209] In this way, the travel planning support system of the present invention is able to automatically generate optimal travel plans for users and provide them with real-time information.

[0210] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0211] Step 1:

[0212] Receiving User Input

[0213] Subject: Device

[0214] Specific operation: The device receives information from the user, such as the travel destination, departure point, desired dates, and favorite tourist spots.

[0215] Input: Travel destination, departure point, desired dates, tourist attractions

[0216] Output: Received user input information

[0217] Data processing: The device converts the received information into an appropriate format and prepares it for transmission to the server.

[0218] Step 2:

[0219] Collection of traffic information and usage statistics

[0220] Subject: Server

[0221] What it does: The server uses the Google Maps API and other external databases to collect traffic information and location app usage statistics.

[0222] Input: External database, Google Maps API endpoint

[0223] Output: Collected traffic information and usage statistics

[0224] Data processing: The server organizes the collected data and stores it in a database, including any necessary filtering and preprocessing.

[0225] Step 3:

[0226] Training a machine learning model

[0227] Subject: Server

[0228] How it works: The server uses historical travel data and real-time usage statistics to train machine learning models.

[0229] Inputs: Historical travel data, real-time usage statistics

[0230] Output: A trained machine learning model

[0231] Data calculation: The server performs training using algorithms such as logistic regression, learning patterns based on past data and optimizing the model parameters.

[0232] Step 4:

[0233] Generating optimal travel routes

[0234] Subject: Server

[0235] What it does: Uses a trained machine learning model to generate optimal travel routes based on the user's criteria.

[0236] Input: User input, trained machine learning model

[0237] Output: Optimal travel route

[0238] Data calculation: The server applies the user's input to the model to predict the best route. It uses an optimization algorithm to evaluate and select travel routes.

[0239] Step 5:

[0240] Automatic travel plan generation

[0241] Subject: Server

[0242] Specific operation: The server automatically generates a travel plan based on the generated travel route, including travel routes, the order in which tourist attractions are visited, and recommended accommodations.

[0243] Input: Optimal travel route

[0244] Output: Auto-generated itinerary

[0245] Data processing: The server incorporates information on tourist attractions and accommodations based on the generated route and formats it into a complete travel plan.

[0246] Step 6:

[0247] View and select your travel plans

[0248] Subject: Device

[0249] Specific operation: The terminal displays the travel plans sent from the server to the user, allowing the user to confirm and select the plans.

[0250] Input: Auto-generated itinerary

[0251] Output: Selected itinerary

[0252] Data processing: Visually display the travel plan through the user interface and accept user selections.

[0253] Step 7:

[0254] Accommodation reservations

[0255] Subject: Server

[0256] Specific operation: Based on the selected travel plan, the server completes the accommodation reservation using the API of the external application.

[0257] Input: Selected travel plan

[0258] Output: Reservation confirmation ID

[0259] Data calculation: Obtain accommodation availability information from the API, complete the reservation process, and generate a confirmation ID.

[0260] Step 8:

[0261] Providing real-time information

[0262] Subject: Server

[0263] Specific operation: The server collects real-time traffic information and congestion status of tourist attractions based on the traveler's current location and sends it to the terminal.

[0264] Input: Current location, real-time information

[0265] Output: User notification

[0266] Data calculation: The server obtains the latest information from the API and pushes it to the device.

[0267] Step 9:

[0268] Get feedback and update the model

[0269] Subject: Server

[0270] What it does: After the trip, the server collects user feedback and uses it as training data for the machine learning model.

[0271] Input: User feedback

[0272] Output: An updated machine learning model

[0273] Data processing: Analyze the feedback and add it to the model's training dataset. Retrain the model as a new model.

[0274] This process enables the travel planning support system to automatically generate optimal travel plans for users and provide real-time information.

[0275] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0276] The present invention relates to a system for assisting in travel planning, and more particularly to a system for providing more personalized travel plans by incorporating an emotion engine that recognizes the emotions of a user. Hereinafter, an embodiment of the present invention will be described in detail.

[0277] Overall system configuration

[0278] This system consists of a server, a terminal, a user, and an emotion engine. The emotion engine has the means to recognize emotions by analyzing the user's voice, facial expressions, and text data. The server generates a travel plan that is most suitable for the user based on the output of this emotion engine. The terminal is responsible for receiving this information from the user and sending it to the server. The user inputs the travel destination, departure point, desired dates, emotional state, etc. via the terminal, and then confirms and selects the travel plan proposed by the server.

[0279] System Operation

[0280] 1. Collection of traffic information

[0281] The server collects traffic information from external databases and the navigation app's API, including public transport schedules and traffic congestion status.

[0282] Example: A server uses the Google Maps API to retrieve Shinkansen schedules and fare information from Tokyo to Kyoto and stores it in a database.

[0283] 2. Route optimization using machine learning

[0284] The server uses the collected traffic information to train a machine learning model that leverages the user's past travel data and real-time usage statistics to suggest optimal travel routes.

[0285] Example: A server analyzes past travel data and determines, based on traveler behavior patterns, that taking the Shinkansen is the quickest and most cost-effective option.

[0286] 3. Emotion Recognition by Emotion Engine

[0287] The terminal collects the user's voice data, facial expression data, and text data and sends them to the emotion engine, which analyzes the data and recognizes the user's current emotional state.

[0288] Example: The emotion engine analyzes the voice and facial expressions of a user speaking into the device's camera and determines that the user is in a "relaxed" state.

[0289] 4. Receiving and Sending User Input

[0290] The user inputs travel conditions such as the destination, desired itinerary, current emotional state, and favorite tourist spots through the terminal, and this information is sent from the terminal to the server.

[0291] Example: A user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into an input form on a smartphone app and clicks the submit button.

[0292] 5. Automatic travel plan generation

[0293] The server generates an optimal travel route and sightseeing plan for the user based on the output of the machine learning model and emotion engine, including the route, the order in which tourist spots are visited, and recommended accommodations.

[0294] Example: The server considers the user's relaxed emotional state and tourist spot preferences to suggest a relaxing route in Kyoto.

[0295] 6. Check and select your travel plan

[0296] The terminal displays the travel plans sent from the server to the user, who then checks the proposed travel plans, selects the one they like, and confirms it.

[0297] Example: A user reviews suggested travel plans on their smartphone screen and selects one to visit Kiyomizu-dera Temple.

[0298] 7. Accommodation reservations

[0299] The server automatically processes the accommodation reservation based on the travel plan selected by the user, and completes the reservation in cooperation with an external accommodation reservation system.

[0300] Example: The server uses the API of a partner accommodation booking site to reserve a room at a selected hotel and obtain a reservation confirmation ID.

[0301] 8. Real-time information provision

[0302] The server provides real-time traffic and tourist information to the user while he or she is traveling, and this information is sent to the user via the terminal.

[0303] Example: While traveling, a user receives the latest traffic information and crowding status of tourist spots on their smartphone.

[0304] 9. Usage history and feedback collection

[0305] The server collects user travel history and feedback and uses it to generate future travel plans. This information is used to update the machine learning model.

[0306] Example: After completing a trip, a user enters an evaluation of the plan provided by the app, and the data is stored on the server.

[0307] In this way, the travel planning support system of the present invention can significantly improve convenience for travelers by proposing optimal travel routes taking into account the user's emotional state, automatically generating travel plans, and seamlessly booking accommodations.

[0308] The processing flow will be explained below.

[0309] Step 1:

[0310] The server collects traffic information from external databases and the navigation app's API.

[0311] Specific operation: The server uses the Google Maps API to obtain Shinkansen schedules and fare information from Tokyo to Kyoto and stores it in a database.

[0312] Step 2:

[0313] The server trains a machine learning model based on the collected traffic information.

[0314] Specific operation: The server takes in past travel data and user behavior patterns and runs an algorithm to learn the optimal travel route that minimizes travel time and costs.

[0315] Step 3:

[0316] The user uses the terminal to input conditions such as the travel destination, departure point, desired dates, and favorite tourist spots.

[0317] Specific operation: The user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into the input form of the smartphone app and clicks the send button.

[0318] Step 4:

[0319] The terminal converts the user's input data into JSON format and sends it to the server.

[0320] Specific operation: The device packages the input data in JSON format and sends it to the server using an HTTP request.

[0321] Step 5:

[0322] The server generates an optimal travel route and sightseeing plan based on the received user's conditions.

[0323] Specific operation: The server uses a machine learning model to generate a travel plan that includes the Shinkansen route that best suits the user's requirements, tourist spots along the way, and recommended accommodations.

[0324] Step 6:

[0325] The server converts the generated travel plan into JSON format and sends it to the user's device.

[0326] Specific operation: The server encodes the generated travel plan in JSON format and sends it to the terminal as an HTTP response.

[0327] Step 7:

[0328] The terminal displays the travel plan received from the server to the user.

[0329] Specific operation: The device parses the contents of the travel plan and displays it on the screen in a format that is easy for the user to view.

[0330] Step 8:

[0331] The user checks the displayed travel plans and selects the plan they want.

[0332] Specific operation: The user checks the suggested tourist spots and accommodations on the smartphone screen and taps to select the plan they like.

[0333] Step 9:

[0334] The device converts the user's selected plan information into JSON format and sends it to the server.

[0335] Specific behavior: The device packages the selected travel plan in JSON format and sends it to the server using an HTTP request.

[0336] Step 10:

[0337] The server then processes the reservation for the accommodation based on the user's selection.

[0338] Specific operation: The server uses the information of the selected accommodation to call the API of the affiliated reservation site, completes the reservation procedure, and obtains the reservation confirmation ID.

[0339] Step 11:

[0340] The server sends information including the reservation confirmation ID to the terminal and notifies the user.

[0341] Specific operation: The server creates a response including the reservation confirmation ID and sends it to the terminal as an HTTP response. The terminal receives this and notifies the user.

[0342] Step 12:

[0343] The server allows the user to receive real-time traffic information and tourist spot updates while traveling and transmits them to the terminal.

[0344] Specific operation: The server collects and updates real-time traffic information and tourist spot information, and sends it to the user's device as a push notification.

[0345] Step 13:

[0346] The server collects the user's travel history and feedback after the trip is completed.

[0347] What happens: A user fills out a feedback form in the app and clicks the submit button. The data is sent to the server and stored in the database.

[0348] Step 14:

[0349] The terminal collects the user's voice data, facial expression data, and text data and sends them to the emotion engine.

[0350] Specific operation: Using the device's camera and microphone, the user's voice and facial expressions are recorded and sent to the emotion engine along with the text data.

[0351] Step 15:

[0352] The emotion engine analyzes the user's voice, facial expression, and text data to recognize the user's current emotional state.

[0353] Specific operation: The emotion engine uses an analytical algorithm to determine emotions such as "relaxed," "excited," and "stressed" from the collected data.

[0354] Step 16:

[0355] The server adjusts the contents of the travel plan based on the emotional state recognized by the emotion engine.

[0356] Specific operation: The server generates a plan that includes tourist spots and experiences that correspond to the emotional state of "wanting to relax."

[0357] Step 17:

[0358] The server updates the travel plan in real time according to changes in the emotional state and sends it to the terminal.

[0359] Specific operation: When a user feels stressed during a trip, the server generates a plan including new relaxation spots and sends it to the terminal.

[0360] In this way, the travel planning support system of the present invention can significantly improve convenience for travelers by proposing optimal travel routes taking into account the user's emotional state, automatically generating travel plans, and seamlessly booking accommodations.

[0361] Example 2

[0362] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0363] Conventional travel planning support systems have the problem of being unable to provide personalized travel plans based on users' emotional state or real-time fluctuating information, which has led to travelers being unable to have a comfortable and satisfying travel experience.

[0364] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0365] In this invention, the server includes means for collecting traffic information from an external database and integrating usage statistics of a navigation app, means for generating an optimal travel route using a machine learning model based on the statistics, means for utilizing an emotion recognition engine that recognizes emotions by analyzing the traveler's voice, facial expression, and text data, means for automatically generating a travel plan based on the output of the emotion recognition engine, means for reserving accommodations based on the travel plan, and means for collecting the traveler's usage history and feedback and updating the model for generating the next travel plan. This makes it possible to provide an optimal and personalized travel plan based on the user's current emotional state and past travel history.

[0366] "Traffic information" refers to data such as public transportation schedules and traffic congestion conditions.

[0367] "Navigation app usage statistics" refers to data on the behavioral patterns and real-time usage status of users of navigation apps.

[0368] A "machine learning model" refers to an algorithm that automatically learns from large amounts of data and makes predictions and classifications.

[0369] An "emotion recognition engine" refers to a system that recognizes emotions by analyzing a user's voice, facial expressions, and text data.

[0370] A "travel plan" refers to a detailed travel plan that includes travel routes, the order in which tourist spots are visited, recommended accommodations, etc.

[0371] "Accommodation Booking" refers to the process of reserving accommodation for a traveler.

[0372] "Usage history" refers to travel plans and behavioral data that a user has used in the past.

[0373] "Feedback" refers to the evaluations and opinions given by users regarding the services and plans provided.

[0374] "Updating a model" refers to the process of improving the performance of an existing machine learning model based on new data.

[0375] The present invention relates to a system for assisting in travel planning, and more particularly to a system for providing more personalized travel plans by incorporating an emotion engine that recognizes the emotions of a user. Hereinafter, an embodiment of the present invention will be described in detail.

[0376] This system consists of a server, a terminal, a user, and an emotion engine. The emotion engine has the means to recognize emotions by analyzing the user's voice, facial expressions, and text data. The server generates a travel plan that is most suitable for the user based on the output of this emotion engine. The terminal is responsible for receiving this information from the user and sending it to the server. The user inputs the travel destination, departure point, desired dates, emotional state, etc. via the terminal, and then confirms and selects the travel plan proposed by the server.

[0377] System Configuration

[0378] 1. Means of collecting traffic information

[0379] The server collects traffic information using external databases and the navigation app's API, including public transport schedules and traffic congestion status. The specific software used is Google Maps API.

[0380] 2. Using Machine Learning Models

[0381] The server uses the collected traffic information to train a machine learning model that uses the user's past travel data and real-time usage statistics to suggest optimal travel routes using software including TensorFlow.

[0382] 3. Emotion recognition

[0383] The device collects the user's voice, facial expression, and text data and sends it to the emotion engine, which analyzes this data and recognizes the user's current emotional state using Microsoft Azure Cognitive Services.

[0384] 4. Automatic generation of travel plans

[0385] The server generates an optimal travel route and sightseeing plan for the user based on the output of the machine learning model and emotion engine, including the route, the order in which tourist spots are visited, and recommended accommodations.

[0386] 5. Accommodation reservations

[0387] The server automatically makes reservations for accommodation based on the travel plan. At this time, it completes the reservation by linking with an external accommodation reservation system. Specifically, it uses the Booking.com API.

[0388] 6. Collecting User Usage History and Feedback

[0389] The server collects user travel history and feedback and uses it to generate future travel plans. This information is also used to update the machine learning model.

[0390] Specific examples

[0391] For example, if a user uses a smartphone app to input "Kyoto, March 15, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple," and the emotion recognition engine recognizes the user's relaxed state, the server will use this information to generate a travel plan that includes relaxing tourist spots and accommodations.The server then uses the Booking.com API to make reservations for the accommodations and notify the user.

[0392] Prompt Sentence Examples

[0393] "How do I use the Google Maps API to plan my trip?"

[0394] In this way, the travel planning support system of the present invention can provide an optimal and personalized travel plan that takes into account the emotional state of the user.

[0395] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0396] Step 1:

[0397] The server collects traffic information using an external database and the navigation app's API. As input, it receives traffic information from the external database and the navigation app. This provides data such as public transport schedules and traffic congestion conditions. The server stores this information in a database. Specifically, the server calls the Google Maps API to obtain the Shinkansen schedule and fare information from Tokyo to Kyoto, and stores this information in the database.

[0398] Step 2:

[0399] The server trains a machine learning model based on the collected traffic information. It uses the traffic information collected in step 1 and past travel data as input. This creates a predictive model for generating optimal travel routes. The server uses the TensorFlow library to train the machine learning model and prepares optimal travel route suggestions. Specifically, the server analyzes past travel data and learns user behavior patterns.

[0400] Step 3:

[0401] The device collects the user's voice data, facial expression data, and text data, and sends this data to the emotion engine. The user's voice, facial expression, and text data are used as input. This allows the emotion engine to recognize the user's current emotional state. The emotion engine uses Microsoft Azure Cognitive Services to perform emotion analysis. Specifically, data collected using the device's camera and microphone is sent to the Microsoft Azure Cognitive Services API, which recognizes the user's emotional state, such as "relaxed" or "tense."

[0402] Step 4:

[0403] The user inputs conditions such as the travel destination, desired schedule, emotional state, and favorite tourist spots through the device. The travel information specified by the user is received as input. This information is sent from the device to the server. Specifically, the user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into the input form of the smartphone app and clicks the send button.

[0404] Step 5:

[0405] The server generates the optimal travel route and sightseeing plan for the user based on the output of the machine learning model and emotion engine. As input, it uses the output of the machine learning model in step 2, the result of the emotion engine in step 3, and the user's travel conditions sent in step 4. This creates the optimal travel plan. Specifically, the server considers the user's relaxed emotional state and tourist spot preferences, and proposes a travel route that includes relaxing tourist spots and accommodations.

[0406] Step 6:

[0407] The terminal displays the travel plan sent from the server to the user. The terminal receives the travel plan from the server as input. Based on this information, the user checks the proposed plans, selects the one they like, and confirms it. Specifically, the user checks the proposed travel plan on the smartphone screen, selects the plan to visit Kiyomizu-dera Temple and Kinkaku-ji Temple, and clicks the confirm button.

[0408] Step 7:

[0409] The server automatically makes accommodation reservations based on the travel plan selected by the user. It receives the confirmed travel plan as input and completes the reservation by connecting with an external accommodation reservation system. Specifically, the server uses the Booking.com API to reserve a room at the selected hotel and obtains a reservation confirmation ID.

[0410] Step 8:

[0411] The server provides real-time traffic and tourist spot information to users traveling. The server references the user's current location and travel plan as input, allowing it to continue providing real-time information. Specifically, the server obtains the latest traffic congestion information and congestion status at tourist spots, and notifies the traveling user via their smartphone.

[0412] Step 9:

[0413] The server collects the user's travel history and feedback and uses it to generate future travel plans. This information is also used to update the machine learning model. User feedback and travel history data are taken as input. Specifically, after the user completes their trip, they enter an evaluation of the plan provided by the app, and this data is stored on the server. The server uses this data as training data for a new machine learning model.

[0414] (Application example 2)

[0415] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0416] Conventional travel planning support systems were unable to generate travel plans that took into account the user's individual emotional state, and were limited to providing standard plans. This made it difficult to provide travel plans that were tailored to the user's needs and emotions. Furthermore, in the shopping experience, there was also the issue of being unable to provide personalized product suggestions based on the user's emotions and real-time situation.

[0417] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0418] In this invention, the server includes means for integrating traffic information collected from an external database and usage statistics of the navigation app, means for performing machine learning based on the statistics to generate an optimal travel route, and means for automatically generating a travel plan based on the generated travel route, thereby making it possible to analyze the user's emotional state in real time and provide personalized travel plans and product suggestions.

[0419] An "external database" is a collection of information or data accessible through the Internet or other network.

[0420] A "navigation app" is software that provides maps and traffic information and guides users to the optimal route from their current location to their destination.

[0421] "Usage Statistics" refers to the compilation and analysis of data based on a user's past usage history and behavioral patterns.

[0422] "Machine learning" is a technology in which a computer learns patterns and rules based on data and automatically makes predictions and decisions.

[0423] A "travel route" refers to a travel route from a user's departure point to a destination.

[0424] A "travel plan" is a plan that includes travel destinations, transportation, accommodations, tourist spots, etc.

[0425] An "accommodation reservation system" is a system for making online reservations for accommodations such as hotels and inns.

[0426] "Usage history" refers to a record of trips the user has taken and a history of services they have used.

[0427] "Feedback" refers to a user providing an evaluation or opinion about a service or product.

[0428] An "emotion recognition engine" is a technology that analyzes a user's voice, facial expressions, text data, etc. to recognize their emotional state.

[0429] "Personalization" refers to providing services and products tailored to the individual preferences and emotional state of each user.

[0430] "Brick and mortar store" refers to a store that has a physical presence.

[0431] "Smart glasses" are glasses-type devices that have display and camera functions and can display and collect information.

[0432] A "smartphone" is a mobile phone that has advanced computing power, multiple functions, and can run apps.

[0433] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following describes an embodiment of the present invention.

[0434] Overall system overview

[0435] The embodiment of the invention comprises an external database, a server, a terminal, a user, and an emotion recognition engine. The server collects traffic information and navigation app usage statistics from the external database, and uses machine learning to generate optimal travel routes based on this information. Furthermore, the emotion recognition engine analyzes the user's emotional state and provides personalized travel plans and product recommendations in physical stores.

[0436] Program Overview

[0437] This system uses programming languages ​​such as Python to collect various data, perform machine learning, and recognize emotions. It also provides information to users via smart glasses or smartphones. Specifically, it uses the following hardware and software:

[0438] 1. Hardware

[0439] Smart glasses or smartphones: Collect the user's voice, facial expression, and text data and send it to an emotion recognition engine.

[0440] Server: Based on the collected data, it integrates traffic information, performs machine learning, generates itineraries, and makes personalized product recommendations.

[0441] 2. Software

[0442] Emotion recognition engine: Recognizes the user's emotional state by analyzing their voice, facial expressions, and text data. For example, it can be built using TensorFlow or Keras.

[0443] Machine learning model: Generates optimal travel routes based on traffic information and usage statistics. Uses Scikit-learn and TensorFlow.

[0444] API: Traffic information is collected using external databases such as Google Maps API.

[0445] System Operation

[0446] The server communicates with external databases to collect external data, and uses machine learning to generate optimal travel routes and plans. Users' emotional state is analyzed through smart glasses or smartphones, and they receive personalized travel plans or in-store product suggestions based on that analysis.

[0447] As a concrete example, there is a system in which a user walks around a store using smart glasses and receives product suggestions based on their emotional state at the time. For example, if the user is feeling stressed, relaxation items are suggested, and if the user is having fun, the latest fashion items are suggested.

[0448] Prompt Sentence Examples

[0449] "The user is currently in the emotional state 'happy'. Please generate a description for the following product:

[0450] Product 1: Latest fashion item (Description: The latest trendy fashion item for the new season.)

[0451] Product 2: Accessories (Description: Luxury accessories.)

[0452] The system can provide personalized travel plans and shopping experiences that take into account the user's emotional state.

[0453] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0454] Step 1:

[0455] The device collects the user's voice, facial expression, and text data in real time. This data is captured using a camera and microphone. The input data is the raw material for recognizing the user's emotions, and the output is this raw data.

[0456] Step 2:

[0457] The device sends the collected voice, facial expression, and text data to an emotion recognition engine. The emotion recognition engine is built using TensorFlow and Keras, and analyzes this data to recognize the user's emotions. The input is the data sent from the device, and the output is the user's emotional state (e.g., "happy," "sad," "angry," etc.). Specifically, the analysis algorithm determines the user's facial muscle movements and tone of voice.

[0458] Step 3:

[0459] The server receives the emotional state output by the emotion recognition engine and generates personalized travel plans or product suggestions based on it. The input is the recognized emotional state, and the output is a detailed travel plan or product list tailored to the user's emotions. Specifically, the server analyzes the characteristics of tourist attractions and products in the database and selects the best ones for the user.

[0460] Step 4:

[0461] The server collects real-time traffic and transportation information using external databases and APIs (e.g., Google Maps API). The input is information obtained from the external database, and the output is integrated traffic information. Specifically, the server makes API calls and stores the obtained data in its internal database.

[0462] Step 5:

[0463] The server performs machine learning based on the collected traffic information and usage statistics to generate the optimal travel route. The input is traffic information and past usage statistics, and the output is an optimized travel route. Specifically, the server trains the model using Scikit-learn and TensorFlow to calculate the optimal route.

[0464] Step 6:

[0465] The terminal displays the travel plan and product suggestions received from the server to the user. The input is the personalized plan sent from the server, and the output is the information displayed on the terminal's display. Specifically, the terminal displays the plan details and product list on the screen.

[0466] Step 7:

[0467] The user checks the displayed travel plans and product suggestions, and makes selections and reservations as necessary. The input is the information displayed on the terminal, and the output is the user's selection. Specifically, the user makes selections using a touch panel or voice commands.

[0468] Step 8:

[0469] Based on the user's selection, the server connects with an external accommodation reservation system to complete the reservation procedure. The input is the user's selection, and the output is reservation confirmation information. Specifically, the server uses the reservation system's API to confirm the reservation and obtain a confirmation ID.

[0470] Step 9:

[0471] The server provides real-time traffic and tourist spot information to users while they are traveling. The input is traffic information and tourist spot data obtained in real time, and the output is the latest information provided to the user. Specifically, the server periodically calls the API and sends information to the device.

[0472] Step 10:

[0473] The server collects user feedback after the trip and updates the machine learning model for the next itinerary generation. The input is the user feedback, and the output is the updated machine learning model. Specifically, the server analyzes the feedback data and adds it to the training dataset of the model.

[0474] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0475] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0476] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0477] [Second embodiment]

[0478] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0479] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0480] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0481] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0482] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0483] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0484] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0485] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0486] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0487] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0488] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0489] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0490] 1. Field of the Invention The present invention relates to a system for assisting in travel planning. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to the drawings.

[0491] Overall system configuration

[0492] This system consists of a server, a terminal, and a user. The server collects traffic information and usage statistics of navigation apps and uses this information to train a machine learning model. The terminal receives input from the user and sends it to the server. The user inputs their destination and travel conditions, and then checks and selects travel plans suggested by the server.

[0493] System Operation

[0494] 1. Collection of traffic information

[0495] The server collects traffic information from external databases and the navigation app's API, including public transport schedules and traffic congestion status.

[0496] Example: A server uses the Google Maps API to retrieve information about Shinkansen schedules and fares from Tokyo to Kyoto.

[0497] 2. Route optimization using machine learning

[0498] The server uses the collected traffic information to train a machine learning model that leverages the user's past travel data and real-time usage statistics to suggest optimal travel routes.

[0499] Example: A server analyzes past travel data and determines, based on traveler behavior patterns, that taking the Shinkansen is the quickest and most cost-effective option.

[0500] 3. Automatic generation of travel plans

[0501] The server uses the trained model to automatically generate an itinerary that best suits the user's requirements, including routes, the order in which tourist spots are visited, and recommended accommodations.

[0502] Example: A server generates a travel plan from Tokyo to Kyoto, outputting the Shinkansen travel time, tourist spots along the way, and a list of recommended hotels.

[0503] 4. Receiving and Sending User Input

[0504] The user inputs travel conditions such as the destination and departure point, desired itinerary, and favorite tourist spots through the terminal, and this input information is sent from the terminal to the server.

[0505] Example: A user uses a smartphone app to enter the following information: "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple."

[0506] 5. Check and select your travel plan

[0507] The terminal displays the travel plans sent from the server to the user, who then checks the proposed travel plans, selects the one they like, and confirms it.

[0508] Example: A user reviews suggested travel plans on their smartphone screen and selects one to visit Kiyomizu-dera Temple.

[0509] 6. Accommodation reservations

[0510] The server automatically processes the accommodation reservation based on the travel plan selected by the user, and completes the reservation in cooperation with an external accommodation reservation system.

[0511] Example: The server uses the API of a partner accommodation booking site to reserve a room at a selected hotel and obtain a reservation confirmation ID.

[0512] 7. Real-time information provision

[0513] The server provides real-time traffic and tourist information to the user while he or she is traveling, and this information is sent to the user via the terminal.

[0514] Example: While traveling, a user receives the latest traffic information and crowding status of tourist spots on their smartphone.

[0515] 8. Usage history and feedback collection

[0516] The server collects user travel history and feedback and uses it to generate future travel plans. This information is used to update the machine learning model.

[0517] Example: After completing a trip, a user enters an evaluation of the plan provided by the app, and the data is stored on the server.

[0518] In this way, the travel planning support system of the present invention can significantly improve convenience for travelers by suggesting optimal travel routes to users, automatically generating travel plans, and seamlessly booking accommodations.

[0519] The processing flow will be explained below.

[0520] Step 1:

[0521] The server collects traffic information from external databases and the navigation app's API.

[0522] Specific operation: The server uses the Google Maps API to obtain Shinkansen schedules and fare information from Tokyo to Kyoto and stores it in a database.

[0523] Step 2:

[0524] The server trains a machine learning model based on the collected traffic information.

[0525] Specific operation: The server takes in past travel data and user behavior patterns and runs an algorithm to learn the optimal travel route that minimizes travel time and costs.

[0526] Step 3:

[0527] Users use their device to input travel conditions such as travel destination, departure point, desired dates, and favorite tourist spots into the "Reserve-san" app.

[0528] Specific operation: The user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into the input form of the smartphone app and clicks the send button.

[0529] Step 4:

[0530] The terminal converts the user's input data into JSON format and sends it to the server.

[0531] Specific operation: The device packages the input data in JSON format and sends it to the server using an HTTP request.

[0532] Step 5:

[0533] The server generates an optimal travel route and sightseeing plan based on the received user's conditions.

[0534] Specific operation: The server uses a machine learning model to generate a travel plan that includes the Shinkansen route that best suits the user's requirements, tourist spots along the way, and recommended accommodations.

[0535] Step 6:

[0536] The server converts the generated travel plan into JSON format and sends it to the user's device.

[0537] Specific operation: The server encodes the generated travel plan in JSON format and sends it to the terminal as an HTTP response.

[0538] Step 7:

[0539] The terminal displays the travel plan received from the server to the user.

[0540] Specific operation: The device parses the contents of the travel plan and displays it on the screen in a format that is easy for the user to view.

[0541] Step 8:

[0542] The user checks the displayed travel plans and selects the plan they want.

[0543] Specific operation: The user checks the suggested tourist spots and accommodations on the smartphone screen and taps to select the plan they like.

[0544] Step 9:

[0545] The device converts the user's selected plan information into JSON format and sends it to the server.

[0546] Specific behavior: The device packages the selected travel plan in JSON format and sends it to the server using an HTTP request.

[0547] Step 10:

[0548] The server then processes the reservation for the accommodation based on the user's selection.

[0549] Specific operation: The server uses the information of the selected accommodation to call the API of the affiliated reservation site, completes the reservation procedure, and obtains the reservation confirmation ID.

[0550] Step 11:

[0551] The server sends information including the reservation confirmation ID to the terminal and notifies the user.

[0552] Specific operation: The server creates a response including the reservation confirmation ID and sends it to the terminal as an HTTP response. The terminal receives this and notifies the user.

[0553] Step 12:

[0554] The server allows the user to receive real-time traffic information and tourist spot updates while traveling and transmits them to the terminal.

[0555] Specific operation: The server collects and updates real-time traffic information and tourist spot information, and sends it to the user's device as a push notification.

[0556] Step 13:

[0557] The server collects the user's travel history and feedback after the trip is completed.

[0558] What happens: A user fills out a feedback form in the app and clicks the submit button. The data is sent to the server and stored in the database.

[0559] Step 14:

[0560] The server updates and improves the machine learning model based on collected travel history and feedback.

[0561] What it does: The server uses the newly collected data to retrain the machine learning algorithm and improve the accuracy of the next itinerary generation.

[0562] Example 1

[0563] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0564] Conventional travel planning systems provide limited information on transportation and tourist attractions, and lack real-time updates. Furthermore, travel plans are often generated manually, making it difficult to provide efficient and optimal plans. Furthermore, they are unable to effectively utilize traveler usage history and feedback, making it difficult to reflect this information in the generation of next travel plans.

[0565] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0566] In this invention, the server includes means for integrating traffic information collected from an external database and usage statistics of the navigation app, means for generating an optimal travel route using a machine learning model based on the integrated information, means for automatically generating a travel plan based on the generated travel route, means for collecting traveler usage history and feedback and updating the machine learning model for generating the next travel plan, and means for providing traffic information and tourist spot information in real time, thereby enabling the user to be provided with an efficient and optimal travel plan and real-time information during the trip.

[0567] An "external database" is an information source that provides transportation information and tourist spot information that can be accessed via the Internet.

[0568] A "navigation app" is a software application that provides users with travel routes and traffic conditions.

[0569] "Traffic information" refers to information related to travel, such as public transportation schedules and road congestion conditions.

[0570] "Usage statistics" refers to statistical data including user behavior data and access frequency for navigation applications.

[0571] A "machine learning model" is a mathematical model that learns patterns from data and makes predictions and classifications.

[0572] The "optimal travel route" refers to the most efficient and comfortable travel route, taking into consideration traffic conditions, past user data, costs, etc.

[0573] "Auto-generation" refers to the process of automatically generating information or plans through a program.

[0574] "Accommodation" means a place such as a hotel or lodging facility where travelers can stay.

[0575] A "reservation system" is a system that manages reservations for accommodation, transportation, etc. online.

[0576] "Usage history" refers to a record of the user's past travels and services used.

[0577] "Feedback" refers to opinions such as ratings and impressions provided by users.

[0578] "Real-time information" refers to information that is updated immediately based on the current time.

[0579] The present invention relates to a system for assisting in travel planning, and this system is composed of a server, a terminal, and a user. The following describes in detail an embodiment of the present invention.

[0580] Overall system configuration

[0581] The server collects traffic information and usage statistics from external databases and the navigation app's API, and uses this data to train the machine learning model. The device receives input from the user and sends it to the server. The user enters their destination and travel conditions, and then confirms and selects the travel plan suggested by the server. The entire system automatically generates an efficient and optimal travel plan by linking each function together.

[0582] Hardware and software used

[0583] This invention utilizes a server, a terminal, and an external API. Specifically, the server is a computer system with a high-performance CPU and large memory capacity, and uses a programming language such as Python to train machine learning models and process data. It also collects traffic information using the Google Maps API and public transportation schedule APIs. The terminals are smartphones and tablet devices, and applications running on these devices can be native iOS or Android apps.

[0584] Data processing and calculation

[0585] The server stores traffic information obtained from external databases and the navigation app's API in a database and performs data preprocessing. For example, the server uses the Google Maps API to obtain Shinkansen schedules and fare information from Tokyo to Kyoto. The server then trains a machine learning model using libraries such as scikit-learn to generate optimal travel routes. This model may include random forests or neural networks.

[0586] The server uses the trained model to automatically generate an optimal travel plan based on the user's criteria. For example, it uses past travel data to suggest a travel route from Tokyo to Kyoto and generates a plan that includes Shinkansen travel times, tourist spots along the way, and recommended accommodations. The user enters the destination, departure point, desired dates, and favorite tourist spots into the smartphone app, and sends the information from the device to the server. An example of a prompt sentence could be "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple."

[0587] Accommodation reservations

[0588] The server automatically reserves accommodations using the API of an external accommodation reservation system based on the travel plan selected by the user. For example, it uses the Booking.com API to search for hotels that meet the user's requirements and makes a reservation. Once the reservation is complete, the server obtains a reservation confirmation ID and notifies the user.

[0589] Providing real-time information

[0590] To provide real-time traffic and tourist spot information to users while traveling, the server periodically calls the navigation app's API to obtain the latest information and sends push notifications to the device, allowing users to stay up to date with the latest information during their trip.

[0591] Collection of usage history and feedback

[0592] The server collects the user's travel history and feedback and updates the machine learning model for future travel plan generation. For example, if a user enters a plan rating in the app after completing a trip, that data is stored on the server and used for the next model training.

[0593] By integrating these functions, the travel planning support system of the present invention can provide users with efficient and optimal travel plans and real-time information during travel.

[0594] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0595] Step 1:

[0596] The server collects traffic information and usage statistics from external databases and the navigation app's API. Input data includes transportation schedules and road traffic conditions. This information is collected and stored in a database. Specifically, the server calls the Google Maps API to obtain information on the schedule and traffic congestion for the specified section.

[0597] Step 2:

[0598] The server trains a machine learning model based on the collected traffic information and usage statistics. Input data includes historical travel data and real-time traffic information. It preprocesses the data and trains a model to suggest optimal travel routes. Specifically, the server preprocesses the data using the scikit-learn library, trains the model using the random forest algorithm, and saves the model.

[0599] Step 3:

[0600] The server uses a trained machine learning model to automatically generate an optimal travel plan based on the user's criteria. Input data includes the user's travel destination, departure point, desired dates, and favorite tourist spots. The model applies the criteria to generate the optimal travel route. Specifically, the server uses the trained model to organize the plan based on the user's criteria into a pandas data frame and send it to the terminal in JSON format.

[0601] Step 4:

[0602] The user inputs travel conditions such as the destination and departure point, desired dates, and tourist spots through the device. The input information is sent to the server. Specifically, the user enters the information into the smartphone app and presses the send button. The device then sends this information to the server as an API request.

[0603] Step 5:

[0604] The device displays the travel plan sent from the server to the user. The user reviews the proposed plans, selects the one they like, and confirms it. The input data includes the travel plan from the server. The travel plan selected by the user is sent to the server as output data. Specifically, the JSON data received from the server is passed to the device app, which displays the details of the travel plan in a list view. The selected information is sent back to the server when the user taps "Confirm" and "Select."

[0605] Step 6:

[0606] The server reserves accommodation based on the travel plan selected by the user. The input data includes the travel plan selected by the user. The output data includes the accommodation reservation confirmation ID. Specifically, the server calls the API of an external accommodation reservation system to complete the reservation. It then obtains the reservation confirmation ID and notifies it to the user.

[0607] Step 7:

[0608] The server provides real-time traffic information and tourist spot information to the user. Input data includes current traffic conditions and the congestion status of tourist spots. Updated information is sent to the device as output data. Specifically, the server periodically calls the navigation app's API, obtains the latest information, and sends it to the device via push notification.

[0609] Step 8:

[0610] The server collects the user's travel history and feedback and updates the machine learning model to generate future travel plans. The input data includes the user's usage history and feedback. The output data includes the updated model. Specifically, after completing a trip, the user enters a rating in the app, and the data is saved on the server. This data is used the next time the model is trained.

[0611] (Application example 1)

[0612] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0613] Conventional travel planning support systems have had difficulty providing travelers with the real-time information they need or automatically generating optimal travel plans based on their detailed preferences. Furthermore, the complicated procedures for changing travel plans and reserving accommodations make them less user-friendly for travelers.

[0614] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0615] In this invention, the server includes means for integrating traffic information collected from an external database and usage statistics of location information apps, means for performing machine learning based on the statistics to generate an optimal travel route, means for automatically generating a travel plan based on the generated travel route, means for booking accommodations based on the travel plan, means for collecting traveler usage history and opinions and updating a model for generating the next travel plan, and means for providing the latest information on the traveler's current location and travel destination in real time, thereby enabling the automatic generation of an optimal travel plan for a traveler and the provision of real-time information.

[0616] A "travel planning support system" is a system that optimizes travelers' travel plans and supports automatic generation and accommodation reservation procedures.

[0617] "Traffic information" refers to information collected from external databases, including public transportation schedules and traffic congestion status.

[0618] A "location-based app" is an application that identifies a user's geographic location and provides navigation and surrounding information to the user.

[0619] "Usage statistics" refers to statistical data based on a user's application usage history and behavioral patterns.

[0620] "Machine learning" is a technology that uses algorithms to learn data and make predictions and classifications.

[0621] "Travel route" refers to route information that indicates the means of transportation and travel route used by a traveler.

[0622] A "travel plan" is a plan that includes travel routes, the order in which tourist spots are visited, recommended accommodations, etc., generated based on conditions specified by the traveler.

[0623] "Accommodation facilities" refer to facilities such as hotels and inns where travelers can stay.

[0624] A "reservation" is a procedure for securing accommodation, transportation, etc. in advance.

[0625] "Usage history" refers to historical information about trips taken by a user in the past and services used by the user.

[0626] "Feedback" refers to user opinions and evaluations provided after using the service.

[0627] "Real-time information" refers to information that is updated hourly, such as the latest traffic conditions and the current congestion status of tourist spots.

[0628] The present invention is a system for assisting in travel planning, and its configuration and operation steps will be described in detail. First, an outline of the program required to realize this system will be created, and the processing based on this program will be described.

[0629] Overall system configuration

[0630] This system consists of a server, a terminal, and a user.

[0631] Server Roles

[0632] Collecting information from external databases: The server collects traffic information and usage statistics from external databases and location app APIs.

[0633] Data analysis with machine learning: Train machine learning models based on collected data to generate optimal travel routes.

[0634] Automatic itinerary generation: Using the trained model, we automatically generate itineraries that best fit the user's requirements.

[0635] Providing real-time information: Providing real-time updates on a traveler's current location and destination.

[0636] Collect usage history and feedback: Collect user usage history and feedback and update the model for the next itinerary generation.

[0637] Accommodation reservations: We will process your accommodation reservations based on your proposed travel plans.

[0638] Device Role

[0639] Receive user input: Enter travel destinations, departure points, desired dates, favorite attractions, and other conditions.

[0640] Display itinerary: Display the itinerary sent from the server to the user, and confirm and select the proposed itinerary.

[0641] User Roles

[0642] Entering travel conditions: Enter travel conditions through the terminal.

[0643] Choose your itinerary: Review the suggested itineraries and choose the one you like best.

[0644] Providing feedback: Provide feedback after your trip and contribute to improving the system.

[0645] Hardware and software configuration

[0646] Hardware: Smartphone (iOS / Android), server

[0647] Software: Google Maps API, HotelBooking API, Machine Learning algorithms (e.g. Logistic Regression)

[0648] Details of data processing and calculation

[0649] 1. Data collection: The server collects traffic and location information from Google Maps API and other external databases.

[0650] 2. Data analysis: Using machine learning models (e.g., Logistic Regression) to analyze users' historical data and generate optimal travel routes.

[0651] 3. Real-time information provision: Based on the traveler's current location, the server periodically calls the API to provide the latest traffic information and congestion status of tourist attractions in real time.

[0652] 4. Obtaining feedback and updating the model: After the trip, obtain feedback from the user and use it as training data for the machine learning model.

[0653] Specific examples

[0654] Travel planning

[0655] If a user enters "I want to travel to Kyoto on March 15th," the system will prompt "Please enter the dates and places of interest." If a user enters "I want to visit Kiyomizu-dera Temple and Kinkaku-ji Temple," the system will generate and suggest the optimal itinerary.

[0656] Prompt sentences for generative AI models (examples)

[0657] A user has entered that they would like to visit Kyoto on March 15th. Their tourist attractions of interest are Kiyomizu-dera Temple and Kinkaku-ji Temple. Please suggest the best travel route and accommodation. Please create a detailed plan, including tourist attractions along the way.

[0658] In this way, the travel planning support system of the present invention is able to automatically generate optimal travel plans for users and provide them with real-time information.

[0659] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0660] Step 1:

[0661] Receiving User Input

[0662] Subject: Device

[0663] Specific operation: The device receives information from the user, such as the travel destination, departure point, desired dates, and favorite tourist spots.

[0664] Input: Travel destination, departure point, desired dates, tourist attractions

[0665] Output: Received user input information

[0666] Data processing: The device converts the received information into an appropriate format and prepares it for transmission to the server.

[0667] Step 2:

[0668] Collection of traffic information and usage statistics

[0669] Subject: Server

[0670] What it does: The server uses the Google Maps API and other external databases to collect traffic information and location app usage statistics.

[0671] Input: External database, Google Maps API endpoint

[0672] Output: Collected traffic information and usage statistics

[0673] Data processing: The server organizes the collected data and stores it in a database, including any necessary filtering and preprocessing.

[0674] Step 3:

[0675] Training a machine learning model

[0676] Subject: Server

[0677] How it works: The server uses historical travel data and real-time usage statistics to train machine learning models.

[0678] Inputs: Historical travel data, real-time usage statistics

[0679] Output: A trained machine learning model

[0680] Data calculation: The server performs training using algorithms such as logistic regression, learning patterns based on past data and optimizing the model parameters.

[0681] Step 4:

[0682] Generating optimal travel routes

[0683] Subject: Server

[0684] What it does: Uses a trained machine learning model to generate optimal travel routes based on the user's criteria.

[0685] Input: User input, trained machine learning model

[0686] Output: Optimal travel route

[0687] Data calculation: The server applies the user's input to the model to predict the best route. It uses an optimization algorithm to evaluate and select travel routes.

[0688] Step 5:

[0689] Automatic travel plan generation

[0690] Subject: Server

[0691] Specific operation: The server automatically generates a travel plan based on the generated travel route, including travel routes, the order in which tourist attractions are visited, and recommended accommodations.

[0692] Input: Optimal travel route

[0693] Output: Auto-generated itinerary

[0694] Data processing: The server incorporates information on tourist attractions and accommodations based on the generated route and formats it into a complete travel plan.

[0695] Step 6:

[0696] View and select your travel plans

[0697] Subject: Device

[0698] Specific operation: The terminal displays the travel plans sent from the server to the user, allowing the user to confirm and select the plans.

[0699] Input: Auto-generated itinerary

[0700] Output: Selected itinerary

[0701] Data processing: Visually display the travel plan through the user interface and accept user selections.

[0702] Step 7:

[0703] Accommodation reservations

[0704] Subject: Server

[0705] Specific operation: Based on the selected travel plan, the server completes the accommodation reservation using the API of the external application.

[0706] Input: Selected travel plan

[0707] Output: Reservation confirmation ID

[0708] Data calculation: Obtain accommodation availability information from the API, complete the reservation process, and generate a confirmation ID.

[0709] Step 8:

[0710] Providing real-time information

[0711] Subject: Server

[0712] Specific operation: The server collects real-time traffic information and congestion status of tourist attractions based on the traveler's current location and sends it to the terminal.

[0713] Input: Current location, real-time information

[0714] Output: User notification

[0715] Data calculation: The server obtains the latest information from the API and pushes it to the device.

[0716] Step 9:

[0717] Get feedback and update the model

[0718] Subject: Server

[0719] What it does: After the trip, the server collects user feedback and uses it as training data for the machine learning model.

[0720] Input: User feedback

[0721] Output: An updated machine learning model

[0722] Data processing: Analyze the feedback and add it to the model's training dataset. Retrain the model as a new model.

[0723] This process enables the travel planning support system to automatically generate optimal travel plans for users and provide real-time information.

[0724] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0725] The present invention relates to a system for assisting in travel planning, and more particularly to a system for providing more personalized travel plans by incorporating an emotion engine that recognizes the emotions of a user. Hereinafter, an embodiment of the present invention will be described in detail.

[0726] Overall system configuration

[0727] This system consists of a server, a terminal, a user, and an emotion engine. The emotion engine has the means to recognize emotions by analyzing the user's voice, facial expressions, and text data. The server generates a travel plan that is most suitable for the user based on the output of this emotion engine. The terminal is responsible for receiving this information from the user and sending it to the server. The user inputs the travel destination, departure point, desired dates, emotional state, etc. via the terminal, and then confirms and selects the travel plan proposed by the server.

[0728] System Operation

[0729] 1. Collection of traffic information

[0730] The server collects traffic information from external databases and the navigation app's API, including public transport schedules and traffic congestion status.

[0731] Example: A server uses the Google Maps API to retrieve Shinkansen schedules and fare information from Tokyo to Kyoto and stores it in a database.

[0732] 2. Route optimization using machine learning

[0733] The server uses the collected traffic information to train a machine learning model that leverages the user's past travel data and real-time usage statistics to suggest optimal travel routes.

[0734] Example: A server analyzes past travel data and determines, based on traveler behavior patterns, that taking the Shinkansen is the quickest and most cost-effective option.

[0735] 3. Emotion Recognition by Emotion Engine

[0736] The terminal collects the user's voice data, facial expression data, and text data and sends them to the emotion engine, which analyzes the data and recognizes the user's current emotional state.

[0737] Example: The emotion engine analyzes the voice and facial expressions of a user speaking into the device's camera and determines that the user is in a "relaxed" state.

[0738] 4. Receiving and Sending User Input

[0739] The user inputs travel conditions such as the destination, desired itinerary, current emotional state, and favorite tourist spots through the terminal, and this information is sent from the terminal to the server.

[0740] Example: A user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into an input form on a smartphone app and clicks the submit button.

[0741] 5. Automatic travel plan generation

[0742] The server generates an optimal travel route and sightseeing plan for the user based on the output of the machine learning model and emotion engine, including the route, the order in which tourist spots are visited, and recommended accommodations.

[0743] Example: The server considers the user's relaxed emotional state and tourist spot preferences to suggest a relaxing route in Kyoto.

[0744] 6. Check and select your travel plan

[0745] The terminal displays the travel plans sent from the server to the user, who then checks the proposed travel plans, selects the one they like, and confirms it.

[0746] Example: A user reviews suggested travel plans on their smartphone screen and selects one to visit Kiyomizu-dera Temple.

[0747] 7. Accommodation reservations

[0748] The server automatically processes the accommodation reservation based on the travel plan selected by the user, and completes the reservation in cooperation with an external accommodation reservation system.

[0749] Example: The server uses the API of a partner accommodation booking site to reserve a room at a selected hotel and obtain a reservation confirmation ID.

[0750] 8. Real-time information provision

[0751] The server provides real-time traffic and tourist information to the user while he or she is traveling, and this information is sent to the user via the terminal.

[0752] Example: While traveling, a user receives the latest traffic information and crowding status of tourist spots on their smartphone.

[0753] 9. Usage history and feedback collection

[0754] The server collects user travel history and feedback and uses it to generate future travel plans. This information is used to update the machine learning model.

[0755] Example: After completing a trip, a user enters an evaluation of the plan provided by the app, and the data is stored on the server.

[0756] In this way, the travel planning support system of the present invention can significantly improve convenience for travelers by proposing optimal travel routes taking into account the user's emotional state, automatically generating travel plans, and seamlessly booking accommodations.

[0757] The processing flow will be explained below.

[0758] Step 1:

[0759] The server collects traffic information from external databases and the navigation app's API.

[0760] Specific operation: The server uses the Google Maps API to obtain Shinkansen schedules and fare information from Tokyo to Kyoto and stores it in a database.

[0761] Step 2:

[0762] The server trains a machine learning model based on the collected traffic information.

[0763] Specific operation: The server takes in past travel data and user behavior patterns and runs an algorithm to learn the optimal travel route that minimizes travel time and costs.

[0764] Step 3:

[0765] The user uses the terminal to input conditions such as the travel destination, departure point, desired dates, and favorite tourist spots.

[0766] Specific operation: The user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into the input form of the smartphone app and clicks the send button.

[0767] Step 4:

[0768] The terminal converts the user's input data into JSON format and sends it to the server.

[0769] Specific operation: The device packages the input data in JSON format and sends it to the server using an HTTP request.

[0770] Step 5:

[0771] The server generates an optimal travel route and sightseeing plan based on the received user's conditions.

[0772] Specific operation: The server uses a machine learning model to generate a travel plan that includes the Shinkansen route that best suits the user's requirements, tourist spots along the way, and recommended accommodations.

[0773] Step 6:

[0774] The server converts the generated travel plan into JSON format and sends it to the user's device.

[0775] Specific operation: The server encodes the generated travel plan in JSON format and sends it to the terminal as an HTTP response.

[0776] Step 7:

[0777] The terminal displays the travel plan received from the server to the user.

[0778] Specific operation: The device parses the contents of the travel plan and displays it on the screen in a format that is easy for the user to view.

[0779] Step 8:

[0780] The user checks the displayed travel plans and selects the plan they want.

[0781] Specific operation: The user checks the suggested tourist spots and accommodations on the smartphone screen and taps to select the plan they like.

[0782] Step 9:

[0783] The device converts the user's selected plan information into JSON format and sends it to the server.

[0784] Specific behavior: The device packages the selected travel plan in JSON format and sends it to the server using an HTTP request.

[0785] Step 10:

[0786] The server then processes the reservation for the accommodation based on the user's selection.

[0787] Specific operation: The server uses the information of the selected accommodation to call the API of the affiliated reservation site, completes the reservation procedure, and obtains the reservation confirmation ID.

[0788] Step 11:

[0789] The server sends information including the reservation confirmation ID to the terminal and notifies the user.

[0790] Specific operation: The server creates a response including the reservation confirmation ID and sends it to the terminal as an HTTP response. The terminal receives this and notifies the user.

[0791] Step 12:

[0792] The server allows the user to receive real-time traffic information and tourist spot updates while traveling and transmits them to the terminal.

[0793] Specific operation: The server collects and updates real-time traffic information and tourist spot information, and sends it to the user's device as a push notification.

[0794] Step 13:

[0795] The server collects the user's travel history and feedback after the trip is completed.

[0796] What happens: A user fills out a feedback form in the app and clicks the submit button. The data is sent to the server and stored in the database.

[0797] Step 14:

[0798] The terminal collects the user's voice data, facial expression data, and text data and sends them to the emotion engine.

[0799] Specific operation: Using the device's camera and microphone, the user's voice and facial expressions are recorded and sent to the emotion engine along with the text data.

[0800] Step 15:

[0801] The emotion engine analyzes the user's voice, facial expression, and text data to recognize the user's current emotional state.

[0802] Specific operation: The emotion engine uses an analytical algorithm to determine emotions such as "relaxed," "excited," and "stressed" from the collected data.

[0803] Step 16:

[0804] The server adjusts the contents of the travel plan based on the emotional state recognized by the emotion engine.

[0805] Specific operation: The server generates a plan that includes tourist spots and experiences that correspond to the emotional state of "wanting to relax."

[0806] Step 17:

[0807] The server updates the travel plan in real time according to changes in the emotional state and sends it to the terminal.

[0808] Specific operation: When a user feels stressed during a trip, the server generates a plan including new relaxation spots and sends it to the terminal.

[0809] In this way, the travel planning support system of the present invention can significantly improve convenience for travelers by proposing optimal travel routes taking into account the user's emotional state, automatically generating travel plans, and seamlessly booking accommodations.

[0810] Example 2

[0811] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0812] Conventional travel planning support systems have the problem of being unable to provide personalized travel plans based on users' emotional state or real-time fluctuating information, which has led to travelers being unable to have a comfortable and satisfying travel experience.

[0813] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0814] In this invention, the server includes means for collecting traffic information from an external database and integrating usage statistics of a navigation app, means for generating an optimal travel route using a machine learning model based on the statistics, means for utilizing an emotion recognition engine that recognizes emotions by analyzing the traveler's voice, facial expression, and text data, means for automatically generating a travel plan based on the output of the emotion recognition engine, means for reserving accommodations based on the travel plan, and means for collecting the traveler's usage history and feedback and updating the model for generating the next travel plan. This makes it possible to provide an optimal and personalized travel plan based on the user's current emotional state and past travel history.

[0815] "Traffic information" refers to data such as public transportation schedules and traffic congestion conditions.

[0816] "Navigation app usage statistics" refers to data on the behavioral patterns and real-time usage status of users of navigation apps.

[0817] A "machine learning model" refers to an algorithm that automatically learns from large amounts of data and makes predictions and classifications.

[0818] An "emotion recognition engine" refers to a system that recognizes emotions by analyzing a user's voice, facial expressions, and text data.

[0819] A "travel plan" refers to a detailed travel plan that includes travel routes, the order in which tourist spots are visited, recommended accommodations, etc.

[0820] "Accommodation Booking" refers to the process of reserving accommodation for a traveler.

[0821] "Usage history" refers to travel plans and behavioral data that a user has used in the past.

[0822] "Feedback" refers to the evaluations and opinions given by users regarding the services and plans provided.

[0823] "Updating a model" refers to the process of improving the performance of an existing machine learning model based on new data.

[0824] The present invention relates to a system for assisting in travel planning, and more particularly to a system for providing more personalized travel plans by incorporating an emotion engine that recognizes the emotions of a user. Hereinafter, an embodiment of the present invention will be described in detail.

[0825] This system consists of a server, a terminal, a user, and an emotion engine. The emotion engine has the means to recognize emotions by analyzing the user's voice, facial expressions, and text data. The server generates a travel plan that is most suitable for the user based on the output of this emotion engine. The terminal is responsible for receiving this information from the user and sending it to the server. The user inputs the travel destination, departure point, desired dates, emotional state, etc. via the terminal, and then confirms and selects the travel plan proposed by the server.

[0826] System Configuration

[0827] 1. Means of collecting traffic information

[0828] The server collects traffic information using external databases and the navigation app's API, including public transport schedules and traffic congestion status. The specific software used is Google Maps API.

[0829] 2. Using Machine Learning Models

[0830] The server uses the collected traffic information to train a machine learning model that uses the user's past travel data and real-time usage statistics to suggest optimal travel routes using software including TensorFlow.

[0831] 3. Emotion recognition

[0832] The device collects the user's voice, facial expression, and text data and sends it to the emotion engine, which analyzes this data and recognizes the user's current emotional state using Microsoft Azure Cognitive Services.

[0833] 4. Automatic generation of travel plans

[0834] The server generates an optimal travel route and sightseeing plan for the user based on the output of the machine learning model and emotion engine, including the route, the order in which tourist spots are visited, and recommended accommodations.

[0835] 5. Accommodation reservations

[0836] The server automatically makes reservations for accommodation based on the travel plan. At this time, it completes the reservation by linking with an external accommodation reservation system. Specifically, it uses the Booking.com API.

[0837] 6. Collecting User Usage History and Feedback

[0838] The server collects user travel history and feedback and uses it to generate future travel plans. This information is also used to update the machine learning model.

[0839] Specific examples

[0840] For example, if a user uses a smartphone app to input "Kyoto, March 15, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple," and the emotion recognition engine recognizes the user's relaxed state, the server will use this information to generate a travel plan that includes relaxing tourist spots and accommodations.The server then uses the Booking.com API to make reservations for the accommodations and notify the user.

[0841] Prompt Sentence Examples

[0842] "How do I use the Google Maps API to plan my trip?"

[0843] In this way, the travel planning support system of the present invention can provide an optimal and personalized travel plan that takes into account the emotional state of the user.

[0844] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0845] Step 1:

[0846] The server collects traffic information using an external database and the navigation app's API. As input, it receives traffic information from the external database and the navigation app. This provides data such as public transport schedules and traffic congestion conditions. The server stores this information in a database. Specifically, the server calls the Google Maps API to obtain the Shinkansen schedule and fare information from Tokyo to Kyoto, and stores this information in the database.

[0847] Step 2:

[0848] The server trains a machine learning model based on the collected traffic information. It uses the traffic information collected in step 1 and past travel data as input. This creates a predictive model for generating optimal travel routes. The server uses the TensorFlow library to train the machine learning model and prepares optimal travel route suggestions. Specifically, the server analyzes past travel data and learns user behavior patterns.

[0849] Step 3:

[0850] The device collects the user's voice data, facial expression data, and text data, and sends this data to the emotion engine. The user's voice, facial expression, and text data are used as input. This allows the emotion engine to recognize the user's current emotional state. The emotion engine uses Microsoft Azure Cognitive Services to perform emotion analysis. Specifically, data collected using the device's camera and microphone is sent to the Microsoft Azure Cognitive Services API, which recognizes the user's emotional state, such as "relaxed" or "tense."

[0851] Step 4:

[0852] The user inputs conditions such as the travel destination, desired schedule, emotional state, and favorite tourist spots through the device. The travel information specified by the user is received as input. This information is sent from the device to the server. Specifically, the user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into the input form of the smartphone app and clicks the send button.

[0853] Step 5:

[0854] The server generates the optimal travel route and sightseeing plan for the user based on the output of the machine learning model and emotion engine. As input, it uses the output of the machine learning model in step 2, the result of the emotion engine in step 3, and the user's travel conditions sent in step 4. This creates the optimal travel plan. Specifically, the server considers the user's relaxed emotional state and tourist spot preferences, and proposes a travel route that includes relaxing tourist spots and accommodations.

[0855] Step 6:

[0856] The terminal displays the travel plan sent from the server to the user. The terminal receives the travel plan from the server as input. Based on this information, the user checks the proposed plans, selects the one they like, and confirms it. Specifically, the user checks the proposed travel plan on the smartphone screen, selects the plan to visit Kiyomizu-dera Temple and Kinkaku-ji Temple, and clicks the confirm button.

[0857] Step 7:

[0858] The server automatically makes accommodation reservations based on the travel plan selected by the user. It receives the confirmed travel plan as input and completes the reservation by connecting with an external accommodation reservation system. Specifically, the server uses the Booking.com API to reserve a room at the selected hotel and obtains a reservation confirmation ID.

[0859] Step 8:

[0860] The server provides real-time traffic and tourist spot information to users traveling. The server references the user's current location and travel plan as input, allowing it to continue providing real-time information. Specifically, the server obtains the latest traffic congestion information and congestion status at tourist spots, and notifies the traveling user via their smartphone.

[0861] Step 9:

[0862] The server collects the user's travel history and feedback and uses it to generate future travel plans. This information is also used to update the machine learning model. User feedback and travel history data are taken as input. Specifically, after the user completes their trip, they enter an evaluation of the plan provided by the app, and this data is stored on the server. The server uses this data as training data for a new machine learning model.

[0863] (Application example 2)

[0864] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0865] Conventional travel planning support systems were unable to generate travel plans that took into account the user's individual emotional state, and were limited to providing standard plans. This made it difficult to provide travel plans that were tailored to the user's needs and emotions. Furthermore, in the shopping experience, there was also the issue of being unable to provide personalized product suggestions based on the user's emotions and real-time situation.

[0866] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0867] In this invention, the server includes means for integrating traffic information collected from an external database and usage statistics of the navigation app, means for performing machine learning based on the statistics to generate an optimal travel route, and means for automatically generating a travel plan based on the generated travel route, thereby making it possible to analyze the user's emotional state in real time and provide personalized travel plans and product suggestions.

[0868] An "external database" is a collection of information or data accessible through the Internet or other network.

[0869] A "navigation app" is software that provides maps and traffic information and guides users to the optimal route from their current location to their destination.

[0870] "Usage Statistics" refers to the compilation and analysis of data based on a user's past usage history and behavioral patterns.

[0871] "Machine learning" is a technology in which a computer learns patterns and rules based on data and automatically makes predictions and decisions.

[0872] A "travel route" refers to a travel route from a user's departure point to a destination.

[0873] A "travel plan" is a plan that includes travel destinations, transportation, accommodations, tourist spots, etc.

[0874] An "accommodation reservation system" is a system for making online reservations for accommodations such as hotels and inns.

[0875] "Usage history" refers to a record of trips the user has taken and a history of services they have used.

[0876] "Feedback" refers to a user providing an evaluation or opinion about a service or product.

[0877] An "emotion recognition engine" is a technology that analyzes a user's voice, facial expressions, text data, etc. to recognize their emotional state.

[0878] "Personalization" refers to providing services and products tailored to the individual preferences and emotional state of each user.

[0879] "Brick and mortar store" refers to a store that has a physical presence.

[0880] "Smart glasses" are glasses-type devices that have display and camera functions and can display and collect information.

[0881] A "smartphone" is a mobile phone that has advanced computing power, multiple functions, and can run apps.

[0882] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following describes an embodiment of the present invention.

[0883] Overall system overview

[0884] The embodiment of the invention comprises an external database, a server, a terminal, a user, and an emotion recognition engine. The server collects traffic information and navigation app usage statistics from the external database, and uses machine learning to generate optimal travel routes based on this information. Furthermore, the emotion recognition engine analyzes the user's emotional state and provides personalized travel plans and product recommendations in physical stores.

[0885] Program Overview

[0886] This system uses programming languages ​​such as Python to collect various data, perform machine learning, and recognize emotions. It also provides information to users via smart glasses or smartphones. Specifically, it uses the following hardware and software:

[0887] 1. Hardware

[0888] Smart glasses or smartphones: Collect the user's voice, facial expression, and text data and send it to an emotion recognition engine.

[0889] Server: Based on the collected data, it integrates traffic information, performs machine learning, generates itineraries, and makes personalized product recommendations.

[0890] 2. Software

[0891] Emotion recognition engine: Recognizes the user's emotional state by analyzing their voice, facial expressions, and text data. For example, it can be built using TensorFlow or Keras.

[0892] Machine learning model: Generates optimal travel routes based on traffic information and usage statistics. Uses Scikit-learn and TensorFlow.

[0893] API: Traffic information is collected using external databases such as Google Maps API.

[0894] System Operation

[0895] The server communicates with external databases to collect external data, and uses machine learning to generate optimal travel routes and plans. Users' emotional state is analyzed through smart glasses or smartphones, and they receive personalized travel plans or in-store product suggestions based on that analysis.

[0896] As a concrete example, there is a system in which a user walks around a store using smart glasses and receives product suggestions based on their emotional state at the time. For example, if the user is feeling stressed, relaxation items are suggested, and if the user is having fun, the latest fashion items are suggested.

[0897] Prompt Sentence Examples

[0898] "The user is currently in the emotional state 'happy'. Please generate a description for the following product:

[0899] Product 1: Latest fashion item (Description: The latest trendy fashion item for the new season.)

[0900] Product 2: Accessories (Description: Luxury accessories.)

[0901] The system can provide personalized travel plans and shopping experiences that take into account the user's emotional state.

[0902] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0903] Step 1:

[0904] The device collects the user's voice, facial expression, and text data in real time. This data is captured using a camera and microphone. The input data is the raw material for recognizing the user's emotions, and the output is this raw data.

[0905] Step 2:

[0906] The device sends the collected voice, facial expression, and text data to an emotion recognition engine. The emotion recognition engine is built using TensorFlow and Keras, and analyzes this data to recognize the user's emotions. The input is the data sent from the device, and the output is the user's emotional state (e.g., "happy," "sad," "angry," etc.). Specifically, the analysis algorithm determines the user's facial muscle movements and tone of voice.

[0907] Step 3:

[0908] The server receives the emotional state output by the emotion recognition engine and generates personalized travel plans or product suggestions based on it. The input is the recognized emotional state, and the output is a detailed travel plan or product list tailored to the user's emotions. Specifically, the server analyzes the characteristics of tourist attractions and products in the database and selects the best ones for the user.

[0909] Step 4:

[0910] The server collects real-time traffic and transportation information using external databases and APIs (e.g., Google Maps API). The input is information obtained from the external database, and the output is integrated traffic information. Specifically, the server makes API calls and stores the obtained data in its internal database.

[0911] Step 5:

[0912] The server performs machine learning based on the collected traffic information and usage statistics to generate the optimal travel route. The input is traffic information and past usage statistics, and the output is an optimized travel route. Specifically, the server trains the model using Scikit-learn and TensorFlow to calculate the optimal route.

[0913] Step 6:

[0914] The terminal displays the travel plan and product suggestions received from the server to the user. The input is the personalized plan sent from the server, and the output is the information displayed on the terminal's display. Specifically, the terminal displays the plan details and product list on the screen.

[0915] Step 7:

[0916] The user checks the displayed travel plans and product suggestions, and makes selections and reservations as necessary. The input is the information displayed on the terminal, and the output is the user's selection. Specifically, the user makes selections using a touch panel or voice commands.

[0917] Step 8:

[0918] Based on the user's selection, the server connects with an external accommodation reservation system to complete the reservation procedure. The input is the user's selection, and the output is reservation confirmation information. Specifically, the server uses the reservation system's API to confirm the reservation and obtain a confirmation ID.

[0919] Step 9:

[0920] The server provides real-time traffic and tourist spot information to users while they are traveling. The input is traffic information and tourist spot data obtained in real time, and the output is the latest information provided to the user. Specifically, the server periodically calls the API and sends information to the device.

[0921] Step 10:

[0922] The server collects user feedback after the trip and updates the machine learning model for the next itinerary generation. The input is the user feedback, and the output is the updated machine learning model. Specifically, the server analyzes the feedback data and adds it to the training dataset of the model.

[0923] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0924] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0925] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0926] [Third embodiment]

[0927] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0928] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0929] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0930] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0931] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0932] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0933] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0934] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0935] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0936] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0937] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0938] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0939] 1. Field of the Invention The present invention relates to a system for assisting in travel planning. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to the drawings.

[0940] Overall system configuration

[0941] This system consists of a server, a terminal, and a user. The server collects traffic information and usage statistics of navigation apps and uses this information to train a machine learning model. The terminal receives input from the user and sends it to the server. The user inputs their destination and travel conditions, and then checks and selects travel plans suggested by the server.

[0942] System Operation

[0943] 1. Collection of traffic information

[0944] The server collects traffic information from external databases and the navigation app's API, including public transport schedules and traffic congestion status.

[0945] Example: A server uses the Google Maps API to retrieve information about Shinkansen schedules and fares from Tokyo to Kyoto.

[0946] 2. Route optimization using machine learning

[0947] The server uses the collected traffic information to train a machine learning model that leverages the user's past travel data and real-time usage statistics to suggest optimal travel routes.

[0948] Example: A server analyzes past travel data and determines, based on traveler behavior patterns, that taking the Shinkansen is the quickest and most cost-effective option.

[0949] 3. Automatic generation of travel plans

[0950] The server uses the trained model to automatically generate an itinerary that best suits the user's requirements, including routes, the order in which tourist spots are visited, and recommended accommodations.

[0951] Example: A server generates a travel plan from Tokyo to Kyoto, outputting the Shinkansen travel time, tourist spots along the way, and a list of recommended hotels.

[0952] 4. Receiving and Sending User Input

[0953] The user inputs travel conditions such as the destination and departure point, desired itinerary, and favorite tourist spots through the terminal, and this input information is sent from the terminal to the server.

[0954] Example: A user uses a smartphone app to enter the following information: "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple."

[0955] 5. Check and select your travel plan

[0956] The terminal displays the travel plans sent from the server to the user, who then checks the proposed travel plans, selects the one they like, and confirms it.

[0957] Example: A user reviews suggested travel plans on their smartphone screen and selects one to visit Kiyomizu-dera Temple.

[0958] 6. Accommodation reservations

[0959] The server automatically processes the accommodation reservation based on the travel plan selected by the user, and completes the reservation in cooperation with an external accommodation reservation system.

[0960] Example: The server uses the API of a partner accommodation booking site to reserve a room at a selected hotel and obtain a reservation confirmation ID.

[0961] 7. Real-time information provision

[0962] The server provides real-time traffic and tourist information to the user while he or she is traveling, and this information is sent to the user via the terminal.

[0963] Example: While traveling, a user receives the latest traffic information and crowding status of tourist spots on their smartphone.

[0964] 8. Usage history and feedback collection

[0965] The server collects user travel history and feedback and uses it to generate future travel plans. This information is used to update the machine learning model.

[0966] Example: After completing a trip, a user enters an evaluation of the plan provided by the app, and the data is stored on the server.

[0967] In this way, the travel planning support system of the present invention can significantly improve convenience for travelers by suggesting optimal travel routes to users, automatically generating travel plans, and seamlessly booking accommodations.

[0968] The processing flow will be explained below.

[0969] Step 1:

[0970] The server collects traffic information from external databases and the navigation app's API.

[0971] Specific operation: The server uses the Google Maps API to obtain Shinkansen schedules and fare information from Tokyo to Kyoto and stores it in a database.

[0972] Step 2:

[0973] The server trains a machine learning model based on the collected traffic information.

[0974] Specific operation: The server takes in past travel data and user behavior patterns and runs an algorithm to learn the optimal travel route that minimizes travel time and costs.

[0975] Step 3:

[0976] Users use their device to input travel conditions such as travel destination, departure point, desired dates, and favorite tourist spots into the "Reserve-san" app.

[0977] Specific operation: The user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into the input form of the smartphone app and clicks the send button.

[0978] Step 4:

[0979] The terminal converts the user's input data into JSON format and sends it to the server.

[0980] Specific operation: The device packages the input data in JSON format and sends it to the server using an HTTP request.

[0981] Step 5:

[0982] The server generates an optimal travel route and sightseeing plan based on the received user's conditions.

[0983] Specific operation: The server uses a machine learning model to generate a travel plan that includes the Shinkansen route that best suits the user's requirements, tourist spots along the way, and recommended accommodations.

[0984] Step 6:

[0985] The server converts the generated travel plan into JSON format and sends it to the user's device.

[0986] Specific operation: The server encodes the generated travel plan in JSON format and sends it to the terminal as an HTTP response.

[0987] Step 7:

[0988] The terminal displays the travel plan received from the server to the user.

[0989] Specific operation: The device parses the contents of the travel plan and displays it on the screen in a format that is easy for the user to view.

[0990] Step 8:

[0991] The user checks the displayed travel plans and selects the plan they want.

[0992] Specific operation: The user checks the suggested tourist spots and accommodations on the smartphone screen and taps to select the plan they like.

[0993] Step 9:

[0994] The device converts the user's selected plan information into JSON format and sends it to the server.

[0995] Specific behavior: The device packages the selected travel plan in JSON format and sends it to the server using an HTTP request.

[0996] Step 10:

[0997] The server then processes the reservation for the accommodation based on the user's selection.

[0998] Specific operation: The server uses the information of the selected accommodation to call the API of the affiliated reservation site, completes the reservation procedure, and obtains the reservation confirmation ID.

[0999] Step 11:

[1000] The server sends information including the reservation confirmation ID to the terminal and notifies the user.

[1001] Specific operation: The server creates a response including the reservation confirmation ID and sends it to the terminal as an HTTP response. The terminal receives this and notifies the user.

[1002] Step 12:

[1003] The server allows the user to receive real-time traffic information and tourist spot updates while traveling and transmits them to the terminal.

[1004] Specific operation: The server collects and updates real-time traffic information and tourist spot information, and sends it to the user's device as a push notification.

[1005] Step 13:

[1006] The server collects the user's travel history and feedback after the trip is completed.

[1007] What happens: A user fills out a feedback form in the app and clicks the submit button. The data is sent to the server and stored in the database.

[1008] Step 14:

[1009] The server updates and improves the machine learning model based on collected travel history and feedback.

[1010] What it does: The server uses the newly collected data to retrain the machine learning algorithm and improve the accuracy of the next itinerary generation.

[1011] Example 1

[1012] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1013] Conventional travel planning systems provide limited information on transportation and tourist attractions, and lack real-time updates. Furthermore, travel plans are often generated manually, making it difficult to provide efficient and optimal plans. Furthermore, they are unable to effectively utilize traveler usage history and feedback, making it difficult to reflect this information in the generation of next travel plans.

[1014] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1015] In this invention, the server includes means for integrating traffic information collected from an external database and usage statistics of the navigation app, means for generating an optimal travel route using a machine learning model based on the integrated information, means for automatically generating a travel plan based on the generated travel route, means for collecting traveler usage history and feedback and updating the machine learning model for generating the next travel plan, and means for providing traffic information and tourist spot information in real time, thereby enabling the user to be provided with an efficient and optimal travel plan and real-time information during the trip.

[1016] An "external database" is an information source that provides transportation information and tourist spot information that can be accessed via the Internet.

[1017] A "navigation app" is a software application that provides users with travel routes and traffic conditions.

[1018] "Traffic information" refers to information related to travel, such as public transportation schedules and road congestion conditions.

[1019] "Usage statistics" refers to statistical data including user behavior data and access frequency for navigation applications.

[1020] A "machine learning model" is a mathematical model that learns patterns from data and makes predictions and classifications.

[1021] The "optimal travel route" refers to the most efficient and comfortable travel route, taking into consideration traffic conditions, past user data, costs, etc.

[1022] "Auto-generation" refers to the process of automatically generating information or plans through a program.

[1023] "Accommodation" means a place such as a hotel or lodging facility where travelers can stay.

[1024] A "reservation system" is a system that manages reservations for accommodation, transportation, etc. online.

[1025] "Usage history" refers to a record of the user's past travels and services used.

[1026] "Feedback" refers to opinions such as ratings and impressions provided by users.

[1027] "Real-time information" refers to information that is updated immediately based on the current time.

[1028] The present invention relates to a system for assisting in travel planning, and this system is composed of a server, a terminal, and a user. The following describes in detail an embodiment of the present invention.

[1029] Overall system configuration

[1030] The server collects traffic information and usage statistics from external databases and the navigation app's API, and uses this data to train the machine learning model. The device receives input from the user and sends it to the server. The user enters their destination and travel conditions, and then confirms and selects the travel plan suggested by the server. The entire system automatically generates an efficient and optimal travel plan by linking each function together.

[1031] Hardware and software used

[1032] This invention utilizes a server, a terminal, and an external API. Specifically, the server is a computer system with a high-performance CPU and large memory capacity, and uses a programming language such as Python to train machine learning models and process data. It also collects traffic information using the Google Maps API and public transportation schedule APIs. The terminals are smartphones and tablet devices, and applications running on these devices can be native iOS or Android apps.

[1033] Data processing and calculation

[1034] The server stores traffic information obtained from external databases and the navigation app's API in a database and performs data preprocessing. For example, the server uses the Google Maps API to obtain Shinkansen schedules and fare information from Tokyo to Kyoto. The server then trains a machine learning model using libraries such as scikit-learn to generate optimal travel routes. This model may include random forests or neural networks.

[1035] The server uses the trained model to automatically generate an optimal travel plan based on the user's criteria. For example, it uses past travel data to suggest a travel route from Tokyo to Kyoto and generates a plan that includes Shinkansen travel times, tourist spots along the way, and recommended accommodations. The user enters the destination, departure point, desired dates, and favorite tourist spots into the smartphone app, and sends the information from the device to the server. An example of a prompt sentence could be "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple."

[1036] Accommodation reservations

[1037] The server automatically reserves accommodations using the API of an external accommodation reservation system based on the travel plan selected by the user. For example, it uses the Booking.com API to search for hotels that meet the user's requirements and makes a reservation. Once the reservation is complete, the server obtains a reservation confirmation ID and notifies the user.

[1038] Providing real-time information

[1039] To provide real-time traffic and tourist spot information to users while traveling, the server periodically calls the navigation app's API to obtain the latest information and sends push notifications to the device, allowing users to stay up to date with the latest information during their trip.

[1040] Collection of usage history and feedback

[1041] The server collects the user's travel history and feedback and updates the machine learning model for future travel plan generation. For example, if a user enters a plan rating in the app after completing a trip, that data is stored on the server and used for the next model training.

[1042] By integrating these functions, the travel planning support system of the present invention can provide users with efficient and optimal travel plans and real-time information during travel.

[1043] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1044] Step 1:

[1045] The server collects traffic information and usage statistics from external databases and the navigation app's API. Input data includes transportation schedules and road traffic conditions. This information is collected and stored in a database. Specifically, the server calls the Google Maps API to obtain information on the schedule and traffic congestion for the specified section.

[1046] Step 2:

[1047] The server trains a machine learning model based on the collected traffic information and usage statistics. Input data includes historical travel data and real-time traffic information. It preprocesses the data and trains a model to suggest optimal travel routes. Specifically, the server preprocesses the data using the scikit-learn library, trains the model using the random forest algorithm, and saves the model.

[1048] Step 3:

[1049] The server uses a trained machine learning model to automatically generate an optimal travel plan based on the user's criteria. Input data includes the user's travel destination, departure point, desired dates, and favorite tourist spots. The model applies the criteria to generate the optimal travel route. Specifically, the server uses the trained model to organize the plan based on the user's criteria into a pandas data frame and send it to the terminal in JSON format.

[1050] Step 4:

[1051] The user inputs travel conditions such as the destination and departure point, desired dates, and tourist spots through the device. The input information is sent to the server. Specifically, the user enters the information into the smartphone app and presses the send button. The device then sends this information to the server as an API request.

[1052] Step 5:

[1053] The device displays the travel plan sent from the server to the user. The user reviews the proposed plans, selects the one they like, and confirms it. The input data includes the travel plan from the server. The travel plan selected by the user is sent to the server as output data. Specifically, the JSON data received from the server is passed to the device app, which displays the details of the travel plan in a list view. The selected information is sent back to the server when the user taps "Confirm" and "Select."

[1054] Step 6:

[1055] The server reserves accommodation based on the travel plan selected by the user. The input data includes the travel plan selected by the user. The output data includes the accommodation reservation confirmation ID. Specifically, the server calls the API of an external accommodation reservation system to complete the reservation. It then obtains the reservation confirmation ID and notifies it to the user.

[1056] Step 7:

[1057] The server provides real-time traffic information and tourist spot information to the user. Input data includes current traffic conditions and the congestion status of tourist spots. Updated information is sent to the device as output data. Specifically, the server periodically calls the navigation app's API, obtains the latest information, and sends it to the device via push notification.

[1058] Step 8:

[1059] The server collects the user's travel history and feedback and updates the machine learning model to generate future travel plans. The input data includes the user's usage history and feedback. The output data includes the updated model. Specifically, after completing a trip, the user enters a rating in the app, and the data is saved on the server. This data is used the next time the model is trained.

[1060] (Application example 1)

[1061] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1062] Conventional travel planning support systems have had difficulty providing travelers with the real-time information they need or automatically generating optimal travel plans based on their detailed preferences. Furthermore, the complicated procedures for changing travel plans and reserving accommodations make them less user-friendly for travelers.

[1063] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1064] In this invention, the server includes means for integrating traffic information collected from an external database and usage statistics of location information apps, means for performing machine learning based on the statistics to generate an optimal travel route, means for automatically generating a travel plan based on the generated travel route, means for booking accommodations based on the travel plan, means for collecting traveler usage history and opinions and updating a model for generating the next travel plan, and means for providing the latest information on the traveler's current location and travel destination in real time, thereby enabling the automatic generation of an optimal travel plan for a traveler and the provision of real-time information.

[1065] A "travel planning support system" is a system that optimizes travelers' travel plans and supports automatic generation and accommodation reservation procedures.

[1066] "Traffic information" refers to information collected from external databases, including public transportation schedules and traffic congestion status.

[1067] A "location-based app" is an application that identifies a user's geographic location and provides navigation and surrounding information to the user.

[1068] "Usage statistics" refers to statistical data based on a user's application usage history and behavioral patterns.

[1069] "Machine learning" is a technology that uses algorithms to learn data and make predictions and classifications.

[1070] "Travel route" refers to route information that indicates the means of transportation and travel route used by a traveler.

[1071] A "travel plan" is a plan that includes travel routes, the order in which tourist spots are visited, recommended accommodations, etc., generated based on conditions specified by the traveler.

[1072] "Accommodation facilities" refer to facilities such as hotels and inns where travelers can stay.

[1073] A "reservation" is a procedure for securing accommodation, transportation, etc. in advance.

[1074] "Usage history" refers to historical information about trips taken by a user in the past and services used by the user.

[1075] "Feedback" refers to user opinions and evaluations provided after using the service.

[1076] "Real-time information" refers to information that is updated hourly, such as the latest traffic conditions and the current congestion status of tourist spots.

[1077] The present invention is a system for assisting in travel planning, and its configuration and operation steps will be described in detail. First, an outline of the program required to realize this system will be created, and the processing based on this program will be described.

[1078] Overall system configuration

[1079] This system consists of a server, a terminal, and a user.

[1080] Server Roles

[1081] Collecting information from external databases: The server collects traffic information and usage statistics from external databases and location app APIs.

[1082] Data analysis with machine learning: Train machine learning models based on collected data to generate optimal travel routes.

[1083] Automatic itinerary generation: Using the trained model, we automatically generate itineraries that best fit the user's requirements.

[1084] Providing real-time information: Providing real-time updates on a traveler's current location and destination.

[1085] Collect usage history and feedback: Collect user usage history and feedback and update the model for the next itinerary generation.

[1086] Accommodation reservations: We will process your accommodation reservations based on your proposed travel plans.

[1087] Device Role

[1088] Receive user input: Enter travel destinations, departure points, desired dates, favorite attractions, and other conditions.

[1089] Display itinerary: Display the itinerary sent from the server to the user, and confirm and select the proposed itinerary.

[1090] User Roles

[1091] Entering travel conditions: Enter travel conditions through the terminal.

[1092] Choose your itinerary: Review the suggested itineraries and choose the one you like best.

[1093] Providing feedback: Provide feedback after your trip and contribute to improving the system.

[1094] Hardware and software configuration

[1095] Hardware: Smartphone (iOS / Android), server

[1096] Software: Google Maps API, HotelBooking API, Machine Learning algorithms (e.g. Logistic Regression)

[1097] Details of data processing and calculation

[1098] 1. Data collection: The server collects traffic and location information from Google Maps API and other external databases.

[1099] 2. Data analysis: Using machine learning models (e.g., Logistic Regression) to analyze users' historical data and generate optimal travel routes.

[1100] 3. Real-time information provision: Based on the traveler's current location, the server periodically calls the API to provide the latest traffic information and congestion status of tourist attractions in real time.

[1101] 4. Obtaining feedback and updating the model: After the trip, obtain feedback from the user and use it as training data for the machine learning model.

[1102] Specific examples

[1103] Travel planning

[1104] If a user enters "I want to travel to Kyoto on March 15th," the system will prompt "Please enter the dates and places of interest." If a user enters "I want to visit Kiyomizu-dera Temple and Kinkaku-ji Temple," the system will generate and suggest the optimal itinerary.

[1105] Prompt sentences for generative AI models (examples)

[1106] A user has entered that they would like to visit Kyoto on March 15th. Their tourist attractions of interest are Kiyomizu-dera Temple and Kinkaku-ji Temple. Please suggest the best travel route and accommodation. Please create a detailed plan, including tourist attractions along the way.

[1107] In this way, the travel planning support system of the present invention is able to automatically generate optimal travel plans for users and provide them with real-time information.

[1108] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1109] Step 1:

[1110] Receiving User Input

[1111] Subject: Device

[1112] Specific operation: The device receives information from the user, such as the travel destination, departure point, desired dates, and favorite tourist spots.

[1113] Input: Travel destination, departure point, desired dates, tourist attractions

[1114] Output: Received user input information

[1115] Data processing: The device converts the received information into an appropriate format and prepares it for transmission to the server.

[1116] Step 2:

[1117] Collection of traffic information and usage statistics

[1118] Subject: Server

[1119] What it does: The server uses the Google Maps API and other external databases to collect traffic information and location app usage statistics.

[1120] Input: External database, Google Maps API endpoint

[1121] Output: Collected traffic information and usage statistics

[1122] Data processing: The server organizes the collected data and stores it in a database, including any necessary filtering and preprocessing.

[1123] Step 3:

[1124] Training a machine learning model

[1125] Subject: Server

[1126] How it works: The server uses historical travel data and real-time usage statistics to train machine learning models.

[1127] Inputs: Historical travel data, real-time usage statistics

[1128] Output: A trained machine learning model

[1129] Data calculation: The server performs training using algorithms such as logistic regression, learning patterns based on past data and optimizing the model parameters.

[1130] Step 4:

[1131] Generating optimal travel routes

[1132] Subject: Server

[1133] What it does: Uses a trained machine learning model to generate optimal travel routes based on the user's criteria.

[1134] Input: User input, trained machine learning model

[1135] Output: Optimal travel route

[1136] Data calculation: The server applies the user's input to the model to predict the best route. It uses an optimization algorithm to evaluate and select travel routes.

[1137] Step 5:

[1138] Automatic travel plan generation

[1139] Subject: Server

[1140] Specific operation: The server automatically generates a travel plan based on the generated travel route, including travel routes, the order in which tourist attractions are visited, and recommended accommodations.

[1141] Input: Optimal travel route

[1142] Output: Auto-generated itinerary

[1143] Data processing: The server incorporates information on tourist attractions and accommodations based on the generated route and formats it into a complete travel plan.

[1144] Step 6:

[1145] View and select your travel plans

[1146] Subject: Device

[1147] Specific operation: The terminal displays the travel plans sent from the server to the user, allowing the user to confirm and select the plans.

[1148] Input: Auto-generated itinerary

[1149] Output: Selected itinerary

[1150] Data processing: Visually display the travel plan through the user interface and accept user selections.

[1151] Step 7:

[1152] Accommodation reservations

[1153] Subject: Server

[1154] Specific operation: Based on the selected travel plan, the server completes the accommodation reservation using the API of the external application.

[1155] Input: Selected travel plan

[1156] Output: Reservation confirmation ID

[1157] Data calculation: Obtain accommodation availability information from the API, complete the reservation process, and generate a confirmation ID.

[1158] Step 8:

[1159] Providing real-time information

[1160] Subject: Server

[1161] Specific operation: The server collects real-time traffic information and congestion status of tourist attractions based on the traveler's current location and sends it to the terminal.

[1162] Input: Current location, real-time information

[1163] Output: User notification

[1164] Data calculation: The server obtains the latest information from the API and pushes it to the device.

[1165] Step 9:

[1166] Get feedback and update the model

[1167] Subject: Server

[1168] What it does: After the trip, the server collects user feedback and uses it as training data for the machine learning model.

[1169] Input: User feedback

[1170] Output: An updated machine learning model

[1171] Data processing: Analyze the feedback and add it to the model's training dataset. Retrain the model as a new model.

[1172] This process enables the travel planning support system to automatically generate optimal travel plans for users and provide real-time information.

[1173] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1174] The present invention relates to a system for assisting in travel planning, and more particularly to a system for providing more personalized travel plans by incorporating an emotion engine that recognizes the emotions of a user. Hereinafter, an embodiment of the present invention will be described in detail.

[1175] Overall system configuration

[1176] This system consists of a server, a terminal, a user, and an emotion engine. The emotion engine has the means to recognize emotions by analyzing the user's voice, facial expressions, and text data. The server generates a travel plan that is most suitable for the user based on the output of this emotion engine. The terminal is responsible for receiving this information from the user and sending it to the server. The user inputs the travel destination, departure point, desired dates, emotional state, etc. via the terminal, and then confirms and selects the travel plan proposed by the server.

[1177] System Operation

[1178] 1. Collection of traffic information

[1179] The server collects traffic information from external databases and the navigation app's API, including public transport schedules and traffic congestion status.

[1180] Example: A server uses the Google Maps API to retrieve Shinkansen schedules and fare information from Tokyo to Kyoto and stores it in a database.

[1181] 2. Route optimization using machine learning

[1182] The server uses the collected traffic information to train a machine learning model that leverages the user's past travel data and real-time usage statistics to suggest optimal travel routes.

[1183] Example: A server analyzes past travel data and determines, based on traveler behavior patterns, that taking the Shinkansen is the quickest and most cost-effective option.

[1184] 3. Emotion Recognition by Emotion Engine

[1185] The terminal collects the user's voice data, facial expression data, and text data and sends them to the emotion engine, which analyzes the data and recognizes the user's current emotional state.

[1186] Example: The emotion engine analyzes the voice and facial expressions of a user speaking into the device's camera and determines that the user is in a "relaxed" state.

[1187] 4. Receiving and Sending User Input

[1188] The user inputs travel conditions such as the destination, desired itinerary, current emotional state, and favorite tourist spots through the terminal, and this information is sent from the terminal to the server.

[1189] Example: A user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into an input form on a smartphone app and clicks the submit button.

[1190] 5. Automatic travel plan generation

[1191] The server generates an optimal travel route and sightseeing plan for the user based on the output of the machine learning model and emotion engine, including the route, the order in which tourist spots are visited, and recommended accommodations.

[1192] Example: The server considers the user's relaxed emotional state and tourist spot preferences to suggest a relaxing route in Kyoto.

[1193] 6. Check and select your travel plan

[1194] The terminal displays the travel plans sent from the server to the user, who then checks the proposed travel plans, selects the one they like, and confirms it.

[1195] Example: A user reviews suggested travel plans on their smartphone screen and selects one to visit Kiyomizu-dera Temple.

[1196] 7. Accommodation reservations

[1197] The server automatically processes the accommodation reservation based on the travel plan selected by the user, and completes the reservation in cooperation with an external accommodation reservation system.

[1198] Example: The server uses the API of a partner accommodation booking site to reserve a room at a selected hotel and obtain a reservation confirmation ID.

[1199] 8. Real-time information provision

[1200] The server provides real-time traffic and tourist information to the user while he or she is traveling, and this information is sent to the user via the terminal.

[1201] Example: While traveling, a user receives the latest traffic information and crowding status of tourist spots on their smartphone.

[1202] 9. Usage history and feedback collection

[1203] The server collects user travel history and feedback and uses it to generate future travel plans. This information is used to update the machine learning model.

[1204] Example: After completing a trip, a user enters an evaluation of the plan provided by the app, and the data is stored on the server.

[1205] In this way, the travel planning support system of the present invention can significantly improve convenience for travelers by proposing optimal travel routes taking into account the user's emotional state, automatically generating travel plans, and seamlessly booking accommodations.

[1206] The processing flow will be explained below.

[1207] Step 1:

[1208] The server collects traffic information from external databases and the navigation app's API.

[1209] Specific operation: The server uses the Google Maps API to obtain Shinkansen schedules and fare information from Tokyo to Kyoto and stores it in a database.

[1210] Step 2:

[1211] The server trains a machine learning model based on the collected traffic information.

[1212] Specific operation: The server takes in past travel data and user behavior patterns and runs an algorithm to learn the optimal travel route that minimizes travel time and costs.

[1213] Step 3:

[1214] The user uses the terminal to input conditions such as the travel destination, departure point, desired dates, and favorite tourist spots.

[1215] Specific operation: The user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into the input form of the smartphone app and clicks the send button.

[1216] Step 4:

[1217] The terminal converts the user's input data into JSON format and sends it to the server.

[1218] Specific operation: The device packages the input data in JSON format and sends it to the server using an HTTP request.

[1219] Step 5:

[1220] The server generates an optimal travel route and sightseeing plan based on the received user's conditions.

[1221] Specific operation: The server uses a machine learning model to generate a travel plan that includes the Shinkansen route that best suits the user's requirements, tourist spots along the way, and recommended accommodations.

[1222] Step 6:

[1223] The server converts the generated travel plan into JSON format and sends it to the user's device.

[1224] Specific operation: The server encodes the generated travel plan in JSON format and sends it to the terminal as an HTTP response.

[1225] Step 7:

[1226] The terminal displays the travel plan received from the server to the user.

[1227] Specific operation: The device parses the contents of the travel plan and displays it on the screen in a format that is easy for the user to view.

[1228] Step 8:

[1229] The user checks the displayed travel plans and selects the plan they want.

[1230] Specific operation: The user checks the suggested tourist spots and accommodations on the smartphone screen and taps to select the plan they like.

[1231] Step 9:

[1232] The device converts the user's selected plan information into JSON format and sends it to the server.

[1233] Specific behavior: The device packages the selected travel plan in JSON format and sends it to the server using an HTTP request.

[1234] Step 10:

[1235] The server then processes the reservation for the accommodation based on the user's selection.

[1236] Specific operation: The server uses the information of the selected accommodation to call the API of the affiliated reservation site, completes the reservation procedure, and obtains the reservation confirmation ID.

[1237] Step 11:

[1238] The server sends information including the reservation confirmation ID to the terminal and notifies the user.

[1239] Specific operation: The server creates a response including the reservation confirmation ID and sends it to the terminal as an HTTP response. The terminal receives this and notifies the user.

[1240] Step 12:

[1241] The server allows the user to receive real-time traffic information and tourist spot updates while traveling and transmits them to the terminal.

[1242] Specific operation: The server collects and updates real-time traffic information and tourist spot information, and sends it to the user's device as a push notification.

[1243] Step 13:

[1244] The server collects the user's travel history and feedback after the trip is completed.

[1245] What happens: A user fills out a feedback form in the app and clicks the submit button. The data is sent to the server and stored in the database.

[1246] Step 14:

[1247] The terminal collects the user's voice data, facial expression data, and text data and sends them to the emotion engine.

[1248] Specific operation: Using the device's camera and microphone, the user's voice and facial expressions are recorded and sent to the emotion engine along with the text data.

[1249] Step 15:

[1250] The emotion engine analyzes the user's voice, facial expression, and text data to recognize the user's current emotional state.

[1251] Specific operation: The emotion engine uses an analytical algorithm to determine emotions such as "relaxed," "excited," and "stressed" from the collected data.

[1252] Step 16:

[1253] The server adjusts the contents of the travel plan based on the emotional state recognized by the emotion engine.

[1254] Specific operation: The server generates a plan that includes tourist spots and experiences that correspond to the emotional state of "wanting to relax."

[1255] Step 17:

[1256] The server updates the travel plan in real time according to changes in the emotional state and sends it to the terminal.

[1257] Specific operation: When a user feels stressed during a trip, the server generates a plan including new relaxation spots and sends it to the terminal.

[1258] In this way, the travel planning support system of the present invention can significantly improve convenience for travelers by proposing optimal travel routes taking into account the user's emotional state, automatically generating travel plans, and seamlessly booking accommodations.

[1259] Example 2

[1260] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1261] Conventional travel planning support systems have the problem of being unable to provide personalized travel plans based on users' emotional state or real-time fluctuating information, which has led to travelers being unable to have a comfortable and satisfying travel experience.

[1262] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1263] In this invention, the server includes means for collecting traffic information from an external database and integrating usage statistics of a navigation app, means for generating an optimal travel route using a machine learning model based on the statistics, means for utilizing an emotion recognition engine that recognizes emotions by analyzing the traveler's voice, facial expression, and text data, means for automatically generating a travel plan based on the output of the emotion recognition engine, means for reserving accommodations based on the travel plan, and means for collecting the traveler's usage history and feedback and updating the model for generating the next travel plan. This makes it possible to provide an optimal and personalized travel plan based on the user's current emotional state and past travel history.

[1264] "Traffic information" refers to data such as public transportation schedules and traffic congestion conditions.

[1265] "Navigation app usage statistics" refers to data on the behavioral patterns and real-time usage status of users of navigation apps.

[1266] A "machine learning model" refers to an algorithm that automatically learns from large amounts of data and makes predictions and classifications.

[1267] An "emotion recognition engine" refers to a system that recognizes emotions by analyzing a user's voice, facial expressions, and text data.

[1268] A "travel plan" refers to a detailed travel plan that includes travel routes, the order in which tourist spots are visited, recommended accommodations, etc.

[1269] "Accommodation Booking" refers to the process of reserving accommodation for a traveler.

[1270] "Usage history" refers to travel plans and behavioral data that a user has used in the past.

[1271] "Feedback" refers to the evaluations and opinions given by users regarding the services and plans provided.

[1272] "Updating a model" refers to the process of improving the performance of an existing machine learning model based on new data.

[1273] The present invention relates to a system for assisting in travel planning, and more particularly to a system for providing more personalized travel plans by incorporating an emotion engine that recognizes the emotions of a user. Hereinafter, an embodiment of the present invention will be described in detail.

[1274] This system consists of a server, a terminal, a user, and an emotion engine. The emotion engine has the means to recognize emotions by analyzing the user's voice, facial expressions, and text data. The server generates a travel plan that is most suitable for the user based on the output of this emotion engine. The terminal is responsible for receiving this information from the user and sending it to the server. The user inputs the travel destination, departure point, desired dates, emotional state, etc. via the terminal, and then confirms and selects the travel plan proposed by the server.

[1275] System Configuration

[1276] 1. Means of collecting traffic information

[1277] The server collects traffic information using external databases and the navigation app's API, including public transport schedules and traffic congestion status. The specific software used is Google Maps API.

[1278] 2. Using Machine Learning Models

[1279] The server uses the collected traffic information to train a machine learning model that uses the user's past travel data and real-time usage statistics to suggest optimal travel routes using software including TensorFlow.

[1280] 3. Emotion recognition

[1281] The device collects the user's voice, facial expression, and text data and sends it to the emotion engine, which analyzes this data and recognizes the user's current emotional state using Microsoft Azure Cognitive Services.

[1282] 4. Automatic generation of travel plans

[1283] The server generates an optimal travel route and sightseeing plan for the user based on the output of the machine learning model and emotion engine, including the route, the order in which tourist spots are visited, and recommended accommodations.

[1284] 5. Accommodation reservations

[1285] The server automatically makes reservations for accommodation based on the travel plan. At this time, it completes the reservation by linking with an external accommodation reservation system. Specifically, it uses the Booking.com API.

[1286] 6. Collecting User Usage History and Feedback

[1287] The server collects user travel history and feedback and uses it to generate future travel plans. This information is also used to update the machine learning model.

[1288] Specific examples

[1289] For example, if a user uses a smartphone app to input "Kyoto, March 15, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple," and the emotion recognition engine recognizes the user's relaxed state, the server will use this information to generate a travel plan that includes relaxing tourist spots and accommodations.The server then uses the Booking.com API to make reservations for the accommodations and notify the user.

[1290] Prompt Sentence Examples

[1291] "How do I use the Google Maps API to plan my trip?"

[1292] In this way, the travel planning support system of the present invention can provide an optimal and personalized travel plan that takes into account the emotional state of the user.

[1293] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1294] Step 1:

[1295] The server collects traffic information using an external database and the navigation app's API. As input, it receives traffic information from the external database and the navigation app. This provides data such as public transport schedules and traffic congestion conditions. The server stores this information in a database. Specifically, the server calls the Google Maps API to obtain the Shinkansen schedule and fare information from Tokyo to Kyoto, and stores this information in the database.

[1296] Step 2:

[1297] The server trains a machine learning model based on the collected traffic information. It uses the traffic information collected in step 1 and past travel data as input. This creates a predictive model for generating optimal travel routes. The server uses the TensorFlow library to train the machine learning model and prepares optimal travel route suggestions. Specifically, the server analyzes past travel data and learns user behavior patterns.

[1298] Step 3:

[1299] The device collects the user's voice data, facial expression data, and text data, and sends this data to the emotion engine. The user's voice, facial expression, and text data are used as input. This allows the emotion engine to recognize the user's current emotional state. The emotion engine uses Microsoft Azure Cognitive Services to perform emotion analysis. Specifically, data collected using the device's camera and microphone is sent to the Microsoft Azure Cognitive Services API, which recognizes the user's emotional state, such as "relaxed" or "tense."

[1300] Step 4:

[1301] The user inputs conditions such as the travel destination, desired schedule, emotional state, and favorite tourist spots through the device. The travel information specified by the user is received as input. This information is sent from the device to the server. Specifically, the user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into the input form of the smartphone app and clicks the send button.

[1302] Step 5:

[1303] The server generates the optimal travel route and sightseeing plan for the user based on the output of the machine learning model and emotion engine. As input, it uses the output of the machine learning model in step 2, the result of the emotion engine in step 3, and the user's travel conditions sent in step 4. This creates the optimal travel plan. Specifically, the server considers the user's relaxed emotional state and tourist spot preferences, and proposes a travel route that includes relaxing tourist spots and accommodations.

[1304] Step 6:

[1305] The terminal displays the travel plan sent from the server to the user. The terminal receives the travel plan from the server as input. Based on this information, the user checks the proposed plans, selects the one they like, and confirms it. Specifically, the user checks the proposed travel plan on the smartphone screen, selects the plan to visit Kiyomizu-dera Temple and Kinkaku-ji Temple, and clicks the confirm button.

[1306] Step 7:

[1307] The server automatically makes accommodation reservations based on the travel plan selected by the user. It receives the confirmed travel plan as input and completes the reservation by connecting with an external accommodation reservation system. Specifically, the server uses the Booking.com API to reserve a room at the selected hotel and obtains a reservation confirmation ID.

[1308] Step 8:

[1309] The server provides real-time traffic and tourist spot information to users traveling. The server references the user's current location and travel plan as input, allowing it to continue providing real-time information. Specifically, the server obtains the latest traffic congestion information and congestion status at tourist spots, and notifies the traveling user via their smartphone.

[1310] Step 9:

[1311] The server collects the user's travel history and feedback and uses it to generate future travel plans. This information is also used to update the machine learning model. User feedback and travel history data are taken as input. Specifically, after the user completes their trip, they enter an evaluation of the plan provided by the app, and this data is stored on the server. The server uses this data as training data for a new machine learning model.

[1312] (Application example 2)

[1313] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1314] Conventional travel planning support systems were unable to generate travel plans that took into account the user's individual emotional state, and were limited to providing standard plans. This made it difficult to provide travel plans that were tailored to the user's needs and emotions. Furthermore, in the shopping experience, there was also the issue of being unable to provide personalized product suggestions based on the user's emotions and real-time situation.

[1315] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1316] In this invention, the server includes means for integrating traffic information collected from an external database and usage statistics of the navigation app, means for performing machine learning based on the statistics to generate an optimal travel route, and means for automatically generating a travel plan based on the generated travel route, thereby making it possible to analyze the user's emotional state in real time and provide personalized travel plans and product suggestions.

[1317] An "external database" is a collection of information or data accessible through the Internet or other network.

[1318] A "navigation app" is software that provides maps and traffic information and guides users to the optimal route from their current location to their destination.

[1319] "Usage Statistics" refers to the compilation and analysis of data based on a user's past usage history and behavioral patterns.

[1320] "Machine learning" is a technology in which a computer learns patterns and rules based on data and automatically makes predictions and decisions.

[1321] A "travel route" refers to a travel route from a user's departure point to a destination.

[1322] A "travel plan" is a plan that includes travel destinations, transportation, accommodations, tourist spots, etc.

[1323] An "accommodation reservation system" is a system for making online reservations for accommodations such as hotels and inns.

[1324] "Usage history" refers to a record of trips the user has taken and a history of services they have used.

[1325] "Feedback" refers to a user providing an evaluation or opinion about a service or product.

[1326] An "emotion recognition engine" is a technology that analyzes a user's voice, facial expressions, text data, etc. to recognize their emotional state.

[1327] "Personalization" refers to providing services and products tailored to the individual preferences and emotional state of each user.

[1328] "Brick and mortar store" refers to a store that has a physical presence.

[1329] "Smart glasses" are glasses-type devices that have display and camera functions and can display and collect information.

[1330] A "smartphone" is a mobile phone that has advanced computing power, multiple functions, and can run apps.

[1331] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following describes an embodiment of the present invention.

[1332] Overall system overview

[1333] The embodiment of the invention comprises an external database, a server, a terminal, a user, and an emotion recognition engine. The server collects traffic information and navigation app usage statistics from the external database, and uses machine learning to generate optimal travel routes based on this information. Furthermore, the emotion recognition engine analyzes the user's emotional state and provides personalized travel plans and product recommendations in physical stores.

[1334] Program Overview

[1335] This system uses programming languages ​​such as Python to collect various data, perform machine learning, and recognize emotions. It also provides information to users via smart glasses or smartphones. Specifically, it uses the following hardware and software:

[1336] 1. Hardware

[1337] Smart glasses or smartphones: Collect the user's voice, facial expression, and text data and send it to an emotion recognition engine.

[1338] Server: Based on the collected data, it integrates traffic information, performs machine learning, generates itineraries, and makes personalized product recommendations.

[1339] 2. Software

[1340] Emotion recognition engine: Recognizes the user's emotional state by analyzing their voice, facial expressions, and text data. For example, it can be built using TensorFlow or Keras.

[1341] Machine learning model: Generates optimal travel routes based on traffic information and usage statistics. Uses Scikit-learn and TensorFlow.

[1342] API: Traffic information is collected using external databases such as Google Maps API.

[1343] System Operation

[1344] The server communicates with external databases to collect external data, and uses machine learning to generate optimal travel routes and plans. Users' emotional state is analyzed through smart glasses or smartphones, and they receive personalized travel plans or in-store product suggestions based on that analysis.

[1345] As a concrete example, there is a system in which a user walks around a store using smart glasses and receives product suggestions based on their emotional state at the time. For example, if the user is feeling stressed, relaxation items are suggested, and if the user is having fun, the latest fashion items are suggested.

[1346] Prompt Sentence Examples

[1347] "The user is currently in the emotional state 'happy'. Please generate a description for the following product:

[1348] Product 1: Latest fashion item (Description: The latest trendy fashion item for the new season.)

[1349] Product 2: Accessories (Description: Luxury accessories.)

[1350] The system can provide personalized travel plans and shopping experiences that take into account the user's emotional state.

[1351] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1352] Step 1:

[1353] The device collects the user's voice, facial expression, and text data in real time. This data is captured using a camera and microphone. The input data is the raw material for recognizing the user's emotions, and the output is this raw data.

[1354] Step 2:

[1355] The device sends the collected voice, facial expression, and text data to an emotion recognition engine. The emotion recognition engine is built using TensorFlow and Keras, and analyzes this data to recognize the user's emotions. The input is the data sent from the device, and the output is the user's emotional state (e.g., "happy," "sad," "angry," etc.). Specifically, the analysis algorithm determines the user's facial muscle movements and tone of voice.

[1356] Step 3:

[1357] The server receives the emotional state output by the emotion recognition engine and generates personalized travel plans or product suggestions based on it. The input is the recognized emotional state, and the output is a detailed travel plan or product list tailored to the user's emotions. Specifically, the server analyzes the characteristics of tourist attractions and products in the database and selects the best ones for the user.

[1358] Step 4:

[1359] The server collects real-time traffic and transportation information using external databases and APIs (e.g., Google Maps API). The input is information obtained from the external database, and the output is integrated traffic information. Specifically, the server makes API calls and stores the obtained data in its internal database.

[1360] Step 5:

[1361] The server performs machine learning based on the collected traffic information and usage statistics to generate the optimal travel route. The input is traffic information and past usage statistics, and the output is an optimized travel route. Specifically, the server trains the model using Scikit-learn and TensorFlow to calculate the optimal route.

[1362] Step 6:

[1363] The terminal displays the travel plan and product suggestions received from the server to the user. The input is the personalized plan sent from the server, and the output is the information displayed on the terminal's display. Specifically, the terminal displays the plan details and product list on the screen.

[1364] Step 7:

[1365] The user checks the displayed travel plans and product suggestions, and makes selections and reservations as necessary. The input is the information displayed on the terminal, and the output is the user's selection. Specifically, the user makes selections using a touch panel or voice commands.

[1366] Step 8:

[1367] Based on the user's selection, the server connects with an external accommodation reservation system to complete the reservation procedure. The input is the user's selection, and the output is reservation confirmation information. Specifically, the server uses the reservation system's API to confirm the reservation and obtain a confirmation ID.

[1368] Step 9:

[1369] The server provides real-time traffic and tourist spot information to users while they are traveling. The input is traffic information and tourist spot data obtained in real time, and the output is the latest information provided to the user. Specifically, the server periodically calls the API and sends information to the device.

[1370] Step 10:

[1371] The server collects user feedback after the trip and updates the machine learning model for the next itinerary generation. The input is the user feedback, and the output is the updated machine learning model. Specifically, the server analyzes the feedback data and adds it to the training dataset of the model.

[1372] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1373] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1374] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1375] [Fourth embodiment]

[1376] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1377] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1378] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1379] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1380] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1381] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1382] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1383] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1384] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1385] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1386] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1387] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1388] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1389] 1. Field of the Invention The present invention relates to a system for assisting in travel planning. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to the drawings.

[1390] Overall system configuration

[1391] This system consists of a server, a terminal, and a user. The server collects traffic information and usage statistics of navigation apps and uses this information to train a machine learning model. The terminal receives input from the user and sends it to the server. The user inputs their destination and travel conditions, and then checks and selects travel plans suggested by the server.

[1392] System Operation

[1393] 1. Collection of traffic information

[1394] The server collects traffic information from external databases and the navigation app's API, including public transport schedules and traffic congestion status.

[1395] Example: A server uses the Google Maps API to retrieve information about Shinkansen schedules and fares from Tokyo to Kyoto.

[1396] 2. Route optimization using machine learning

[1397] The server uses the collected traffic information to train a machine learning model that leverages the user's past travel data and real-time usage statistics to suggest optimal travel routes.

[1398] Example: A server analyzes past travel data and determines, based on traveler behavior patterns, that taking the Shinkansen is the quickest and most cost-effective option.

[1399] 3. Automatic generation of travel plans

[1400] The server uses the trained model to automatically generate an itinerary that best suits the user's requirements, including routes, the order in which tourist spots are visited, and recommended accommodations.

[1401] Example: A server generates a travel plan from Tokyo to Kyoto, outputting the Shinkansen travel time, tourist spots along the way, and a list of recommended hotels.

[1402] 4. Receiving and Sending User Input

[1403] The user inputs travel conditions such as the destination and departure point, desired itinerary, and favorite tourist spots through the terminal, and this input information is sent from the terminal to the server.

[1404] Example: A user uses a smartphone app to enter the following information: "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple."

[1405] 5. Check and select your travel plan

[1406] The terminal displays the travel plans sent from the server to the user, who then checks the proposed travel plans, selects the one they like, and confirms it.

[1407] Example: A user reviews suggested travel plans on their smartphone screen and selects one to visit Kiyomizu-dera Temple.

[1408] 6. Accommodation reservations

[1409] The server automatically processes the accommodation reservation based on the travel plan selected by the user, and completes the reservation in cooperation with an external accommodation reservation system.

[1410] Example: The server uses the API of a partner accommodation booking site to reserve a room at a selected hotel and obtain a reservation confirmation ID.

[1411] 7. Real-time information provision

[1412] The server provides real-time traffic and tourist information to the user while he or she is traveling, and this information is sent to the user via the terminal.

[1413] Example: While traveling, a user receives the latest traffic information and crowding status of tourist spots on their smartphone.

[1414] 8. Usage history and feedback collection

[1415] The server collects user travel history and feedback and uses it to generate future travel plans. This information is used to update the machine learning model.

[1416] Example: After completing a trip, a user enters an evaluation of the plan provided by the app, and the data is stored on the server.

[1417] In this way, the travel planning support system of the present invention can significantly improve convenience for travelers by suggesting optimal travel routes to users, automatically generating travel plans, and seamlessly booking accommodations.

[1418] The processing flow will be explained below.

[1419] Step 1:

[1420] The server collects traffic information from external databases and the navigation app's API.

[1421] Specific operation: The server uses the Google Maps API to obtain Shinkansen schedules and fare information from Tokyo to Kyoto and stores it in a database.

[1422] Step 2:

[1423] The server trains a machine learning model based on the collected traffic information.

[1424] Specific operation: The server takes in past travel data and user behavior patterns and runs an algorithm to learn the optimal travel route that minimizes travel time and costs.

[1425] Step 3:

[1426] Users use their device to input travel conditions such as travel destination, departure point, desired dates, and favorite tourist spots into the "Reserve-san" app.

[1427] Specific operation: The user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into the input form of the smartphone app and clicks the send button.

[1428] Step 4:

[1429] The terminal converts the user's input data into JSON format and sends it to the server.

[1430] Specific operation: The device packages the input data in JSON format and sends it to the server using an HTTP request.

[1431] Step 5:

[1432] The server generates an optimal travel route and sightseeing plan based on the received user's conditions.

[1433] Specific operation: The server uses a machine learning model to generate a travel plan that includes the Shinkansen route that best suits the user's requirements, tourist spots along the way, and recommended accommodations.

[1434] Step 6:

[1435] The server converts the generated travel plan into JSON format and sends it to the user's device.

[1436] Specific operation: The server encodes the generated travel plan in JSON format and sends it to the terminal as an HTTP response.

[1437] Step 7:

[1438] The terminal displays the travel plan received from the server to the user.

[1439] Specific operation: The device parses the contents of the travel plan and displays it on the screen in a format that is easy for the user to view.

[1440] Step 8:

[1441] The user checks the displayed travel plans and selects the plan they want.

[1442] Specific operation: The user checks the suggested tourist spots and accommodations on the smartphone screen and taps to select the plan they like.

[1443] Step 9:

[1444] The device converts the user's selected plan information into JSON format and sends it to the server.

[1445] Specific behavior: The device packages the selected travel plan in JSON format and sends it to the server using an HTTP request.

[1446] Step 10:

[1447] The server then processes the reservation for the accommodation based on the user's selection.

[1448] Specific operation: The server uses the information of the selected accommodation to call the API of the affiliated reservation site, completes the reservation procedure, and obtains the reservation confirmation ID.

[1449] Step 11:

[1450] The server sends information including the reservation confirmation ID to the terminal and notifies the user.

[1451] Specific operation: The server creates a response including the reservation confirmation ID and sends it to the terminal as an HTTP response. The terminal receives this and notifies the user.

[1452] Step 12:

[1453] The server allows the user to receive real-time traffic information and tourist spot updates while traveling and transmits them to the terminal.

[1454] Specific operation: The server collects and updates real-time traffic information and tourist spot information, and sends it to the user's device as a push notification.

[1455] Step 13:

[1456] The server collects the user's travel history and feedback after the trip is completed.

[1457] What happens: A user fills out a feedback form in the app and clicks the submit button. The data is sent to the server and stored in the database.

[1458] Step 14:

[1459] The server updates and improves the machine learning model based on collected travel history and feedback.

[1460] What it does: The server uses the newly collected data to retrain the machine learning algorithm and improve the accuracy of the next itinerary generation.

[1461] Example 1

[1462] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1463] Conventional travel planning systems provide limited information on transportation and tourist attractions, and lack real-time updates. Furthermore, travel plans are often generated manually, making it difficult to provide efficient and optimal plans. Furthermore, they are unable to effectively utilize traveler usage history and feedback, making it difficult to reflect this information in the generation of next travel plans.

[1464] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1465] In this invention, the server includes means for integrating traffic information collected from an external database and usage statistics of the navigation app, means for generating an optimal travel route using a machine learning model based on the integrated information, means for automatically generating a travel plan based on the generated travel route, means for collecting traveler usage history and feedback and updating the machine learning model for generating the next travel plan, and means for providing traffic information and tourist spot information in real time, thereby enabling the user to be provided with an efficient and optimal travel plan and real-time information during the trip.

[1466] An "external database" is an information source that provides transportation information and tourist spot information that can be accessed via the Internet.

[1467] A "navigation app" is a software application that provides users with travel routes and traffic conditions.

[1468] "Traffic information" refers to information related to travel, such as public transportation schedules and road congestion conditions.

[1469] "Usage statistics" refers to statistical data including user behavior data and access frequency for navigation applications.

[1470] A "machine learning model" is a mathematical model that learns patterns from data and makes predictions and classifications.

[1471] The "optimal travel route" refers to the most efficient and comfortable travel route, taking into consideration traffic conditions, past user data, costs, etc.

[1472] "Auto-generation" refers to the process of automatically generating information or plans through a program.

[1473] "Accommodation" means a place such as a hotel or lodging facility where travelers can stay.

[1474] A "reservation system" is a system that manages reservations for accommodation, transportation, etc. online.

[1475] "Usage history" refers to a record of the user's past travels and services used.

[1476] "Feedback" refers to opinions such as ratings and impressions provided by users.

[1477] "Real-time information" refers to information that is updated immediately based on the current time.

[1478] The present invention relates to a system for assisting in travel planning, and this system is composed of a server, a terminal, and a user. The following describes in detail an embodiment of the present invention.

[1479] Overall system configuration

[1480] The server collects traffic information and usage statistics from external databases and the navigation app's API, and uses this data to train the machine learning model. The device receives input from the user and sends it to the server. The user enters their destination and travel conditions, and then confirms and selects the travel plan suggested by the server. The entire system automatically generates an efficient and optimal travel plan by linking each function together.

[1481] Hardware and software used

[1482] This invention utilizes a server, a terminal, and an external API. Specifically, the server is a computer system with a high-performance CPU and large memory capacity, and uses a programming language such as Python to train machine learning models and process data. It also collects traffic information using the Google Maps API and public transportation schedule APIs. The terminals are smartphones and tablet devices, and applications running on these devices can be native iOS or Android apps.

[1483] Data processing and calculation

[1484] The server stores traffic information obtained from external databases and the navigation app's API in a database and performs data preprocessing. For example, the server uses the Google Maps API to obtain Shinkansen schedules and fare information from Tokyo to Kyoto. The server then trains a machine learning model using libraries such as scikit-learn to generate optimal travel routes. This model may include random forests or neural networks.

[1485] The server uses the trained model to automatically generate an optimal travel plan based on the user's criteria. For example, it uses past travel data to suggest a travel route from Tokyo to Kyoto and generates a plan that includes Shinkansen travel times, tourist spots along the way, and recommended accommodations. The user enters the destination, departure point, desired dates, and favorite tourist spots into the smartphone app, and sends the information from the device to the server. An example of a prompt sentence could be "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple."

[1486] Accommodation reservations

[1487] The server automatically reserves accommodations using the API of an external accommodation reservation system based on the travel plan selected by the user. For example, it uses the Booking.com API to search for hotels that meet the user's requirements and makes a reservation. Once the reservation is complete, the server obtains a reservation confirmation ID and notifies the user.

[1488] Providing real-time information

[1489] To provide real-time traffic and tourist spot information to users while traveling, the server periodically calls the navigation app's API to obtain the latest information and sends push notifications to the device, allowing users to stay up to date with the latest information during their trip.

[1490] Collection of usage history and feedback

[1491] The server collects the user's travel history and feedback and updates the machine learning model for future travel plan generation. For example, if a user enters a plan rating in the app after completing a trip, that data is stored on the server and used for the next model training.

[1492] By integrating these functions, the travel planning support system of the present invention can provide users with efficient and optimal travel plans and real-time information during travel.

[1493] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1494] Step 1:

[1495] The server collects traffic information and usage statistics from external databases and the navigation app's API. Input data includes transportation schedules and road traffic conditions. This information is collected and stored in a database. Specifically, the server calls the Google Maps API to obtain information on the schedule and traffic congestion for the specified section.

[1496] Step 2:

[1497] The server trains a machine learning model based on the collected traffic information and usage statistics. Input data includes historical travel data and real-time traffic information. It preprocesses the data and trains a model to suggest optimal travel routes. Specifically, the server preprocesses the data using the scikit-learn library, trains the model using the random forest algorithm, and saves the model.

[1498] Step 3:

[1499] The server uses a trained machine learning model to automatically generate an optimal travel plan based on the user's criteria. Input data includes the user's travel destination, departure point, desired dates, and favorite tourist spots. The model applies the criteria to generate the optimal travel route. Specifically, the server uses the trained model to organize the plan based on the user's criteria into a pandas data frame and send it to the terminal in JSON format.

[1500] Step 4:

[1501] The user inputs travel conditions such as the destination and departure point, desired dates, and tourist spots through the device. The input information is sent to the server. Specifically, the user enters the information into the smartphone app and presses the send button. The device then sends this information to the server as an API request.

[1502] Step 5:

[1503] The device displays the travel plan sent from the server to the user. The user reviews the proposed plans, selects the one they like, and confirms it. The input data includes the travel plan from the server. The travel plan selected by the user is sent to the server as output data. Specifically, the JSON data received from the server is passed to the device app, which displays the details of the travel plan in a list view. The selected information is sent back to the server when the user taps "Confirm" and "Select."

[1504] Step 6:

[1505] The server reserves accommodation based on the travel plan selected by the user. The input data includes the travel plan selected by the user. The output data includes the accommodation reservation confirmation ID. Specifically, the server calls the API of an external accommodation reservation system to complete the reservation. It then obtains the reservation confirmation ID and notifies it to the user.

[1506] Step 7:

[1507] The server provides real-time traffic information and tourist spot information to the user. Input data includes current traffic conditions and the congestion status of tourist spots. Updated information is sent to the device as output data. Specifically, the server periodically calls the navigation app's API, obtains the latest information, and sends it to the device via push notification.

[1508] Step 8:

[1509] The server collects the user's travel history and feedback and updates the machine learning model to generate future travel plans. The input data includes the user's usage history and feedback. The output data includes the updated model. Specifically, after completing a trip, the user enters a rating in the app, and the data is saved on the server. This data is used the next time the model is trained.

[1510] (Application example 1)

[1511] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1512] Conventional travel planning support systems have had difficulty providing travelers with the real-time information they need or automatically generating optimal travel plans based on their detailed preferences. Furthermore, the complicated procedures for changing travel plans and reserving accommodations make them less user-friendly for travelers.

[1513] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1514] In this invention, the server includes means for integrating traffic information collected from an external database and usage statistics of location information apps, means for performing machine learning based on the statistics to generate an optimal travel route, means for automatically generating a travel plan based on the generated travel route, means for booking accommodations based on the travel plan, means for collecting traveler usage history and opinions and updating a model for generating the next travel plan, and means for providing the latest information on the traveler's current location and travel destination in real time, thereby enabling the automatic generation of an optimal travel plan for a traveler and the provision of real-time information.

[1515] A "travel planning support system" is a system that optimizes travelers' travel plans and supports automatic generation and accommodation reservation procedures.

[1516] "Traffic information" refers to information collected from external databases, including public transportation schedules and traffic congestion status.

[1517] A "location-based app" is an application that identifies a user's geographic location and provides navigation and surrounding information to the user.

[1518] "Usage statistics" refers to statistical data based on a user's application usage history and behavioral patterns.

[1519] "Machine learning" is a technology that uses algorithms to learn data and make predictions and classifications.

[1520] "Travel route" refers to route information that indicates the means of transportation and travel route used by a traveler.

[1521] A "travel plan" is a plan that includes travel routes, the order in which tourist spots are visited, recommended accommodations, etc., generated based on conditions specified by the traveler.

[1522] "Accommodation facilities" refer to facilities such as hotels and inns where travelers can stay.

[1523] A "reservation" is a procedure for securing accommodation, transportation, etc. in advance.

[1524] "Usage history" refers to historical information about trips taken by a user in the past and services used by the user.

[1525] "Feedback" refers to user opinions and evaluations provided after using the service.

[1526] "Real-time information" refers to information that is updated hourly, such as the latest traffic conditions and the current congestion status of tourist spots.

[1527] The present invention is a system for assisting in travel planning, and its configuration and operation steps will be described in detail. First, an outline of the program required to realize this system will be created, and the processing based on this program will be described.

[1528] Overall system configuration

[1529] This system consists of a server, a terminal, and a user.

[1530] Server Roles

[1531] Collecting information from external databases: The server collects traffic information and usage statistics from external databases and location app APIs.

[1532] Data analysis with machine learning: Train machine learning models based on collected data to generate optimal travel routes.

[1533] Automatic itinerary generation: Using the trained model, we automatically generate itineraries that best fit the user's requirements.

[1534] Providing real-time information: Providing real-time updates on a traveler's current location and destination.

[1535] Collect usage history and feedback: Collect user usage history and feedback and update the model for the next itinerary generation.

[1536] Accommodation reservations: We will process your accommodation reservations based on your proposed travel plans.

[1537] Device Role

[1538] Receive user input: Enter travel destinations, departure points, desired dates, favorite attractions, and other conditions.

[1539] Display itinerary: Display the itinerary sent from the server to the user, and confirm and select the proposed itinerary.

[1540] User Roles

[1541] Entering travel conditions: Enter travel conditions through the terminal.

[1542] Choose your itinerary: Review the suggested itineraries and choose the one you like best.

[1543] Providing feedback: Provide feedback after your trip and contribute to improving the system.

[1544] Hardware and software configuration

[1545] Hardware: Smartphone (iOS / Android), server

[1546] Software: Google Maps API, HotelBooking API, Machine Learning algorithms (e.g. Logistic Regression)

[1547] Details of data processing and calculation

[1548] 1. Data collection: The server collects traffic and location information from Google Maps API and other external databases.

[1549] 2. Data analysis: Using machine learning models (e.g., Logistic Regression) to analyze users' historical data and generate optimal travel routes.

[1550] 3. Real-time information provision: Based on the traveler's current location, the server periodically calls the API to provide the latest traffic information and congestion status of tourist attractions in real time.

[1551] 4. Obtaining feedback and updating the model: After the trip, obtain feedback from the user and use it as training data for the machine learning model.

[1552] Specific examples

[1553] Travel planning

[1554] If a user enters "I want to travel to Kyoto on March 15th," the system will prompt "Please enter the dates and places of interest." If a user enters "I want to visit Kiyomizu-dera Temple and Kinkaku-ji Temple," the system will generate and suggest the optimal itinerary.

[1555] Prompt sentences for generative AI models (examples)

[1556] A user has entered that they would like to visit Kyoto on March 15th. Their tourist attractions of interest are Kiyomizu-dera Temple and Kinkaku-ji Temple. Please suggest the best travel route and accommodation. Please create a detailed plan, including tourist attractions along the way.

[1557] In this way, the travel planning support system of the present invention is able to automatically generate optimal travel plans for users and provide them with real-time information.

[1558] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1559] Step 1:

[1560] Receiving User Input

[1561] Subject: Device

[1562] Specific operation: The device receives information from the user, such as the travel destination, departure point, desired dates, and favorite tourist spots.

[1563] Input: Travel destination, departure point, desired dates, tourist attractions

[1564] Output: Received user input information

[1565] Data processing: The device converts the received information into an appropriate format and prepares it for transmission to the server.

[1566] Step 2:

[1567] Collection of traffic information and usage statistics

[1568] Subject: Server

[1569] What it does: The server uses the Google Maps API and other external databases to collect traffic information and location app usage statistics.

[1570] Input: External database, Google Maps API endpoint

[1571] Output: Collected traffic information and usage statistics

[1572] Data processing: The server organizes the collected data and stores it in a database, including any necessary filtering and preprocessing.

[1573] Step 3:

[1574] Training a machine learning model

[1575] Subject: Server

[1576] How it works: The server uses historical travel data and real-time usage statistics to train machine learning models.

[1577] Inputs: Historical travel data, real-time usage statistics

[1578] Output: A trained machine learning model

[1579] Data calculation: The server performs training using algorithms such as logistic regression, learning patterns based on past data and optimizing the model parameters.

[1580] Step 4:

[1581] Generating optimal travel routes

[1582] Subject: Server

[1583] What it does: Uses a trained machine learning model to generate optimal travel routes based on the user's criteria.

[1584] Input: User input, trained machine learning model

[1585] Output: Optimal travel route

[1586] Data calculation: The server applies the user's input to the model to predict the best route. It uses an optimization algorithm to evaluate and select travel routes.

[1587] Step 5:

[1588] Automatic travel plan generation

[1589] Subject: Server

[1590] Specific operation: The server automatically generates a travel plan based on the generated travel route, including travel routes, the order in which tourist attractions are visited, and recommended accommodations.

[1591] Input: Optimal travel route

[1592] Output: Auto-generated itinerary

[1593] Data processing: The server incorporates information on tourist attractions and accommodations based on the generated route and formats it into a complete travel plan.

[1594] Step 6:

[1595] View and select your travel plans

[1596] Subject: Device

[1597] Specific operation: The terminal displays the travel plans sent from the server to the user, allowing the user to confirm and select the plans.

[1598] Input: Auto-generated itinerary

[1599] Output: Selected itinerary

[1600] Data processing: Visually display the travel plan through the user interface and accept user selections.

[1601] Step 7:

[1602] Accommodation reservations

[1603] Subject: Server

[1604] Specific operation: Based on the selected travel plan, the server completes the accommodation reservation using the API of the external application.

[1605] Input: Selected travel plan

[1606] Output: Reservation confirmation ID

[1607] Data calculation: Obtain accommodation availability information from the API, complete the reservation process, and generate a confirmation ID.

[1608] Step 8:

[1609] Providing real-time information

[1610] Subject: Server

[1611] Specific operation: The server collects real-time traffic information and congestion status of tourist attractions based on the traveler's current location and sends it to the terminal.

[1612] Input: Current location, real-time information

[1613] Output: User notification

[1614] Data calculation: The server obtains the latest information from the API and pushes it to the device.

[1615] Step 9:

[1616] Get feedback and update the model

[1617] Subject: Server

[1618] What it does: After the trip, the server collects user feedback and uses it as training data for the machine learning model.

[1619] Input: User feedback

[1620] Output: An updated machine learning model

[1621] Data processing: Analyze the feedback and add it to the model's training dataset. Retrain the model as a new model.

[1622] This process enables the travel planning support system to automatically generate optimal travel plans for users and provide real-time information.

[1623] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1624] The present invention relates to a system for assisting in travel planning, and more particularly to a system for providing more personalized travel plans by incorporating an emotion engine that recognizes the emotions of a user. Hereinafter, an embodiment of the present invention will be described in detail.

[1625] Overall system configuration

[1626] This system consists of a server, a terminal, a user, and an emotion engine. The emotion engine has the means to recognize emotions by analyzing the user's voice, facial expressions, and text data. The server generates a travel plan that is most suitable for the user based on the output of this emotion engine. The terminal is responsible for receiving this information from the user and sending it to the server. The user inputs the travel destination, departure point, desired dates, emotional state, etc. via the terminal, and then confirms and selects the travel plan proposed by the server.

[1627] System Operation

[1628] 1. Collection of traffic information

[1629] The server collects traffic information from external databases and the navigation app's API, including public transport schedules and traffic congestion status.

[1630] Example: A server uses the Google Maps API to retrieve Shinkansen schedules and fare information from Tokyo to Kyoto and stores it in a database.

[1631] 2. Route optimization using machine learning

[1632] The server uses the collected traffic information to train a machine learning model that leverages the user's past travel data and real-time usage statistics to suggest optimal travel routes.

[1633] Example: A server analyzes past travel data and determines, based on traveler behavior patterns, that taking the Shinkansen is the quickest and most cost-effective option.

[1634] 3. Emotion Recognition by Emotion Engine

[1635] The terminal collects the user's voice data, facial expression data, and text data and sends them to the emotion engine, which analyzes the data and recognizes the user's current emotional state.

[1636] Example: The emotion engine analyzes the voice and facial expressions of a user speaking into the device's camera and determines that the user is in a "relaxed" state.

[1637] 4. Receiving and Sending User Input

[1638] The user inputs travel conditions such as the destination, desired itinerary, current emotional state, and favorite tourist spots through the terminal, and this information is sent from the terminal to the server.

[1639] Example: A user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into an input form on a smartphone app and clicks the submit button.

[1640] 5. Automatic travel plan generation

[1641] The server generates an optimal travel route and sightseeing plan for the user based on the output of the machine learning model and emotion engine, including the route, the order in which tourist spots are visited, and recommended accommodations.

[1642] Example: The server considers the user's relaxed emotional state and tourist spot preferences to suggest a relaxing route in Kyoto.

[1643] 6. Check and select your travel plan

[1644] The terminal displays the travel plans sent from the server to the user, who then checks the proposed travel plans, selects the one they like, and confirms it.

[1645] Example: A user reviews suggested travel plans on their smartphone screen and selects one to visit Kiyomizu-dera Temple.

[1646] 7. Accommodation reservations

[1647] The server automatically processes the accommodation reservation based on the travel plan selected by the user, and completes the reservation in cooperation with an external accommodation reservation system.

[1648] Example: The server uses the API of a partner accommodation booking site to reserve a room at a selected hotel and obtain a reservation confirmation ID.

[1649] 8. Real-time information provision

[1650] The server provides real-time traffic and tourist information to the user while he or she is traveling, and this information is sent to the user via the terminal.

[1651] Example: While traveling, a user receives the latest traffic information and crowding status of tourist spots on their smartphone.

[1652] 9. Usage history and feedback collection

[1653] The server collects user travel history and feedback and uses it to generate future travel plans. This information is used to update the machine learning model.

[1654] Example: After completing a trip, a user enters an evaluation of the plan provided by the app, and the data is stored on the server.

[1655] In this way, the travel planning support system of the present invention can significantly improve convenience for travelers by proposing optimal travel routes taking into account the user's emotional state, automatically generating travel plans, and seamlessly booking accommodations.

[1656] The processing flow will be explained below.

[1657] Step 1:

[1658] The server collects traffic information from external databases and the navigation app's API.

[1659] Specific operation: The server uses the Google Maps API to obtain Shinkansen schedules and fare information from Tokyo to Kyoto and stores it in a database.

[1660] Step 2:

[1661] The server trains a machine learning model based on the collected traffic information.

[1662] Specific operation: The server takes in past travel data and user behavior patterns and runs an algorithm to learn the optimal travel route that minimizes travel time and costs.

[1663] Step 3:

[1664] The user uses the terminal to input conditions such as the travel destination, departure point, desired dates, and favorite tourist spots.

[1665] Specific operation: The user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into the input form of the smartphone app and clicks the send button.

[1666] Step 4:

[1667] The terminal converts the user's input data into JSON format and sends it to the server.

[1668] Specific operation: The device packages the input data in JSON format and sends it to the server using an HTTP request.

[1669] Step 5:

[1670] The server generates an optimal travel route and sightseeing plan based on the received user's conditions.

[1671] Specific operation: The server uses a machine learning model to generate a travel plan that includes the Shinkansen route that best suits the user's requirements, tourist spots along the way, and recommended accommodations.

[1672] Step 6:

[1673] The server converts the generated travel plan into JSON format and sends it to the user's device.

[1674] Specific operation: The server encodes the generated travel plan in JSON format and sends it to the terminal as an HTTP response.

[1675] Step 7:

[1676] The terminal displays the travel plan received from the server to the user.

[1677] Specific operation: The device parses the contents of the travel plan and displays it on the screen in a format that is easy for the user to view.

[1678] Step 8:

[1679] The user checks the displayed travel plans and selects the plan they want.

[1680] Specific operation: The user checks the suggested tourist spots and accommodations on the smartphone screen and taps to select the plan they like.

[1681] Step 9:

[1682] The device converts the user's selected plan information into JSON format and sends it to the server.

[1683] Specific behavior: The device packages the selected travel plan in JSON format and sends it to the server using an HTTP request.

[1684] Step 10:

[1685] The server then processes the reservation for the accommodation based on the user's selection.

[1686] Specific operation: The server uses the information of the selected accommodation to call the API of the affiliated reservation site, completes the reservation procedure, and obtains the reservation confirmation ID.

[1687] Step 11:

[1688] The server sends information including the reservation confirmation ID to the terminal and notifies the user.

[1689] Specific operation: The server creates a response including the reservation confirmation ID and sends it to the terminal as an HTTP response. The terminal receives this and notifies the user.

[1690] Step 12:

[1691] The server allows the user to receive real-time traffic information and tourist spot updates while traveling and transmits them to the terminal.

[1692] Specific operation: The server collects and updates real-time traffic information and tourist spot information, and sends it to the user's device as a push notification.

[1693] Step 13:

[1694] The server collects the user's travel history and feedback after the trip is completed.

[1695] What happens: A user fills out a feedback form in the app and clicks the submit button. The data is sent to the server and stored in the database.

[1696] Step 14:

[1697] The terminal collects the user's voice data, facial expression data, and text data and sends them to the emotion engine.

[1698] Specific operation: Using the device's camera and microphone, the user's voice and facial expressions are recorded and sent to the emotion engine along with the text data.

[1699] Step 15:

[1700] The emotion engine analyzes the user's voice, facial expression, and text data to recognize the user's current emotional state.

[1701] Specific operation: The emotion engine uses an analytical algorithm to determine emotions such as "relaxed," "excited," and "stressed" from the collected data.

[1702] Step 16:

[1703] The server adjusts the contents of the travel plan based on the emotional state recognized by the emotion engine.

[1704] Specific operation: The server generates a plan that includes tourist spots and experiences that correspond to the emotional state of "wanting to relax."

[1705] Step 17:

[1706] The server updates the travel plan in real time according to changes in the emotional state and sends it to the terminal.

[1707] Specific operation: When a user feels stressed during a trip, the server generates a plan including new relaxation spots and sends it to the terminal.

[1708] In this way, the travel planning support system of the present invention can significantly improve convenience for travelers by proposing optimal travel routes taking into account the user's emotional state, automatically generating travel plans, and seamlessly booking accommodations.

[1709] Example 2

[1710] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1711] Conventional travel planning support systems have the problem of being unable to provide personalized travel plans based on users' emotional state or real-time fluctuating information, which has led to travelers being unable to have a comfortable and satisfying travel experience.

[1712] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1713] In this invention, the server includes means for collecting traffic information from an external database and integrating usage statistics of a navigation app, means for generating an optimal travel route using a machine learning model based on the statistics, means for utilizing an emotion recognition engine that recognizes emotions by analyzing the traveler's voice, facial expression, and text data, means for automatically generating a travel plan based on the output of the emotion recognition engine, means for reserving accommodations based on the travel plan, and means for collecting the traveler's usage history and feedback and updating the model for generating the next travel plan. This makes it possible to provide an optimal and personalized travel plan based on the user's current emotional state and past travel history.

[1714] "Traffic information" refers to data such as public transportation schedules and traffic congestion conditions.

[1715] "Navigation app usage statistics" refers to data on the behavioral patterns and real-time usage status of users of navigation apps.

[1716] A "machine learning model" refers to an algorithm that automatically learns from large amounts of data and makes predictions and classifications.

[1717] An "emotion recognition engine" refers to a system that recognizes emotions by analyzing a user's voice, facial expressions, and text data.

[1718] A "travel plan" refers to a detailed travel plan that includes travel routes, the order in which tourist spots are visited, recommended accommodations, etc.

[1719] "Accommodation Booking" refers to the process of reserving accommodation for a traveler.

[1720] "Usage history" refers to travel plans and behavioral data that a user has used in the past.

[1721] "Feedback" refers to the evaluations and opinions given by users regarding the services and plans provided.

[1722] "Updating a model" refers to the process of improving the performance of an existing machine learning model based on new data.

[1723] The present invention relates to a system for assisting in travel planning, and more particularly to a system for providing more personalized travel plans by incorporating an emotion engine that recognizes the emotions of a user. Hereinafter, an embodiment of the present invention will be described in detail.

[1724] This system consists of a server, a terminal, a user, and an emotion engine. The emotion engine has the means to recognize emotions by analyzing the user's voice, facial expressions, and text data. The server generates a travel plan that is most suitable for the user based on the output of this emotion engine. The terminal is responsible for receiving this information from the user and sending it to the server. The user inputs the travel destination, departure point, desired dates, emotional state, etc. via the terminal, and then confirms and selects the travel plan proposed by the server.

[1725] System Configuration

[1726] 1. Means of collecting traffic information

[1727] The server collects traffic information using external databases and the navigation app's API, including public transport schedules and traffic congestion status. The specific software used is Google Maps API.

[1728] 2. Using Machine Learning Models

[1729] The server uses the collected traffic information to train a machine learning model that uses the user's past travel data and real-time usage statistics to suggest optimal travel routes using software including TensorFlow.

[1730] 3. Emotion recognition

[1731] The device collects the user's voice, facial expression, and text data and sends it to the emotion engine, which analyzes this data and recognizes the user's current emotional state using Microsoft Azure Cognitive Services.

[1732] 4. Automatic generation of travel plans

[1733] The server generates an optimal travel route and sightseeing plan for the user based on the output of the machine learning model and emotion engine, including the route, the order in which tourist spots are visited, and recommended accommodations.

[1734] 5. Accommodation reservations

[1735] The server automatically makes reservations for accommodation based on the travel plan. At this time, it completes the reservation by linking with an external accommodation reservation system. Specifically, it uses the Booking.com API.

[1736] 6. Collecting User Usage History and Feedback

[1737] The server collects user travel history and feedback and uses it to generate future travel plans. This information is also used to update the machine learning model.

[1738] Specific examples

[1739] For example, if a user uses a smartphone app to input "Kyoto, March 15, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple," and the emotion recognition engine recognizes the user's relaxed state, the server will use this information to generate a travel plan that includes relaxing tourist spots and accommodations.The server then uses the Booking.com API to make reservations for the accommodations and notify the user.

[1740] Prompt Sentence Examples

[1741] "How do I use the Google Maps API to plan my trip?"

[1742] In this way, the travel planning support system of the present invention can provide an optimal and personalized travel plan that takes into account the emotional state of the user.

[1743] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1744] Step 1:

[1745] The server collects traffic information using an external database and the navigation app's API. As input, it receives traffic information from the external database and the navigation app. This provides data such as public transport schedules and traffic congestion conditions. The server stores this information in a database. Specifically, the server calls the Google Maps API to obtain the Shinkansen schedule and fare information from Tokyo to Kyoto, and stores this information in the database.

[1746] Step 2:

[1747] The server trains a machine learning model based on the collected traffic information. It uses the traffic information collected in step 1 and past travel data as input. This creates a predictive model for generating optimal travel routes. The server uses the TensorFlow library to train the machine learning model and prepares optimal travel route suggestions. Specifically, the server analyzes past travel data and learns user behavior patterns.

[1748] Step 3:

[1749] The device collects the user's voice data, facial expression data, and text data, and sends this data to the emotion engine. The user's voice, facial expression, and text data are used as input. This allows the emotion engine to recognize the user's current emotional state. The emotion engine uses Microsoft Azure Cognitive Services to perform emotion analysis. Specifically, data collected using the device's camera and microphone is sent to the Microsoft Azure Cognitive Services API, which recognizes the user's emotional state, such as "relaxed" or "tense."

[1750] Step 4:

[1751] The user inputs conditions such as the travel destination, desired schedule, emotional state, and favorite tourist spots through the device. The travel information specified by the user is received as input. This information is sent from the device to the server. Specifically, the user enters "Kyoto, March 15th, departure point: Tokyo, tourist spots: Kiyomizu-dera Temple, Kinkaku-ji Temple" into the input form of the smartphone app and clicks the send button.

[1752] Step 5:

[1753] The server generates the optimal travel route and sightseeing plan for the user based on the output of the machine learning model and emotion engine. As input, it uses the output of the machine learning model in step 2, the result of the emotion engine in step 3, and the user's travel conditions sent in step 4. This creates the optimal travel plan. Specifically, the server considers the user's relaxed emotional state and tourist spot preferences, and proposes a travel route that includes relaxing tourist spots and accommodations.

[1754] Step 6:

[1755] The terminal displays the travel plan sent from the server to the user. The terminal receives the travel plan from the server as input. Based on this information, the user checks the proposed plans, selects the one they like, and confirms it. Specifically, the user checks the proposed travel plan on the smartphone screen, selects the plan to visit Kiyomizu-dera Temple and Kinkaku-ji Temple, and clicks the confirm button.

[1756] Step 7:

[1757] The server automatically makes accommodation reservations based on the travel plan selected by the user. It receives the confirmed travel plan as input and completes the reservation by connecting with an external accommodation reservation system. Specifically, the server uses the Booking.com API to reserve a room at the selected hotel and obtains a reservation confirmation ID.

[1758] Step 8:

[1759] The server provides real-time traffic and tourist spot information to users traveling. The server references the user's current location and travel plan as input, allowing it to continue providing real-time information. Specifically, the server obtains the latest traffic congestion information and congestion status at tourist spots, and notifies the traveling user via their smartphone.

[1760] Step 9:

[1761] The server collects the user's travel history and feedback and uses it to generate future travel plans. This information is also used to update the machine learning model. User feedback and travel history data are taken as input. Specifically, after the user completes their trip, they enter an evaluation of the plan provided by the app, and this data is stored on the server. The server uses this data as training data for a new machine learning model.

[1762] (Application example 2)

[1763] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1764] Conventional travel planning support systems were unable to generate travel plans that took into account the user's individual emotional state, and were limited to providing standard plans. This made it difficult to provide travel plans that were tailored to the user's needs and emotions. Furthermore, in the shopping experience, there was also the issue of being unable to provide personalized product suggestions based on the user's emotions and real-time situation.

[1765] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1766] In this invention, the server includes means for integrating traffic information collected from an external database and usage statistics of the navigation app, means for performing machine learning based on the statistics to generate an optimal travel route, and means for automatically generating a travel plan based on the generated travel route, thereby making it possible to analyze the user's emotional state in real time and provide personalized travel plans and product suggestions.

[1767] An "external database" is a collection of information or data accessible through the Internet or other network.

[1768] A "navigation app" is software that provides maps and traffic information and guides users to the optimal route from their current location to their destination.

[1769] "Usage Statistics" refers to the compilation and analysis of data based on a user's past usage history and behavioral patterns.

[1770] "Machine learning" is a technology in which a computer learns patterns and rules based on data and automatically makes predictions and decisions.

[1771] A "travel route" refers to a travel route from a user's departure point to a destination.

[1772] A "travel plan" is a plan that includes travel destinations, transportation, accommodations, tourist spots, etc.

[1773] An "accommodation reservation system" is a system for making online reservations for accommodations such as hotels and inns.

[1774] "Usage history" refers to a record of trips the user has taken and a history of services they have used.

[1775] "Feedback" refers to a user providing an evaluation or opinion about a service or product.

[1776] An "emotion recognition engine" is a technology that analyzes a user's voice, facial expressions, text data, etc. to recognize their emotional state.

[1777] "Personalization" refers to providing services and products tailored to the individual preferences and emotional state of each user.

[1778] "Brick and mortar store" refers to a store that has a physical presence.

[1779] "Smart glasses" are glasses-type devices that have display and camera functions and can display and collect information.

[1780] A "smartphone" is a mobile phone that has advanced computing power, multiple functions, and can run apps.

[1781] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following describes an embodiment of the present invention.

[1782] Overall system overview

[1783] The embodiment of the invention comprises an external database, a server, a terminal, a user, and an emotion recognition engine. The server collects traffic information and navigation app usage statistics from the external database, and uses machine learning to generate optimal travel routes based on this information. Furthermore, the emotion recognition engine analyzes the user's emotional state and provides personalized travel plans and product recommendations in physical stores.

[1784] Program Overview

[1785] This system uses programming languages ​​such as Python to collect various data, perform machine learning, and recognize emotions. It also provides information to users via smart glasses or smartphones. Specifically, it uses the following hardware and software:

[1786] 1. Hardware

[1787] Smart glasses or smartphones: Collect the user's voice, facial expression, and text data and send it to an emotion recognition engine.

[1788] Server: Based on the collected data, it integrates traffic information, performs machine learning, generates itineraries, and makes personalized product recommendations.

[1789] 2. Software

[1790] Emotion recognition engine: Recognizes the user's emotional state by analyzing their voice, facial expressions, and text data. For example, it can be built using TensorFlow or Keras.

[1791] Machine learning model: Generates optimal travel routes based on traffic information and usage statistics. Uses Scikit-learn and TensorFlow.

[1792] API: Traffic information is collected using external databases such as Google Maps API.

[1793] System Operation

[1794] The server communicates with external databases to collect external data, and uses machine learning to generate optimal travel routes and plans. Users' emotional state is analyzed through smart glasses or smartphones, and they receive personalized travel plans or in-store product suggestions based on that analysis.

[1795] As a concrete example, there is a system in which a user walks around a store using smart glasses and receives product suggestions based on their emotional state at the time. For example, if the user is feeling stressed, relaxation items are suggested, and if the user is having fun, the latest fashion items are suggested.

[1796] Prompt Sentence Examples

[1797] "The user is currently in the emotional state 'happy'. Please generate a description for the following product:

[1798] Product 1: Latest fashion item (Description: The latest trendy fashion item for the new season.)

[1799] Product 2: Accessories (Description: Luxury accessories.)

[1800] The system can provide personalized travel plans and shopping experiences that take into account the user's emotional state.

[1801] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1802] Step 1:

[1803] The device collects the user's voice, facial expression, and text data in real time. This data is captured using a camera and microphone. The input data is the raw material for recognizing the user's emotions, and the output is this raw data.

[1804] Step 2:

[1805] The device sends the collected voice, facial expression, and text data to an emotion recognition engine. The emotion recognition engine is built using TensorFlow and Keras, and analyzes this data to recognize the user's emotions. The input is the data sent from the device, and the output is the user's emotional state (e.g., "happy," "sad," "angry," etc.). Specifically, the analysis algorithm determines the user's facial muscle movements and tone of voice.

[1806] Step 3:

[1807] The server receives the emotional state output by the emotion recognition engine and generates personalized travel plans or product suggestions based on it. The input is the recognized emotional state, and the output is a detailed travel plan or product list tailored to the user's emotions. Specifically, the server analyzes the characteristics of tourist attractions and products in the database and selects the best ones for the user.

[1808] Step 4:

[1809] The server collects real-time traffic and transportation information using external databases and APIs (e.g., Google Maps API). The input is information obtained from the external database, and the output is integrated traffic information. Specifically, the server makes API calls and stores the obtained data in its internal database.

[1810] Step 5:

[1811] The server performs machine learning based on the collected traffic information and usage statistics to generate the optimal travel route. The input is traffic information and past usage statistics, and the output is an optimized travel route. Specifically, the server trains the model using Scikit-learn and TensorFlow to calculate the optimal route.

[1812] Step 6:

[1813] The terminal displays the travel plan and product suggestions received from the server to the user. The input is the personalized plan sent from the server, and the output is the information displayed on the terminal's display. Specifically, the terminal displays the plan details and product list on the screen.

[1814] Step 7:

[1815] The user checks the displayed travel plans and product suggestions, and makes selections and reservations as necessary. The input is the information displayed on the terminal, and the output is the user's selection. Specifically, the user makes selections using a touch panel or voice commands.

[1816] Step 8:

[1817] Based on the user's selection, the server connects with an external accommodation reservation system to complete the reservation procedure. The input is the user's selection, and the output is reservation confirmation information. Specifically, the server uses the reservation system's API to confirm the reservation and obtain a confirmation ID.

[1818] Step 9:

[1819] The server provides real-time traffic and tourist spot information to users while they are traveling. The input is traffic information and tourist spot data obtained in real time, and the output is the latest information provided to the user. Specifically, the server periodically calls the API and sends information to the device.

[1820] Step 10:

[1821] The server collects user feedback after the trip and updates the machine learning model for the next itinerary generation. The input is the user feedback, and the output is the updated machine learning model. Specifically, the server analyzes the feedback data and adds it to the training dataset of the model.

[1822] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1823] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1824] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1825] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1826] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1827] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1828] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1829] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1830] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1831] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1832] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1833] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1834] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1835] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1836] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1837] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1838] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1839] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1840] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1841] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1842] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard ...

Claims

1. A system for assisting with travel planning, comprising: a means for integrating traffic information and navigation application usage statistics collected from external databases; A means for performing machine learning based on the statistics to generate an optimal travel route; means for automatically generating a travel plan based on the generated travel route; A means for making reservations for accommodations based on the travel plan; A means for collecting traveler usage history and feedback and updating the model for next itinerary generation; A system including:

2. 10. The system of claim 1, further comprising means for receiving traveler input information and generating the optimal travel route and travel plan based on the information.

3. The system according to claim 1 , further comprising means for completing a reservation procedure in cooperation with an external accommodation reservation system based on the travel plan.

4. 10. The system of claim 1, further comprising means for providing real-time traffic and tourist spot information to the user during the trip.

5. The system of claim 1 , further comprising means for categorizing a plurality of user data and providing a personalized travel plan for each user.

Images