System

Abstract

A system is provided.SOLUTION: A system comprising: means for collectively uploading photographs taken by a traveler to a server; means for automatically tabulating a travel route of the traveler from position information and time stamps of the photographs; means for automatically generating a travel diary of the traveler based on the travel route; means for generating a new travel route incorporating highly evaluated restaurants and tourist spots based on the generated travel diary; and means for generating a popularity ranking based on the evaluation and giving a reward to a person with a high ranking.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Travelers take many photos during their trips, but organizing them later and compiling them into a travel record can be a time-consuming process. While travelers often plan their trips in advance, standard itineraries are often useless because arrival times and planned stay times vary for each traveler. Foreign travelers, in particular, face a lack of information in their native language, making it difficult to customize their travel plans to suit their interests and needs. This invention aims to automatically organize photos after a trip, making it easy to create a travel record, and to provide new tour plans that incorporate highly rated restaurants and tourist attractions, thereby enabling travelers to experience the optimal travel experience tailored to their diverse needs. [Means for solving the problem]

[0005] This invention provides a system that includes a means for uploading photos taken by travelers to a server in bulk, a means for automatically compiling the traveler's travel route based on the location information and timestamps of the photos, a means for automatically generating the traveler's travel diary based on the travel route, a means for generating a new travel route based on the generated travel diary and incorporating highly rated restaurants and tourist attractions, a means for users to view and rate the generated travel route, and a means for generating a popularity ranking based on the rating and awarding rewards to the top ranked users. The system also includes a means for analyzing the location information and timestamps of the uploaded photos and sorting them in chronological order, and a means for displaying the new travel route on a map and providing navigation. This eliminates the need for travelers to organize photos later, allowing them to easily create travel plans tailored to their individual needs, and share and rate them with other travelers.

[0006] A "traveler" is someone who takes photographs while traveling and creates a travel record based on those photographs.

[0007] The "server" is a computer system that analyzes the uploaded photo data and generates and stores travel routes and travel diaries.

[0008] A "photo" is an image file taken by a traveler, and is digital data that includes location information and a timestamp.

[0009] "Location information" refers to the geographic coordinate data (GPS data) where the photo was taken.

[0010] A "timestamp" is data that indicates the date and time a photo was taken.

[0011] The "travel route" is the traveler's route calculated based on the location information and timestamp of the photo.

[0012] A "travel diary" is an automatically generated travel record that combines travel routes and highlight events.

[0013] "Generative AI" is an artificial intelligence system that creates new travel routes based on travel diaries.

[0014] "Highly rated restaurants and tourist attractions" are restaurants and tourist spots that have been highly rated by other users.

[0015] A "tour plan" is a new travel route or travel plan provided by the generation AI.

[0016] The "popularity ranking" is a list that aggregates evaluation points from users and identifies the top tour plans.

[0017] "Rewards" are incentives given to tour plan contributors who rank highly in popularity.

[0018] "Navigation" is a system that displays new travel routes on a map and provides audio and visual guidance. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] MODE FOR CARRYING OUT THE INVENTION

[0041] System Overview

[0042] This system automatically compiles travel routes and generates travel diaries based on photos taken by travelers after their trip. It also uses AI to propose new tour plans and provides a function that allows users to rate and share them.

[0043] Bulk photo posting

[0044] 1. After the trip, the user opens the dedicated app and selects all the photos taken during the trip.

[0045] 2. The device extracts the location information (GPS data) and timestamp (date and time of shooting) of the selected photo.

[0046] 3. The device uploads the photo file, its location information, and timestamp to the server.

[0047] Specific examples

[0048] Users select photos they have taken around Mount Fuji in the app and upload them to the server all at once.

[0049] When uploading, the device automatically extracts the GPS data and shooting date and time contained in the photo and sends them to the server.

[0050] Automatic calculation of travel route

[0051] 1. The server analyzes the received photo data and checks the location information and timestamp.

[0052] 2. The server calculates the user's route in chronological order based on the location information and timestamp.

[0053] 3. The server stores the calculated route in a database.

[0054] Specific examples

[0055] The server analyzes the user's itinerary and plots the route from the base of Mount Fuji to the summit on a map.

[0056] Based on the location information, each photographed point is connected in a continuous manner to form a route.

[0057] Travel diary generation

[0058] 1. The server extracts highlights and events from the calculated travel route.

[0059] 2. The server automatically generates a travel diary based on the extracted highlights, combining photos and descriptions.

[0060] 3. The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[0061] Specific examples

[0062] The server automatically inserts photos of summits and particularly beautiful views into the travel diary as highlights and generates detailed descriptions.

[0063] The travel diary is saved in the user's profile, allowing them to follow the overall picture of their Mount Fuji climb chronologically.

[0064] Creating a new tour plan

[0065] 1. The server learns the user's preferences and patterns of places visited from the contents of their travel diary.

[0066] 2. The generating AI analyzes rating data from other users and identifies highly rated restaurants and tourist attractions.

[0067] 3. The generation AI combines the user's preferences and high rating points to generate the optimal travel route as a new tour plan.

[0068] 4. The server provides the newly generated travel route to the user.

[0069] Specific examples

[0070] The generative AI learns the user's preferences and suggests new tour plans that incorporate other tourist attractions and restaurants.

[0071] For example, if a user likes the view from a mountaintop, they can add other scenic spots.

[0072] User ratings and popularity rankings

[0073] 1. Users view tour plans published by other users.

[0074] 2. Users can "like" tour plans they find good or post comments.

[0075] 3. The server will tally the evaluation points for each tour plan and update the rankings in real time.

[0076] 4. The server awards rewards to the top-ranked users and notifies them of the results.

[0077] Specific examples

[0078] Users view Yamanashi Prefecture tour plans published by other users and "like" them.

[0079] The server tallys likes and comments in real time and updates the popularity rankings.

[0080] Map display and navigation

[0081] 1. The server integrates the generated travel route with map data.

[0082] 2. The device will display the route on a map and enter navigation mode.

[0083] 3. The device provides voice guidance to support the user's movement.

[0084] Specific examples

[0085] The device displays the route of the new tour plan on the user's smartphone and begins navigation in real time.

[0086] Voice guidance will explain how to reach your destination and where to stop.

[0087] Such a system allows travelers to effortlessly record their trips, get travel plans tailored to their individual preferences, and improve the quality of their travel experience by sharing information and ratings with other travelers.

[0088] The processing flow will be explained below.

[0089] Program processing flow

[0090] Bulk photo posting

[0091] Step 1:

[0092] After traveling, users open a dedicated app and select all the photos they want to post at once.

[0093] Step 2:

[0094] The terminal displays a list of the selected photos and asks the user for confirmation.

[0095] Step 3:

[0096] The device extracts the location information (GPS data) and timestamp (date and time of photo capture) for each selected photo.

[0097] Step 4:

[0098] The device sends the photo file, its location information, and a timestamp to the server.

[0099] Automatic calculation of travel route

[0100] Step 5:

[0101] The server analyzes the received photo data and checks the location information and timestamp.

[0102] Step 6:

[0103] The server sorts the photos in chronological order based on the location information and timestamps, and calculates the user's travel route.

[0104] Step 7:

[0105] The server plots the calculated travel route on map data to create a visual route.

[0106] Step 8:

[0107] The server stores the calculated travel route data in a database.

[0108] Travel diary generation

[0109] Step 9:

[0110] The server extracts highlights and events from the trip based on the stored travel route data.

[0111] Step 10:

[0112] The server combines the extracted highlights and photos to automatically generate a travel diary with explanatory text.

[0113] Step 11:

[0114] The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[0115] Creating a new tour plan

[0116] Step 12:

[0117] The server learns the user's preferences and patterns of places visited from the contents of the user's travel diary.

[0118] Step 13:

[0119] The generative AI analyzes rating data from other users to identify highly rated restaurants and tourist attractions.

[0120] Step 14:

[0121] The generation AI combines user preferences and high rating points to generate new travel routes.

[0122] Step 15:

[0123] The server saves the generated new travel route in the user's profile and makes it available for the user to view.

[0124] User ratings and popularity rankings

[0125] Step 16:

[0126] A user can view tour plans published by other users.

[0127] Step 17:

[0128] Users can "like" tour plans they find good and post comments.

[0129] Step 18:

[0130] The server tally up the evaluation points for each tour plan and update the rankings in real time.

[0131] Step 19:

[0132] The server gives rewards to the top ranked tour plan posters and notifies them of the results.

[0133] Map display and navigation

[0134] Step 20:

[0135] The server integrates the generated travel route with map data.

[0136] Step 21:

[0137] The device displays the route on a map and prompts the user to begin navigation.

[0138] Step 22:

[0139] The terminal starts voice guidance to support the user's movement.

[0140] Step 23:

[0141] The user uses the navigation system to enjoy sightseeing and eating and drinking along the generated travel route.

[0142] The above is the specific flow of program processing for this system. Each step incorporates a detailed process that includes the specific operations of the user, terminal, and server.

[0143] Example 1

[0144] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0145] There is a need for a system that can efficiently create detailed travel records based on photos taken by travelers, suggest new travel plans, and share information and ratings with other travelers in real time. However, conventional systems require the time-consuming task of manually classifying and organizing photos, making it difficult to automatically generate detailed travel records including trip highlights. It is also difficult to automatically generate new travel plans tailored to individual preferences and provide navigation functions integrated with map data.

[0146] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0147] In this invention, the server includes a means for uploading image data taken by a traveler to the server in bulk, a means for automatically tallying up the travel route of the traveler from the location information and timestamp of the image data, and a means for automatically generating a travel record of the traveler based on the travel route. This makes it possible to effectively utilize the photos taken by the traveler and automatically generate a detailed travel record based on a chronological order.

[0148] "Image data" refers to digital data of photographs and videos taken by travelers.

[0149] "Server" refers to a computer system for receiving and processing data from terminals connected over a network.

[0150] "Location Information" refers to geographic latitude and longitude data obtained using technologies such as GPS.

[0151] A "timestamp" refers to date and time information that indicates when digital data was created, modified, or accessed at a certain time.

[0152] A "travel route" refers to a route that connects multiple locations visited by a traveler within a specific period in chronological order.

[0153] A "travel journal" is a record of the places and events visited during a trip, compiled in photographs and text.

[0154] A "generative AI model" refers to an artificial intelligence algorithm that uses machine learning techniques to generate new data and plans.

[0155] "Restaurant" refers to a commercial establishment that sells food and beverages and allows customers to eat and drink.

[0156] "Tourist attractions" refer to popular tourist destinations and historical and cultural sites.

[0157] "Smart devices" refer to portable electronic devices that can connect to the Internet, such as smartphones and tablets.

[0158] "Navigation" refers to a system and its functions that indicate the optimal route from the current location to the destination.

[0159] "Evaluation" refers to the act of rating a tour plan or service created by another user after using the service.

[0160] "Popularity ranking" refers to a ranking list created by compiling feedback such as user ratings and "likes."

[0161] "Rewards" refers to incentives such as points or vouchers that the system gives to users.

[0162] This system is designed to automatically generate travel records and propose new travel plans based on photos taken by travelers. The system is mainly composed of three elements: a server, a terminal, and a user, and performs the following processes:

[0163] Bulk photo posting

[0164] After the trip, the user opens the app and selects photos taken during the trip from the gallery. For example, select photos taken during a trip to Mount Fuji. The device extracts the location information (GPS data) and timestamp (date and time of the photo) from the selected photo. Using this information, the device uploads the photo file and its associated location information and timestamp to the server.

[0165] Automatic calculation of travel route

[0166] The server analyzes the received photo data and checks the location and timestamp of each photo. Based on the location and timestamp, it calculates the user's movement route in chronological order and stores this movement route in a database. For example, it generates a route from the base of Mount Fuji to the summit and plots it on a map.

[0167] Travel diary generation

[0168] The server extracts highlights and events from the trip based on the calculated route. Using these highlights, the server automatically generates a travel log by combining corresponding photos and descriptions. The generated travel log is saved in the user's profile and can be viewed freely by the user.

[0169] Creating a new tour plan

[0170] The server learns the user's preferences and patterns of places visited from the contents of the user's travel log. Then, using a generative AI model, it analyzes the rating data from other users and identifies highly rated restaurants and tourist spots. Based on this, it generates an optimal travel route by combining the user's preferences and high rating points. This newly generated travel route is provided to the user by the server.

[0171] User ratings and popularity rankings

[0172] Users can browse new travel plans published by other users, click "like" on the ones they like, and post comments. The server aggregates this evaluation data and updates the popularity rankings in real time. It then awards rewards to users who rank highly and notifies them of the results.

[0173] Map display and navigation

[0174] The server integrates the generated route with map data and provides navigation functions. The device displays the route on a map and provides voice guidance to support the user's movement. For example, when navigation begins, voice guidance such as "Turn left" or "You have arrived at the next tourist spot" is provided.

[0175] Prompt Sentence Examples

[0176] "Upload photos from your Mount Fuji trip and create a travel route."

[0177] "Generate a new tour plan based on your travel records."

[0178] "Please rate tour plans created by other users."

[0179] In this way, the system allows travelers to easily organize their travel records, automatically suggest new itineraries based on their individual preferences, and improve the quality of their travel experience by sharing and rating information with other travelers.

[0180] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0181] Step 1:

[0182] After the trip, the user opens the dedicated app and selects all the image data taken during the trip from the gallery.

[0183] Input: Multiple image files selected by the user.

[0184] Output: A list of selected image files.

[0185] Specific operation: The user taps the "Upload Photos" button in the app and selects multiple images taken during their trip. The device stores the image files as a list.

[0186] Step 2:

[0187] The device extracts location information (GPS data) and timestamps (date and time of photo) from each selected image file.

[0188] Input: A list of selected image files.

[0189] Output: Image metadata including location and timestamp.

[0190] How it works: The device analyzes the metadata of each image file to obtain location information and the date and time the image was taken, and temporarily stores this data in memory.

[0191] Step 3:

[0192] The device uploads them to a server along with an image file containing the extracted location information and timestamp.

[0193] Input: Image metadata including location and timestamp.

[0194] Output: Image data uploaded to the server.

[0195] Specific operation: The device packs image files containing location information and timestamps into packets and sends them via the server's upload API.

[0196] Step 4:

[0197] The server analyzes the received image data and checks the location information and timestamp of each file.

[0198] Input: Image data uploaded to the server.

[0199] Output: Parsed location and timestamp.

[0200] Specific operation: The server decodes the received network packets and stores the binary data of the image file, including location information and timestamp, in a database.

[0201] Step 5:

[0202] The server calculates the user's movement route in chronological order based on the location information and timestamp.

[0203] Input: Parsed location and timestamp.

[0204] Output: Travel path data sorted in chronological order.

[0205] Specific operation: The server analyzes the location information as latitude and longitude data, sorts it in chronological order by the date and time of the photo, and connects the points on the map to form a movement route.

[0206] Step 6:

[0207] The server stores the calculated travel route data in a database.

[0208] Input: Travel path data arranged in chronological order.

[0209] Output: Travel path data stored in a database.

[0210] Specific operation: The server inserts the generated travel route data into a dedicated travel route table and stores it in association with each user.

[0211] Step 7:

[0212] Based on the calculated route, the server extracts highlights and events of the trip.

[0213] Input: Time-series travel path data.

[0214] Output: Highlight extracted data.

[0215] Specific operation: The server identifies the most frequently visited or long-stayed locations from photos taken at different locations and lists them as highlights.

[0216] Step 8:

[0217] Based on the highlights, the server automatically generates a travel record by combining corresponding photos and descriptions.

[0218] Input: Highlight extraction data.

[0219] Output: An automatically generated trip log.

[0220] What it does: The server uses a text generation API to generate a description of each highlight and formats it along with a photo in HTML or another display format.

[0221] Step 9:

[0222] The server stores the generated travel record in the user's profile and makes it available for the user to view.

[0223] Input: Auto-generated trip log.

[0224] Output: Trip records stored in the user profile.

[0225] What it does: The server associates and stores the trip record data in the user's profile table and provides a link to display it via the app's UI.

[0226] Step 10:

[0227] The server learns the user's preferences and patterns of places visited from the contents of their travel log.

[0228] Input: Saved trip log data.

[0229] Output: User preferences and visiting place patterns.

[0230] Specific operation: The server analyzes the user's past travel history and uses a machine learning algorithm to analyze the categories and frequency of visited places.

[0231] Step 11:

[0232] The generative AI model analyzes rating data from other users to identify highly rated restaurants and tourist spots.

[0233] Input: Rating data from other users.

[0234] Output: A list of highly rated restaurants and attractions.

[0235] How it works: The generative AI model aggregates reviews and rating points from other users and lists highly rated spots.

[0236] Step 12:

[0237] The generative AI model combines user preferences and high-rated points to generate the optimal tour plan as a new travel route.

[0238] Input: User preferences, visiting location patterns, and high rating points.

[0239] Output: The newly generated tour plan.

[0240] How it works: The generative AI model integrates the user's preferred patterns and highly rated spots to create an efficient tour plan that takes into account transportation options and time of day.

[0241] Step 13:

[0242] The server provides the generated travel route to the user.

[0243] Input: The newly generated tour plan.

[0244] Output: The tour plan provided to the user account.

[0245] Specific operation: The server associates the newly generated tour plan with the user's account and notifies the user via the app's notification function.

[0246] Step 14:

[0247] Users can browse tour plans published by other users, click "like" on plans they like, and post comments.

[0248] Input: Published tour itinerary.

[0249] Output: Rating data such as likes and comments.

[0250] What happens: A user opens the app's "Popular Plans" section, browses the list of tour plans published by other users, and enters a rating.

[0251] Step 15:

[0252] The server compiles the evaluation data and updates the popularity rankings in real time.

[0253] Input: Rating data such as likes and comments.

[0254] Output: Updated popularity rankings.

[0255] Specific operation: The server periodically aggregates the evaluation data, calculates the score for each tour plan, and updates the ranking table.

[0256] Step 16:

[0257] The server awards rewards to top-ranked users and notifies them of the results.

[0258] Input: Updated popularity rankings.

[0259] Output: The user account to which the reward was granted.

[0260] Specific operation: The server will add points or vouchers to the top-ranked user accounts and notify the results through the notification system.

[0261] Step 17:

[0262] The server integrates the generated travel route with map data and provides navigation functions.

[0263] Input: Generated travel path data.

[0264] Output: Travel path integrated with map data.

[0265] Specific operation: The server uses a map service (e.g., Google Maps API) to plot the generated route on an interactive map.

[0266] Step 18:

[0267] The device displays the route on a map and provides real-time navigation with voice guidance.

[0268] Input: Travel path integrated with map data.

[0269] Output: Route displayed on map and voice guidance.

[0270] Specific operation: The user starts navigation mode, and the device tracks GPS information in real time and provides voice guidance.

[0271] (Application example 1)

[0272] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0273] Modern travelers take many photos during their trips and want to use them to record their journeys. However, manually organizing photos and creating travel routes and travel diaries after a trip is tedious and time-consuming. Furthermore, there are limited ways to efficiently gather information on new routes, tourist spots, and restaurants for the next trip. Conventional methods make it difficult to easily obtain reviews from other travelers and obtain individually optimized tourist information. Furthermore, there is a lack of means to obtain current location information in real time and efficiently record visited places. To solve these problems, a system is needed that allows travelers to easily record their trips and provides highly personalized tourist information in real time.

[0274] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0275] In this invention, the server includes: means for uploading photos taken by travelers to the server in bulk; means for automatically compiling the travel route of the traveler based on the location information and timestamps of the photos; means for automatically generating a travel diary for the traveler based on the travel route; means for generating a new travel route incorporating highly rated facilities and attractions based on the generated travel diary; means for the user to view and rate the generated travel route; means for generating a popularity ranking based on the ranking and awarding rewards to top-ranked users; means for tracking and recording places visited by the user using a smart device in real time; and means for displaying information about the vicinity of the user's current location on the smart device in real time and providing recommended tourist spots and shops. This allows travelers to automatically record their travels without hassle, obtain useful tourist information in real time, and efficiently plan their next trip by sharing information and rating with other travelers.

[0276] "Photo upload" refers to the operation of sending image data taken by a traveler to a server all at once.

[0277] "Location information" refers to data that indicates the geographic coordinates (latitude and longitude) of the location where a photograph was taken.

[0278] A "timestamp" is information that indicates the date and time a photo was taken.

[0279] "Automatic travel route compilation" refers to the process of automatically calculating and organizing the order of a traveler's visits and routes based on the location information and timestamps of photos.

[0280] A "travel diary" is a record of photographs taken during a trip, with accompanying descriptions, arranged in chronological order.

[0281] "Highly rated facilities and attractions" refers to tourist destinations and stores that have received high ratings based on user ratings and reviews.

[0282] "Generation of new travel routes" refers to the process of creating new recommended travel routes based on existing travel diaries and high-rating information.

[0283] "User ratings" refer to feedback such as "likes" and comments on travel routes and travel diaries shared by other users.

[0284] "Popularity ranking" refers to a list of travel routes and travel diaries ranked based on user ratings.

[0285] "Giving rewards" means providing rewards such as benefits or points to those who rank highly.

[0286] "Smart device" refers to a portable electronic device that can provide and display digital information in real time, such as a smartphone, smart glasses, or head-mounted display.

[0287] "Place tracking" refers to the process of recording the places visited by a user in real time using a smart device.

[0288] "Displaying surrounding information" means displaying information about nearby tourist attractions and stores on the smart device display based on the current location.

[0289] "Recommended tourist attractions and stores" refer to tourist attractions and stores that are attractive to users and selected by the generative AI model.

[0290] This invention realizes a system that automatically generates travel diaries based on photos taken by travelers and provides new tourist information. The system is composed of the following elements.

[0291] Bulk upload and analysis of photos

[0292] The server provides a means for travelers to upload photos taken in bulk. After their trip, travelers open a dedicated app, select all photos taken during their trip, and upload them. The device extracts the location information (GPS data) and timestamp (date and time of photo) of the selected photos, and sends the photo files, along with their location information and timestamp, to the server. This reduces the user's workload and enables efficient data transmission.

[0293] Automatic calculation of travel route

[0294] The server analyzes the received photo data and checks the location information and timestamp. This allows it to calculate the traveler's route in chronological order. For analysis, Python and TensorFlow are used, for example, to process the data based on the location information and timestamp. The calculated route is then stored in a database. This automatically manages a detailed record of the places the traveler has visited.

[0295] Automatic generation of travel diary

[0296] The server extracts highlights and events from the trip based on the calculated travel route. A generative AI model is used to extract the highlights and generate detailed descriptions for each point. This automatically generates a travel diary that combines photos and descriptions. The generated travel diary is saved in the user's profile and can be viewed by the user.

[0297] Real-time tourist information

[0298] The server uses the generated travel diary content and rating data from other users to identify highly rated facilities and attractions. Using a generative AI model, the server analyzes the user's preferences and ratings from other users to generate new travel routes. This allows travelers to receive optimal sightseeing information for their next trip in real time. Additionally, if the user is using a smart device (smartphone, smart glasses, etc.), the terminal displays real-time information about the area around the current location and provides recommended tourist spots and stores.

[0299] User ratings and rankings

[0300] The server provides a means for users to view and rate publicly posted travel routes and travel diaries. Users can "like" tour plans they find good or post comments, and rating points are tallied in real time. The server then generates popularity rankings based on the ratings and operates a system that awards rewards to those at the top of the rankings. This promotes the sharing of travel information and the activation of the community.

[0301] Specific examples

[0302] Visiting tourist attractions: When a user visits Mount Fuji or a temple, the route traveled is automatically recorded, and the travel diary generated after the trip can be viewed on the smart glasses.

[0303] Restaurant guide: An example in which AI provides real-time guidance on highly rated restaurants near tourist spots and displays it on the smart glasses.

[0304] Prompt Sentence Examples

[0305] "When a user visits a particular tourist spot, the smart glasses will display the history and background of that place in real time, record the data of the places visited, and automatically generate a travel diary. In addition, the glasses will provide personalized recommendations based on other users' ratings."

[0306] In this way, the specific embodiment of the present invention allows travelers to efficiently keep a record of their trip and obtain useful tourist information in real time.

[0307] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0308] Step 1:

[0309] Bulk upload photos

[0310] Input: Photos taken by the user during their trip

[0311] The server provides a means for users to select photos taken during their trip in bulk through a dedicated app and upload them all to the server. The device extracts the location information (GPS data) and timestamp (date and time of photo shooting) of the selected photos and sends them to the server along with the photo files. This streamlines the upload process and aggregates photo data on the server.

[0312] Output: Photo data uploaded to the server

[0313] Step 2:

[0314] Automatic calculation of travel route

[0315] Input: Uploaded photo data (including location and timestamp)

[0316] The server analyzes the location information and timestamps of the received photo data and sorts them in chronological order. It then automatically compiles the traveler's route based on the location information and timestamps. Specifically, it calculates the distance between each photo position using Python's Geopy library and plots the photo data as a continuous route.

[0317] Output: Calculated travel route

[0318] Step 3:

[0319] Automatic generation of travel diary

[0320] Input: Calculated travel route

[0321] The server extracts highlights and events from the trip based on the calculated route. It uses a generative AI model to generate detailed descriptions for each point. It then combines the photos and descriptions to automatically generate a travel diary. The generated travel diary is saved in the user's profile and can be viewed later.

[0322] Output: Automatically generated travel diary

[0323] Step 4:

[0324] Real-time tourist information

[0325] Input: User's current location information, other users' rating data

[0326] The server identifies highly rated facilities and tourist attractions based on the user's current location information and rating data from other users. Using a generative AI model, it generates new sightseeing routes by analyzing the user's preferences and ratings from other users. The terminal also displays information about the area around the user's current location in real time via the smart device, providing guidance to recommended tourist spots and stores.

[0327] Output: Real-time generated tourist information and new travel routes

[0328] Step 5:

[0329] User ratings and rankings

[0330] Input: Travel routes and travel diaries published by other users, user feedback data

[0331] The server provides a means for users to view and rate publicly posted travel routes and travel diaries. Users can "like" routes they find good and post comments. The server tally evaluation points in real time and generate popularity rankings. It also operates a system that awards rewards to those at the top of the rankings.

[0332] Output: Generated ranking and reward information

[0333] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0334] MODE FOR CARRYING OUT THE INVENTION

[0335] System Overview

[0336] This system automatically compiles travel routes and generates travel diaries based on photos taken by travelers after their trip. It also utilizes generative AI to propose new tour plans, allowing users to rate and share them. It also combines an emotion engine that recognizes users' emotions to provide a more personalized travel experience.

[0337] Bulk photo posting

[0338] 1. After the trip, the user opens the dedicated app and selects all the photos taken during the trip.

[0339] 2. The device extracts the location information (GPS data) and timestamp (date and time of shooting) of the selected photo.

[0340] 3. The device uploads the photo file, its location information, and timestamp to the server.

[0341] Specific examples

[0342] The app allows users to select all photos they have taken during their trip to Europe and upload them to the server.

[0343] When uploading, the device automatically extracts the GPS data and shooting date and time contained in the photo and sends them to the server.

[0344] Automatic calculation of travel route

[0345] 1. The server analyzes the received photo data and checks the location information and timestamp.

[0346] 2. The server calculates the user's route in chronological order based on the location information and timestamp.

[0347] 3. The server plots the calculated travel route on map data to create a visual route.

[0348] 4. The server stores the calculated route data in a database.

[0349] Specific examples

[0350] The server analyzes the user's itinerary and plots the route traveled, for example, from France to Italy, on a map.

[0351] Based on the location information, each photographed point is connected in a continuous manner to form a route.

[0352] Travel diary generation

[0353] 1. The server extracts highlights and events from the trip based on the saved travel route data.

[0354] 2. The server combines the extracted highlights and photos to automatically generate a travel diary with explanatory text.

[0355] 3. The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[0356] Specific examples

[0357] The server automatically selects highlights such as the Eiffel Tower and the Colosseum and generates detailed descriptions.

[0358] Travel diaries are saved in a user's profile, allowing them to track their entire journey in chronological order.

[0359] Creating a new tour plan

[0360] 1. The server learns the user's preferences and patterns of places visited from the contents of the user's travel diary.

[0361] 2. The generating AI analyzes other users' rating data and identifies highly rated restaurants and tourist attractions.

[0362] 3. The generation AI combines the user's preferences and high rating points to generate a new travel route.

[0363] 4. The server saves the generated new travel route in the user's profile and makes it available for the user to view.

[0364] Specific examples

[0365] Generative AI learns the user's preferences and suggests new tour plans that incorporate highly rated restaurants and tourist attractions.

[0366] For example, if the user loves pasta, add a popular Italian restaurant to the route.

[0367] Optimization by Emotion Engine

[0368] 1. The device uses an emotion engine to analyze the user's emotional state from photo and audio data.

[0369] 2. The emotion engine optimizes travel routes and travel diaries based on the user's emotional state (joy, surprise, sadness, etc.).

[0370] 3. The emotion engine makes suggestions based on the user's current emotions.

[0371] Specific examples

[0372] The emotion engine analyzes the user's travel photos and comments, and if it finds that they are particularly happy at a particular place, it will suggest a new travel plan that includes that place.

[0373] For example, if the user has many photos of themselves having fun at Huis Ten Bosch, a tour plan will be generated that includes other tourist spots nearby.

[0374] User ratings and popularity rankings

[0375] 1. Users view tour plans published by other users.

[0376] 2. Users can "like" tour plans they find good or post comments.

[0377] 3. The server will tally the evaluation points for each tour plan and update the rankings in real time.

[0378] 4. The server will award rewards to the top-ranked tour plan contributors and notify them of the results.

[0379] Specific examples

[0380] A user browses Tokyo tour plans published by other users and "likes" them.

[0381] The server tallys likes and comments in real time and updates the popularity rankings.

[0382] Map display and navigation

[0383] 1. The server integrates the generated travel route with map data.

[0384] 2. The device displays the route on a map and prompts the user to begin navigation.

[0385] 3. The device starts voice guidance to support the user's movement.

[0386] Specific examples

[0387] The device displays the route of the new tour plan on the user's smartphone and begins navigation in real time.

[0388] Voice guidance will explain how to reach your destination and where to stop.

[0389] In this way, a system incorporating an emotion engine allows travelers to receive more personalized travel plans, providing an optimal travel experience based on the user's emotions and preferences.

[0390] The processing flow will be explained below.

[0391] MODE FOR CARRYING OUT THE INVENTION

[0392] Bulk photo posting

[0393] Step 1:

[0394] After traveling, users open a dedicated app and select all the photos they want to post at once.

[0395] Step 2:

[0396] The terminal displays a list of the selected photos and asks the user for confirmation.

[0397] Step 3:

[0398] The device extracts the location information (GPS data) and timestamp (date and time of photo capture) for each selected photo.

[0399] Step 4:

[0400] The device sends the photo file, its location information, and a timestamp to the server.

[0401] Automatic calculation of travel route

[0402] Step 5:

[0403] The server analyzes the received photo data and checks the location information and timestamp.

[0404] Step 6:

[0405] The server sorts the photos in chronological order based on the location information and timestamps, and calculates the user's travel route.

[0406] Step 7:

[0407] The server plots the calculated travel route on map data to create a visual route.

[0408] Step 8:

[0409] The server stores the calculated travel route data in a database.

[0410] Travel diary generation

[0411] Step 9:

[0412] The server extracts highlights and events from the trip based on the stored travel route data.

[0413] Step 10:

[0414] The server combines the extracted highlights and photos to automatically generate a travel diary with explanatory text.

[0415] Step 11:

[0416] The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[0417] Creating a new tour plan

[0418] Step 12:

[0419] The server learns the user's preferences and patterns of places visited from the contents of the user's travel diary.

[0420] Step 13:

[0421] The generative AI analyzes other users' rating data to identify highly rated restaurants and tourist attractions.

[0422] Step 14:

[0423] The generation AI combines user preferences and high rating points to generate new travel routes.

[0424] Step 15:

[0425] The server saves the generated new travel route in the user's profile and makes it available for the user to view.

[0426] Optimization by Emotion Engine

[0427] Step 16:

[0428] The device uses an emotion engine to analyze the user's emotional state from photo and audio data.

[0429] Step 17:

[0430] The emotion engine optimizes travel routes and travel diaries based on the user's emotional state (joy, surprise, sadness, etc.).

[0431] Step 18:

[0432] The emotion engine selects new travel routes and travel highlights based on the user's current emotions and reflects them in the travel diary.

[0433] Step 19:

[0434] The server generates a new optimized travel route based on the user's emotions and saves it in the profile.

[0435] User ratings and popularity rankings

[0436] Step 20:

[0437] A user can view tour plans published by other users.

[0438] Step 21:

[0439] Users can "like" tour plans that they find good and post comments.

[0440] Step 22:

[0441] The server tally up the evaluation points for each tour plan and update the rankings in real time.

[0442] Step 23:

[0443] The server gives rewards to the top ranked tour plan posters and notifies them of the results.

[0444] Map display and navigation

[0445] Step 24:

[0446] The server integrates the generated travel route with map data.

[0447] Step 25:

[0448] The device displays the route on a map and prompts the user to begin navigation.

[0449] Step 26:

[0450] The terminal starts voice guidance to support the user's movement.

[0451] Step 27:

[0452] The user uses the navigation system to enjoy sightseeing and eating and drinking along the generated travel route.

[0453] The above is the specific program processing flow of the system that combines the emotion engine. Each step incorporates a detailed process that includes the specific operations of the user, terminal, and server.

[0454] Example 2

[0455] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0456] In recent years, there has been a growing demand for creating travel records based on photos taken during travel and sharing travel experiences. However, organizing photos, compiling route records, and creating travel diaries is time-consuming. Automating each of these processes would enable users to create and share travel records efficiently. However, current systems do not provide a means to process these processes in an integrated manner. Furthermore, it is difficult to provide personalized services that combine travel plan suggestions, evaluations, and rankings based on users' preferences and emotions. Therefore, there is a need to enable efficient creation and sharing of travel records based on travelers' photos, and to provide an optimal travel experience that suits users' emotions and preferences.

[0457] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0458] In this invention, the server includes: means for uploading photos taken by travelers to the server in bulk using a terminal; means for the terminal to extract location information and timestamps of the photos and for the server to analyze them; means for automatically compiling the traveler's travel route from the location information and timestamps of the photos; means for plotting the travel route on map data and creating a visual route; means for automatically generating the traveler's travel diary based on the travel route; means for learning based on the user's preferences and visiting trends and using a generation AI to generate a new travel route combining highly rated restaurants and tourist attractions; means for the user to view, rate, and share the generated travel route and travel diary; means for generating a popularity ranking based on the rating and awarding rewards to the top ranked users; means for analyzing the user's emotional state and optimizing the travel route and travel diary based on the emotion; and means for the terminal to display the generated travel route on a map and provide navigation. This enables efficient creation and sharing of travel records based on photos taken by travelers and further enables providing an optimal travel experience tailored to the user's emotions and preferences.

[0459] A "traveler" is an individual who takes photographs while traveling and intends to create a travel record using the photographic data.

[0460] A "terminal" is an electronic device used by a user, such as a mobile information terminal, smartphone, or tablet, which collects, analyzes, and uploads captured photo data to a server.

[0461] A "server" is a computer system that receives, analyzes, stores, and processes data sent from travelers' terminals as necessary.

[0462] "Location information" is information about the location where a photo was taken, and is geographical data recorded in the form of GPS data or the like.

[0463] A "timestamp" is data that indicates the date and time a photo was taken, and is stored as metadata for the photo.

[0464] "Travel route" is data showing the route traveled by a traveler during a trip, and is arranged in chronological order based on location information and timestamps.

[0465] "Map data" means geographical map information provided by a geographic information system (GIS) and is data used to visually display location information and travel routes.

[0466] A "travel diary" is an automatically generated travel record that integrates a traveler's route, highlights, photos, and descriptions.

[0467] "Generative AI" refers to artificial intelligence that learns from large amounts of data and generates new information and suggestions.

[0468] A "new travel route" is a recommended route for the next trip created by the generation AI based on the user's preferences and evaluation data from other users.

[0469] "User preferences" refers to the personal tastes and interests that a traveler has based on past travel and rating data.

[0470] "Top-rated restaurants and attractions" refers to restaurants and attractions that have been highly rated by other travelers.

[0471] "Evaluation" refers to evaluation points that a user gives to a tour plan published by another user by giving it a "like" or comment.

[0472] The "popularity ranking" is a ranking list generated based on the evaluation points obtained by each tour plan, and indicates the order of the most popular tour plans.

[0473] "Emotional state" refers to the emotions such as joy, surprise, or sadness that a traveler feels in response to a particular scene or photo, and is data estimated through emotion analysis.

[0474] "Navigation" is a function that provides audio and visual guidance to help a user reach a destination using a terminal.

[0475] MODE FOR CARRYING OUT THE INVENTION

[0476] This invention is a system that automatically creates a travel log based on photos taken by a traveler and proposes a travel plan that matches the user's preferences and feelings. Specific embodiments of this system will be described below.

[0477] System configuration

[0478] Users use a mobile information terminal (hereinafter referred to as "terminal") such as a smartphone or tablet to select all photos taken during their trip and upload them through a dedicated app. The terminal extracts the location information (GPS data) and timestamp (date and time of photo) embedded in the photos and sends this data to a server.

[0479] The server analyzes the received photo data and calculates the user's travel route in chronological order based on the location information and timestamp. The calculated travel route is plotted on map data, visually representing the route. This makes the user's travel route clear.

[0480] The server also extracts highlights and events from the saved travel route data and automatically generates a travel diary with explanatory text based on the extracted information. The generated travel diary is saved in the user's profile and can be viewed by the user at any time.

[0481] Creating new tour plans and optimizing them with an emotion engine

[0482] The server learns the user's preferences and visiting place patterns from the contents of the user's travel diary, and then analyzes other users' rating data using a generative AI model. From this analysis, it identifies highly rated restaurants and tourist attractions, and proposes new travel routes by combining the user's preferences with these highly rated points. The new tour plan is saved in the user's profile.

[0483] Furthermore, the device uses an emotion engine to analyze the user's emotional state from photos and voice data. The emotion engine evaluates the degree to which the user felt emotions such as joy or surprise at a particular location, and optimizes travel routes and travel diaries based on this. For example, if the device determines that the user was particularly pleased at a particular tourist spot, it will suggest a new tour plan that includes that tourist spot.

[0484] User ratings and popularity rankings

[0485] Users can browse tour plans published by other users, click "like" on the ones they like, and post comments. The server tally up the evaluation points of each tour plan and generate a popularity ranking in real time. Tour plan posters who rank highly are awarded rewards.

[0486] Navigation Features

[0487] The terminal displays the travel route generated by the server on a map and prompts the user to start navigation. The terminal then starts voice guidance and provides directions to reach the destination specified by the user. For example, the terminal displays the route of a new tour plan in real time and provides voice guidance on the way to the destination.

[0488] Examples and prompts

[0489] Example 1: A user selects photos taken during a trip to Europe and uploads them to a server. When uploading, the device automatically extracts the GPS data and the date and time of the photo and sends them to the server.

[0490] Example 2: The server analyzes the user's itinerary and plots the route, for example, from France to Italy, on a map. Based on the location information, each photographed point is connected in a continuous manner to form a route.

[0491] Example prompt: "How can I automatically compile travel routes and generate a travel diary based on travel photos taken by the user?"

[0492] This system allows travelers to efficiently create and share travel records with other users, and also allows them to enjoy personalized travel plans.

[0493] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0494] Step 1:

[0495] The user selects a photo

[0496] Input: Travel photos taken by the user

[0497] Output: A list of selected photo files

[0498] Specific operation: The user opens the dedicated app and selects all the photos taken during the trip at once. The app selects multiple photos from the gallery and adds them to a list for uploading at once.

[0499] Step 2:

[0500] The device extracts the location and timestamp

[0501] Input: A list of photo files selected by the user

[0502] Output: Location information (GPS data) and timestamp (date and time of photo) for each photo

[0503] What it does: Your device reads the GPS data and timestamp from the photo's metadata and stores this information in temporary memory.

[0504] Step 3:

[0505] The device uploads data to the server

[0506] Input: Location and timestamp of each photo, photo file

[0507] Output: Photo data and metadata sent to the server

[0508] Specific operation: The device sends an HTTP request to the server, sending the photo data and associated metadata. It receives a success response from the server and notifies the user of the upload status.

[0509] Step 4:

[0510] The server analyzes the photo data

[0511] Input: Photo data and metadata uploaded to the server

[0512] Output: Chronologically ordered locations and timestamps

[0513] What it does: The server retrieves the photo data stored in the database and analyzes the metadata of each photo to check the location and timestamp.

[0514] Step 5:

[0515] The server calculates the route.

[0516] Input: Organized location and timestamp

[0517] Output: Calculated travel route

[0518] Specific operation: The server sorts the location information based on time series, and uses a specific algorithm to connect consecutive points to form the user's travel route.

[0519] Step 6:

[0520] The server plots the map data

[0521] Input: Calculated travel route

[0522] Output: Plotted map data

[0523] Specific operation: The server calls a map API (e.g., Google Maps API) to plot the route and reflects the plotted data in the user interface.

[0524] Step 7:

[0525] The server stores the travel route data

[0526] Input: Plotted map data and travel route information

[0527] Output: Travel route data stored in a database

[0528] Specific operation: The server inserts the travel route data into a specific table in the database and checks whether the insertion is successful.

[0529] Step 8:

[0530] The server extracts the highlights

[0531] Input: Travel route data

[0532] Output: Extracted highlight points and events

[0533] Specific operation: The server runs a highlight extraction algorithm to derive and list notable points.

[0534] Step 9:

[0535] The server automatically generates a travel diary

[0536] Input: Extracted highlight locations and events, corresponding photo data

[0537] Output: Automatically generated travel diary

[0538] Specific operation: The server uses a text generation model based on the highlight points to combine appropriate photos and descriptions to create a travel diary.

[0539] Step 10:

[0540] The server stores the travel diary and provides it to the user.

[0541] Input: Auto-generated travel diary

[0542] Output: Travel diary saved in the user's profile

[0543] Specific operation: The server saves the travel diary data in the database and then reflects it on the user's profile page.

[0544] Step 11:

[0545] The server learns the user's preferences

[0546] Input: User's travel diary data, rating data

[0547] Output: Learned user preferences and visiting habits

[0548] What it does: The server uses machine learning algorithms to analyze your behavioral patterns and add your preferences to your profile.

[0549] Step 12:

[0550] Generative AI analyzes the evaluation data

[0551] Input: Other users' ratings data

[0552] Output: A list of highly rated restaurants and attractions

[0553] What it does: The generative AI queries user rating data, performs statistical analysis, and lists specific places and services.

[0554] Step 13:

[0555] Generation AI generates new travel routes

[0556] Input: User preferences, likes

[0557] Output: Newly generated travel route

[0558] Specific operation: The generation AI runs a route generation algorithm based on preferences and high rating points, then verifies and optimizes the generated route.

[0559] Step 14:

[0560] The server stores the route and serves it to the user.

[0561] Input: New tour plan

[0562] Output: A new tour plan saved in the user's profile.

[0563] Specific behavior: The server saves the new tour plan in the database and displays the saved tour plan in the user interface.

[0564] Step 15:

[0565] The device performs emotion analysis

[0566] Input: Photo and audio data

[0567] Output: Emotion analysis results

[0568] Specific operation: The device sends photo and audio data to the emotion analysis module, receives and logs the emotion analysis results.

[0569] Step 16:

[0570] Emotion engine optimizes routes and travel diaries

[0571] Input: Sentiment analysis results

[0572] Output: Optimized travel route and travel diary

[0573] What it does: The emotion engine reevaluates highlights, routes, and descriptions based on emotion data and makes adjustments as needed.

[0574] Step 17:

[0575] Emotion engine makes suggestions

[0576] Input: Sentiment analysis results

[0577] Output: Improved tour plan

[0578] Specific behavior: The emotion engine generates a customized tour plan and notifies the user of the new tour plan.

[0579] Step 18:

[0580] A user browses tour plans

[0581] Input: Published tour plan

[0582] Output: Rating and comments

[0583] Specific operation: Users can view tour plans published by other users, select the tour plan they want to view, and rate and comment on it.

[0584] Step 19:

[0585] The server updates the rankings

[0586] Input: Reputation points

[0587] Output: Updated ranking list

[0588] Specific operation: The server retrieves reputation points from the database, aggregates them, and updates the ranking list.

[0589] Step 20:

[0590] The server will grant the reward

[0591] Input: Updated ranking list

[0592] Output: Reward granted notification

[0593] Specific operation: The server inserts reward data into the top ranking users and notifies them of the reward via the notification system.

[0594] Step 21:

[0595] The device displays the route

[0596] Input: Route data from the server

[0597] Output: Route map displayed on the terminal

[0598] Specific operation: The device displays map data on the screen and prompts the user to start navigation.

[0599] Step 22:

[0600] The device will provide voice guidance

[0601] Input: Current location data and route information

[0602] Output: Voice navigation

[0603] Specific operation: The device determines the current location based on GPS data and provides voice guidance to the destination.

[0604] (Application example 2)

[0605] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0606] Currently, there is a lack of means to automatically generate travel records and effectively manage photos and routes after a trip. Furthermore, there is no system that can suggest new routes and services that reflect the traveler's feelings and preferences. This makes the process of recording experiences during and after a trip cumbersome, making it difficult to provide optimal services to travelers.

[0607] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for uploading photos taken by travelers to the data processing device in bulk; means for automatically aggregating the traveler's travel route from the location information and timestamps of the photos; means for automatically generating the travel record of the traveler based on the travel route; means for generating a new travel route incorporating highly rated restaurants and tourist attractions based on the generated travel log; means for users to view and rate the generated travel route; means for generating a popularity index based on the evaluation and awarding rewards to those with the highest index; means for displaying the photos on a map based on the locations where they were taken and visualizing the visit route; and means for analyzing the user's emotional state from the photo data and proposing new services based on the analysis. This allows travelers to automatically generate and manage their travel routes and travel records, and further enables optimal service suggestions based on their emotions during the trip.

[0608] "Photos" are image data taken by travelers during their trip.

[0609] "Data Processing Device" means a combination of hardware and software that allows travelers to upload photos taken in bulk.

[0610] "Location information" is the geographic coordinate data where the photo was taken.

[0611] A "timestamp" is data that indicates the date and time a photo was taken.

[0612] A "travel route" is a traveler's travel history calculated based on location information and timestamps.

[0613] A "travel log" is a document automatically generated based on a traveler's travel route and photographs.

[0614] "Food and beverage establishments" refers to restaurants and bars along the travel route of travelers.

[0615] "Tourist destination" refers to a tourist attraction that a tourist visits.

[0616] "User" refers to a traveler or user of the system.

[0617] "Evaluation" refers to feedback provided by users regarding generated travel routes and services.

[0618] "Popularity index" is ranking data compiled from evaluation points.

[0619] "Rewards" are perks and incentives given to those who rank highly in popularity indicators.

[0620] A "visit route" is a route on a map that connects the places visited by a traveler in order.

[0621] "Emotional state" refers to the emotional state of a traveler analyzed from photo data.

[0622] "New services" refers to new services and benefits that are proposed based on the analyzed emotional state.

[0623] MODE FOR CARRYING OUT THE INVENTION

[0624] The system for implementing this invention automatically compiles travel routes based on photos taken by travelers during their trips, generates travel records, and utilizes AI to propose new services based on the travelers' emotional state.

[0625] 1. System Program Description

[0626] Hardware and software used:

[0627] Hardware:

[0628] Smartphones (for travelers to take and upload photos)

[0629] Servers (for data processing and storage)

[0630] Databases (to store traveller data and travel records)

[0631] software:

[0632] Pillow (a Python library for extracting photo metadata)

[0633] geopy (a Python library for analyzing geolocation data)

[0634] Basemap (a Python library for displaying travel routes on a map)

[0635] Emotion Engine API (external service for analyzing emotions from photos)

[0636] 2. System operation explanation

[0637] User Action:

[0638] 1. The user uses a dedicated app on their smartphone to select and upload all the photos they took during their trip.

[0639] 2. Automatically extract metadata (location and timestamp) from uploaded photos and send it to the server.

[0640] Server Action:

[0641] 1. The server analyzes the received photo data, checks the location information and timestamp, and calculates the user's movement route in chronological order based on this.

[0642] 2. The calculated travel path is plotted on map data to generate a visual route, which is then used to automatically generate a travel record for the traveler.

[0643] Sentiment analysis and service suggestions:

[0644] 1. The server analyzes the user's emotional state from the photo data using an emotion engine.

[0645] 2. Based on the analysis results, new services are proposed according to the user's emotions. For example, the system suggests nearby tourist spots and restaurants based on the places where the user feels particularly happy.

[0646] Specific examples:

[0647] 1. A tourist visits Paris, takes a photo, and uploads it. The photo includes location information such as the Eiffel Tower and the Louvre.

[0648] 2. The server uses an emotion engine to analyze the photos and determine what the traveler is enjoying, and recommends nearby cafes and museums as the next travel destination.

[0649] Example prompt sentence:

[0650] Suggest new services and places to visit based on the location and emotion data of places the user has visited. The user would like to see new places and services offered near areas where they have experienced particularly high levels of joy, such as:

[0651] 1. Highly rated cafes and restaurants

[0652] 2. Popular tourist destinations

[0653] 3. Promotions and offers available

[0654] This system allows travelers to automatically generate and manage their travel routes and travel records, and also allows them to receive recommendations for optimal services based on their emotions while traveling.

[0655] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0656] Step 1:

[0657] Users use a dedicated smartphone app to select and upload multiple photos taken during their trip. The input is the photo files selected by the user, and the output is the selected photo files being sent to the server.

[0658] Step 2:

[0659] The device automatically extracts location information (GPS data) and timestamp (date and time of photo) from uploaded photo files. The input is the photo file, and the output is metadata including location information and timestamp. This metadata is sent to the server along with the photo file.

[0660] Step 3:

[0661] The server analyzes the received photo data and checks the location information and timestamp for each photo. The input is the uploaded photo data and its metadata, and the output is a list of the location information and timestamp corresponding to each photo data.

[0662] Step 4:

[0663] The server calculates the user's route in chronological order based on the location information and timestamps. The input is a list of location information and timestamps, and the output is the user's route data sorted in chronological order. This is plotted on map data to generate a visual route.

[0664] Step 5:

[0665] The server automatically generates a travel record for the traveler based on the generated travel route. The input is travel route data, and the output is text and image data of the travel record.

[0666] Step 6:

[0667] The server sends the photo data to the emotion engine API to analyze the user's emotional state. The input is the photo data and its metadata, and the output is the emotion analysis result, which is a specific emotion (e.g., joy, surprise, sadness) that reflects the user's travel experience.

[0668] Step 7:

[0669] The server proposes new services and travel routes based on the emotion analysis results. The input is the emotion analysis results and travel records, and the output is a list of recommended new services and travel routes.

[0670] Step 8:

[0671] The terminal notifies the user of new services and travel routes provided by the server and allows the user to view them. The input is a recommendation list sent from the server, and the output is recommendation information displayed on the user's smartphone.

[0672] Step 9:

[0673] Users rate new services and travel routes provided and provide feedback. The input is the user's ratings and comments, and the output is the rating data sent to the server.

[0674] Step 10:

[0675] The server aggregates user rating data, generates a popularity index, updates rankings based on that index, and awards rewards to top users. The input is user rating data, and the output is updated rankings and reward notifications.

[0676] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0677] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0678] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0679] [Second embodiment]

[0680] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0681] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0682] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0683] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0684] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0685] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0686] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0687] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0688] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0689] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0690] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0691] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0692] MODE FOR CARRYING OUT THE INVENTION

[0693] System Overview

[0694] This system automatically compiles travel routes and generates travel diaries based on photos taken by travelers after their trip. It also uses AI to propose new tour plans and provides a function that allows users to rate and share them.

[0695] Bulk photo posting

[0696] 1. After the trip, the user opens the dedicated app and selects all the photos taken during the trip.

[0697] 2. The device extracts the location information (GPS data) and timestamp (date and time of shooting) of the selected photo.

[0698] 3. The device uploads the photo file, its location information, and timestamp to the server.

[0699] Specific examples

[0700] Users select photos they have taken around Mount Fuji in the app and upload them to the server all at once.

[0701] When uploading, the device automatically extracts the GPS data and shooting date and time contained in the photo and sends them to the server.

[0702] Automatic calculation of travel route

[0703] 1. The server analyzes the received photo data and checks the location information and timestamp.

[0704] 2. The server calculates the user's route in chronological order based on the location information and timestamp.

[0705] 3. The server stores the calculated route in a database.

[0706] Specific examples

[0707] The server analyzes the user's itinerary and plots the route from the base of Mount Fuji to the summit on a map.

[0708] Based on the location information, each photographed point is connected in a continuous manner to form a route.

[0709] Travel diary generation

[0710] 1. The server extracts highlights and events from the calculated travel route.

[0711] 2. The server automatically generates a travel diary based on the extracted highlights, combining photos and descriptions.

[0712] 3. The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[0713] Specific examples

[0714] The server automatically inserts photos of summits and particularly beautiful views into the travel diary as highlights and generates detailed descriptions.

[0715] The travel diary is saved in the user's profile, allowing them to follow the overall picture of their Mount Fuji climb chronologically.

[0716] Creating a new tour plan

[0717] 1. The server learns the user's preferences and patterns of places visited from the contents of their travel diary.

[0718] 2. The generating AI analyzes rating data from other users and identifies highly rated restaurants and tourist attractions.

[0719] 3. The generation AI combines the user's preferences and high rating points to generate the optimal travel route as a new tour plan.

[0720] 4. The server provides the newly generated travel route to the user.

[0721] Specific examples

[0722] The generative AI learns the user's preferences and suggests new tour plans that incorporate other tourist attractions and restaurants.

[0723] For example, if a user likes the view from a mountaintop, they can add other scenic spots.

[0724] User ratings and popularity rankings

[0725] 1. Users view tour plans published by other users.

[0726] 2. Users can "like" tour plans they find good or post comments.

[0727] 3. The server will tally the evaluation points for each tour plan and update the rankings in real time.

[0728] 4. The server awards rewards to the top-ranked users and notifies them of the results.

[0729] Specific examples

[0730] Users view Yamanashi Prefecture tour plans published by other users and "like" them.

[0731] The server tallys likes and comments in real time and updates the popularity rankings.

[0732] Map display and navigation

[0733] 1. The server integrates the generated travel route with map data.

[0734] 2. The device will display the route on a map and enter navigation mode.

[0735] 3. The device provides voice guidance to support the user's movement.

[0736] Specific examples

[0737] The device displays the route of the new tour plan on the user's smartphone and begins navigation in real time.

[0738] Voice guidance will explain how to reach your destination and where to stop.

[0739] Such a system allows travelers to effortlessly record their trips, get travel plans tailored to their individual preferences, and improve the quality of their travel experience by sharing information and ratings with other travelers.

[0740] The processing flow will be explained below.

[0741] Program processing flow

[0742] Bulk photo posting

[0743] Step 1:

[0744] After traveling, users open a dedicated app and select all the photos they want to post at once.

[0745] Step 2:

[0746] The terminal displays a list of the selected photos and asks the user for confirmation.

[0747] Step 3:

[0748] The device extracts the location information (GPS data) and timestamp (date and time of photo capture) for each selected photo.

[0749] Step 4:

[0750] The device sends the photo file, its location information, and a timestamp to the server.

[0751] Automatic calculation of travel route

[0752] Step 5:

[0753] The server analyzes the received photo data and checks the location information and timestamp.

[0754] Step 6:

[0755] The server sorts the photos in chronological order based on the location information and timestamps, and calculates the user's travel route.

[0756] Step 7:

[0757] The server plots the calculated travel route on map data to create a visual route.

[0758] Step 8:

[0759] The server stores the calculated travel route data in a database.

[0760] Travel diary generation

[0761] Step 9:

[0762] The server extracts highlights and events from the trip based on the stored travel route data.

[0763] Step 10:

[0764] The server combines the extracted highlights and photos to automatically generate a travel diary with explanatory text.

[0765] Step 11:

[0766] The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[0767] Creating a new tour plan

[0768] Step 12:

[0769] The server learns the user's preferences and patterns of places visited from the contents of the user's travel diary.

[0770] Step 13:

[0771] The generative AI analyzes rating data from other users to identify highly rated restaurants and tourist attractions.

[0772] Step 14:

[0773] The generation AI combines user preferences and high rating points to generate new travel routes.

[0774] Step 15:

[0775] The server saves the generated new travel route in the user's profile and makes it available for the user to view.

[0776] User ratings and popularity rankings

[0777] Step 16:

[0778] A user can view tour plans published by other users.

[0779] Step 17:

[0780] Users can "like" tour plans they find good and post comments.

[0781] Step 18:

[0782] The server tally up the evaluation points for each tour plan and update the rankings in real time.

[0783] Step 19:

[0784] The server gives rewards to the top ranked tour plan posters and notifies them of the results.

[0785] Map display and navigation

[0786] Step 20:

[0787] The server integrates the generated travel route with map data.

[0788] Step 21:

[0789] The device displays the route on a map and prompts the user to begin navigation.

[0790] Step 22:

[0791] The terminal starts voice guidance to support the user's movement.

[0792] Step 23:

[0793] The user uses the navigation system to enjoy sightseeing and eating and drinking along the generated travel route.

[0794] The above is the specific flow of program processing for this system. Each step incorporates a detailed process that includes the specific operations of the user, terminal, and server.

[0795] Example 1

[0796] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0797] There is a need for a system that can efficiently create detailed travel records based on photos taken by travelers, suggest new travel plans, and share information and ratings with other travelers in real time. However, conventional systems require the time-consuming task of manually classifying and organizing photos, making it difficult to automatically generate detailed travel records including trip highlights. It is also difficult to automatically generate new travel plans tailored to individual preferences and provide navigation functions integrated with map data.

[0798] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0799] In this invention, the server includes a means for uploading image data taken by a traveler to the server in bulk, a means for automatically tallying up the travel route of the traveler from the location information and timestamp of the image data, and a means for automatically generating a travel record of the traveler based on the travel route. This makes it possible to effectively utilize the photos taken by the traveler and automatically generate a detailed travel record based on a chronological order.

[0800] "Image data" refers to digital data of photographs and videos taken by travelers.

[0801] "Server" refers to a computer system for receiving and processing data from terminals connected over a network.

[0802] "Location Information" refers to geographic latitude and longitude data obtained using technologies such as GPS.

[0803] A "timestamp" refers to date and time information that indicates when digital data was created, modified, or accessed at a certain time.

[0804] A "travel route" refers to a route that connects multiple locations visited by a traveler within a specific period in chronological order.

[0805] A "travel journal" is a record of the places and events visited during a trip, compiled in photographs and text.

[0806] A "generative AI model" refers to an artificial intelligence algorithm that uses machine learning techniques to generate new data and plans.

[0807] "Restaurant" refers to a commercial establishment that sells food and beverages and allows customers to eat and drink.

[0808] "Tourist attractions" refer to popular tourist destinations and historical and cultural sites.

[0809] "Smart devices" refer to portable electronic devices that can connect to the Internet, such as smartphones and tablets.

[0810] "Navigation" refers to a system and its functions that indicate the optimal route from the current location to the destination.

[0811] "Evaluation" refers to the act of rating a tour plan or service created by another user after using the service.

[0812] "Popularity ranking" refers to a ranking list created by compiling feedback such as user ratings and "likes."

[0813] "Rewards" refers to incentives such as points or vouchers that the system gives to users.

[0814] This system is designed to automatically generate travel records and propose new travel plans based on photos taken by travelers. The system is mainly composed of three elements: a server, a terminal, and a user, and performs the following processes:

[0815] Bulk photo posting

[0816] After the trip, the user opens the app and selects photos taken during the trip from the gallery. For example, select photos taken during a trip to Mount Fuji. The device extracts the location information (GPS data) and timestamp (date and time of the photo) from the selected photo. Using this information, the device uploads the photo file and its associated location information and timestamp to the server.

[0817] Automatic calculation of travel route

[0818] The server analyzes the received photo data and checks the location and timestamp of each photo. Based on the location and timestamp, it calculates the user's movement route in chronological order and stores this movement route in a database. For example, it generates a route from the base of Mount Fuji to the summit and plots it on a map.

[0819] Travel diary generation

[0820] The server extracts highlights and events from the trip based on the calculated route. Using these highlights, the server automatically generates a travel log by combining corresponding photos and descriptions. The generated travel log is saved in the user's profile and can be viewed freely by the user.

[0821] Creating a new tour plan

[0822] The server learns the user's preferences and patterns of places visited from the contents of the user's travel log. Then, using a generative AI model, it analyzes the rating data from other users and identifies highly rated restaurants and tourist spots. Based on this, it generates an optimal travel route by combining the user's preferences and high rating points. This newly generated travel route is provided to the user by the server.

[0823] User ratings and popularity rankings

[0824] Users can browse new travel plans published by other users, click "like" on the ones they like, and post comments. The server aggregates this evaluation data and updates the popularity rankings in real time. It then awards rewards to users who rank highly and notifies them of the results.

[0825] Map display and navigation

[0826] The server integrates the generated route with map data and provides navigation functions. The device displays the route on a map and provides voice guidance to support the user's movement. For example, when navigation begins, voice guidance such as "Turn left" or "You have arrived at the next tourist spot" is provided.

[0827] Prompt Sentence Examples

[0828] "Upload photos from your Mount Fuji trip and create a travel route."

[0829] "Generate a new tour plan based on your travel records."

[0830] "Please rate tour plans created by other users."

[0831] In this way, the system allows travelers to easily organize their travel records, automatically suggest new itineraries based on their individual preferences, and improve the quality of their travel experience by sharing and rating information with other travelers.

[0832] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0833] Step 1:

[0834] After the trip, the user opens the dedicated app and selects all the image data taken during the trip from the gallery.

[0835] Input: Multiple image files selected by the user.

[0836] Output: A list of selected image files.

[0837] Specific operation: The user taps the "Upload Photos" button in the app and selects multiple images taken during their trip. The device stores the image files as a list.

[0838] Step 2:

[0839] The device extracts location information (GPS data) and timestamps (date and time of photo) from each selected image file.

[0840] Input: A list of selected image files.

[0841] Output: Image metadata including location and timestamp.

[0842] How it works: The device analyzes the metadata of each image file to obtain location information and the date and time the image was taken, and temporarily stores this data in memory.

[0843] Step 3:

[0844] The device uploads them to a server along with an image file containing the extracted location information and timestamp.

[0845] Input: Image metadata including location and timestamp.

[0846] Output: Image data uploaded to the server.

[0847] Specific operation: The device packs image files containing location information and timestamps into packets and sends them via the server's upload API.

[0848] Step 4:

[0849] The server analyzes the received image data and checks the location information and timestamp of each file.

[0850] Input: Image data uploaded to the server.

[0851] Output: Parsed location and timestamp.

[0852] Specific operation: The server decodes the received network packets and stores the binary data of the image file, including location information and timestamp, in a database.

[0853] Step 5:

[0854] The server calculates the user's movement route in chronological order based on the location information and timestamp.

[0855] Input: Parsed location and timestamp.

[0856] Output: Travel path data sorted in chronological order.

[0857] Specific operation: The server analyzes the location information as latitude and longitude data, sorts it in chronological order by the date and time of the photo, and connects the points on the map to form a movement route.

[0858] Step 6:

[0859] The server stores the calculated travel route data in a database.

[0860] Input: Travel path data arranged in chronological order.

[0861] Output: Travel path data stored in a database.

[0862] Specific operation: The server inserts the generated travel route data into a dedicated travel route table and stores it in association with each user.

[0863] Step 7:

[0864] Based on the calculated route, the server extracts highlights and events of the trip.

[0865] Input: Time-series travel path data.

[0866] Output: Highlight extracted data.

[0867] Specific operation: The server identifies the most frequently visited or long-stayed locations from photos taken at different locations and lists them as highlights.

[0868] Step 8:

[0869] Based on the highlights, the server automatically generates a travel record by combining corresponding photos and descriptions.

[0870] Input: Highlight extraction data.

[0871] Output: An automatically generated trip log.

[0872] What it does: The server uses a text generation API to generate a description of each highlight and formats it along with a photo in HTML or another display format.

[0873] Step 9:

[0874] The server stores the generated travel record in the user's profile and makes it available for the user to view.

[0875] Input: Auto-generated trip log.

[0876] Output: Trip records stored in the user profile.

[0877] What it does: The server associates and stores the trip record data in the user's profile table and provides a link to display it via the app's UI.

[0878] Step 10:

[0879] The server learns user preferences and patterns of places visited from the contents of the user's travel log.

[0880] Input: Saved trip log data.

[0881] Output: User preferences and visiting place patterns.

[0882] Specific operation: The server analyzes the user's past travel history and uses a machine learning algorithm to analyze the categories and frequency of visited places.

[0883] Step 11:

[0884] The generative AI model analyzes rating data from other users to identify highly rated restaurants and tourist spots.

[0885] Input: Rating data from other users.

[0886] Output: A list of highly rated restaurants and attractions.

[0887] How it works: The generative AI model aggregates reviews and rating points from other users and lists highly rated spots.

[0888] Step 12:

[0889] The generative AI model combines user preferences and high-rated points to generate the optimal tour plan as a new travel route.

[0890] Input: User preferences, visiting location patterns, and high rating points.

[0891] Output: The newly generated tour plan.

[0892] How it works: The generative AI model integrates the user's preferred patterns and highly rated spots to create an efficient tour plan that takes into account transportation options and time of day.

[0893] Step 13:

[0894] The server provides the generated travel route to the user.

[0895] Input: The newly generated tour plan.

[0896] Output: The tour plan provided to the user account.

[0897] Specific operation: The server associates the newly generated tour plan with the user's account and notifies the user via the app's notification function.

[0898] Step 14:

[0899] Users can browse tour plans published by other users, click "like" on plans they like, and post comments.

[0900] Input: Published tour itinerary.

[0901] Output: Rating data such as likes and comments.

[0902] What happens: A user opens the app's "Popular Plans" section, browses the list of tour plans published by other users, and enters a rating.

[0903] Step 15:

[0904] The server compiles the evaluation data and updates the popularity rankings in real time.

[0905] Input: Rating data such as likes and comments.

[0906] Output: Updated popularity rankings.

[0907] Specific operation: The server periodically aggregates the evaluation data, calculates the score for each tour plan, and updates the ranking table.

[0908] Step 16:

[0909] The server awards rewards to top-ranked users and notifies them of the results.

[0910] Input: Updated popularity rankings.

[0911] Output: The user account to which the reward was granted.

[0912] Specific operation: The server will add points or vouchers to the top-ranked user accounts and notify the results through the notification system.

[0913] Step 17:

[0914] The server integrates the generated travel route with map data and provides navigation functions.

[0915] Input: Generated travel path data.

[0916] Output: Travel path integrated with map data.

[0917] Specific operation: The server uses a map service (e.g., Google Maps API) to plot the generated route on an interactive map.

[0918] Step 18:

[0919] The device displays the route on a map and provides real-time navigation with voice guidance.

[0920] Input: Travel path integrated with map data.

[0921] Output: Route displayed on map and voice guidance.

[0922] Specific operation: The user starts navigation mode, and the device tracks GPS information in real time and provides voice guidance.

[0923] (Application example 1)

[0924] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0925] Modern travelers take many photos during their trips and want to use them to record their journeys. However, manually organizing photos and creating travel routes and travel diaries after a trip is tedious and time-consuming. Furthermore, there are limited ways to efficiently gather information on new routes, tourist spots, and restaurants for the next trip. Conventional methods make it difficult to easily obtain reviews from other travelers and obtain individually optimized tourist information. Furthermore, there is a lack of means to obtain current location information in real time and efficiently record visited places. To solve these problems, a system is needed that allows travelers to easily record their trips and provides highly personalized tourist information in real time.

[0926] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0927] In this invention, the server includes: means for uploading photos taken by travelers to the server in bulk; means for automatically compiling the travel route of the traveler based on the location information and timestamps of the photos; means for automatically generating a travel diary for the traveler based on the travel route; means for generating a new travel route incorporating highly rated facilities and attractions based on the generated travel diary; means for the user to view and rate the generated travel route; means for generating a popularity ranking based on the ranking and awarding rewards to top-ranked users; means for tracking and recording places visited by the user using a smart device in real time; and means for displaying information about the vicinity of the current location on the smart device in real time and providing recommended tourist spots and shops. This allows travelers to automatically record their travels without hassle, obtain useful tourist information in real time, and efficiently plan their next trip by sharing information and ratings with other travelers.

[0928] "Photo upload" refers to the operation of sending image data taken by a traveler to a server all at once.

[0929] "Location information" refers to data that indicates the geographic coordinates (latitude and longitude) of the location where a photograph was taken.

[0930] A "timestamp" is information that indicates the date and time a photo was taken.

[0931] "Automatic travel route compilation" refers to the process of automatically calculating and organizing the order of a traveler's visits and routes based on the location information and timestamps of photos.

[0932] A "travel diary" is a record of photographs taken during a trip, with accompanying descriptions, arranged in chronological order.

[0933] "Highly rated facilities and attractions" refers to tourist destinations and stores that have received high ratings based on user ratings and reviews.

[0934] "Generation of new travel routes" refers to the process of creating new recommended travel routes based on existing travel diaries and high-rating information.

[0935] "User ratings" refer to feedback such as "likes" and comments on travel routes and travel diaries shared by other users.

[0936] "Popularity ranking" refers to a list of travel routes and travel diaries ranked based on user ratings.

[0937] "Giving rewards" means providing rewards such as benefits or points to those who rank highly.

[0938] "Smart device" refers to a portable electronic device that can provide and display digital information in real time, such as a smartphone, smart glasses, or head-mounted display.

[0939] "Place tracking" refers to the process of recording the places visited by a user in real time using a smart device.

[0940] "Displaying surrounding information" means displaying information about nearby tourist attractions and stores on the smart device display based on the current location.

[0941] "Recommended tourist attractions and stores" refer to tourist attractions and stores that are attractive to users and selected by the generative AI model.

[0942] This invention realizes a system that automatically generates travel diaries based on photos taken by travelers and provides new tourist information. The system is composed of the following elements.

[0943] Bulk upload and analysis of photos

[0944] The server provides a means for travelers to upload photos taken in bulk. After their trip, travelers open a dedicated app, select all photos taken during their trip, and upload them. The device extracts the location information (GPS data) and timestamp (date and time of photo) of the selected photos, and sends the photo files, along with their location information and timestamp, to the server. This reduces the user's workload and enables efficient data transmission.

[0945] Automatic calculation of travel route

[0946] The server analyzes the received photo data and checks the location information and timestamp. This allows it to calculate the traveler's route in chronological order. For analysis, Python and TensorFlow are used, for example, to process the data based on the location information and timestamp. The calculated route is then stored in a database. This automatically manages a detailed record of the places the traveler has visited.

[0947] Automatic generation of travel diary

[0948] The server extracts highlights and events from the trip based on the calculated travel route. A generative AI model is used to extract the highlights and generate detailed descriptions for each point. This automatically generates a travel diary that combines photos and descriptions. The generated travel diary is saved in the user's profile and can be viewed by the user.

[0949] Real-time tourist information

[0950] The server uses the generated travel diary content and rating data from other users to identify highly rated facilities and attractions. Using a generative AI model, the server analyzes the user's preferences and ratings from other users to generate new travel routes. This allows travelers to receive optimal sightseeing information for their next trip in real time. Additionally, if the user is using a smart device (smartphone, smart glasses, etc.), the terminal displays real-time information about the area around the current location and provides recommended tourist spots and stores.

[0951] User ratings and rankings

[0952] The server provides a means for users to view and rate publicly posted travel routes and travel diaries. Users can "like" tour plans they find good or post comments, and rating points are tallied in real time. The server then generates popularity rankings based on the ratings and operates a system that awards rewards to those at the top of the rankings. This promotes the sharing of travel information and the activation of the community.

[0953] Specific examples

[0954] Visiting tourist attractions: When a user visits Mount Fuji or a temple, the route traveled is automatically recorded, and the travel diary generated after the trip can be viewed on the smart glasses.

[0955] Restaurant guide: An example in which AI provides real-time guidance on highly rated restaurants near tourist spots and displays it on the smart glasses.

[0956] Prompt Sentence Examples

[0957] "When a user visits a particular tourist spot, the smart glasses will display the history and background of that place in real time, record the data of the visited places, and automatically generate a travel diary. In addition, the glasses will provide personalized recommendations based on other users' ratings."

[0958] In this way, the specific embodiment of the present invention allows travelers to efficiently keep a record of their trip and obtain useful tourist information in real time.

[0959] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0960] Step 1:

[0961] Bulk upload photos

[0962] Input: Photos taken by the user during their trip

[0963] The server provides a means for users to select photos taken during their trip in bulk through a dedicated app and upload them all to the server. The device extracts the location information (GPS data) and timestamp (date and time of photo shooting) of the selected photos and sends them to the server along with the photo files. This streamlines the upload process and aggregates photo data on the server.

[0964] Output: Photo data uploaded to the server

[0965] Step 2:

[0966] Automatic calculation of travel route

[0967] Input: Uploaded photo data (including location and timestamp)

[0968] The server analyzes the location information and timestamps of the received photo data and sorts them in chronological order. It then automatically compiles the traveler's route based on the location information and timestamps. Specifically, it calculates the distance between each photo position using Python's Geopy library and plots the photo data as a continuous route.

[0969] Output: Calculated travel route

[0970] Step 3:

[0971] Automatic generation of travel diary

[0972] Input: Calculated travel route

[0973] The server extracts highlights and events from the trip based on the calculated route. It uses a generative AI model to generate detailed descriptions for each point. It then combines the photos and descriptions to automatically generate a travel diary. The generated travel diary is saved in the user's profile and can be viewed later.

[0974] Output: Automatically generated travel diary

[0975] Step 4:

[0976] Real-time tourist information

[0977] Input: User's current location information, other users' rating data

[0978] The server identifies highly rated facilities and tourist attractions based on the user's current location information and rating data from other users. Using a generative AI model, it generates new sightseeing routes by analyzing the user's preferences and ratings from other users. The terminal also displays information about the area around the user's current location in real time via the smart device, providing guidance to recommended tourist spots and stores.

[0979] Output: Real-time generated tourist information and new travel routes

[0980] Step 5:

[0981] User ratings and rankings

[0982] Input: Travel routes and travel diaries published by other users, user feedback data

[0983] The server provides a means for users to view and rate publicly posted travel routes and travel diaries. Users can "like" routes they find good and post comments. The server tally evaluation points in real time and generate popularity rankings. It also operates a system that awards rewards to those at the top of the rankings.

[0984] Output: Generated ranking and reward information

[0985] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0986] MODE FOR CARRYING OUT THE INVENTION

[0987] System Overview

[0988] This system automatically compiles travel routes and generates travel diaries based on photos taken by travelers after their trip. It also utilizes generative AI to propose new tour plans, allowing users to rate and share them. It also combines an emotion engine that recognizes users' emotions to provide a more personalized travel experience.

[0989] Bulk photo posting

[0990] 1. After the trip, the user opens the dedicated app and selects all the photos taken during the trip.

[0991] 2. The device extracts the location information (GPS data) and timestamp (date and time of shooting) of the selected photo.

[0992] 3. The device uploads the photo file, its location information, and timestamp to the server.

[0993] Specific examples

[0994] The app allows users to select all photos they have taken during their trip to Europe and upload them to the server.

[0995] When uploading, the device automatically extracts the GPS data and shooting date and time contained in the photo and sends them to the server.

[0996] Automatic calculation of travel route

[0997] 1. The server analyzes the received photo data and checks the location information and timestamp.

[0998] 2. The server calculates the user's route in chronological order based on the location information and timestamp.

[0999] 3. The server plots the calculated travel route on map data to create a visual route.

[1000] 4. The server stores the calculated route data in a database.

[1001] Specific examples

[1002] The server analyzes the user's itinerary and plots the route traveled, for example, from France to Italy, on a map.

[1003] Based on the location information, each photographed point is connected in a continuous manner to form a route.

[1004] Travel diary generation

[1005] 1. The server extracts highlights and events from the trip based on the saved travel route data.

[1006] 2. The server combines the extracted highlights and photos to automatically generate a travel diary with explanatory text.

[1007] 3. The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[1008] Specific examples

[1009] The server automatically selects highlights such as the Eiffel Tower and the Colosseum and generates detailed descriptions.

[1010] Travel diaries are saved in a user's profile, allowing them to track their entire journey in chronological order.

[1011] Creating a new tour plan

[1012] 1. The server learns the user's preferences and patterns of places visited from the contents of the user's travel diary.

[1013] 2. The generating AI analyzes other users' rating data and identifies highly rated restaurants and tourist attractions.

[1014] 3. The generation AI combines the user's preferences and high rating points to generate a new travel route.

[1015] 4. The server saves the generated new travel route in the user's profile and makes it available for the user to view.

[1016] Specific examples

[1017] Generative AI learns the user's preferences and suggests new tour plans that incorporate highly rated restaurants and tourist attractions.

[1018] For example, if the user loves pasta, add a popular Italian restaurant to the route.

[1019] Optimization by Emotion Engine

[1020] 1. The device uses an emotion engine to analyze the user's emotional state from photo and audio data.

[1021] 2. The emotion engine optimizes travel routes and travel diaries based on the user's emotional state (joy, surprise, sadness, etc.).

[1022] 3. The emotion engine makes suggestions based on the user's current emotions.

[1023] Specific examples

[1024] The emotion engine analyzes the user's travel photos and comments, and if it finds that they are particularly happy at a particular place, it will suggest a new travel plan that includes that place.

[1025] For example, if the user has many photos of themselves having fun at Huis Ten Bosch, a tour plan will be generated that includes other tourist spots nearby.

[1026] User ratings and popularity rankings

[1027] 1. Users view tour plans published by other users.

[1028] 2. Users can "like" tour plans they find good or post comments.

[1029] 3. The server will tally the evaluation points for each tour plan and update the rankings in real time.

[1030] 4. The server will award rewards to the top-ranked tour plan contributors and notify them of the results.

[1031] Specific examples

[1032] A user browses Tokyo tour plans published by other users and "likes" them.

[1033] The server tallys likes and comments in real time and updates the popularity rankings.

[1034] Map display and navigation

[1035] 1. The server integrates the generated travel route with map data.

[1036] 2. The device displays the route on a map and prompts the user to begin navigation.

[1037] 3. The device starts voice guidance to support the user's movement.

[1038] Specific examples

[1039] The device displays the route of the new tour plan on the user's smartphone and begins navigation in real time.

[1040] Voice guidance will explain how to reach your destination and where to stop.

[1041] In this way, a system incorporating an emotion engine allows travelers to receive more personalized travel plans, providing an optimal travel experience based on the user's emotions and preferences.

[1042] The processing flow will be explained below.

[1043] MODE FOR CARRYING OUT THE INVENTION

[1044] Bulk photo posting

[1045] Step 1:

[1046] After traveling, users open a dedicated app and select all the photos they want to post at once.

[1047] Step 2:

[1048] The terminal displays a list of the selected photos and asks the user for confirmation.

[1049] Step 3:

[1050] The device extracts the location information (GPS data) and timestamp (date and time of photo capture) for each selected photo.

[1051] Step 4:

[1052] The device sends the photo file, its location information, and a timestamp to the server.

[1053] Automatic calculation of travel route

[1054] Step 5:

[1055] The server analyzes the received photo data and checks the location information and timestamp.

[1056] Step 6:

[1057] The server sorts the photos in chronological order based on the location information and timestamps, and calculates the user's travel route.

[1058] Step 7:

[1059] The server plots the calculated travel route on map data to create a visual route.

[1060] Step 8:

[1061] The server stores the calculated travel route data in a database.

[1062] Travel diary generation

[1063] Step 9:

[1064] The server extracts highlights and events from the trip based on the stored travel route data.

[1065] Step 10:

[1066] The server combines the extracted highlights and photos to automatically generate a travel diary with explanatory text.

[1067] Step 11:

[1068] The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[1069] Creating a new tour plan

[1070] Step 12:

[1071] The server learns the user's preferences and patterns of places visited from the contents of the user's travel diary.

[1072] Step 13:

[1073] The generative AI analyzes other users' rating data to identify highly rated restaurants and tourist attractions.

[1074] Step 14:

[1075] The generation AI combines user preferences and high rating points to generate new travel routes.

[1076] Step 15:

[1077] The server saves the generated new travel route in the user's profile and makes it available for the user to view.

[1078] Optimization by Emotion Engine

[1079] Step 16:

[1080] The device uses an emotion engine to analyze the user's emotional state from photo and audio data.

[1081] Step 17:

[1082] The emotion engine optimizes travel routes and travel diaries based on the user's emotional state (joy, surprise, sadness, etc.).

[1083] Step 18:

[1084] The emotion engine selects new travel routes and travel highlights based on the user's current emotions and reflects them in the travel diary.

[1085] Step 19:

[1086] The server generates a new optimized travel route based on the user's emotions and saves it in the profile.

[1087] User ratings and popularity rankings

[1088] Step 20:

[1089] A user can view tour plans published by other users.

[1090] Step 21:

[1091] Users can "like" tour plans that they find good and post comments.

[1092] Step 22:

[1093] The server tally up the evaluation points for each tour plan and update the rankings in real time.

[1094] Step 23:

[1095] The server gives rewards to the top ranked tour plan posters and notifies them of the results.

[1096] Map display and navigation

[1097] Step 24:

[1098] The server integrates the generated travel route with map data.

[1099] Step 25:

[1100] The device displays the route on a map and prompts the user to begin navigation.

[1101] Step 26:

[1102] The terminal starts voice guidance to support the user's movement.

[1103] Step 27:

[1104] The user uses the navigation system to enjoy sightseeing and eating and drinking along the generated travel route.

[1105] The above is the specific program processing flow of the system that combines the emotion engine. Each step incorporates a detailed process that includes the specific operations of the user, terminal, and server.

[1106] Example 2

[1107] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1108] In recent years, there has been a growing demand for creating travel records based on photos taken during travel and sharing travel experiences. However, organizing photos, compiling route records, and creating travel diaries is time-consuming. Automating each of these processes would enable users to create and share travel records efficiently. However, current systems do not provide a means to process these processes in an integrated manner. Furthermore, it is difficult to provide personalized services that combine travel plan suggestions, evaluations, and rankings based on users' preferences and emotions. Therefore, there is a need to enable efficient creation and sharing of travel records based on travelers' photos, and to provide an optimal travel experience that suits users' emotions and preferences.

[1109] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1110] In this invention, the server includes: means for uploading photos taken by travelers to the server in bulk using a terminal; means for the terminal to extract location information and timestamps of the photos and for the server to analyze them; means for automatically compiling the traveler's travel route from the location information and timestamps of the photos; means for plotting the travel route on map data and creating a visual route; means for automatically generating the travel diary of the traveler based on the travel route; means for learning based on the user's preferences and visiting trends and using a generation AI to generate a new travel route combining highly rated restaurants and tourist attractions; means for the user to view, rate, and share the generated travel route and travel diary; means for generating a popularity ranking based on the rating and awarding rewards to the top ranked users; means for analyzing the user's emotional state and optimizing the travel route and travel diary based on the emotion; and means for the terminal to display the generated travel route on a map and provide navigation. This enables efficient creation and sharing of travel records based on photos taken by travelers and further enables providing an optimal travel experience tailored to the user's emotions and preferences.

[1111] A "traveler" is an individual who takes photographs while traveling and intends to create a travel record using the photographic data.

[1112] A "terminal" is an electronic device used by a user, such as a mobile information terminal, smartphone, or tablet, which collects, analyzes, and uploads captured photo data to a server.

[1113] A "server" is a computer system that receives, analyzes, stores, and processes data sent from travelers' terminals as necessary.

[1114] "Location information" is information about the location where a photo was taken, and is geographical data recorded in the form of GPS data or the like.

[1115] A "timestamp" is data that indicates the date and time a photo was taken, and is stored as metadata for the photo.

[1116] "Travel route" is data showing the route traveled by a traveler during a trip, and is arranged in chronological order based on location information and timestamps.

[1117] "Map data" means geographical map information provided by a geographic information system (GIS) and is data used to visually display location information and travel routes.

[1118] A "travel diary" is an automatically generated travel record that integrates a traveler's route, highlights, photos, and descriptions.

[1119] "Generative AI" refers to artificial intelligence that learns from large amounts of data and generates new information and suggestions.

[1120] A "new travel route" is a recommended route for the next trip created by the generation AI based on the user's preferences and evaluation data from other users.

[1121] "User preferences" refers to the personal tastes and interests that a traveler has based on past travel and rating data.

[1122] "Top-rated restaurants and attractions" refers to restaurants and attractions that have been highly rated by other travelers.

[1123] "Evaluation" refers to evaluation points that a user gives to a tour plan published by another user by giving it a "like" or comment.

[1124] The "popularity ranking" is a ranking list generated based on the evaluation points obtained by each tour plan, and indicates the order of the most popular tour plans.

[1125] "Emotional state" refers to the emotions such as joy, surprise, or sadness that a traveler feels in response to a particular scene or photo, and is data estimated through emotion analysis.

[1126] "Navigation" is a function that provides audio and visual guidance to help a user reach a destination using a terminal.

[1127] MODE FOR CARRYING OUT THE INVENTION

[1128] This invention is a system that automatically creates a travel log based on photos taken by a traveler and proposes a travel plan that matches the user's preferences and feelings. Specific embodiments of this system will be described below.

[1129] System configuration

[1130] Users use a mobile information terminal (hereinafter referred to as "terminal") such as a smartphone or tablet to select all photos taken during their trip and upload them through a dedicated app. The terminal extracts the location information (GPS data) and timestamp (date and time of photo) embedded in the photos and sends this data to a server.

[1131] The server analyzes the received photo data and calculates the user's travel route in chronological order based on the location information and timestamp. The calculated travel route is plotted on map data, visually representing the route. This makes the user's travel route clear.

[1132] The server also extracts highlights and events from the saved travel route data and automatically generates a travel diary with explanatory text based on the extracted information. The generated travel diary is saved in the user's profile and can be viewed by the user at any time.

[1133] Creating new tour plans and optimizing them with an emotion engine

[1134] The server learns the user's preferences and visiting place patterns from the contents of the user's travel diary, and then analyzes other users' rating data using a generative AI model. From this analysis, it identifies highly rated restaurants and tourist attractions, and proposes new travel routes by combining the user's preferences with these highly rated points. The new tour plan is saved in the user's profile.

[1135] Furthermore, the device uses an emotion engine to analyze the user's emotional state from photos and voice data. The emotion engine evaluates the degree to which the user felt emotions such as joy or surprise at a particular location, and optimizes travel routes and travel diaries based on this. For example, if the device determines that the user was particularly pleased at a particular tourist spot, it will suggest a new tour plan that includes that tourist spot.

[1136] User ratings and popularity rankings

[1137] Users can browse tour plans published by other users, click "like" on the ones they like, and post comments. The server tally up the evaluation points of each tour plan and generate a popularity ranking in real time. Tour plan posters who rank highly are awarded rewards.

[1138] Navigation Features

[1139] The terminal displays the travel route generated by the server on a map and prompts the user to start navigation. The terminal then starts voice guidance and provides directions to reach the destination specified by the user. For example, the terminal displays the route of a new tour plan in real time and provides voice guidance on the way to the destination.

[1140] Examples and prompts

[1141] Example 1: A user selects photos taken during a trip to Europe and uploads them to a server. When uploading, the device automatically extracts the GPS data and the date and time of the photo and sends them to the server.

[1142] Example 2: The server analyzes the user's itinerary and plots the route, for example, from France to Italy, on a map. Based on the location information, each photographed point is connected in a continuous manner to form the route.

[1143] Example prompt: "How can I automatically compile travel routes and generate a travel diary based on travel photos taken by the user?"

[1144] This system allows travelers to efficiently create and share travel records with other users, and also allows them to enjoy personalized travel plans.

[1145] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1146] Step 1:

[1147] The user selects a photo

[1148] Input: Travel photos taken by the user

[1149] Output: A list of selected photo files

[1150] Specific operation: The user opens the dedicated app and selects all the photos taken during the trip at once. Select multiple photos from the app's gallery and add them to a list for uploading at once.

[1151] Step 2:

[1152] The device extracts the location and timestamp

[1153] Input: A list of photo files selected by the user

[1154] Output: Location information (GPS data) and timestamp (date and time of photo) for each photo

[1155] What it does: Your device reads the GPS data and timestamp from the photo's metadata and stores this information in temporary memory.

[1156] Step 3:

[1157] The device uploads data to the server

[1158] Input: Location and timestamp of each photo, photo file

[1159] Output: Photo data and metadata sent to the server

[1160] Specific operation: The device sends an HTTP request to the server, sending the photo data and associated metadata. It receives a success response from the server and notifies the user of the upload status.

[1161] Step 4:

[1162] The server analyzes the photo data

[1163] Input: Photo data and metadata uploaded to the server

[1164] Output: Locations and timestamps, organized chronologically

[1165] What it does: The server retrieves the photo data stored in the database and analyzes the metadata of each photo to check the location and timestamp.

[1166] Step 5:

[1167] The server calculates the route.

[1168] Input: Organized location and timestamp

[1169] Output: Calculated travel route

[1170] Specific operation: The server sorts the location information based on time series, and uses a specific algorithm to connect consecutive points to form the user's travel route.

[1171] Step 6:

[1172] The server plots the map data

[1173] Input: Calculated travel route

[1174] Output: Plotted map data

[1175] Specific operation: The server calls a map API (e.g., Google Maps API) to plot the route and reflects the plotted data in the user interface.

[1176] Step 7:

[1177] The server stores the travel route data

[1178] Input: Plotted map data and travel route information

[1179] Output: Travel route data stored in a database

[1180] Specific operation: The server inserts the travel route data into a specific table in the database and checks whether the insertion is successful.

[1181] Step 8:

[1182] The server extracts the highlights

[1183] Input: Travel route data

[1184] Output: Extracted highlight points and events

[1185] Specific operation: The server runs a highlight extraction algorithm to derive and list notable points.

[1186] Step 9:

[1187] The server automatically generates a travel diary

[1188] Input: Extracted highlight locations and events, corresponding photo data

[1189] Output: Automatically generated travel diary

[1190] Specific operation: The server uses a text generation model based on the highlight points to combine appropriate photos and descriptions to create a travel diary.

[1191] Step 10:

[1192] The server stores the travel diary and provides it to the user.

[1193] Input: Auto-generated travel diary

[1194] Output: Travel diary saved in the user's profile

[1195] Specific operation: The server saves the travel diary data in the database and then reflects it on the user's profile page.

[1196] Step 11:

[1197] The server learns the user's preferences

[1198] Input: User's travel diary data, rating data

[1199] Output: Learned user preferences and visiting habits

[1200] What it does: The server uses machine learning algorithms to analyze your behavioral patterns and add your preferences to your profile.

[1201] Step 12:

[1202] Generative AI analyzes the evaluation data

[1203] Input: Other users' ratings data

[1204] Output: A list of highly rated restaurants and attractions

[1205] What it does: The generative AI queries user rating data, performs statistical analysis, and lists specific places and services.

[1206] Step 13:

[1207] Generation AI generates new travel routes

[1208] Input: User preferences, likes

[1209] Output: Newly generated travel route

[1210] Specific operation: The generation AI runs a route generation algorithm based on preferences and high rating points, then verifies and optimizes the generated route.

[1211] Step 14:

[1212] The server stores the route and serves it to the user.

[1213] Input: New tour plan

[1214] Output: A new tour plan saved in the user's profile.

[1215] Specific behavior: The server saves the new tour plan in the database and displays the saved tour plan in the user interface.

[1216] Step 15:

[1217] The device performs emotion analysis

[1218] Input: Photo and audio data

[1219] Output: Emotion analysis results

[1220] Specific operation: The device sends photo and audio data to the emotion analysis module, receives and logs the emotion analysis results.

[1221] Step 16:

[1222] Emotion engine optimizes routes and travel diaries

[1223] Input: Sentiment analysis results

[1224] Output: Optimized travel route and travel diary

[1225] What it does: The emotion engine reevaluates highlights, routes, and descriptions based on emotion data and makes adjustments as needed.

[1226] Step 17:

[1227] Emotion engine makes suggestions

[1228] Input: Sentiment analysis results

[1229] Output: Improved tour plan

[1230] Specific behavior: The emotion engine generates a customized tour plan and notifies the user of the new tour plan.

[1231] Step 18:

[1232] A user browses tour plans

[1233] Input: Published tour plan

[1234] Output: Rating and comments

[1235] Specific operation: Users can view tour plans published by other users, select the tour plan they want to view, and rate and comment on it.

[1236] Step 19:

[1237] The server updates the rankings

[1238] Input: Reputation points

[1239] Output: Updated ranking list

[1240] Specific operation: The server retrieves reputation points from the database, aggregates them, and updates the ranking list.

[1241] Step 20:

[1242] The server will grant the reward

[1243] Input: Updated ranking list

[1244] Output: Reward granted notification

[1245] Specific operation: The server inserts reward data into the top ranking users and notifies them of the reward via the notification system.

[1246] Step 21:

[1247] The device displays the route

[1248] Input: Route data from the server

[1249] Output: Route map displayed on the terminal

[1250] Specific operation: The device displays map data on the screen and prompts the user to start navigation.

[1251] Step 22:

[1252] The device will provide voice guidance

[1253] Input: Current location data and route information

[1254] Output: Voice navigation

[1255] Specific operation: The device determines the current location based on GPS data and provides voice guidance to the destination.

[1256] (Application example 2)

[1257] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1258] Currently, there is a lack of means to automatically generate travel records and effectively manage photos and routes after a trip. Furthermore, there is no system that can suggest new routes and services that reflect the traveler's feelings and preferences. This makes the process of recording experiences during and after a trip cumbersome, making it difficult to provide optimal services to travelers.

[1259] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for uploading photos taken by travelers to the data processing device in bulk; means for automatically aggregating the traveler's travel route from the location information and timestamps of the photos; means for automatically generating the travel record of the traveler based on the travel route; means for generating a new travel route incorporating highly rated restaurants and tourist attractions based on the generated travel log; means for users to view and rate the generated travel route; means for generating a popularity index based on the evaluation and awarding rewards to those with the highest index; means for displaying the photos on a map based on the locations where they were taken and visualizing the visit route; and means for analyzing the user's emotional state from the photo data and proposing new services based on the analysis. This allows travelers to automatically generate and manage their travel routes and travel records, and further enables optimal service suggestions based on their emotions during the trip.

[1260] "Photos" are image data taken by travelers during their trip.

[1261] "Data Processing Device" means a combination of hardware and software that allows travelers to upload photos taken in bulk.

[1262] "Location information" is the geographic coordinate data where the photo was taken.

[1263] A "timestamp" is data that indicates the date and time a photo was taken.

[1264] A "travel route" is a traveler's travel history calculated based on location information and timestamps.

[1265] A "travel log" is a document automatically generated based on a traveler's travel route and photographs.

[1266] "Food and beverage establishments" refers to restaurants and bars along the travel route of travelers.

[1267] "Tourist destination" refers to a tourist attraction that a tourist visits.

[1268] "User" refers to a traveler or user of the system.

[1269] "Evaluation" refers to feedback provided by users regarding generated travel routes and services.

[1270] "Popularity index" is ranking data compiled from evaluation points.

[1271] "Rewards" are perks and incentives given to those who rank highly in popularity indicators.

[1272] A "visit route" is a route on a map that connects the places visited by a traveler in order.

[1273] "Emotional state" refers to the emotional state of a traveler analyzed from photo data.

[1274] "New services" refers to new services and benefits that are proposed based on the analyzed emotional state.

[1275] MODE FOR CARRYING OUT THE INVENTION

[1276] The system for implementing this invention automatically compiles travel routes based on photos taken by travelers during their trips, generates travel records, and utilizes AI to propose new services based on the travelers' emotional state.

[1277] 1. System Program Description

[1278] Hardware and software used:

[1279] Hardware:

[1280] Smartphones (for travelers to take and upload photos)

[1281] Servers (for data processing and storage)

[1282] Databases (to store traveller data and travel records)

[1283] software:

[1284] Pillow (a Python library for extracting photo metadata)

[1285] geopy (a Python library for analyzing geolocation data)

[1286] Basemap (a Python library for displaying travel routes on a map)

[1287] Emotion Engine API (external service for analyzing emotions from photos)

[1288] 2. System operation explanation

[1289] User Action:

[1290] 1. The user uses a dedicated app on their smartphone to select and upload all the photos they took during their trip.

[1291] 2. Automatically extract metadata (location and timestamp) from uploaded photos and send it to the server.

[1292] Server Action:

[1293] 1. The server analyzes the received photo data, checks the location information and timestamp, and calculates the user's movement route in chronological order based on this.

[1294] 2. The calculated travel path is plotted on map data to generate a visual route, which is then used to automatically generate a travel record for the traveler.

[1295] Sentiment analysis and service suggestions:

[1296] 1. The server analyzes the user's emotional state from the photo data using an emotion engine.

[1297] 2. Based on the analysis results, new services are proposed according to the user's emotions. For example, the system suggests nearby tourist spots and restaurants based on the places where the user feels particularly happy.

[1298] Specific examples:

[1299] 1. A tourist visits Paris, takes a photo, and uploads it. The photo includes location information such as the Eiffel Tower and the Louvre.

[1300] 2. The server uses an emotion engine to analyze the photos and determine what the traveler is enjoying, and recommends nearby cafes and museums as the next travel destination.

[1301] Example prompt sentence:

[1302] Suggest new services and places to visit based on the location and emotion data of places the user has visited. The user would like to see new places and services offered near areas where they have experienced particularly high levels of joy, such as:

[1303] 1. Highly rated cafes and restaurants

[1304] 2. Popular tourist destinations

[1305] 3. Promotions and offers available

[1306] This system allows travelers to automatically generate and manage their travel routes and travel records, and also allows them to receive recommendations for optimal services based on their emotions while traveling.

[1307] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1308] Step 1:

[1309] Users use a dedicated smartphone app to select and upload multiple photos taken during their trip. The input is the photo files selected by the user, and the output is the selected photo files being sent to the server.

[1310] Step 2:

[1311] The device automatically extracts location information (GPS data) and timestamp (date and time of photo) from uploaded photo files. The input is the photo file, and the output is metadata including location information and timestamp. This metadata is sent to the server along with the photo file.

[1312] Step 3:

[1313] The server analyzes the received photo data and checks the location information and timestamp for each photo. The input is the uploaded photo data and its metadata, and the output is a list of the location information and timestamp corresponding to each photo data.

[1314] Step 4:

[1315] The server calculates the user's route in chronological order based on the location information and timestamps. The input is a list of location information and timestamps, and the output is the user's route data sorted in chronological order. This is plotted on map data to generate a visual route.

[1316] Step 5:

[1317] The server automatically generates a travel record for the traveler based on the generated travel route. The input is travel route data, and the output is text and image data of the travel record.

[1318] Step 6:

[1319] The server sends the photo data to the emotion engine API to analyze the user's emotional state. The input is the photo data and its metadata, and the output is the emotion analysis result, which is a specific emotion (e.g., joy, surprise, sadness) that reflects the user's travel experience.

[1320] Step 7:

[1321] The server proposes new services and travel routes based on the emotion analysis results. The input is the emotion analysis results and travel records, and the output is a list of recommended new services and travel routes.

[1322] Step 8:

[1323] The terminal notifies the user of new services and travel routes provided by the server and allows the user to view them. The input is a recommendation list sent from the server, and the output is recommendation information displayed on the user's smartphone.

[1324] Step 9:

[1325] Users rate new services and travel routes provided and provide feedback. The input is the user's ratings and comments, and the output is the rating data sent to the server.

[1326] Step 10:

[1327] The server aggregates user rating data, generates a popularity index, updates rankings based on that index, and awards rewards to top users. The input is user rating data, and the output is updated rankings and reward notifications.

[1328] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1329] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1330] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1331] [Third embodiment]

[1332] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1333] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1334] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1335] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1336] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1337] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1338] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1339] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1340] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1341] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1342] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1343] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1344] MODE FOR CARRYING OUT THE INVENTION

[1345] System Overview

[1346] This system automatically compiles travel routes and generates travel diaries based on photos taken by travelers after their trip. It also uses AI to propose new tour plans and provides a function that allows users to rate and share them.

[1347] Bulk photo posting

[1348] 1. After the trip, the user opens the dedicated app and selects all the photos taken during the trip.

[1349] 2. The device extracts the location information (GPS data) and timestamp (date and time of shooting) of the selected photo.

[1350] 3. The device uploads the photo file, its location information, and timestamp to the server.

[1351] Specific examples

[1352] Users select photos they have taken around Mount Fuji in the app and upload them to the server all at once.

[1353] When uploading, the device automatically extracts the GPS data and shooting date and time contained in the photo and sends them to the server.

[1354] Automatic calculation of travel route

[1355] 1. The server analyzes the received photo data and checks the location information and timestamp.

[1356] 2. The server calculates the user's route in chronological order based on the location information and timestamp.

[1357] 3. The server stores the calculated route in a database.

[1358] Specific examples

[1359] The server analyzes the user's itinerary and plots the route from the base of Mount Fuji to the summit on a map.

[1360] Based on the location information, each photographed point is connected in a continuous manner to form a route.

[1361] Travel diary generation

[1362] 1. The server extracts highlights and events from the calculated travel route.

[1363] 2. The server automatically generates a travel diary based on the extracted highlights, combining photos and descriptions.

[1364] 3. The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[1365] Specific examples

[1366] The server automatically inserts photos of summits and particularly beautiful views into the travel diary as highlights and generates detailed descriptions.

[1367] The travel diary is saved in the user's profile, allowing them to follow the overall picture of their Mount Fuji climb chronologically.

[1368] Creating a new tour plan

[1369] 1. The server learns the user's preferences and patterns of places visited from the contents of their travel diary.

[1370] 2. The generating AI analyzes rating data from other users and identifies highly rated restaurants and tourist attractions.

[1371] 3. The generation AI combines the user's preferences and high rating points to generate the optimal travel route as a new tour plan.

[1372] 4. The server provides the newly generated travel route to the user.

[1373] Specific examples

[1374] The generative AI learns the user's preferences and suggests new tour plans that incorporate other tourist attractions and restaurants.

[1375] For example, if a user likes the view from a mountaintop, they can add other scenic spots.

[1376] User ratings and popularity rankings

[1377] 1. Users view tour plans published by other users.

[1378] 2. Users can "like" tour plans they find good or post comments.

[1379] 3. The server will tally the evaluation points for each tour plan and update the rankings in real time.

[1380] 4. The server awards rewards to the top-ranked users and notifies them of the results.

[1381] Specific examples

[1382] Users view Yamanashi Prefecture tour plans published by other users and "like" them.

[1383] The server tallys likes and comments in real time and updates the popularity rankings.

[1384] Map display and navigation

[1385] 1. The server integrates the generated travel route with map data.

[1386] 2. The device will display the route on a map and enter navigation mode.

[1387] 3. The device provides voice guidance to support the user's movement.

[1388] Specific examples

[1389] The device displays the route of the new tour plan on the user's smartphone and begins navigation in real time.

[1390] Voice guidance will explain how to reach your destination and where to stop.

[1391] Such a system allows travelers to effortlessly record their trips, get travel plans tailored to their individual preferences, and improve the quality of their travel experience by sharing information and ratings with other travelers.

[1392] The processing flow will be explained below.

[1393] Program processing flow

[1394] Bulk photo posting

[1395] Step 1:

[1396] After traveling, users open a dedicated app and select all the photos they want to post at once.

[1397] Step 2:

[1398] The terminal displays a list of the selected photos and asks the user for confirmation.

[1399] Step 3:

[1400] The device extracts the location information (GPS data) and timestamp (date and time of photo capture) for each selected photo.

[1401] Step 4:

[1402] The device sends the photo file, its location information, and a timestamp to the server.

[1403] Automatic calculation of travel route

[1404] Step 5:

[1405] The server analyzes the received photo data and checks the location information and timestamp.

[1406] Step 6:

[1407] The server sorts the photos in chronological order based on the location information and timestamps, and calculates the user's travel route.

[1408] Step 7:

[1409] The server plots the calculated travel route on map data to create a visual route.

[1410] Step 8:

[1411] The server stores the calculated travel route data in a database.

[1412] Travel diary generation

[1413] Step 9:

[1414] The server extracts highlights and events from the trip based on the stored travel route data.

[1415] Step 10:

[1416] The server combines the extracted highlights and photos to automatically generate a travel diary with explanatory text.

[1417] Step 11:

[1418] The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[1419] Creating a new tour plan

[1420] Step 12:

[1421] The server learns the user's preferences and patterns of places visited from the contents of the user's travel diary.

[1422] Step 13:

[1423] The generative AI analyzes rating data from other users to identify highly rated restaurants and tourist attractions.

[1424] Step 14:

[1425] The generation AI combines user preferences and high rating points to generate new travel routes.

[1426] Step 15:

[1427] The server saves the generated new travel route in the user's profile and makes it available for the user to view.

[1428] User ratings and popularity rankings

[1429] Step 16:

[1430] A user can view tour plans published by other users.

[1431] Step 17:

[1432] Users can "like" tour plans they find good and post comments.

[1433] Step 18:

[1434] The server tally up the evaluation points for each tour plan and update the rankings in real time.

[1435] Step 19:

[1436] The server gives rewards to the top ranked tour plan posters and notifies them of the results.

[1437] Map display and navigation

[1438] Step 20:

[1439] The server integrates the generated travel route with map data.

[1440] Step 21:

[1441] The device displays the route on a map and prompts the user to begin navigation.

[1442] Step 22:

[1443] The terminal starts voice guidance to support the user's movement.

[1444] Step 23:

[1445] The user uses the navigation system to enjoy sightseeing and eating and drinking along the generated travel route.

[1446] The above is the specific flow of program processing for this system. Each step incorporates a detailed process that includes the specific operations of the user, terminal, and server.

[1447] Example 1

[1448] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1449] There is a need for a system that can efficiently create detailed travel records based on photos taken by travelers, suggest new travel plans, and share information and ratings with other travelers in real time. However, conventional systems require the time-consuming task of manually classifying and organizing photos, making it difficult to automatically generate detailed travel records including trip highlights. It is also difficult to automatically generate new travel plans tailored to individual preferences and provide navigation functions integrated with map data.

[1450] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1451] In this invention, the server includes a means for uploading image data taken by a traveler to the server in bulk, a means for automatically tallying up the travel route of the traveler from the location information and timestamp of the image data, and a means for automatically generating a travel record of the traveler based on the travel route. This makes it possible to effectively utilize the photos taken by the traveler and automatically generate a detailed travel record based on a chronological order.

[1452] "Image data" refers to digital data of photographs and videos taken by travelers.

[1453] "Server" refers to a computer system for receiving and processing data from terminals connected over a network.

[1454] "Location Information" refers to geographic latitude and longitude data obtained using technologies such as GPS.

[1455] A "timestamp" refers to date and time information that indicates when digital data was created, modified, or accessed at a certain time.

[1456] A "travel route" refers to a route that connects multiple locations visited by a traveler within a specific period in chronological order.

[1457] A "travel journal" is a record of the places and events visited during a trip, compiled in photographs and text.

[1458] A "generative AI model" refers to an artificial intelligence algorithm that uses machine learning techniques to generate new data and plans.

[1459] "Restaurant" refers to a commercial establishment that sells food and beverages and allows customers to eat and drink.

[1460] "Tourist attractions" refer to popular tourist destinations and historical and cultural sites.

[1461] "Smart devices" refer to portable electronic devices that can connect to the Internet, such as smartphones and tablets.

[1462] "Navigation" refers to a system and its functions that indicate the optimal route from the current location to the destination.

[1463] "Evaluation" refers to the act of rating a tour plan or service created by another user after using the service.

[1464] "Popularity ranking" refers to a ranking list created by compiling feedback such as user ratings and "likes."

[1465] "Rewards" refers to incentives such as points or vouchers that the system gives to users.

[1466] This system is designed to automatically generate travel records and propose new travel plans based on photos taken by travelers. The system is mainly composed of three elements: a server, a terminal, and a user, and performs the following processes:

[1467] Bulk photo posting

[1468] After the trip, the user opens the app and selects photos taken during the trip from the gallery. For example, select photos taken during a trip to Mount Fuji. The device extracts the location information (GPS data) and timestamp (date and time of the photo) from the selected photo. Using this information, the device uploads the photo file and its associated location information and timestamp to the server.

[1469] Automatic calculation of travel route

[1470] The server analyzes the received photo data and checks the location and timestamp of each photo. Based on the location and timestamp, it calculates the user's movement route in chronological order and stores this movement route in a database. For example, it generates a route from the base of Mount Fuji to the summit and plots it on a map.

[1471] Travel diary generation

[1472] The server extracts highlights and events from the trip based on the calculated route. Using these highlights, the server automatically generates a travel log by combining corresponding photos and descriptions. The generated travel log is saved in the user's profile and can be viewed freely by the user.

[1473] Creating a new tour plan

[1474] The server learns the user's preferences and patterns of places visited from the contents of the user's travel log. Then, using a generative AI model, it analyzes the rating data from other users and identifies highly rated restaurants and tourist spots. Based on this, it generates an optimal travel route by combining the user's preferences and high rating points. This newly generated travel route is provided to the user by the server.

[1475] User ratings and popularity rankings

[1476] Users can browse new travel plans published by other users, click "like" on the ones they like, and post comments. The server aggregates this evaluation data and updates the popularity rankings in real time. It then awards rewards to users who rank highly and notifies them of the results.

[1477] Map display and navigation

[1478] The server integrates the generated route with map data and provides navigation functions. The device displays the route on a map and provides voice guidance to support the user's movement. For example, when navigation begins, voice guidance such as "Turn left" or "You have arrived at the next tourist spot" is provided.

[1479] Prompt Sentence Examples

[1480] "Upload photos from your Mount Fuji trip and create a travel route."

[1481] "Generate a new tour plan based on your travel records."

[1482] "Please rate tour plans created by other users."

[1483] In this way, the system allows travelers to easily organize their travel records, automatically suggest new itineraries based on their individual preferences, and improve the quality of their travel experience by sharing and rating information with other travelers.

[1484] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1485] Step 1:

[1486] After the trip, the user opens the dedicated app and selects all the image data taken during the trip from the gallery.

[1487] Input: Multiple image files selected by the user.

[1488] Output: A list of selected image files.

[1489] Specific operation: The user taps the "Upload Photos" button in the app and selects multiple images taken during their trip. The device stores the image files as a list.

[1490] Step 2:

[1491] The device extracts location information (GPS data) and timestamps (date and time of photo) from each selected image file.

[1492] Input: A list of selected image files.

[1493] Output: Image metadata including location and timestamp.

[1494] How it works: The device analyzes the metadata of each image file to obtain location information and the date and time the image was taken, and temporarily stores this data in memory.

[1495] Step 3:

[1496] The device uploads them to a server along with an image file containing the extracted location information and timestamp.

[1497] Input: Image metadata including location and timestamp.

[1498] Output: Image data uploaded to the server.

[1499] Specific operation: The device packs image files containing location information and timestamps into packets and sends them via the server's upload API.

[1500] Step 4:

[1501] The server analyzes the received image data and checks the location information and timestamp of each file.

[1502] Input: Image data uploaded to the server.

[1503] Output: Parsed location and timestamp.

[1504] Specific operation: The server decodes the received network packets and stores the binary data of the image file, including location information and timestamp, in a database.

[1505] Step 5:

[1506] The server calculates the user's movement route in chronological order based on the location information and timestamp.

[1507] Input: Parsed location and timestamp.

[1508] Output: Travel path data sorted in chronological order.

[1509] Specific operation: The server analyzes the location information as latitude and longitude data, sorts it in chronological order by the date and time of the photo, and connects the points on the map to form a movement route.

[1510] Step 6:

[1511] The server stores the calculated travel route data in a database.

[1512] Input: Travel path data arranged in chronological order.

[1513] Output: Travel path data stored in a database.

[1514] Specific operation: The server inserts the generated travel route data into a dedicated travel route table and stores it in association with each user.

[1515] Step 7:

[1516] Based on the calculated route, the server extracts highlights and events of the trip.

[1517] Input: Time-series travel path data.

[1518] Output: Highlight extracted data.

[1519] Specific operation: The server identifies the most frequently visited or long-stayed locations from photos taken at different locations and lists them as highlights.

[1520] Step 8:

[1521] Based on the highlights, the server automatically generates a travel record by combining corresponding photos and descriptions.

[1522] Input: Highlight extraction data.

[1523] Output: An automatically generated trip log.

[1524] What it does: The server uses a text generation API to generate a description of each highlight and formats it along with a photo in HTML or another display format.

[1525] Step 9:

[1526] The server stores the generated travel record in the user's profile and makes it available for the user to view.

[1527] Input: Auto-generated trip log.

[1528] Output: Trip records stored in the user profile.

[1529] What it does: The server associates and stores the trip record data in the user's profile table and provides a link to display it via the app's UI.

[1530] Step 10:

[1531] The server learns the user's preferences and patterns of places visited from the contents of their travel log.

[1532] Input: Saved trip log data.

[1533] Output: User preferences and visiting place patterns.

[1534] Specific operation: The server analyzes the user's past travel history and uses a machine learning algorithm to analyze the categories and frequency of visited places.

[1535] Step 11:

[1536] The generative AI model analyzes rating data from other users to identify highly rated restaurants and tourist spots.

[1537] Input: Rating data from other users.

[1538] Output: A list of highly rated restaurants and attractions.

[1539] How it works: The generative AI model aggregates reviews and rating points from other users and lists highly rated spots.

[1540] Step 12:

[1541] The generative AI model combines user preferences and high-rated points to generate the optimal tour plan as a new travel route.

[1542] Input: User preferences, visiting location patterns, and high rating points.

[1543] Output: The newly generated tour plan.

[1544] How it works: The generative AI model integrates the user's preferred patterns and highly rated spots to create an efficient tour plan that takes into account transportation options and time of day.

[1545] Step 13:

[1546] The server provides the generated travel route to the user.

[1547] Input: The newly generated tour plan.

[1548] Output: The tour plan provided to the user account.

[1549] Specific operation: The server associates the newly generated tour plan with the user's account and notifies the user via the app's notification function.

[1550] Step 14:

[1551] Users can browse tour plans published by other users, click "like" on plans they like, and post comments.

[1552] Input: Published tour itinerary.

[1553] Output: Rating data such as likes and comments.

[1554] What happens: A user opens the app's "Popular Plans" section, browses the list of tour plans published by other users, and enters a rating.

[1555] Step 15:

[1556] The server compiles the evaluation data and updates the popularity rankings in real time.

[1557] Input: Rating data such as likes and comments.

[1558] Output: Updated popularity rankings.

[1559] Specific operation: The server periodically aggregates the evaluation data, calculates the score for each tour plan, and updates the ranking table.

[1560] Step 16:

[1561] The server awards rewards to top-ranked users and notifies them of the results.

[1562] Input: Updated popularity rankings.

[1563] Output: The user account to which the reward was granted.

[1564] Specific operation: The server will add points or vouchers to the top-ranked user accounts and notify the results through the notification system.

[1565] Step 17:

[1566] The server integrates the generated travel route with map data and provides navigation functions.

[1567] Input: Generated travel path data.

[1568] Output: Travel path integrated with map data.

[1569] Specific operation: The server uses a map service (e.g., Google Maps API) to plot the generated route on an interactive map.

[1570] Step 18:

[1571] The device displays the route on a map and provides real-time navigation with voice guidance.

[1572] Input: Travel path integrated with map data.

[1573] Output: Route displayed on map and voice guidance.

[1574] Specific operation: The user starts navigation mode, and the device tracks GPS information in real time and provides voice guidance.

[1575] (Application example 1)

[1576] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1577] Modern travelers take many photos during their trips and want to use them to record their journeys. However, manually organizing photos and creating travel routes and travel diaries after a trip is tedious and time-consuming. Furthermore, there are limited ways to efficiently gather information on new routes, tourist spots, and restaurants for the next trip. Conventional methods make it difficult to easily obtain reviews from other travelers and obtain individually optimized tourist information. Furthermore, there is a lack of means to obtain current location information in real time and efficiently record visited places. To solve these problems, a system is needed that allows travelers to easily record their trips and provides highly personalized tourist information in real time.

[1578] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1579] In this invention, the server includes: means for uploading photos taken by travelers to the server in bulk; means for automatically compiling the travel route of the traveler based on the location information and timestamps of the photos; means for automatically generating a travel diary for the traveler based on the travel route; means for generating a new travel route incorporating highly rated facilities and attractions based on the generated travel diary; means for the user to view and rate the generated travel route; means for generating a popularity ranking based on the ranking and awarding rewards to top-ranked users; means for tracking and recording places visited by the user using a smart device in real time; and means for displaying information about the vicinity of the user's current location on the smart device in real time and providing recommended tourist spots and shops. This allows travelers to automatically record their travels without hassle, obtain useful tourist information in real time, and efficiently plan their next trip by sharing information and rating with other travelers.

[1580] "Photo upload" refers to the operation of sending image data taken by a traveler to a server all at once.

[1581] "Location information" refers to data that indicates the geographic coordinates (latitude and longitude) of the location where a photograph was taken.

[1582] A "timestamp" is information that indicates the date and time a photo was taken.

[1583] "Automatic travel route compilation" refers to the process of automatically calculating and organizing the order of a traveler's visits and routes based on the location information and timestamps of photos.

[1584] A "travel diary" is a record of photographs taken during a trip, with accompanying descriptions, arranged in chronological order.

[1585] "Highly rated facilities and attractions" refers to tourist destinations and stores that have received high ratings based on user ratings and reviews.

[1586] "Generation of new travel routes" refers to the process of creating new recommended travel routes based on existing travel diaries and high-rating information.

[1587] "User ratings" refer to feedback such as "likes" and comments on travel routes and travel diaries shared by other users.

[1588] "Popularity ranking" refers to a list of travel routes and travel diaries ranked based on user ratings.

[1589] "Giving rewards" means providing rewards such as benefits or points to those who rank highly.

[1590] "Smart device" refers to a portable electronic device that can provide and display digital information in real time, such as a smartphone, smart glasses, or head-mounted display.

[1591] "Place tracking" refers to the process of recording the places visited by a user in real time using a smart device.

[1592] "Displaying surrounding information" means displaying information about nearby tourist attractions and stores on the smart device display based on the current location.

[1593] "Recommended tourist attractions and stores" refer to tourist attractions and stores that are attractive to users and selected by the generative AI model.

[1594] This invention realizes a system that automatically generates travel diaries based on photos taken by travelers and provides new tourist information. The system is composed of the following elements.

[1595] Bulk upload and analysis of photos

[1596] The server provides a means for travelers to upload photos taken in bulk. After their trip, travelers open a dedicated app, select all photos taken during their trip, and upload them. The device extracts the location information (GPS data) and timestamp (date and time of photo) of the selected photos, and sends the photo files, along with their location information and timestamp, to the server. This reduces the user's workload and enables efficient data transmission.

[1597] Automatic calculation of travel route

[1598] The server analyzes the received photo data and checks the location information and timestamp. This allows it to calculate the traveler's route in chronological order. For analysis, Python and TensorFlow are used, for example, to process the data based on the location information and timestamp. The calculated route is then stored in a database. This automatically manages a detailed record of the places the traveler has visited.

[1599] Automatic generation of travel diary

[1600] The server extracts highlights and events from the trip based on the calculated travel route. A generative AI model is used to extract the highlights and generate detailed descriptions for each point. This automatically generates a travel diary that combines photos and descriptions. The generated travel diary is saved in the user's profile and can be viewed by the user.

[1601] Real-time tourist information

[1602] The server uses the generated travel diary content and rating data from other users to identify highly rated facilities and attractions. Using a generative AI model, the server analyzes the user's preferences and ratings from other users to generate new travel routes. This allows travelers to receive optimal sightseeing information for their next trip in real time. Additionally, if the user is using a smart device (smartphone, smart glasses, etc.), the terminal displays real-time information about the area around the current location and provides recommended tourist spots and stores.

[1603] User ratings and rankings

[1604] The server provides a means for users to view and rate publicly posted travel routes and travel diaries. Users can "like" tour plans they find good or post comments, and rating points are tallied in real time. The server then generates popularity rankings based on the ratings and operates a system that awards rewards to those at the top of the rankings. This promotes the sharing of travel information and the activation of the community.

[1605] Specific examples

[1606] Visiting tourist attractions: When a user visits Mount Fuji or a temple, the route traveled is automatically recorded, and the travel diary generated after the trip can be viewed on the smart glasses.

[1607] Restaurant guide: An example in which AI provides real-time guidance on highly rated restaurants near tourist spots and displays it on the smart glasses.

[1608] Prompt Sentence Examples

[1609] "When a user visits a particular tourist spot, the smart glasses will display the history and background of that place in real time, record the data of the places visited, and automatically generate a travel diary. In addition, the glasses will provide personalized recommendations based on other users' ratings."

[1610] In this way, the specific embodiment of the present invention allows travelers to efficiently keep a record of their trip and obtain useful tourist information in real time.

[1611] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1612] Step 1:

[1613] Bulk upload photos

[1614] Input: Photos taken by the user during their trip

[1615] The server provides a means for users to select photos taken during their trip in bulk through a dedicated app and upload them all to the server. The device extracts the location information (GPS data) and timestamp (date and time of photo shooting) of the selected photos and sends them to the server along with the photo files. This streamlines the upload process and aggregates photo data on the server.

[1616] Output: Photo data uploaded to the server

[1617] Step 2:

[1618] Automatic calculation of travel route

[1619] Input: Uploaded photo data (including location and timestamp)

[1620] The server analyzes the location information and timestamps of the received photo data and sorts them in chronological order. It then automatically compiles the traveler's route based on the location information and timestamps. Specifically, it calculates the distance between each photo position using Python's Geopy library and plots the photo data as a continuous route.

[1621] Output: Calculated travel route

[1622] Step 3:

[1623] Automatic generation of travel diary

[1624] Input: Calculated travel route

[1625] The server extracts highlights and events from the trip based on the calculated route. It uses a generative AI model to generate detailed descriptions for each point. It then combines the photos and descriptions to automatically generate a travel diary. The generated travel diary is saved in the user's profile and can be viewed later.

[1626] Output: Automatically generated travel diary

[1627] Step 4:

[1628] Real-time tourist information

[1629] Input: User's current location information, other users' rating data

[1630] The server identifies highly rated facilities and tourist attractions based on the user's current location information and rating data from other users. Using a generative AI model, it generates new sightseeing routes by analyzing the user's preferences and ratings from other users. The terminal also displays information about the area around the user's current location in real time via the smart device, providing guidance to recommended tourist spots and stores.

[1631] Output: Real-time generated tourist information and new travel routes

[1632] Step 5:

[1633] User ratings and rankings

[1634] Input: Travel routes and travel diaries published by other users, user feedback data

[1635] The server provides a means for users to view and rate publicly posted travel routes and travel diaries. Users can "like" routes they find good and post comments. The server tally evaluation points in real time and generate popularity rankings. It also operates a system that awards rewards to those at the top of the rankings.

[1636] Output: Generated ranking and reward information

[1637] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1638] MODE FOR CARRYING OUT THE INVENTION

[1639] System Overview

[1640] This system automatically compiles travel routes and generates travel diaries based on photos taken by travelers after their trip. It also utilizes generative AI to propose new tour plans, allowing users to rate and share them. It also combines an emotion engine that recognizes users' emotions to provide a more personalized travel experience.

[1641] Bulk photo posting

[1642] 1. After the trip, the user opens the dedicated app and selects all the photos taken during the trip.

[1643] 2. The device extracts the location information (GPS data) and timestamp (date and time of shooting) of the selected photo.

[1644] 3. The device uploads the photo file, its location information, and timestamp to the server.

[1645] Specific examples

[1646] The app allows users to select all photos they have taken during their trip to Europe and upload them to the server.

[1647] When uploading, the device automatically extracts the GPS data and shooting date and time contained in the photo and sends them to the server.

[1648] Automatic calculation of travel route

[1649] 1. The server analyzes the received photo data and checks the location information and timestamp.

[1650] 2. The server calculates the user's route in chronological order based on the location information and timestamp.

[1651] 3. The server plots the calculated travel route on map data to create a visual route.

[1652] 4. The server stores the calculated route data in a database.

[1653] Specific examples

[1654] The server analyzes the user's itinerary and plots the route traveled, for example, from France to Italy, on a map.

[1655] Based on the location information, each photographed point is connected in a continuous manner to form a route.

[1656] Travel diary generation

[1657] 1. The server extracts highlights and events from the trip based on the saved travel route data.

[1658] 2. The server combines the extracted highlights and photos to automatically generate a travel diary with explanatory text.

[1659] 3. The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[1660] Specific examples

[1661] The server automatically selects highlights such as the Eiffel Tower and the Colosseum and generates detailed descriptions.

[1662] Travel diaries are saved in a user's profile, allowing them to track their entire journey in chronological order.

[1663] Creating a new tour plan

[1664] 1. The server learns the user's preferences and patterns of places visited from the contents of the user's travel diary.

[1665] 2. The generating AI analyzes other users' rating data and identifies highly rated restaurants and tourist attractions.

[1666] 3. The generation AI combines the user's preferences and high rating points to generate a new travel route.

[1667] 4. The server saves the generated new travel route in the user's profile and makes it available for the user to view.

[1668] Specific examples

[1669] Generative AI learns the user's preferences and suggests new tour plans that incorporate highly rated restaurants and tourist attractions.

[1670] For example, if the user loves pasta, add a popular Italian restaurant to the route.

[1671] Optimization by Emotion Engine

[1672] 1. The device uses an emotion engine to analyze the user's emotional state from photo and audio data.

[1673] 2. The emotion engine optimizes travel routes and travel diaries based on the user's emotional state (joy, surprise, sadness, etc.).

[1674] 3. The emotion engine makes suggestions based on the user's current emotions.

[1675] Specific examples

[1676] The emotion engine analyzes the user's travel photos and comments, and if it finds that they are particularly happy at a particular place, it will suggest a new travel plan that includes that place.

[1677] For example, if the user has many photos of themselves having fun at Huis Ten Bosch, a tour plan will be generated that includes other tourist spots nearby.

[1678] User ratings and popularity rankings

[1679] 1. Users view tour plans published by other users.

[1680] 2. Users can "like" tour plans they find good or post comments.

[1681] 3. The server will tally the evaluation points for each tour plan and update the rankings in real time.

[1682] 4. The server will award rewards to the top-ranked tour plan contributors and notify them of the results.

[1683] Specific examples

[1684] A user browses Tokyo tour plans published by other users and "likes" them.

[1685] The server tallys likes and comments in real time and updates the popularity rankings.

[1686] Map display and navigation

[1687] 1. The server integrates the generated travel route with map data.

[1688] 2. The device displays the route on a map and prompts the user to begin navigation.

[1689] 3. The device starts voice guidance to support the user's movement.

[1690] Specific examples

[1691] The device displays the route of the new tour plan on the user's smartphone and begins navigation in real time.

[1692] Voice guidance will explain how to reach your destination and where to stop.

[1693] In this way, a system incorporating an emotion engine allows travelers to receive more personalized travel plans, providing an optimal travel experience based on the user's emotions and preferences.

[1694] The processing flow will be explained below.

[1695] MODE FOR CARRYING OUT THE INVENTION

[1696] Bulk photo posting

[1697] Step 1:

[1698] After traveling, users open a dedicated app and select all the photos they want to post at once.

[1699] Step 2:

[1700] The terminal displays a list of the selected photos and asks the user for confirmation.

[1701] Step 3:

[1702] The device extracts the location information (GPS data) and timestamp (date and time of photo capture) for each selected photo.

[1703] Step 4:

[1704] The device sends the photo file, its location information, and a timestamp to the server.

[1705] Automatic calculation of travel route

[1706] Step 5:

[1707] The server analyzes the received photo data and checks the location information and timestamp.

[1708] Step 6:

[1709] The server sorts the photos in chronological order based on the location information and timestamps, and calculates the user's travel route.

[1710] Step 7:

[1711] The server plots the calculated travel route on map data to create a visual route.

[1712] Step 8:

[1713] The server stores the calculated travel route data in a database.

[1714] Travel diary generation

[1715] Step 9:

[1716] The server extracts highlights and events from the trip based on the stored travel route data.

[1717] Step 10:

[1718] The server combines the extracted highlights and photos to automatically generate a travel diary with explanatory text.

[1719] Step 11:

[1720] The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[1721] Creating a new tour plan

[1722] Step 12:

[1723] The server learns the user's preferences and patterns of places visited from the contents of the user's travel diary.

[1724] Step 13:

[1725] The generative AI analyzes other users' rating data to identify highly rated restaurants and tourist attractions.

[1726] Step 14:

[1727] The generation AI combines user preferences and high rating points to generate new travel routes.

[1728] Step 15:

[1729] The server saves the generated new travel route in the user's profile and makes it available for the user to view.

[1730] Optimization by Emotion Engine

[1731] Step 16:

[1732] The device uses an emotion engine to analyze the user's emotional state from photo and audio data.

[1733] Step 17:

[1734] The emotion engine optimizes travel routes and travel diaries based on the user's emotional state (joy, surprise, sadness, etc.).

[1735] Step 18:

[1736] The emotion engine selects new travel routes and travel highlights based on the user's current emotions and reflects them in the travel diary.

[1737] Step 19:

[1738] The server generates a new optimized travel route based on the user's emotions and saves it in the profile.

[1739] User ratings and popularity rankings

[1740] Step 20:

[1741] A user can view tour plans published by other users.

[1742] Step 21:

[1743] Users can "like" tour plans that they find good and post comments.

[1744] Step 22:

[1745] The server tally up the evaluation points for each tour plan and update the rankings in real time.

[1746] Step 23:

[1747] The server gives rewards to the top ranked tour plan posters and notifies them of the results.

[1748] Map display and navigation

[1749] Step 24:

[1750] The server integrates the generated travel route with map data.

[1751] Step 25:

[1752] The device displays the route on a map and prompts the user to begin navigation.

[1753] Step 26:

[1754] The terminal starts voice guidance to support the user's movement.

[1755] Step 27:

[1756] The user uses the navigation system to enjoy sightseeing and eating and drinking along the generated travel route.

[1757] The above is the specific program processing flow of the system that combines the emotion engine. Each step incorporates a detailed process that includes the specific operations of the user, terminal, and server.

[1758] Example 2

[1759] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1760] In recent years, there has been a growing demand for creating travel records based on photos taken during travel and sharing travel experiences. However, organizing photos, compiling route records, and creating travel diaries is time-consuming. Automating each of these processes would enable users to create and share travel records efficiently. However, current systems do not provide a means to process these processes in an integrated manner. Furthermore, it is difficult to provide personalized services that combine travel plan suggestions, evaluations, and rankings based on users' preferences and emotions. Therefore, there is a need to enable efficient creation and sharing of travel records based on travelers' photos, and to provide an optimal travel experience that suits users' emotions and preferences.

[1761] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1762] In this invention, the server includes: means for uploading photos taken by travelers to the server in bulk using a terminal; means for the terminal to extract location information and timestamps of the photos and for the server to analyze them; means for automatically compiling the traveler's travel route from the location information and timestamps of the photos; means for plotting the travel route on map data and creating a visual route; means for automatically generating the travel diary of the traveler based on the travel route; means for learning based on the user's preferences and visiting trends and using a generation AI to generate a new travel route combining highly rated restaurants and tourist attractions; means for the user to view, rate, and share the generated travel route and travel diary; means for generating a popularity ranking based on the rating and awarding rewards to the top ranked users; means for analyzing the user's emotional state and optimizing the travel route and travel diary based on the emotion; and means for the terminal to display the generated travel route on a map and provide navigation. This enables efficient creation and sharing of travel records based on photos taken by travelers and further enables providing an optimal travel experience tailored to the user's emotions and preferences.

[1763] A "traveler" is an individual who takes photographs while traveling and intends to create a travel record using the photographic data.

[1764] A "terminal" is an electronic device used by a user, such as a mobile information terminal, smartphone, or tablet, which collects, analyzes, and uploads captured photo data to a server.

[1765] A "server" is a computer system that receives, analyzes, stores, and processes data sent from travelers' terminals as necessary.

[1766] "Location information" is information about the location where a photo was taken, and is geographical data recorded in the form of GPS data or the like.

[1767] A "timestamp" is data that indicates the date and time a photo was taken, and is stored as metadata for the photo.

[1768] "Travel route" is data showing the route traveled by a traveler during a trip, and is arranged in chronological order based on location information and timestamps.

[1769] "Map data" means geographical map information provided by a geographic information system (GIS) and is data used to visually display location information and travel routes.

[1770] A "travel diary" is an automatically generated travel record that integrates a traveler's route, highlights, photos, and descriptions.

[1771] "Generative AI" refers to artificial intelligence that learns from large amounts of data and generates new information and suggestions.

[1772] A "new travel route" is a recommended route for the next trip created by the generation AI based on the user's preferences and evaluation data from other users.

[1773] "User preferences" refers to the personal tastes and interests that a traveler has based on past travel and rating data.

[1774] "Top-rated restaurants and attractions" refers to restaurants and attractions that have been highly rated by other travelers.

[1775] "Evaluation" refers to evaluation points that a user gives to a tour plan published by another user by giving it a "like" or comment.

[1776] The "popularity ranking" is a ranking list generated based on the evaluation points obtained by each tour plan, and indicates the order of the most popular tour plans.

[1777] "Emotional state" refers to the emotions such as joy, surprise, or sadness that a traveler feels in response to a particular scene or photo, and is data estimated through emotion analysis.

[1778] "Navigation" is a function that provides audio and visual guidance to help a user reach a destination using a terminal.

[1779] MODE FOR CARRYING OUT THE INVENTION

[1780] This invention is a system that automatically creates a travel log based on photos taken by a traveler and proposes a travel plan that matches the user's preferences and feelings. Specific embodiments of this system will be described below.

[1781] System configuration

[1782] Users use a mobile information terminal (hereinafter referred to as "terminal") such as a smartphone or tablet to select all photos taken during their trip and upload them through a dedicated app. The terminal extracts the location information (GPS data) and timestamp (date and time of photo) embedded in the photos and sends this data to a server.

[1783] The server analyzes the received photo data and calculates the user's travel route in chronological order based on the location information and timestamp. The calculated travel route is plotted on map data, visually representing the route. This makes the user's travel route clear.

[1784] The server also extracts highlights and events from the saved travel route data and automatically generates a travel diary with explanatory text based on the extracted information. The generated travel diary is saved in the user's profile and can be viewed by the user at any time.

[1785] Creating new tour plans and optimizing them with an emotion engine

[1786] The server learns the user's preferences and visiting place patterns from the contents of the user's travel diary, and then analyzes other users' rating data using a generative AI model. From this analysis, it identifies highly rated restaurants and tourist attractions, and proposes new travel routes by combining the user's preferences with these highly rated points. The new tour plan is saved in the user's profile.

[1787] Furthermore, the device uses an emotion engine to analyze the user's emotional state from photos and voice data. The emotion engine evaluates the degree to which the user felt emotions such as joy or surprise at a particular location, and optimizes travel routes and travel diaries based on this. For example, if the device determines that the user was particularly pleased at a particular tourist spot, it will suggest a new tour plan that includes that tourist spot.

[1788] User ratings and popularity rankings

[1789] Users can browse tour plans published by other users, click "like" on the ones they like, and post comments. The server tally up the evaluation points of each tour plan and generate a popularity ranking in real time. Tour plan posters who rank highly are awarded rewards.

[1790] Navigation Features

[1791] The terminal displays the travel route generated by the server on a map and prompts the user to start navigation. The terminal then starts voice guidance and provides directions to reach the destination specified by the user. For example, the terminal displays the route of a new tour plan in real time and provides voice guidance on the way to the destination.

[1792] Examples and prompts

[1793] Example 1: A user selects photos taken during a trip to Europe and uploads them to a server. When uploading, the device automatically extracts the GPS data and the date and time of the photo and sends them to the server.

[1794] Example 2: The server analyzes the user's itinerary and plots the route, for example, from France to Italy, on a map. Based on the location information, each photographed point is connected in a continuous manner to form the route.

[1795] Example prompt: "How can I automatically compile travel routes and generate a travel diary based on travel photos taken by the user?"

[1796] This system allows travelers to efficiently create and share travel records with other users, and also allows them to enjoy personalized travel plans.

[1797] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1798] Step 1:

[1799] The user selects a photo

[1800] Input: Travel photos taken by the user

[1801] Output: A list of selected photo files

[1802] Specific operation: The user opens the dedicated app and selects all the photos taken during the trip at once. The app selects multiple photos from the gallery and adds them to a list for uploading at once.

[1803] Step 2:

[1804] The device extracts the location and timestamp

[1805] Input: A list of photo files selected by the user

[1806] Output: Location information (GPS data) and timestamp (date and time of photo) for each photo

[1807] What it does: Your device reads the GPS data and timestamp from the photo's metadata and stores this information in temporary memory.

[1808] Step 3:

[1809] The device uploads data to the server

[1810] Input: Location and timestamp of each photo, photo file

[1811] Output: Photo data and metadata sent to the server

[1812] Specific operation: The device sends an HTTP request to the server, sending the photo data and associated metadata. It receives a success response from the server and notifies the user of the upload status.

[1813] Step 4:

[1814] The server analyzes the photo data

[1815] Input: Photo data and metadata uploaded to the server

[1816] Output: Chronologically ordered locations and timestamps

[1817] What it does: The server retrieves the photo data stored in the database and analyzes the metadata of each photo to check the location and timestamp.

[1818] Step 5:

[1819] The server calculates the route.

[1820] Input: Organized location and timestamp

[1821] Output: Calculated travel route

[1822] Specific operation: The server sorts the location information based on time series, and uses a specific algorithm to connect consecutive points to form the user's travel route.

[1823] Step 6:

[1824] The server plots the map data

[1825] Input: Calculated travel route

[1826] Output: Plotted map data

[1827] Specific operation: The server calls a map API (e.g., Google Maps API) to plot the route and reflects the plotted data in the user interface.

[1828] Step 7:

[1829] The server stores the travel route data

[1830] Input: Plotted map data and travel route information

[1831] Output: Travel route data stored in a database

[1832] Specific operation: The server inserts the travel route data into a specific table in the database and checks whether the insertion is successful.

[1833] Step 8:

[1834] The server extracts the highlights

[1835] Input: Travel route data

[1836] Output: Extracted highlight points and events

[1837] Specific operation: The server runs a highlight extraction algorithm to derive and list notable points.

[1838] Step 9:

[1839] The server automatically generates a travel diary

[1840] Input: Extracted highlight locations and events, corresponding photo data

[1841] Output: Automatically generated travel diary

[1842] Specific operation: The server uses a text generation model based on the highlight points to combine appropriate photos and descriptions to create a travel diary.

[1843] Step 10:

[1844] The server stores the travel diary and provides it to the user.

[1845] Input: Auto-generated travel diary

[1846] Output: Travel diary saved in the user's profile

[1847] Specific operation: The server saves the travel diary data in the database and then reflects it on the user's profile page.

[1848] Step 11:

[1849] The server learns the user's preferences

[1850] Input: User's travel diary data, rating data

[1851] Output: Learned user preferences and visiting habits

[1852] What it does: The server uses machine learning algorithms to analyze your behavioral patterns and add your preferences to your profile.

[1853] Step 12:

[1854] Generative AI analyzes the evaluation data

[1855] Input: Other users' ratings data

[1856] Output: A list of highly rated restaurants and attractions

[1857] What it does: The generative AI queries user rating data, performs statistical analysis, and lists specific places and services.

[1858] Step 13:

[1859] Generation AI generates new travel routes

[1860] Input: User preferences, likes

[1861] Output: Newly generated travel route

[1862] Specific operation: The generation AI runs a route generation algorithm based on preferences and high rating points, then verifies and optimizes the generated route.

[1863] Step 14:

[1864] The server stores the route and serves it to the user.

[1865] Input: New tour plan

[1866] Output: A new tour plan saved in the user's profile.

[1867] Specific behavior: The server saves the new tour plan in the database and displays the saved tour plan in the user interface.

[1868] Step 15:

[1869] The device performs emotion analysis

[1870] Input: Photo and audio data

[1871] Output: Emotion analysis results

[1872] Specific operation: The device sends photo and audio data to the emotion analysis module, receives and logs the emotion analysis results.

[1873] Step 16:

[1874] Emotion engine optimizes routes and travel diaries

[1875] Input: Sentiment analysis results

[1876] Output: Optimized travel route and travel diary

[1877] What it does: The emotion engine reevaluates highlights, routes, and descriptions based on emotion data and makes adjustments as needed.

[1878] Step 17:

[1879] Emotion engine makes suggestions

[1880] Input: Sentiment analysis results

[1881] Output: Improved tour plan

[1882] Specific behavior: The emotion engine generates a customized tour plan and notifies the user of the new tour plan.

[1883] Step 18:

[1884] A user browses tour plans

[1885] Input: Published tour plan

[1886] Output: Rating and comments

[1887] Specific operation: Users can view tour plans published by other users, select the tour plan they want to view, and rate and comment on it.

[1888] Step 19:

[1889] The server updates the rankings

[1890] Input: Reputation points

[1891] Output: Updated ranking list

[1892] Specific operation: The server retrieves reputation points from the database, aggregates them, and updates the ranking list.

[1893] Step 20:

[1894] The server will grant the reward

[1895] Input: Updated ranking list

[1896] Output: Reward granted notification

[1897] Specific operation: The server inserts reward data into the top ranking users and notifies them of the reward via the notification system.

[1898] Step 21:

[1899] The device displays the route

[1900] Input: Route data from the server

[1901] Output: Route map displayed on the terminal

[1902] Specific operation: The device displays map data on the screen and prompts the user to start navigation.

[1903] Step 22:

[1904] The device will provide voice guidance

[1905] Input: Current location data and route information

[1906] Output: Voice navigation

[1907] Specific operation: The device determines the current location based on GPS data and provides voice guidance to the destination.

[1908] (Application example 2)

[1909] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1910] Currently, there is a lack of means to automatically generate travel records and effectively manage photos and routes after a trip. Furthermore, there is no system that can suggest new routes and services that reflect the traveler's feelings and preferences. This makes the process of recording experiences during and after a trip cumbersome, making it difficult to provide optimal services to travelers.

[1911] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for uploading photos taken by travelers to the data processing device in bulk; means for automatically aggregating the traveler's travel route from the location information and timestamps of the photos; means for automatically generating the travel record of the traveler based on the travel route; means for generating a new travel route incorporating highly rated restaurants and tourist attractions based on the generated travel log; means for users to view and rate the generated travel route; means for generating a popularity index based on the evaluation and awarding rewards to those with the highest index; means for displaying the photos on a map based on the locations where they were taken and visualizing the visit route; and means for analyzing the user's emotional state from the photo data and proposing new services based on the analysis. This allows travelers to automatically generate and manage their travel routes and travel records, and further enables optimal service suggestions based on their emotions during the trip.

[1912] "Photos" are image data taken by travelers during their trip.

[1913] "Data Processing Device" means a combination of hardware and software that allows travelers to upload photos taken in bulk.

[1914] "Location information" is the geographic coordinate data where the photo was taken.

[1915] A "timestamp" is data that indicates the date and time a photo was taken.

[1916] A "travel route" is a traveler's travel history calculated based on location information and timestamps.

[1917] A "travel log" is a document automatically generated based on a traveler's travel route and photographs.

[1918] "Food and beverage establishments" refers to restaurants and bars along the travel route of travelers.

[1919] "Tourist destination" refers to a tourist attraction that a tourist visits.

[1920] "User" refers to a traveler or user of the system.

[1921] "Evaluation" refers to feedback provided by users regarding generated travel routes and services.

[1922] "Popularity index" is ranking data compiled from evaluation points.

[1923] "Rewards" are perks and incentives given to those who rank highly in popularity indicators.

[1924] A "visit route" is a route on a map that connects the places visited by a traveler in order.

[1925] "Emotional state" refers to the emotional state of a traveler analyzed from photo data.

[1926] "New services" refers to new services and benefits that are proposed based on the analyzed emotional state.

[1927] MODE FOR CARRYING OUT THE INVENTION

[1928] The system for implementing this invention automatically compiles travel routes based on photos taken by travelers during their trips, generates travel records, and utilizes AI to propose new services based on the travelers' emotional state.

[1929] 1. System Program Description

[1930] Hardware and software used:

[1931] Hardware:

[1932] Smartphones (for travelers to take and upload photos)

[1933] Servers (for data processing and storage)

[1934] Databases (to store traveller data and travel records)

[1935] software:

[1936] Pillow (a Python library for extracting photo metadata)

[1937] geopy (a Python library for analyzing geolocation data)

[1938] Basemap (a Python library for displaying travel routes on a map)

[1939] Emotion Engine API (external service for analyzing emotions from photos)

[1940] 2. System operation explanation

[1941] User Action:

[1942] 1. The user uses a dedicated app on their smartphone to select and upload all the photos they took during their trip.

[1943] 2. Automatically extract metadata (location and timestamp) from uploaded photos and send it to the server.

[1944] Server Action:

[1945] 1. The server analyzes the received photo data, checks the location information and timestamp, and calculates the user's movement route in chronological order based on this.

[1946] 2. The calculated travel path is plotted on map data to generate a visual route, which is then used to automatically generate a travel record for the traveler.

[1947] Sentiment analysis and service suggestions:

[1948] 1. The server analyzes the user's emotional state from the photo data using an emotion engine.

[1949] 2. Based on the analysis results, new services are proposed according to the user's emotions. For example, the system suggests nearby tourist spots and restaurants based on the places where the user feels particularly happy.

[1950] Specific examples:

[1951] 1. A tourist visits Paris, takes a photo, and uploads it. The photo includes location information such as the Eiffel Tower and the Louvre.

[1952] 2. The server uses an emotion engine to analyze the photos and determine what the traveler is enjoying, and recommends nearby cafes and museums as the next travel destination.

[1953] Example prompt sentence:

[1954] Suggest new services and places to visit based on the location and emotion data of places the user has visited. The user would like to see new places and services offered near areas where they have experienced particularly high levels of joy, such as:

[1955] 1. Highly rated cafes and restaurants

[1956] 2. Popular tourist destinations

[1957] 3. Promotions and offers available

[1958] This system allows travelers to automatically generate and manage their travel routes and travel records, and also allows them to receive recommendations for optimal services based on their emotions while traveling.

[1959] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1960] Step 1:

[1961] Users use a dedicated smartphone app to select and upload multiple photos taken during their trip. The input is the photo files selected by the user, and the output is the selected photo files being sent to the server.

[1962] Step 2:

[1963] The device automatically extracts location information (GPS data) and timestamp (date and time of photo) from uploaded photo files. The input is the photo file, and the output is metadata including location information and timestamp. This metadata is sent to the server along with the photo file.

[1964] Step 3:

[1965] The server analyzes the received photo data and checks the location information and timestamp for each photo. The input is the uploaded photo data and its metadata, and the output is a list of the location information and timestamp corresponding to each photo data.

[1966] Step 4:

[1967] The server calculates the user's route in chronological order based on the location information and timestamps. The input is a list of location information and timestamps, and the output is the user's route data sorted in chronological order. This is plotted on map data to generate a visual route.

[1968] Step 5:

[1969] The server automatically generates a travel record for the traveler based on the generated travel route. The input is travel route data, and the output is text and image data of the travel record.

[1970] Step 6:

[1971] The server sends the photo data to the emotion engine API to analyze the user's emotional state. The input is the photo data and its metadata, and the output is the emotion analysis result, which is a specific emotion (e.g., joy, surprise, sadness) that reflects the user's travel experience.

[1972] Step 7:

[1973] The server proposes new services and travel routes based on the emotion analysis results. The input is the emotion analysis results and travel records, and the output is a list of recommended new services and travel routes.

[1974] Step 8:

[1975] The terminal notifies the user of new services and travel routes provided by the server and allows the user to view them. The input is a recommendation list sent from the server, and the output is recommendation information displayed on the user's smartphone.

[1976] Step 9:

[1977] Users rate new services and travel routes provided and provide feedback. The input is the user's ratings and comments, and the output is the rating data sent to the server.

[1978] Step 10:

[1979] The server aggregates user rating data, generates a popularity index, updates rankings based on that index, and awards rewards to top users. The input is user rating data, and the output is updated rankings and reward notifications.

[1980] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1981] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1982] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1983] [Fourth embodiment]

[1984] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1985] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1986] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1987] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1988] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1989] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1990] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1991] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1992] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1993] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1994] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1995] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1996] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1997] MODE FOR CARRYING OUT THE INVENTION

[1998] System Overview

[1999] This system automatically compiles travel routes and generates travel diaries based on photos taken by travelers after their trip. It also uses AI to propose new tour plans and provides a function that allows users to rate and share them.

[2000] Bulk photo posting

[2001] 1. After the trip, the user opens the dedicated app and selects all the photos taken during the trip.

[2002] 2. The device extracts the location information (GPS data) and timestamp (date and time of shooting) of the selected photo.

[2003] 3. The device uploads the photo file, its location information, and timestamp to the server.

[2004] Specific examples

[2005] Users select photos they have taken around Mount Fuji in the app and upload them to the server all at once.

[2006] When uploading, the device automatically extracts the GPS data and shooting date and time contained in the photo and sends them to the server.

[2007] Automatic calculation of travel route

[2008] 1. The server analyzes the received photo data and checks the location information and timestamp.

[2009] 2. The server calculates the user's route in chronological order based on the location information and timestamp.

[2010] 3. The server stores the calculated route in a database.

[2011] Specific examples

[2012] The server analyzes the user's itinerary and plots the route from the base of Mount Fuji to the summit on a map.

[2013] Based on the location information, each photographed point is connected in a continuous manner to form a route.

[2014] Travel diary generation

[2015] 1. The server extracts highlights and events from the calculated travel route.

[2016] 2. The server automatically generates a travel diary based on the extracted highlights, combining photos and descriptions.

[2017] 3. The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[2018] Specific examples

[2019] The server automatically inserts photos of summits and particularly beautiful views into the travel diary as highlights and generates detailed descriptions.

[2020] The travel diary is saved in the user's profile, allowing them to follow the overall picture of their Mount Fuji climb chronologically.

[2021] Creating a new tour plan

[2022] 1. The server learns the user's preferences and patterns of places visited from the contents of their travel diary.

[2023] 2. The generating AI analyzes rating data from other users and identifies highly rated restaurants and tourist attractions.

[2024] 3. The generation AI combines the user's preferences and high rating points to generate the optimal travel route as a new tour plan.

[2025] 4. The server provides the newly generated travel route to the user.

[2026] Specific examples

[2027] The generative AI learns the user's preferences and suggests new tour plans that incorporate other tourist attractions and restaurants.

[2028] For example, if a user likes the view from a mountaintop, they can add other scenic spots.

[2029] User ratings and popularity rankings

[2030] 1. Users view tour plans published by other users.

[2031] 2. Users can "like" tour plans they find good or post comments.

[2032] 3. The server will tally the evaluation points for each tour plan and update the rankings in real time.

[2033] 4. The server awards rewards to the top-ranked users and notifies them of the results.

[2034] Specific examples

[2035] Users view Yamanashi Prefecture tour plans published by other users and "like" them.

[2036] The server tallys likes and comments in real time and updates the popularity rankings.

[2037] Map display and navigation

[2038] 1. The server integrates the generated travel route with map data.

[2039] 2. The device will display the route on a map and enter navigation mode.

[2040] 3. The device provides voice guidance to support the user's movement.

[2041] Specific examples

[2042] The device displays the route of the new tour plan on the user's smartphone and begins navigation in real time.

[2043] Voice guidance will explain how to reach your destination and where to stop.

[2044] Such a system allows travelers to effortlessly record their trips, get travel plans tailored to their individual preferences, and improve the quality of their travel experience by sharing information and ratings with other travelers.

[2045] The processing flow will be explained below.

[2046] Program processing flow

[2047] Bulk photo posting

[2048] Step 1:

[2049] After traveling, users open a dedicated app and select all the photos they want to post at once.

[2050] Step 2:

[2051] The terminal displays a list of the selected photos and asks the user for confirmation.

[2052] Step 3:

[2053] The device extracts the location information (GPS data) and timestamp (date and time of photo capture) for each selected photo.

[2054] Step 4:

[2055] The device sends the photo file, its location information, and a timestamp to the server.

[2056] Automatic calculation of travel route

[2057] Step 5:

[2058] The server analyzes the received photo data and checks the location information and timestamp.

[2059] Step 6:

[2060] The server sorts the photos in chronological order based on the location information and timestamps, and calculates the user's travel route.

[2061] Step 7:

[2062] The server plots the calculated travel route on map data to create a visual route.

[2063] Step 8:

[2064] The server stores the calculated travel route data in a database.

[2065] Travel diary generation

[2066] Step 9:

[2067] The server extracts highlights and events from the trip based on the stored travel route data.

[2068] Step 10:

[2069] The server combines the extracted highlights and photos to automatically generate a travel diary with explanatory text.

[2070] Step 11:

[2071] The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[2072] Creating a new tour plan

[2073] Step 12:

[2074] The server learns the user's preferences and patterns of places visited from the contents of the user's travel diary.

[2075] Step 13:

[2076] The generative AI analyzes rating data from other users to identify highly rated restaurants and tourist attractions.

[2077] Step 14:

[2078] The generation AI combines user preferences and high rating points to generate new travel routes.

[2079] Step 15:

[2080] The server saves the generated new travel route in the user's profile and makes it available for the user to view.

[2081] User ratings and popularity rankings

[2082] Step 16:

[2083] A user can view tour plans published by other users.

[2084] Step 17:

[2085] Users can "like" tour plans they find good and post comments.

[2086] Step 18:

[2087] The server tally up the evaluation points for each tour plan and update the rankings in real time.

[2088] Step 19:

[2089] The server gives rewards to the top ranked tour plan posters and notifies them of the results.

[2090] Map display and navigation

[2091] Step 20:

[2092] The server integrates the generated travel route with map data.

[2093] Step 21:

[2094] The device displays the route on a map and prompts the user to begin navigation.

[2095] Step 22:

[2096] The terminal starts voice guidance to support the user's movement.

[2097] Step 23:

[2098] The user uses the navigation system to enjoy sightseeing and eating and drinking along the generated travel route.

[2099] The above is the specific flow of program processing for this system. Each step incorporates a detailed process that includes the specific operations of the user, terminal, and server.

[2100] Example 1

[2101] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2102] There is a need for a system that can efficiently create detailed travel records based on photos taken by travelers, suggest new travel plans, and share information and ratings with other travelers in real time. However, conventional systems require the time-consuming task of manually classifying and organizing photos, making it difficult to automatically generate detailed travel records including trip highlights. It is also difficult to automatically generate new travel plans tailored to individual preferences and provide navigation functions integrated with map data.

[2103] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2104] In this invention, the server includes a means for uploading image data taken by a traveler to the server in bulk, a means for automatically tallying up the travel route of the traveler from the location information and timestamp of the image data, and a means for automatically generating a travel record of the traveler based on the travel route. This makes it possible to effectively utilize the photos taken by the traveler and automatically generate a detailed travel record based on a chronological order.

[2105] "Image data" refers to digital data of photographs and videos taken by travelers.

[2106] "Server" refers to a computer system for receiving and processing data from terminals connected over a network.

[2107] "Location Information" refers to geographic latitude and longitude data obtained using technologies such as GPS.

[2108] A "timestamp" refers to date and time information that indicates when digital data was created, modified, or accessed at a certain time.

[2109] A "travel route" refers to a route that connects multiple locations visited by a traveler within a specific period in chronological order.

[2110] A "travel journal" is a record of the places and events visited during a trip, compiled in photographs and text.

[2111] A "generative AI model" refers to an artificial intelligence algorithm that uses machine learning techniques to generate new data and plans.

[2112] "Restaurant" refers to a commercial establishment that sells food and beverages and allows customers to eat and drink.

[2113] "Tourist attractions" refer to popular tourist destinations and historical and cultural sites.

[2114] "Smart devices" refer to portable electronic devices that can connect to the Internet, such as smartphones and tablets.

[2115] "Navigation" refers to a system and its functions that indicate the optimal route from the current location to the destination.

[2116] "Evaluation" refers to the act of rating a tour plan or service created by another user after using the service.

[2117] "Popularity ranking" refers to a ranking list created by compiling feedback such as user ratings and "likes."

[2118] "Rewards" refers to incentives such as points or vouchers that the system gives to users.

[2119] This system is designed to automatically generate travel records and propose new travel plans based on photos taken by travelers. The system is mainly composed of three elements: a server, a terminal, and a user, and performs the following processes:

[2120] Bulk photo posting

[2121] After the trip, the user opens the app and selects photos taken during the trip from the gallery. For example, select photos taken during a trip to Mount Fuji. The device extracts the location information (GPS data) and timestamp (date and time of the photo) from the selected photo. Using this information, the device uploads the photo file and its associated location information and timestamp to the server.

[2122] Automatic calculation of travel route

[2123] The server analyzes the received photo data and checks the location and timestamp of each photo. Based on the location and timestamp, it calculates the user's movement route in chronological order and stores this movement route in a database. For example, it generates a route from the base of Mount Fuji to the summit and plots it on a map.

[2124] Travel diary generation

[2125] The server extracts highlights and events from the trip based on the calculated route. Using these highlights, the server automatically generates a travel log by combining corresponding photos and descriptions. The generated travel log is saved in the user's profile and can be viewed freely by the user.

[2126] Creating a new tour plan

[2127] The server learns the user's preferences and patterns of places visited from the contents of the user's travel log. Then, using a generative AI model, it analyzes the rating data from other users and identifies highly rated restaurants and tourist spots. Based on this, it generates an optimal travel route by combining the user's preferences and high rating points. This newly generated travel route is provided to the user by the server.

[2128] User ratings and popularity rankings

[2129] Users can browse new travel plans published by other users, click "like" on the ones they like, and post comments. The server aggregates this evaluation data and updates the popularity rankings in real time. It then awards rewards to users who rank highly and notifies them of the results.

[2130] Map display and navigation

[2131] The server integrates the generated route with map data and provides navigation functions. The device displays the route on a map and provides voice guidance to support the user's movement. For example, when navigation begins, voice guidance such as "Turn left" or "You have arrived at the next tourist spot" is provided.

[2132] Prompt Sentence Examples

[2133] "Upload photos from your Mount Fuji trip and create a travel route."

[2134] "Generate a new tour plan based on your travel records."

[2135] "Please rate tour plans created by other users."

[2136] In this way, the system allows travelers to easily organize their travel records, automatically suggest new itineraries based on their individual preferences, and improve the quality of their travel experience by sharing and rating information with other travelers.

[2137] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2138] Step 1:

[2139] After the trip, the user opens the dedicated app and selects all the image data taken during the trip from the gallery.

[2140] Input: Multiple image files selected by the user.

[2141] Output: A list of selected image files.

[2142] Specific operation: The user taps the "Upload Photos" button in the app and selects multiple images taken during their trip. The device stores the image files as a list.

[2143] Step 2:

[2144] The device extracts location information (GPS data) and timestamps (date and time of photo) from each selected image file.

[2145] Input: A list of selected image files.

[2146] Output: Image metadata including location and timestamp.

[2147] How it works: The device analyzes the metadata of each image file to obtain location information and the date and time the image was taken, and temporarily stores this data in memory.

[2148] Step 3:

[2149] The device uploads them to a server along with an image file containing the extracted location information and timestamp.

[2150] Input: Image metadata including location and timestamp.

[2151] Output: Image data uploaded to the server.

[2152] Specific operation: The device packs image files containing location information and timestamps into packets and sends them via the server's upload API.

[2153] Step 4:

[2154] The server analyzes the received image data and checks the location information and timestamp of each file.

[2155] Input: Image data uploaded to the server.

[2156] Output: Parsed location and timestamp.

[2157] Specific operation: The server decodes the received network packets and stores the binary data of the image file, including location information and timestamp, in a database.

[2158] Step 5:

[2159] The server calculates the user's movement route in chronological order based on the location information and timestamp.

[2160] Input: Parsed location and timestamp.

[2161] Output: Travel path data sorted in chronological order.

[2162] Specific operation: The server analyzes the location information as latitude and longitude data, sorts it in chronological order by the date and time of the photo, and connects the points on the map to form a movement route.

[2163] Step 6:

[2164] The server stores the calculated travel route data in a database.

[2165] Input: Travel path data arranged in chronological order.

[2166] Output: Travel path data stored in a database.

[2167] Specific operation: The server inserts the generated travel route data into a dedicated travel route table and stores it in association with each user.

[2168] Step 7:

[2169] Based on the calculated route, the server extracts highlights and events of the trip.

[2170] Input: Time-series travel path data.

[2171] Output: Highlight extracted data.

[2172] Specific operation: The server identifies the most frequently visited or long-stayed locations from photos taken at different locations and lists them as highlights.

[2173] Step 8:

[2174] Based on the highlights, the server automatically generates a travel record by combining corresponding photos and descriptions.

[2175] Input: Highlight extraction data.

[2176] Output: An automatically generated trip log.

[2177] What it does: The server uses a text generation API to generate a description of each highlight and formats it along with a photo in HTML or another display format.

[2178] Step 9:

[2179] The server stores the generated travel record in the user's profile and makes it available for the user to view.

[2180] Input: Auto-generated trip log.

[2181] Output: Trip records stored in the user profile.

[2182] What it does: The server associates and stores the trip record data in the user's profile table and provides a link to display it via the app's UI.

[2183] Step 10:

[2184] The server learns the user's preferences and patterns of places visited from the contents of their travel log.

[2185] Input: Saved trip log data.

[2186] Output: User preferences and visiting place patterns.

[2187] Specific operation: The server analyzes the user's past travel history and uses a machine learning algorithm to analyze the categories and frequency of visited places.

[2188] Step 11:

[2189] The generative AI model analyzes rating data from other users to identify highly rated restaurants and tourist spots.

[2190] Input: Rating data from other users.

[2191] Output: A list of highly rated restaurants and attractions.

[2192] How it works: The generative AI model aggregates reviews and rating points from other users and lists highly rated spots.

[2193] Step 12:

[2194] The generative AI model combines user preferences and high-rated points to generate the optimal tour plan as a new travel route.

[2195] Input: User preferences, visiting location patterns, and high rating points.

[2196] Output: The newly generated tour plan.

[2197] How it works: The generative AI model integrates the user's preferred patterns and highly rated spots to create an efficient tour plan that takes into account transportation options and time of day.

[2198] Step 13:

[2199] The server provides the generated travel route to the user.

[2200] Input: The newly generated tour plan.

[2201] Output: The tour plan provided to the user account.

[2202] Specific operation: The server associates the newly generated tour plan with the user's account and notifies the user via the app's notification function.

[2203] Step 14:

[2204] Users can browse tour plans published by other users, click "like" on plans they like, and post comments.

[2205] Input: Published tour itinerary.

[2206] Output: Rating data such as likes and comments.

[2207] What happens: A user opens the app's "Popular Plans" section, browses the list of tour plans published by other users, and enters a rating.

[2208] Step 15:

[2209] The server compiles the evaluation data and updates the popularity rankings in real time.

[2210] Input: Rating data such as likes and comments.

[2211] Output: Updated popularity rankings.

[2212] Specific operation: The server periodically aggregates the evaluation data, calculates the score for each tour plan, and updates the ranking table.

[2213] Step 16:

[2214] The server awards rewards to top-ranked users and notifies them of the results.

[2215] Input: Updated popularity rankings.

[2216] Output: The user account to which the reward was granted.

[2217] Specific operation: The server will add points or vouchers to the top-ranked user accounts and notify the results through the notification system.

[2218] Step 17:

[2219] The server integrates the generated travel route with map data and provides navigation functions.

[2220] Input: Generated travel path data.

[2221] Output: Travel path integrated with map data.

[2222] Specific operation: The server uses a map service (e.g., Google Maps API) to plot the generated route on an interactive map.

[2223] Step 18:

[2224] The device displays the route on a map and provides real-time navigation with voice guidance.

[2225] Input: Travel path integrated with map data.

[2226] Output: Route displayed on map and voice guidance.

[2227] Specific operation: The user starts navigation mode, and the device tracks GPS information in real time and provides voice guidance.

[2228] (Application example 1)

[2229] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2230] Modern travelers take many photos during their trips and want to use them to record their journeys. However, manually organizing photos and creating travel routes and travel diaries after a trip is tedious and time-consuming. Furthermore, there are limited ways to efficiently gather information on new routes, tourist spots, and restaurants for the next trip. Conventional methods make it difficult to easily obtain reviews from other travelers and obtain individually optimized tourist information. Furthermore, there is a lack of means to obtain current location information in real time and efficiently record visited places. To solve these problems, a system is needed that allows travelers to easily record their trips and provides highly personalized tourist information in real time.

[2231] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2232] In this invention, the server includes: means for uploading photos taken by travelers to the server in bulk; means for automatically compiling the travel route of the traveler based on the location information and timestamps of the photos; means for automatically generating a travel diary for the traveler based on the travel route; means for generating a new travel route incorporating highly rated facilities and attractions based on the generated travel diary; means for the user to view and rate the generated travel route; means for generating a popularity ranking based on the ranking and awarding rewards to top-ranked users; means for tracking and recording places visited by the user using a smart device in real time; and means for displaying information about the vicinity of the user's current location on the smart device in real time and providing recommended tourist spots and shops. This allows travelers to automatically record their travels without hassle, obtain useful tourist information in real time, and efficiently plan their next trip by sharing information and rating with other travelers.

[2233] "Photo upload" refers to the operation of sending image data taken by a traveler to a server all at once.

[2234] "Location information" refers to data that indicates the geographic coordinates (latitude and longitude) of the location where a photograph was taken.

[2235] A "timestamp" is information that indicates the date and time a photo was taken.

[2236] "Automatic travel route compilation" refers to the process of automatically calculating and organizing the order of a traveler's visits and routes based on the location information and timestamps of photos.

[2237] A "travel diary" is a record of photographs taken during a trip, with accompanying descriptions, arranged in chronological order.

[2238] "Highly rated facilities and attractions" refers to tourist destinations and stores that have received high ratings based on user ratings and reviews.

[2239] "Generation of new travel routes" refers to the process of creating new recommended travel routes based on existing travel diaries and high-rating information.

[2240] "User ratings" refer to feedback such as "likes" and comments on travel routes and travel diaries shared by other users.

[2241] "Popularity ranking" refers to a list of travel routes and travel diaries ranked based on user ratings.

[2242] "Giving rewards" means providing rewards such as benefits or points to those who rank highly.

[2243] "Smart device" refers to a portable electronic device that can provide and display digital information in real time, such as a smartphone, smart glasses, or head-mounted display.

[2244] "Place tracking" refers to the process of recording the places visited by a user in real time using a smart device.

[2245] "Displaying surrounding information" means displaying information about nearby tourist attractions and stores on the smart device display based on the current location.

[2246] "Recommended tourist attractions and stores" refer to tourist attractions and stores that are attractive to users and selected by the generative AI model.

[2247] This invention realizes a system that automatically generates travel diaries based on photos taken by travelers and provides new tourist information. The system is composed of the following elements.

[2248] Bulk upload and analysis of photos

[2249] The server provides a means for travelers to upload photos taken in bulk. After their trip, travelers open a dedicated app, select all photos taken during their trip, and upload them. The device extracts the location information (GPS data) and timestamp (date and time of photo) of the selected photos, and sends the photo files, along with their location information and timestamp, to the server. This reduces the user's workload and enables efficient data transmission.

[2250] Automatic calculation of travel route

[2251] The server analyzes the received photo data and checks the location information and timestamp. This allows it to calculate the traveler's route in chronological order. For analysis, Python and TensorFlow are used, for example, to process the data based on the location information and timestamp. The calculated route is then stored in a database. This automatically manages a detailed record of the places the traveler has visited.

[2252] Automatic generation of travel diary

[2253] The server extracts highlights and events from the trip based on the calculated travel route. A generative AI model is used to extract the highlights and generate detailed descriptions for each point. This automatically generates a travel diary that combines photos and descriptions. The generated travel diary is saved in the user's profile and can be viewed by the user.

[2254] Real-time tourist information

[2255] The server uses the generated travel diary content and rating data from other users to identify highly rated facilities and attractions. Using a generative AI model, the server analyzes the user's preferences and ratings from other users to generate new travel routes. This allows travelers to receive optimal sightseeing information for their next trip in real time. Additionally, if the user is using a smart device (smartphone, smart glasses, etc.), the terminal displays real-time information about the area around the current location and provides recommended tourist spots and stores.

[2256] User ratings and rankings

[2257] The server provides a means for users to view and rate publicly posted travel routes and travel diaries. Users can "like" tour plans they find good or post comments, and rating points are tallied in real time. The server then generates popularity rankings based on the ratings and operates a system that awards rewards to those at the top of the rankings. This promotes the sharing of travel information and the activation of the community.

[2258] Specific examples

[2259] Visiting tourist attractions: When a user visits Mount Fuji or a temple, the route traveled is automatically recorded, and the travel diary generated after the trip can be viewed on the smart glasses.

[2260] Restaurant guide: An example in which AI provides real-time guidance on highly rated restaurants near tourist spots and displays it on the smart glasses.

[2261] Prompt Sentence Examples

[2262] "When a user visits a particular tourist spot, the smart glasses will display the history and background of that place in real time, record the data of the places visited, and automatically generate a travel diary. In addition, the glasses will provide personalized recommendations based on other users' ratings."

[2263] In this way, the specific embodiment of the present invention allows travelers to efficiently keep a record of their trip and obtain useful tourist information in real time.

[2264] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2265] Step 1:

[2266] Bulk upload photos

[2267] Input: Photos taken by the user during their trip

[2268] The server provides a means for users to select photos taken during their trip in bulk through a dedicated app and upload them all to the server. The device extracts the location information (GPS data) and timestamp (date and time of photo shooting) of the selected photos and sends them to the server along with the photo files. This streamlines the upload process and aggregates photo data on the server.

[2269] Output: Photo data uploaded to the server

[2270] Step 2:

[2271] Automatic calculation of travel route

[2272] Input: Uploaded photo data (including location and timestamp)

[2273] The server analyzes the location information and timestamps of the received photo data and sorts them in chronological order. It then automatically compiles the traveler's route based on the location information and timestamps. Specifically, it calculates the distance between each photo position using Python's Geopy library and plots the photo data as a continuous route.

[2274] Output: Calculated travel route

[2275] Step 3:

[2276] Automatic generation of travel diary

[2277] Input: Calculated travel route

[2278] The server extracts highlights and events from the trip based on the calculated route. It uses a generative AI model to generate detailed descriptions for each point. It then combines the photos and descriptions to automatically generate a travel diary. The generated travel diary is saved in the user's profile and can be viewed later.

[2279] Output: Automatically generated travel diary

[2280] Step 4:

[2281] Real-time tourist information

[2282] Input: User's current location information, other users' rating data

[2283] The server identifies highly rated facilities and tourist attractions based on the user's current location information and rating data from other users. Using a generative AI model, it generates new sightseeing routes by analyzing the user's preferences and ratings from other users. The terminal also displays information about the area around the user's current location in real time via the smart device, providing guidance to recommended tourist spots and stores.

[2284] Output: Real-time generated tourist information and new travel routes

[2285] Step 5:

[2286] User ratings and rankings

[2287] Input: Travel routes and travel diaries published by other users, user feedback data

[2288] The server provides a means for users to view and rate publicly posted travel routes and travel diaries. Users can "like" routes they find good and post comments. The server tally evaluation points in real time and generate popularity rankings. It also operates a system that awards rewards to those at the top of the rankings.

[2289] Output: Generated ranking and reward information

[2290] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2291] MODE FOR CARRYING OUT THE INVENTION

[2292] System Overview

[2293] This system automatically compiles travel routes and generates travel diaries based on photos taken by travelers after their trip. It also utilizes generative AI to propose new tour plans, allowing users to rate and share them. It also combines an emotion engine that recognizes users' emotions to provide a more personalized travel experience.

[2294] Bulk photo posting

[2295] 1. After the trip, the user opens the dedicated app and selects all the photos taken during the trip.

[2296] 2. The device extracts the location information (GPS data) and timestamp (date and time of shooting) of the selected photo.

[2297] 3. The device uploads the photo file, its location information, and timestamp to the server.

[2298] Specific examples

[2299] The app allows users to select all photos they have taken during their trip to Europe and upload them to the server.

[2300] When uploading, the device automatically extracts the GPS data and shooting date and time contained in the photo and sends them to the server.

[2301] Automatic calculation of travel route

[2302] 1. The server analyzes the received photo data and checks the location information and timestamp.

[2303] 2. The server calculates the user's route in chronological order based on the location information and timestamp.

[2304] 3. The server plots the calculated travel route on map data to create a visual route.

[2305] 4. The server stores the calculated route data in a database.

[2306] Specific examples

[2307] The server analyzes the user's itinerary and plots the route traveled, for example, from France to Italy, on a map.

[2308] Based on the location information, each photographed point is connected in a continuous manner to form a route.

[2309] Travel diary generation

[2310] 1. The server extracts highlights and events from the trip based on the saved travel route data.

[2311] 2. The server combines the extracted highlights and photos to automatically generate a travel diary with explanatory text.

[2312] 3. The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[2313] Specific examples

[2314] The server automatically selects highlights such as the Eiffel Tower and the Colosseum and generates detailed descriptions.

[2315] Travel diaries are saved in a user's profile, allowing them to track their entire journey in chronological order.

[2316] Creating a new tour plan

[2317] 1. The server learns the user's preferences and patterns of places visited from the contents of the user's travel diary.

[2318] 2. The generating AI analyzes other users' rating data and identifies highly rated restaurants and tourist attractions.

[2319] 3. The generation AI combines the user's preferences and high rating points to generate a new travel route.

[2320] 4. The server saves the generated new travel route in the user's profile and makes it available for the user to view.

[2321] Specific examples

[2322] Generative AI learns the user's preferences and suggests new tour plans that incorporate highly rated restaurants and tourist attractions.

[2323] For example, if the user loves pasta, add a popular Italian restaurant to the route.

[2324] Optimization by Emotion Engine

[2325] 1. The device uses an emotion engine to analyze the user's emotional state from photo and audio data.

[2326] 2. The emotion engine optimizes travel routes and travel diaries based on the user's emotional state (joy, surprise, sadness, etc.).

[2327] 3. The emotion engine makes suggestions based on the user's current emotions.

[2328] Specific examples

[2329] The emotion engine analyzes the user's travel photos and comments, and if it finds that they are particularly happy at a particular place, it will suggest a new travel plan that includes that place.

[2330] For example, if the user has many photos of themselves having fun at Huis Ten Bosch, a tour plan will be generated that includes other tourist spots nearby.

[2331] User ratings and popularity rankings

[2332] 1. Users view tour plans published by other users.

[2333] 2. Users can "like" tour plans they find good or post comments.

[2334] 3. The server will tally the evaluation points for each tour plan and update the rankings in real time.

[2335] 4. The server will award rewards to the top-ranked tour plan contributors and notify them of the results.

[2336] Specific examples

[2337] A user browses Tokyo tour plans published by other users and "likes" them.

[2338] The server tallys likes and comments in real time and updates the popularity rankings.

[2339] Map display and navigation

[2340] 1. The server integrates the generated travel route with map data.

[2341] 2. The device displays the route on a map and prompts the user to begin navigation.

[2342] 3. The device starts voice guidance to support the user's movement.

[2343] Specific examples

[2344] The device displays the route of the new tour plan on the user's smartphone and begins navigation in real time.

[2345] Voice guidance will explain how to reach your destination and where to stop.

[2346] In this way, a system incorporating an emotion engine allows travelers to receive more personalized travel plans, providing an optimal travel experience based on the user's emotions and preferences.

[2347] The processing flow will be explained below.

[2348] MODE FOR CARRYING OUT THE INVENTION

[2349] Bulk photo posting

[2350] Step 1:

[2351] After traveling, users open a dedicated app and select all the photos they want to post at once.

[2352] Step 2:

[2353] The terminal displays a list of the selected photos and asks the user for confirmation.

[2354] Step 3:

[2355] The device extracts the location information (GPS data) and timestamp (date and time of photo capture) for each selected photo.

[2356] Step 4:

[2357] The device sends the photo file, its location information, and a timestamp to the server.

[2358] Automatic calculation of travel route

[2359] Step 5:

[2360] The server analyzes the received photo data and checks the location information and timestamp.

[2361] Step 6:

[2362] The server sorts the photos in chronological order based on the location information and timestamps, and calculates the user's travel route.

[2363] Step 7:

[2364] The server plots the calculated travel route on map data to create a visual route.

[2365] Step 8:

[2366] The server stores the calculated travel route data in a database.

[2367] Travel diary generation

[2368] Step 9:

[2369] The server extracts highlights and events from the trip based on the stored travel route data.

[2370] Step 10:

[2371] The server combines the extracted highlights and photos to automatically generate a travel diary with explanatory text.

[2372] Step 11:

[2373] The server saves the generated travel diary in the user's profile and makes it available for the user to view.

[2374] Creating a new tour plan

[2375] Step 12:

[2376] The server learns the user's preferences and patterns of places visited from the contents of the user's travel diary.

[2377] Step 13:

[2378] The generative AI analyzes other users' rating data to identify highly rated restaurants and tourist attractions.

[2379] Step 14:

[2380] The generation AI combines user preferences and high rating points to generate new travel routes.

[2381] Step 15:

[2382] The server saves the generated new travel route in the user's profile and makes it available for the user to view.

[2383] Optimization by Emotion Engine

[2384] Step 16:

[2385] The device uses an emotion engine to analyze the user's emotional state from photo and audio data.

[2386] Step 17:

[2387] The emotion engine optimizes travel routes and travel diaries based on the user's emotional state (joy, surprise, sadness, etc.).

[2388] Step 18:

[2389] The emotion engine selects new travel routes and travel highlights based on the user's current emotions and reflects them in the travel diary.

[2390] Step 19:

[2391] The server generates a new optimized travel route based on the user's emotions and saves it in the profile.

[2392] User ratings and popularity rankings

[2393] Step 20:

[2394] A user can view tour plans published by other users.

[2395] Step 21:

[2396] Users can "like" tour plans that they find good and post comments.

[2397] Step 22:

[2398] The server tally up the evaluation points for each tour plan and update the rankings in real time.

[2399] Step 23:

[2400] The server gives rewards to the top ranked tour plan posters and notifies them of the results.

[2401] Map display and navigation

[2402] Step 24:

[2403] The server integrates the generated travel route with map data.

[2404] Step 25:

[2405] The device displays the route on a map and prompts the user to begin navigation.

[2406] Step 26:

[2407] The terminal starts voice guidance to support the user's movement.

[2408] Step 27:

[2409] The user uses the navigation system to enjoy sightseeing and eating and drinking along the generated travel route.

[2410] The above is the specific program processing flow of the system that combines the emotion engine. Each step incorporates a detailed process that includes the specific operations of the user, terminal, and server.

[2411] Example 2

[2412] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2413] In recent years, there has been a growing demand for creating travel records based on photos taken during travel and sharing travel experiences. However, organizing photos, compiling route records, and creating travel diaries is time-consuming. Automating each of these processes would enable users to create and share travel records efficiently. However, current systems do not provide a means to process these processes in an integrated manner. Furthermore, it is difficult to provide personalized services that combine travel plan suggestions, evaluations, and rankings based on users' preferences and emotions. Therefore, there is a need to enable efficient creation and sharing of travel records based on travelers' photos, and to provide an optimal travel experience that suits users' emotions and preferences.

[2414] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2415] In this invention, the server includes: means for uploading photos taken by travelers to the server in bulk using a terminal; means for the terminal to extract location information and timestamps of the photos and for the server to analyze them; means for automatically compiling the traveler's travel route from the location information and timestamps of the photos; means for plotting the travel route on map data and creating a visual route; means for automatically generating the travel diary of the traveler based on the travel route; means for learning based on the user's preferences and visiting trends and using a generation AI to generate a new travel route combining highly rated restaurants and tourist attractions; means for the user to view, rate, and share the generated travel route and travel diary; means for generating a popularity ranking based on the rating and awarding rewards to the top ranked users; means for analyzing the user's emotional state and optimizing the travel route and travel diary based on the emotion; and means for the terminal to display the generated travel route on a map and provide navigation. This enables efficient creation and sharing of travel records based on photos taken by travelers and further enables providing an optimal travel experience tailored to the user's emotions and preferences.

[2416] A "traveler" is an individual who takes photographs while traveling and intends to create a travel record using the photographic data.

[2417] A "terminal" is an electronic device used by a user, such as a mobile information terminal, smartphone, or tablet, which collects, analyzes, and uploads captured photo data to a server.

[2418] A "server" is a computer system that receives, analyzes, stores, and processes data sent from travelers' terminals as necessary.

[2419] "Location information" is information about the location where a photo was taken, and is geographical data recorded in the form of GPS data or the like.

[2420] A "timestamp" is data that indicates the date and time a photo was taken, and is stored as metadata for the photo.

[2421] "Travel route" is data showing the route traveled by a traveler during a trip, and is arranged in chronological order based on location information and timestamps.

[2422] "Map data" means geographical map information provided by a geographic information system (GIS) and is data used to visually display location information and travel routes.

[2423] A "travel diary" is an automatically generated travel record that integrates a traveler's route, highlights, photos, and descriptions.

[2424] "Generative AI" refers to artificial intelligence that learns from large amounts of data and generates new information and suggestions.

[2425] A "new travel route" is a recommended route for the next trip created by the generation AI based on the user's preferences and evaluation data from other users.

[2426] "User preferences" refers to the personal tastes and interests that a traveler has based on past travel and rating data.

[2427] "Top-rated restaurants and attractions" refers to restaurants and attractions that have been highly rated by other travelers.

[2428] "Evaluation" refers to evaluation points that a user gives to a tour plan published by another user by giving it a "like" or comment.

[2429] The "popularity ranking" is a ranking list generated based on the evaluation points obtained by each tour plan, and indicates the order of the most popular tour plans.

[2430] "Emotional state" refers to the emotions such as joy, surprise, or sadness that a traveler feels in response to a particular scene or photo, and is data estimated through emotion analysis.

[2431] "Navigation" is a function that provides audio and visual guidance to help a user reach a destination using a terminal.

[2432] MODE FOR CARRYING OUT THE INVENTION

[2433] This invention is a system that automatically creates a travel log based on photos taken by a traveler and proposes a travel plan that matches the user's preferences and emotions. Specific embodiments of this system will be described below.

[2434] System configuration

[2435] Users use a mobile information terminal (hereinafter referred to as "terminal") such as a smartphone or tablet to select all photos taken during their trip and upload them through a dedicated app. The terminal extracts the location information (GPS data) and timestamp (date and time of photo) embedded in the photos and sends this data to a server.

[2436] The server analyzes the received photo data and calculates the user's travel route in chronological order based on the location information and timestamp. The calculated travel route is plotted on map data, visually representing the route. This makes the user's travel route clear.

[2437] The server also extracts highlights and events from the saved travel route data and automatically generates a travel diary with explanatory text based on the extracted information. The generated travel diary is saved in the user's profile and can be viewed by the user at any time.

[2438] Creating new tour plans and optimizing them with an emotion engine

[2439] The server learns the user's preferences and visiting place patterns from the contents of the user's travel diary, and then analyzes other users' rating data using a generative AI model. From this analysis, it identifies highly rated restaurants and tourist attractions, and proposes new travel routes by combining the user's preferences with these highly rated points. The new tour plan is saved in the user's profile.

[2440] Furthermore, the device uses an emotion engine to analyze the user's emotional state from photos and voice data. The emotion engine evaluates the degree to which the user felt emotions such as joy or surprise at a particular location, and optimizes travel routes and travel diaries based on this. For example, if the device determines that the user was particularly pleased at a particular tourist spot, it will suggest a new tour plan that includes that tourist spot.

[2441] User ratings and popularity rankings

[2442] Users can browse tour plans published by other users, click "like" on the ones they like, and post comments. The server tally up the evaluation points of each tour plan and generate a popularity ranking in real time. Tour plan posters who rank highly are awarded rewards.

[2443] Navigation Features

[2444] The terminal displays the travel route generated by the server on a map and prompts the user to start navigation. The terminal then starts voice guidance and provides directions to reach the destination specified by the user. For example, the terminal displays the route of a new tour plan in real time and provides voice guidance on the way to the destination.

[2445] Examples and prompts

[2446] Example 1: A user selects photos taken during a trip to Europe and uploads them to a server. When uploading, the device automatically extracts the GPS data and the date and time of the photo and sends them to the server.

[2447] Example 2: The server analyzes the user's itinerary and plots the route, for example, from France to Italy, on a map. Based on the location information, each photographed point is connected in a continuous manner to form a route.

[2448] Example prompt: "How can I automatically compile travel routes and generate a travel diary based on travel photos taken by the user?"

[2449] This system allows travelers to efficiently create and share travel records with other users, and also allows them to enjoy personalized travel plans.

[2450] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2451] Step 1:

[2452] The user selects a photo

[2453] Input: Travel photos taken by the user

[2454] Output: A list of selected photo files

[2455] Specific operation: The user opens the dedicated app and selects all the photos taken during the trip at once. The app selects multiple photos from the gallery and adds them to a list for uploading at once.

[2456] Step 2:

[2457] The device extracts the location and timestamp

[2458] Input: A list of photo files selected by the user

[2459] Output: Location information (GPS data) and timestamp (date and time of shooting) for each photo

[2460] What it does: Your device reads the GPS data and timestamp from the photo's metadata and stores this information in temporary memory.

[2461] Step 3:

[2462] The device uploads data to the server

[2463] Input: Location and timestamp of each photo, photo file

[2464] Output: Photo data and metadata sent to the server

[2465] Specific operation: The device sends an HTTP request to the server, sending the photo data and associated metadata. It receives a success response from the server and notifies the user of the upload status.

[2466] Step 4:

[2467] The server analyzes the photo data

[2468] Input: Photo data and metadata uploaded to the server

[2469] Output: Locations and timestamps, organized chronologically

[2470] What it does: The server retrieves the photo data stored in the database and analyzes the metadata of each photo to check the location and timestamp.

[2471] Step 5:

[2472] The server calculates the route.

[2473] Input: Organized location and timestamp

[2474] Output: Calculated travel route

[2475] Specific operation: The server sorts the location information based on time series, and then uses a specific algorithm to connect consecutive points to form the user's travel route.

[2476] Step 6:

[2477] The server plots the map data

[2478] Input: Calculated travel route

[2479] Output: Plotted map data

[2480] Specific operation: The server calls a map API (e.g., Google Maps API) to plot the route and reflects the plotted data in the user interface.

[2481] Step 7: ...

Claims

1. A way for travelers to upload photos they have taken to the server in bulk, A means for automatically collecting travel routes of travelers from the location information and timestamps of the photos; means for automatically generating a travel diary of the traveler based on the travel route; A means for generating new travel routes that incorporate highly rated restaurants and tourist attractions based on the generated travel diary; A means for a user to view and evaluate the generated travel route; a means for generating a popularity ranking based on the evaluation and awarding rewards to those ranked highly; A system including:

2. 2. The system according to claim 1, further comprising means for analyzing location information and time stamps of the uploaded photos and arranging them in chronological order.

3. 2. The system according to claim 1, further comprising means for displaying said new travel route on a map and providing navigation.

Images