System

Abstract

To provide a system for allowing a traveler to efficiently prepare a travel plan useful for not only himself / herself but also other travelers, and to automatically evaluate it.SOLUTION: The system includes a means for collectively uploading photographs taken by travelers, a means for extracting location information and time stamps of the uploaded photographs, a means for automatically aggregating travel routes based on the extracted data, a means for generating travel records from the aggregation results, a means for analyzing the generated travel records and proposing new travel routes, a means for notifying users of the proposed new travel routes, a means for users to actually revisit the new travel routes and publish the results to other users, a means for evaluating the published travel routes and forming a popularity ranking, and a means for providing rewards to creators of highly evaluated travel routes.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When planning a trip, travelers often find it difficult to find an appropriate itinerary that matches their arrival time and length of stay. This difficulty increases especially when information in foreign languages ​​is difficult to obtain. It is difficult to simply use existing itineraries, and finding the best plan for oneself takes considerable time and effort. This often limits the travel experience. Furthermore, there is no efficient way to share excellent travel routes created by other travelers, visualize their value, and evaluate them. To solve these problems, it is necessary to provide a system that allows travelers to efficiently create travel plans that are useful not only to themselves but also to other travelers, and that can be automatically evaluated. [Means for solving the problem]

[0005] This invention provides a system that allows travelers to upload photos taken in bulk, automatically compile travel routes based on the photos' location information and timestamps, and generate a "travel diary." Furthermore, by analyzing this "travel diary," AI can suggest new travel routes that incorporate highly rated restaurants and tourist attractions. Users can then execute the proposed new travel route and publish the results to other users by uploading photos again. Other users rate the published travel routes, and a popularity ranking is formed based on the ratings. Creators of highly rated travel routes are rewarded. In this way, we provide a system that not only enables travelers to efficiently plan their own optimal trips, but also provides useful information to other travelers and visualizes its value. This significantly reduces the time and effort required to plan trips and enhances travel experiences.

[0006] A "traveler" is a person who travels, visiting a particular region or country and engaging in activities such as sightseeing, dining, and leisure.

[0007] A "photograph" is an image of a landscape, person, building, etc., recorded using a photographic device such as a camera or smartphone.

[0008] "Bulk upload" is the act of transferring multiple digital files at once over the Internet to a specific server.

[0009] "Location information" is data indicating the location at the time of photography, and includes GPS coordinates, primarily expressed as latitude and longitude.

[0010] A "timestamp" is data that indicates the exact date and time when a photo was taken, and includes the year, month, day, and time.

[0011] A "travel route" is information indicating the order in which the traveler has visited places and the route he or she has taken.

[0012] "Automatic aggregation" is the act of automatically collecting, organizing, and analyzing data using a system, eliminating the need for manual operations.

[0013] A "travel log" is a document or collection of data that contains detailed information about the places and routes a traveler has visited and the journey.

[0014] "AI" stands for artificial intelligence, a software system that analyzes large amounts of data to find patterns and make decisions and predictions based on them.

[0015] "New travel routes" refer to new travel routes that are based on existing travel routes and incorporate additional tourist attractions, restaurants, etc.

[0016] "Publication" is the act of making information or data available for viewing by many people, not just specific individuals or groups.

[0017] "Evaluation" is the act of judging the value and quality of specific information or data and expressing it in the form of a score or comment.

[0018] A "popularity ranking" is a list of items ranked according to a certain standard based on evaluation data, and indicates the popularity of a particular item.

[0019] "Reward" refers to a reward or prize given for a specific action or achievement. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] overview

[0042] This system automatically compiles travel routes based on the location information and timestamps of photos taken by travelers and creates a "travel diary." Furthermore, AI analyzes this "travel diary" to suggest optimal travel routes, shares the results with other travelers, and based on their ratings, forms a popularity ranking and offers rewards to those with the highest ratings.

[0043] System configuration

[0044] 1. User device: A device such as a smartphone or PC that uploads photos in bulk.

[0045] 2. Server: Collects the received data, generates and stores the travel diary, and uses AI to suggest new travel routes and collect evaluations.

[0046] 3. AI system: Analyzes travel records and suggests new travel routes.

[0047] 4. Database: Storage for high-rated spots and rating data.

[0048] Program processing

[0049] Bulk photo posting and location analysis

[0050] When a user returns from their trip, they upload all the photos they took to a dedicated application. The device reads the metadata of the photos, extracts the location information (latitude and longitude) and timestamp, and sends them to the server.

[0051] Automatically compile travel routes and create travel diaries

[0052] The server automatically compiles the user's route based on the received location information and timestamps, determining the places the user visited, the order in which they visited them, and the duration of their stay. The server then uses this information to generate a "travel diary" containing details of the places visited and relevant photos, which is then saved and made available to the user's account.

[0053] AI-powered new travel route suggestions

[0054] The server provides the generated "travel diary" to the AI ​​analysis system. The AI ​​refers to the "travel diary" and generates a new travel route that includes places the user has not visited, based on a database of highly rated restaurants and tourist attractions. The server notifies the user of this new travel route.

[0055] Revisiting and Publishing New Courses

[0056] The user plans and executes another trip using the proposed new travel route. After the trip, the user uploads photos again and notifies the server that the new travel route has been executed. The server then publishes the results of the new travel route to other users and collects their ratings.

[0057] Creating popularity rankings and offering rewards

[0058] Other users rate the new travel routes that are published. The server aggregates the rating data and generates a popularity ranking. Users who create and publish highly rated travel routes are rewarded.

[0059] Specific examples

[0060] After returning from a week-long trip to Europe, User A uploads all the photos he or she has taken with their smartphone to a dedicated app. The device extracts location information and timestamps from the photos and sends them to the server.

[0061] Based on this, the server automatically calculates the order in which User A visited Paris, London, and Berlin, and the duration of their stay, and generates a "travel diary." This "travel diary" is saved in the user's account and can be viewed.

[0062] The AI ​​system then analyzes the travel diary and suggests a new travel route in Paris that includes highly rated cafes and tourist attractions that have not yet been visited, and the server notifies User A. User A then visits Paris again based on the proposed new route and enjoys the new tourist spots. Afterwards, the system uploads photos again and notifies the server of the results.

[0063] Other users rate User A's new travel route, and the server generates a popularity ranking. Finally, User A who receives a high rating is rewarded. This process allows many travelers to plan their trips efficiently and have a fulfilling travel experience.

[0064] The processing flow will be explained below.

[0065] Specific explanation of program processing

[0066] Bulk photo posting and location analysis

[0067] Step 1: Upload a photo

[0068] After the trip, users upload all the photos they took to a dedicated application.

[0069] (Specific operation) Click the "Upload photos" button within the app, select multiple photo files from the file selection screen, and press the "Upload" button.

[0070] Step 2: Extract location and timestamp

[0071] The device extracts location information (GPS coordinates) and timestamp from the metadata of each uploaded photo.

[0072] (Specific operation) Analyzes files on the device and obtains the latitude, longitude, and shooting date and time from the EXIF ​​information of each photo.

[0073] Step 3: Send data to the server

[0074] The terminal transmits the extracted data, including the location information and timestamp, to the server.

[0075] (Specific operation) The location information and timestamp data are packaged in JSON format and sent to the server's API endpoint as an HTTP POST request.

[0076] Automatically compile travel routes and create travel diaries

[0077] Step 4: Aggregating travel routes

[0078] Based on the received location information and timestamp, the server automatically compiles the user's travel route by chronologically listing the places they visited.

[0079] (Specific operation) Based on location information, the locations to be visited are plotted on a map, and the order of visits is determined by an algorithm based on timestamps.

[0080] Step 5: Generate a travel diary

[0081] The server generates a "travel diary" based on the collected travel routes.

[0082] (Specific Actions) Create a travel record document by combining text and images, including the names of places visited, the date and time of visit, the duration of stay, and key photos.

[0083] Step 6: Save to your user account

[0084] The server saves the generated "travel diary" in the user's account and makes it available for viewing.

[0085] (Specific operation) The travel diary data is associated with the user's account information and saved in a database, and a link to the new travel diary is displayed on the user's dashboard.

[0086] AI-powered new travel route suggestions

[0087] Step 7: Provide to the AI ​​analysis system

[0088] The server provides the generated "travel diary" to the AI ​​analysis system.

[0089] (Specific operation) Send the travel diary data to the analysis endpoint of the AI ​​server.

[0090] Step 8: Create a new travel route

[0091] The AI ​​refers to the travel diary and a database of highly rated restaurants and tourist attractions to generate new travel routes that include places the user has not yet visited.

[0092] (Specific operation) Highly rated spots are retrieved from the database, and a new route is generated by analyzing the order of visits and travel efficiency using an algorithm.

[0093] Step 9: Proposal Notification

[0094] The server notifies the user of the new travel route created by the AI.

[0095] (Specific behavior) Based on the user's notification settings, the app will send new routes via in-app notification or email.

[0096] Revisiting and Publishing New Courses

[0097] Step 10: Performing a revisit

[0098] Users can use the proposed new travel routes to plan and carry out their trip again.

[0099] (Specific actions) Make travel reservations and schedules based on the suggested route.

[0100] Step 11: Re-upload your photos

[0101] The user revisits the new route and uploads the photos again, notifying the server that the AI's suggestions have been implemented.

[0102] (Specific operation) Upload new travel photos to the app in bulk, check that they correspond to the new route based on the metadata at the time of upload, and notify the server.

[0103] Step 12: Publish

[0104] The server publishes the results of the new travel route to other users.

[0105] (Specific operation) Group photos and testimonials of new routes and create a public page that can be viewed by other users.

[0106] Creating popularity rankings and offering rewards

[0107] Step 13: User Rating

[0108] Other users rate the new travel routes that are published.

[0109] (Specific operation) Press the "Rate" button on the public page and post a star rating and comment.

[0110] Step 14: Generate Popularity Rankings

[0111] The server aggregates user ratings and generates a popularity ranking.

[0112] (Specific operation) Obtain evaluation data, calculate scores, and update the ranking page.

[0113] Step 15: Offer Rewards

[0114] The server rewards users who create and publish highly rated travel routes.

[0115] (Specific operations) Check the account information of highly rated users, transfer rewards, send reward acquisition notifications within the app, and record them in the reward history.

[0116] This process allows users to share their travel experiences and allows other users to use them to make better travel plans.

[0117] Example 1

[0118] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0119] It is difficult to efficiently organize the large number of photos taken by travelers after their trip and keep track of the places they visited and the order in which they did so. It is also complicated to suggest new travel routes based on past travel records. Furthermore, there is no consistent system for sharing recommended sightseeing spots and dining places with other travelers, receiving ratings, or providing rankings and rewards based on ratings.

[0120] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0121] In this invention, the server includes: a means for uploading photos taken by travelers in bulk; a means for extracting location information and timestamps from uploaded photos; a means for automatically aggregating travel routes based on the extracted data; a means for generating a travel log from the aggregation results; a means for analyzing the generated travel log and proposing new travel routes based on highly rated locations; a means for notifying users of the proposed new travel routes; a means for users to actually visit the new travel routes and publish the results to other users; a means for evaluating the published travel routes; a means for creating a popularity ranking based on the evaluation data; and a means for providing rewards to creators of highly rated travel routes. This allows travelers to easily organize their travel logs and create high-quality travel plans. Furthermore, by sharing information with other travelers and receiving ratings, travel information useful to many people is accumulated.

[0122] A "traveler" is a person who travels, takes photos at travel destinations, and leaves records of the photos in the system.

[0123] A "bulk photo uploading means" is a function or device that a traveler uses to upload multiple photos taken during a trip to the system at one time.

[0124] "Location information" is the geographic coordinate data of the location where the photo was taken, expressed in the form of latitude and longitude.

[0125] A "timestamp" is data that records the date and time a photo was taken.

[0126] The "extracting means" is a function or device for obtaining location information and timestamps from uploaded photos.

[0127] The "means for automatically compiling travel routes" is a function or device that automatically compiles the order of a traveler's visits and travel routes based on location information and timestamps.

[0128] The "means for generating a travel record" is a function or device that creates a detailed travel record based on the collected travel routes.

[0129] The "analysis means" is a function or device for analyzing the generated travel record and proposing a new travel route based on highly rated places.

[0130] "Means for suggesting new travel routes" refers to a function or device that provides users with travel routes that include new tourist spots and restaurants based on the analysis results.

[0131] "Notification means" refers to a function or device for informing the user of a proposed new travel route.

[0132] A "means for publishing" is a function or device that provides the results of new travel routes visited by a user in a manner that is visible to other users.

[0133] The "means for evaluating" is a function or device that collects feedback from other users regarding the published travel route.

[0134] The "means for forming a popularity ranking" is a function or device that determines the ranking of highly rated travel routes based on the collected evaluation data.

[0135] The "means for providing rewards" is a function or device that provides rewards to users who create highly rated travel routes.

[0136] The system for implementing this invention is composed of multiple components: a user terminal, a server, an AI system, and a database. The specific processing and operation of this system will be described below.

[0137] Bulk photo posting and location analysis

[0138] After returning home from their trip, users can upload all the photos they have taken to a dedicated application using their smartphone, PC, or other device. This application requires an internet connection.

[0139] The device retrieves the Exif data from the uploaded photos, extracting the location (latitude and longitude) and timestamp, and temporarily stores this metadata in a local database.

[0140] The device uses an HTTP POST request to send the extracted metadata to the server in JSON format.

[0141] Automatically compile travel routes and create travel diaries

[0142] The server analyzes the received metadata, which involves matching it with previously stored location information to determine which places a traveler visited and in what order. A distributed processing system such as Hadoop may be used to sort the data chronologically and calculate dwell times.

[0143] The server then generates a "travel diary" containing details of the places visited and related photos based on the travel data collected, which is then stored in the user's individual account and can be viewed through a dedicated application.

[0144] AI-powered new travel route suggestions

[0145] The server uses a REST API to provide the generated travel diary to the AI ​​analysis system.

[0146] The AI ​​system analyzes the travel diary and consults a database of highly rated restaurants and attractions, often using a relational database management system such as PostgreSQL.

[0147] The AI ​​system generates a new route and returns this information to the server via API, which then notifies the user of the newly generated route via push notification or email.

[0148] Revisiting and Publishing New Courses

[0149] When the user actually visits the proposed new route, they take photos again and upload them to the app, following the same process as the first time.

[0150] The server receives the results of new route executions and makes them available to other users via a web interface or an in-app feed.

[0151] Other users rate the new travel routes that are published, and these ratings are collected using buttons and comment functions.

[0152] Creating popularity rankings and offering rewards

[0153] The server aggregates the rating data and generates a popularity ranking, which may involve the use of data analysis tools such as BigQuery.

[0154] The server rewards users who create highly rated travel routes, typically in the form of gift cards or points added to the user's account.

[0155] Examples of concrete examples and prompts

[0156] For example, after returning from a week-long trip to Europe, User A uploads all the photos he or she took to a dedicated app. The device extracts location information and timestamps from the photos and sends them to the server. Based on this information, the server automatically calculates the order of visits to Paris, London, and Berlin, as well as the duration of stay, and generates a "travel diary." This travel diary is saved in the user's account and can be viewed.

[0157] The AI ​​system analyzes the generated "travel diary" and suggests a new travel route that includes highly rated cafes and tourist attractions, which the server notifies User A of. User A visits Paris again based on the new route and uploads the results with photos. The results of the new travel route are made public to other users and are rated. The server compiles the rating data, generates a popularity ranking, and provides rewards to User A who receives high ratings.

[0158] Prompt Sentence Examples

[0159] Please explain a program that automatically compiles the route traveled that day based on the metadata (location information and timestamp) of photos uploaded by users and creates a travel diary. Also, please explain the function of an AI system that analyzes the travel diary and suggests new routes.

[0160] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0161] Step 1:

[0162] After the trip is over, users upload all of the photos they took to a dedicated application.

[0163] Input: Photos taken during travel (image files)

[0164] Output: Upload queue in the application

[0165] How it works: The user launches the app and selects multiple photos from the gallery. The selected photos are added to a queue for uploading within the app. This operation is performed via a GUI.

[0166] Step 2:

[0167] The device retrieves the Exif data from the uploaded photo and extracts the location information (latitude and longitude) and timestamp.

[0168] Input: Uploaded photo (image file)

[0169] Output: Metadata including location and timestamp

[0170] What it does: The application uses the Exif library to extract metadata from each image file, storing the extracted data as a JSON object containing latitude and longitude coordinates and a timestamp.

[0171] Step 3:

[0172] The terminal transmits the extracted metadata to the server.

[0173] Input: Metadata including location and timestamp (JSON format)

[0174] Output: Data sent to the server

[0175] What it does: Your application sends the extracted metadata to the server using an HTTP POST request, to a pre-configured API endpoint.

[0176] Step 4:

[0177] The server analyzes the received metadata to determine the order in which the user visited each location and the duration of their stay.

[0178] Input: Metadata including location and timestamp (JSON format)

[0179] Output: Order of visits and duration

[0180] Specific operation: The server stores the received data in a database and sorts it chronologically. The difference between the previous and next timestamps is used to calculate the stay time. A distributed processing system (e.g., Hadoop) is used to efficiently process large amounts of data.

[0181] Step 5:

[0182] The server generates a "travel diary" containing details of the places visited and associated photos and stores it in the user's account.

[0183] Input: Order of visits and duration, location information and timestamp

[0184] Output: "Travel diary" saved in user account

[0185] Specific operation: The server retrieves detailed information about each visited place (e.g., place name, tourist attraction name) from the database based on the location information. It then combines the retrieved information with related photos and the duration of stay to create a "travel diary" and save it in the user's account.

[0186] Step 6:

[0187] The server provides the generated "travel diary" to an AI analysis system and suggests new travel routes.

[0188] Input: Generated "Travel Diary"

[0189] Output: New travel route proposal

[0190] How it works: The server uses a REST API to send the "travel diary" to the AI ​​analysis system, which analyzes the travel diary and suggests new travel routes including highly rated tourist spots and restaurants based on the database.

[0191] Step 7:

[0192] The server notifies the user of the proposed new travel route.

[0193] Input: New travel route proposal

[0194] Output: User notification

[0195] What happens: The server notifies the user of the new route via push notification or email, including details about the new route and recommended places to visit.

[0196] Step 8:

[0197] The user tries a new route and uploads the results again.

[0198] Enter: New travel photos

[0199] Output: Upload queue and metadata including location and timestamp

[0200] How it works: The user plans a new trip based on a new route, and uploads the photos they take after the trip to the dedicated application. Just like the first time, location information and timestamps are extracted from the photos and sent to the server.

[0201] Step 9:

[0202] The server publishes the results of the new travel route to other users and collects their ratings.

[0203] Input: Execution result of new travel route

[0204] Output: Published execution results and evaluations

[0205] What it does: The server publishes the results of new route executions in a web interface or in-app feed, where other users can view the information and provide feedback using the rating buttons and comment functions.

[0206] Step 10:

[0207] The server aggregates the rating data, generates popularity rankings, and provides rewards.

[0208] Input: Collected assessment data

[0209] Output: Popularity ranking and rewards

[0210] How it works: The server aggregates the rating data using data analysis tools such as BigQuery. Users who create highly rated travel routes are rewarded with gift cards or points.

[0211] (Application example 1)

[0212] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0213] Modern travelers take many photos and videos while traveling, but organizing this data and creating travel routes and travel records is time-consuming. Finding the best route for each destination requires collecting a lot of information and planning on your own. Furthermore, there is no system for sharing these travel records with other travelers and receiving their ratings, making it difficult to improve the quality of the travel experience.

[0214] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0215] In this invention, the server includes: a means for uploading photos and videos taken by travelers in bulk; a means for extracting location information and timestamps from the uploaded photos and videos; a means for automatically aggregating travel routes based on the extracted data; a means for generating a travel log from the aggregation results; a means for analyzing the generated travel log and proposing new travel routes; a means for displaying the proposed new travel routes in a virtual environment; a means for notifying the user; a means for the user to actually revisit the new travel route and publish the results to other users; a means for evaluating the published travel routes and forming a popularity ranking; and a means for providing rewards to creators of highly rated travel routes. This enables the automatic creation of travel logs, the proposal of optimal travel routes, and the sharing of information with other travelers and feedback through evaluations.

[0216] "Tourist" means a person who undertakes a trip.

[0217] "Photos and videos" refers to digital content containing visual information captured using a camera or other photographic device.

[0218] "Bulk upload" refers to the process of transferring multiple photos and videos simultaneously to an online server.

[0219] "Location information" refers to geographic coordinate data such as the latitude and longitude of a location.

[0220] A "timestamp" refers to information that records the date and time that digital data was acquired.

[0221] "Travel route" refers to information showing the places a traveler has visited and the route they took.

[0222] A "travel log" refers to a detailed record of the places a traveler visited, the order in which they visited, and the duration of their stay.

[0223] "Analysis" refers to the process of examining data in detail to reveal its structure and relationships.

[0224] "New travel routes" refer to new routes proposed to improve existing visitor routes.

[0225] "Virtual environment" refers to a virtual space generated by a computer.

[0226] "Notification" refers to the process of conveying information to a recipient.

[0227] "Revisiting" refers to the act of returning to a place that you have visited before.

[0228] "Publishing" refers to the process of making specific information available for others to view.

[0229] "Evaluation" refers to the act of expressing an opinion or value about a particular piece of information or action.

[0230] "Popularity ranking" refers to a list of items arranged in order of highest ratings.

[0231] "Reward" refers to the consideration or prize offered for some action or achievement.

[0232] "Artificial intelligence" refers to the technology that enables computers to judge and process data like humans.

[0233] A "prompt sentence" refers to an instruction sentence to be input into a generative AI model.

[0234] This invention is a system that automatically compiles travel routes based on the location information and timestamps of photos and videos taken by travelers and creates a "travel log." Furthermore, this "travel log" is analyzed, and artificial intelligence suggests new optimal travel routes and notifies the user, supporting efficient travel planning and offering rewards based on evaluations.

[0235] System Configuration

[0236] Hardware

[0237] 1. Smart Glasses:

[0238] Examples: Google Glass, Oculus Quest 2

[0239] Role: Taking photos and videos, recording location and timestamp

[0240] 2. Server:

[0241] Examples: AWS, Google Cloud

[0242] Role: Data collection, processing, analysis, and storage

[0243] 3. PC or smartphone:

[0244] Role: Bulk upload of data, providing user interface

[0245] software

[0246] 1. Application development environment:

[0247] Example: Unity, ARKit / ARCore

[0248] Role: Creating virtual environments and providing virtual tours

[0249] 2. AI analysis system:

[0250] Examples: TensorFlow, PyTorch

[0251] Role: Data analysis, suggesting optimal new travel routes, generating prompts

[0252] 3. Database:

[0253] Examples: Firebase, MongoDB

[0254] Role: Storing highly rated spots and rating data

[0255] Processing steps

[0256] 1. Bulk upload of photos and videos:

[0257] Users can upload photos and videos taken with the smart glasses to a cloud server via their smartphones while traveling. The uploading application is installed on the user's PC or smartphone.

[0258] 2. Location and timestamp analysis:

[0259] The server analyzes the metadata of uploaded photos and videos to extract location information (latitude and longitude) and timestamps.

[0260] 3. Automatically track your travel route and create a trip record:

[0261] The server automatically compiles the traveler's route based on the extracted location information and timestamps, thereby determining the places the user visited, the order in which they visited them, and the duration of their stay, creating a detailed "travel record."

[0262] 4. AI-powered new travel route suggestions:

[0263] The generated "travel log" is provided to an AI analysis system, which then uses a database of highly rated restaurants and tourist attractions to suggest new optimal travel routes. The new routes are then notified to the user.

[0264] 5. Route display in virtual environment:

[0265] The proposed new travel route is displayed in a virtual environment, and users can take part in this virtual tour through smart glasses.

[0266] 6. Revisiting new routes and publishing results:

[0267] Users can plan and carry out their trips again based on the proposed new travel routes and share the results with other users.

[0268] 7. Evaluation and ranking formation:

[0269] Other travelers rate the new travel routes that are published, and the server aggregates the rating data to form a popularity ranking.

[0270] 8. Offering Rewards:

[0271] Users who create and publish highly rated travel routes will be rewarded.

[0272] Specific examples

[0273] After returning from a week-long trip, the user uploads all the photos and videos taken with the smart glasses to a dedicated application. The server extracts location information (latitude and longitude) and timestamps from this data and automatically compiles the user's travel route. This identifies the places the user visited, the order in which they visited them, and the duration of their stay, creating a detailed "travel record."

[0274] The AI ​​system then analyzes the "travel log" and proposes a new travel route that includes highly rated spots that the user has not yet visited. The server notifies the user. The user can then check this new travel route in a virtual environment and plan and carry out the trip again. The results can then be made public to other users and evaluated.

[0275] Prompt Sentence Examples

[0276] "Photos and videos taken with smart glasses while traveling are uploaded to the cloud, and the metadata is analyzed to automatically generate a virtual travel route. Based on data on the places visited, AI proposes new, optimal travel routes and provides them in the form of a virtual tour. These tours can be shared with other users, and rewards are provided for those who rate them highly."

[0277] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0278] Step 1:

[0279] The user takes photos and videos while traveling with the smart glasses and uploads them to a cloud server via their smartphone. Specifically, the user uses a dedicated application to execute the bulk upload function. The input is the photographed data, and the output is the data stored on the cloud server.

[0280] Step 2:

[0281] The server analyzes the metadata of uploaded photos and videos to extract location information (latitude and longitude) and timestamp. Specifically, it uses an image processing library to obtain metadata and analyze location information and timestamp. The input is the uploaded photos and videos, and the output is a list of location information and timestamp.

[0282] Step 3:

[0283] The server automatically compiles the travel route of the traveler based on the extracted location information and timestamps. Specifically, it organizes the location information and timestamp data in chronological order and runs an algorithm to identify the places visited and the order in which they were visited. The input is a list of location information and timestamps, and the output is travel route information.

[0284] Step 4:

[0285] The server generates a detailed "travel record" based on the travel route information. Specifically, it associates photos and videos with each visited location and creates a travel record in a format that records the traveler's activities in detail. The input is the travel route information, and the output is the travel record data.

[0286] Step 5:

[0287] The server uses artificial intelligence to analyze the generated travel log and propose a new optimal travel route. Specifically, it uses an AI analysis system to reference a database of highly rated spots and generate a new route that includes unvisited but highly rated locations. The input is the travel log data, and the output is the proposed new travel route.

[0288] Step 6:

[0289] The server displays the proposed new travel route in a virtual environment. Specifically, it generates a virtual tour using Unity or ARKit / ARCore and creates data to display on smart glasses. The input is the new travel route, and the output is the virtual tour data.

[0290] Step 7:

[0291] The server notifies the user, specifically, by sending the details of the new travel route and a link to the virtual tour to the user through a smartphone application. The input is the virtual tour data, and the output is the notification received by the user.

[0292] Step 8:

[0293] The user plans and executes the trip again based on the proposed new travel route, and uploads the results to the server again. Specifically, the photos and videos taken at the newly visited locations are uploaded to the cloud server. The input is the revisited trip data, and the output is the revisited data stored on the server.

[0294] Step 9:

[0295] The server publishes the revisited data to other users, specifically through a web portal or a sharing function within the application, making the data available for viewing by other users. The input is the revisited data, and the output is the published data.

[0296] Step 10:

[0297] Other travelers rate the published new travel routes. The rating results are compiled on the server, and a popularity ranking is formed. Specifically, user ratings are compiled as points and displayed in a ranking format. The input is the rating data, and the output is a popularity ranking.

[0298] Step 11:

[0299] The server provides rewards to users who create and publish highly rated travel routes. Specifically, it runs a system that provides rewards such as points, cash, and coupons to users who rank highly. The input is the ranking data, and the output is the reward provided.

[0300] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0301] overview

[0302] This system automatically compiles travel routes based on the location information and timestamps of photos taken by travelers and creates a "travel diary." Furthermore, the AI ​​analyzes this "travel diary" to suggest optimal travel routes, and the results are shared with other travelers. Based on their ratings, a popularity ranking is created and rewards are offered to those with the highest ratings. Furthermore, by combining it with an emotion engine that recognizes user emotions, more personalized suggestions can be made.

[0303] System configuration

[0304] 1. User device: A device such as a smartphone or PC that uploads photos in bulk.

[0305] 2. Server: Collects the received data, generates and stores the travel diary, and uses AI to suggest new travel routes and collect evaluations.

[0306] 3. AI system: Analyzes travel records and suggests new travel routes.

[0307] 4. Emotion engine: Recognizes user emotions and optimizes travel routes based on that data.

[0308] 5. Database: Storage for high-rated spots and rating data.

[0309] Program processing

[0310] Bulk photo posting and location analysis

[0311] When a user returns from their trip, they upload all the photos they took to a dedicated application. The device reads the metadata of the photos, extracts the location information (latitude and longitude) and timestamp, and sends them to the server.

[0312] Automatically compile travel routes and create travel diaries

[0313] The server automatically compiles the user's route based on the received location information and timestamps, determining the places the user visited, the order in which they visited them, and the duration of their stay. The server then uses this information to generate a "travel diary" containing details of the places visited and relevant photos, which is then saved and made available to the user's account.

[0314] AI-powered new travel route suggestions

[0315] The server provides the generated "travel diary" to the AI ​​analysis system. The AI ​​refers to the "travel diary" and generates a new travel route that includes places the user has not visited, based on a database of highly rated restaurants and tourist attractions. The server notifies the user of this new travel route.

[0316] Combining Emotion Engines

[0317] The emotion engine extracts emotional data from users' photos and text inputs. The extracted emotional data is sent to the server, which then takes this emotional data into account to optimize the route based on the user's mood and preferences, providing a more personalized experience.

[0318] Revisiting and Publishing New Courses

[0319] The user plans and executes another trip using the proposed new travel route. After the trip, the user uploads photos again and notifies the server that the new travel route has been executed. The server then publishes the results of the new travel route to other users and collects their ratings.

[0320] Creating popularity rankings and offering rewards

[0321] Other users rate the new travel routes that are published. The server aggregates the rating data and generates a popularity ranking. Users who create and publish highly rated travel routes are rewarded.

[0322] Specific examples

[0323] After returning from a week-long trip to Europe, User A uploads all the photos he or she has taken with their smartphone to a dedicated app. The device extracts location information and timestamps from the photos and sends them to the server.

[0324] Based on this, the server automatically calculates the order in which User A visited Paris, London, and Berlin, and the duration of their stay, and generates a "travel diary." This "travel diary" is saved in the user's account and can be viewed.

[0325] The AI ​​system then analyzes the travel diary and suggests a new travel route that includes highly rated cafes and tourist attractions that User A has not yet visited in Paris, and the server notifies User A. Furthermore, the emotion engine analyzes User A's emotional data during the trip and incorporates tourist spots and events that match the user's mood into the new travel route.

[0326] User A visits Paris again based on the proposed new route, enjoys new tourist spots, and then uploads photos again and notifies the server of the results.

[0327] Other users rate User A's new travel route, and the server generates a popularity ranking. Ultimately, User A, who receives a high rating, is rewarded. This process allows many travelers to plan their trips efficiently and have a fulfilling travel experience. Furthermore, by combining it with an emotion engine, it is possible to provide a personalized travel experience that matches each individual user's mood and preferences.

[0328] The processing flow will be explained below.

[0329] Explanation of the specific processing of the program

[0330] Bulk photo posting and location analysis

[0331] Step 1: Upload a photo

[0332] After the trip, users upload all the photos they took to a dedicated application.

[0333] (Specific operation) Click the "Upload photos" button within the app, select multiple photo files from the file selection screen, and press the "Upload" button.

[0334] Step 2: Extract location and timestamp

[0335] The device extracts location information (GPS coordinates) and timestamp from the metadata of each uploaded photo.

[0336] (Specific operation) Analyzes files on the device and obtains the latitude, longitude, and shooting date and time from the EXIF ​​information of each photo.

[0337] Step 3: Send data to the server

[0338] The terminal transmits the extracted data, including the location information and timestamp, to the server.

[0339] (Specific operation) The location information and timestamp data are packaged in JSON format and sent to the server's API endpoint as an HTTP POST request.

[0340] Automatically compile travel routes and create travel diaries

[0341] Step 4: Aggregating travel routes

[0342] Based on the received location information and timestamp, the server automatically compiles the user's travel route by chronologically listing the places they visited.

[0343] (Specific operation) Based on location information, the locations to be visited are plotted on a map, and the order of visits is determined by an algorithm based on timestamps.

[0344] Step 5: Generate a travel diary

[0345] The server generates a "travel diary" based on the collected travel routes.

[0346] (Specific Actions) Create a travel record document by combining text and images, including the names of places visited, the date and time of visit, the duration of stay, and key photos.

[0347] Step 6: Save to your user account

[0348] The server saves the generated "travel diary" in the user's account and makes it available for viewing.

[0349] (Specific operation) The travel diary data is associated with the user's account information and saved in a database, and a link to the new travel diary is displayed on the user's dashboard.

[0350] AI-powered new travel route suggestions

[0351] Step 7: Provide to the AI ​​analysis system

[0352] The server provides the generated "travel diary" to the AI ​​analysis system.

[0353] (Specific operation) Send the travel diary data to the analysis endpoint of the AI ​​server.

[0354] Step 8: Create a new travel route

[0355] The AI ​​refers to the travel diary and a database of highly rated restaurants and tourist attractions to generate new travel routes that include places the user has not yet visited.

[0356] (Specific operation) Highly rated spots are retrieved from the database, and a new route is generated by analyzing the order of visits and travel efficiency using an algorithm.

[0357] Step 9: Proposal Notification

[0358] The server notifies the user of the new travel route created by the AI.

[0359] (Specific behavior) Based on the user's notification settings, the app will send new routes via in-app notification or email.

[0360] Combining Emotion Engines

[0361] Step 10: Extracting Emotion Data

[0362] The emotion engine extracts emotional data from user photos and text inputs.

[0363] (Specific operation) Uploaded photos and text are analyzed, and emotional data (e.g., joy, surprise, sadness, etc.) is extracted using an emotion recognition algorithm.

[0364] Step 11: Sending Emotion Data

[0365] The emotion engine transmits the extracted emotion data to the server.

[0366] (Specific operation) The analysis results are sent in JSON format to the server's API endpoint.

[0367] Step 12: Optimize travel routes

[0368] Based on the received emotional data, the server optimizes the travel route to match the user's mood and preferences.

[0369] (Specific operation) Taking into account the emotional data, the most suitable location is selected from among the highly rated spots, and the optimal travel route is regenerated.

[0370] Revisiting and Publishing New Courses

[0371] Step 13: Performing a revisit

[0372] Users can use the proposed new travel routes to plan and carry out their trip again.

[0373] (Specific actions) Make travel reservations and schedules based on the suggested route.

[0374] Step 14: Re-upload photos

[0375] The user revisits the new route and uploads the photos again, notifying the server that the AI's suggestions have been implemented.

[0376] (Specific operation) Upload new travel photos to the app in bulk, check that they correspond to the new route based on the metadata at the time of upload, and notify the server.

[0377] Step 15: Publish the new travel route

[0378] The server publishes the results of the new travel route to other users.

[0379] (Specific operation) Group photos and testimonials of new routes and create a public page that can be viewed by other users.

[0380] Creating popularity rankings and offering rewards

[0381] Step 16: User Rating

[0382] Other users rate the new travel routes that are published.

[0383] (Specific operation) Press the "Rate" button on the public page and post a star rating and comment.

[0384] Step 17: Generate Popularity Rankings

[0385] The server aggregates user ratings and generates a popularity ranking.

[0386] (Specific operation) Obtain evaluation data, calculate scores, and update the ranking page.

[0387] Step 18: Offer Rewards

[0388] The server rewards users who create and publish highly rated travel routes.

[0389] (Specific operations) Check the account information of highly rated users, transfer rewards, send reward acquisition notifications within the app, and record them in the reward history.

[0390] This process allows users to share their travel experiences, allowing other users to use them to plan more fulfilling trips. Furthermore, by combining it with an emotion engine, it is possible to provide a personalized travel experience that matches each user's mood and preferences.

[0391] Example 2

[0392] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0393] It is difficult to efficiently organize and analyze the vast amount of photo data taken by travelers, and to optimize travel routes based on emotional data during travel. Furthermore, systems that allow users to share their travel experiences with other users and receive feedback through ratings to further enhance travel plans are currently inadequate. Conventional systems struggle to plan trips that take into account travelers' emotions and interests, limiting their ability to provide personalized travel recommendations.

[0394] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for uploading photos taken by travelers in bulk, a means for extracting location information and timestamps from the uploaded photos, a means for automatically aggregating travel routes based on the extracted data, a means for generating a travel log from the aggregation results, a means for analyzing the generated travel log and proposing an optimal travel route, a means for extracting emotional data and optimizing the travel route according to the user's mood and preferences, a means for notifying the user of the proposed new travel route, a means for the user to actually revisit the new travel route and publish the results to other users, a means for evaluating the published travel route and forming a popularity ranking, and a means for providing rewards to creators of highly rated travel routes. This enables efficient organization and analysis of travelers' photo data and personalized travel suggestions. Furthermore, by sharing and rating travel experiences, it is possible to create comprehensive travel plans that can be used as reference by other travelers.

[0395] "Tourist" means an individual or group undertaking a trip.

[0396] "Photographs" refer to digital image data taken by travelers during their trip.

[0397] "Bulk upload" refers to sending multiple photo data to a server over the Internet at once.

[0398] "Location information" refers to the location where the photo was taken, indicated by GPS coordinates.

[0399] A "timestamp" refers to information that indicates the date and time a photo was taken.

[0400] "Extraction" refers to extracting specific metadata (location information and timestamp) from the photo data.

[0401] A "travel route" refers to the order of the places visited by a traveler and the route taken by them.

[0402] "Automatically calculated" means that the program calculates the locations visited, their order, and the duration of stay based on location information and timestamps.

[0403] A "travel log" refers to a digital document that compiles a traveler's route and associated photographs and descriptions.

[0404] "Analysis" refers to the act of examining data in detail to find meaning, particularly using AI and machine learning.

[0405] "Optimal travel route" refers to a route that suggests the most efficient and satisfying travel plan based on the user's preferences and rating data.

[0406] "Emotional data" refers to data that indicates the emotional state of a traveler extracted from their photos and text.

[0407] "Optimization" refers to using emotional data to adjust travel routes and recommendations to best suit a traveler's mood and preferences.

[0408] "Notification" refers to the server sending information to the traveler, especially suggestions for new travel routes.

[0409] "Public" means making the results and photos of the route a user has taken visible to other users.

[0410] "Rating" refers to the act of other users reviewing publicly available travel records and travel routes and judging their quality.

[0411] "Popularity ranking" refers to a list that ranks and displays travel records and travel routes that have received particularly high ratings based on evaluation data.

[0412] "Rewards" refers to incentives (e.g., points, gift cards, cash, etc.) offered to users who receive high ratings.

[0413] System Overview

[0414] This invention is a system that automatically compiles travel routes based on the location information and timestamps of photos taken by travelers and generates a "travel log." Furthermore, it analyzes this "travel log" and uses AI to suggest optimal travel routes. It also includes a mechanism for sharing the results with other users, obtaining ratings, forming a popularity ranking, and offering rewards to those with the highest ratings. Furthermore, by combining it with an emotion engine that recognizes user emotions, it is possible to make more personalized suggestions.

[0415] Hardware and Software Configuration

[0416] The system consists of the following major hardware and software components:

[0417] 1. User device: A device such as a smartphone or PC that uploads photos in bulk.

[0418] 2. Server: Collects the received data, generates and stores travel records, and uses AI to suggest new travel routes and compile evaluations.

[0419] 3. AI system: Analyzes travel records and suggests new travel routes.

[0420] 4. Emotion engine: Recognizes user emotions and optimizes travel routes based on that data.

[0421] 5. Database: Storage for high-rated spots and rating data.

[0422] Operation flow

[0423] When a user returns from a trip, they upload all the photos they took during the trip to a dedicated application. The device analyzes the photo metadata, extracts location information (latitude and longitude) and timestamps, and sends them to the server. The server receives this information, calculates the order of the places the user visited and the duration of their stay, and automatically compiles the user's travel route. The server then generates a travel record from the compilation results, saves it in the user's account, and makes it available for viewing.

[0424] The server then provides the generated travel record to an AI analysis system, which then generates a new travel route that includes places the user has not yet visited, and notifies the user of the new travel route.

[0425] Furthermore, the emotion engine extracts emotional data from the user's photos and text inputs, and this emotional data is sent to the server, which then takes the emotional data into account to optimize the travel route according to the user's mood and preferences, thus enabling more personalized travel suggestions.

[0426] When a user plans and executes a repeat trip using a new travel route and uploads photos again, the server receives the results of the new travel route. This allows the route to be published to other users and ratings to be collected. Finally, the server aggregates the rating data and generates a popularity ranking. Users who create and publish highly rated travel routes can receive rewards.

[0427] Specific examples

[0428] After User A returns from a week-long trip to Europe, he uploads all the photos he took with his smartphone to a dedicated app. The device extracts location information and timestamps from the photos and sends them to the server. Based on this, the server automatically tallys up the order in which User A visited Paris, London, and Berlin, as well as the length of time spent there, and generates a "travel log." This "travel log" is saved in the user's account and can be viewed.

[0429] The AI ​​system then analyzes the "travel log" and proposes a new travel route that includes highly rated cafes and tourist attractions that have not yet been visited in Paris, and the server notifies User A. Furthermore, the emotion engine analyzes User A's emotional data during the trip and reflects tourist attractions and events that match the user's mood in the new travel route. User A then visits Paris again based on the proposed new route and enjoys the new tourist attractions. He then uploads photos again and notifies the server of the execution results. Other users rate User A's new travel route, and the server generates a popularity ranking. Finally, User A, who received a high rating, is provided with a reward.

[0430] Prompt Sentence Examples

[0431] "After returning from a week-long trip to Europe, you upload the photos you took to a dedicated app, sending location information and timestamps to a server. The AI ​​system then suggests a new travel route. This new route includes places you've already visited as well as highly rated spots you haven't yet visited."

[0432] The system allows travelers to share their experiences and receive rewards for positive reviews from other travelers, and leverages AI and an emotion engine to provide more personalized travel plans.

[0433] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0434] Step 1: Bulk upload photos

[0435] When a user returns from a trip, they upload all the photos they took during their trip to a dedicated application by opening the app, tapping the "Upload Photos" button, and selecting the desired photo folder.

[0436] Input: Multiple photos taken by the user.

[0437] Output: Photo data waiting to be uploaded.

[0438] Step 2: Extract location and timestamp

[0439] The device analyzes the metadata (EXIF information) from uploaded photos to extract location information (latitude and longitude) and timestamp. This operation is performed automatically by an internal process on the device. Specifically, the device reads the photo file and reads the latitude, longitude, and timestamp tags.

[0440] Input: Uploaded photo data.

[0441] Output: Location and timestamp data for each photo.

[0442] Step 3: Aggregating travel routes

[0443] The server receives the location information and timestamp sent from the device. Based on this information, the server calculates the order of the locations the user visited and the duration of their stay, and automatically aggregates the user's travel route. Specifically, the server sorts the location information along the time axis and calculates the transition time.

[0444] Input: Location and timestamp data.

[0445] Output: Travel route data including order of visits and duration of visits.

[0446] Step 4: Generate a trip record

[0447] The server generates a "trip log" based on the aggregated travel route information, including details of the places visited and related photos. This "trip log" is saved in the user's account and can be viewed. Specifically, the server saves the travel route information and photos in a database and generates an HTML page to display in the web application.

[0448] Input: Travel route data and photo data.

[0449] Output: Trip log data and HTML page.

[0450] Step 5: AI proposes new travel routes

[0451] The server provides the generated "travel log" to an AI analysis system. The AI ​​system analyzes the "travel log" and generates a new travel route that includes places the user has not yet visited based on a database of highly rated restaurants and tourist attractions. Specifically, it uses a machine learning model to analyze similar traveler patterns and recommend the optimal route.

[0452] Input: Travel record data, restaurant and tourist attraction rating database.

[0453] Output: New travel route data.

[0454] Step 6: Extracting emotion data

[0455] The emotion engine extracts emotional data from photos uploaded by users and the accompanying text input. This process is carried out using image analysis technology and natural language processing (NLP). Specifically, facial recognition technology is used to read emotions from photos, and text analysis is used to extract emotions from sentences.

[0456] Input: Photo data, text input data.

[0457] Output: Extracted emotion data.

[0458] Step 7: Optimize the new travel route

[0459] The server optimizes the travel route based on the user's mood and preferences based on the emotional data sent from the emotion engine. Specifically, it uses the emotional data to re-evaluate the priority of destinations and add tourist spots and events that match the user's emotions to the schedule.

[0460] Input: New travel route data, emotion data.

[0461] Output: Optimized travel route data.

[0462] Step 8: Notification of new travel route

[0463] The server notifies the user of the new optimized route via a dedicated app or email. Specific notification procedures include sending a push notification to the user's app or contacting them by email.

[0464] Input: Optimized travel route data.

[0465] Output: A notification message to the user.

[0466] Step 9: Plan and execute a revisit

[0467] The user plans and executes the trip again, referring to the new travel route notified by the server. Specifically, the user checks the new route in the travel diary app and makes a travel plan. After that, the user actually visits the destination.

[0468] Input: Notification of new travel route.

[0469] Output: User revisit plan and execution.

[0470] Step 10: Notification of the results of the new route

[0471] After returning from the trip, the user uploads the photos again and notifies the server of the results of the new travel route. The device again obtains the location information and timestamp and sends them to the server.

[0472] Input: Newly taken photo data.

[0473] Output: The data resulting from the new travel route.

[0474] Step 11: Collecting ratings and generating popularity rankings

[0475] Other users can rate the new routes published, using dedicated applications or websites, for example, through star rating systems or comments, and the server aggregates these rating data to generate a popularity ranking.

[0476] Input: Rating data from other users.

[0477] Output: Updated popularity rankings.

[0478] Step 12: Offer Rewards

[0479] The server will reward users who have received high ratings based on the popularity rankings. Specific forms of rewards can include a points system, gift cards, cash, etc. To provide rewards, the server may add points to the user's account or send a gift card via email.

[0480] Input: Popularity Rankings.

[0481] Output: Providing a reward to the user.

[0482] Through these steps, the system can provide travelers with an efficient and fulfilling travel experience and achieve more personalized service.

[0483] (Application example 2)

[0484] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0485] Conventional travel record and travel diary systems have difficulty efficiently recording and analyzing travelers' visit history and ratings, and lack the functionality to make personalized suggestions that reflect the emotions and preferences of individual users. Furthermore, there is an inadequate mechanism for users to share and rate information based on this data, making it difficult to create attractive travel itineraries or maximize the value of the information.

[0486] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0487] In this invention, the server includes: a means for uploading photos taken by travelers in bulk; a means for extracting location information and timestamps from the uploaded photos; a means for automatically aggregating travel routes based on the extracted data; a means for generating a travel log from the aggregation result; a means for analyzing the generated travel log and proposing new travel routes; a means for notifying users of the proposed new travel routes; a means for users to actually revisit the new travel routes and publish the results to other users; a means for evaluating the published travel routes and forming a popularity ranking; a means for providing rewards to creators of highly rated travel routes; an emotion analysis means for extracting emotion data from captions of the uploaded photos in bulk; and a means for personalizing the proposed travel routes based on the emotion data. This makes it possible to provide personalized travel suggestions that reflect the individual preferences and emotions of users and to efficiently share ratings among users.

[0488] - "Bulk Upload" refers to a User submitting multiple photos to the system at once.

[0489] "Location information" is data that indicates the latitude and longitude of the location where the photo was taken.

[0490] A "timestamp" is data that indicates the date and time a photo was taken.

[0491] A "travel route" is information that indicates the order and route of the places visited by the user.

[0492] A "travel log" is a document that includes a user's travel route and corresponding photos and comments.

[0493] "Analysis" is the process of examining collected data and extracting new information and insights.

[0494] "Suggestions" are actions or routes that the system recommends to users based on the analysis results.

[0495] "Emotional data" is information about a user's emotions extracted from the user's photo captions and text.

[0496] "Personalization" refers to tailoring content to each individual user based on their preferences and emotions.

[0497] "Publishing" refers to making user-generated information or data available for other users to view.

[0498] "Rating" refers to other users' feedback or ratings on published information or data.

[0499] The "popularity ranking" is a list of items that have received particularly high ratings, created based on the evaluation results.

[0500] A "reward" is a monetary or material incentive offered for a certain behavior or outcome.

[0501] "Sentiment analysis" is the process of extracting and analyzing emotional information from text and photo captions entered by users.

[0502] The embodiment of this invention is a system that utilizes photos taken by users and proposes optimal travel routes through emotion analysis and AI analysis. The specific configuration of this system is shown below.

[0503] First, users upload all the photos they took during their trip using a device such as a smartphone or PC. The software used for this is typically Python, Geopy, Exifread, TextBlob, etc. The device then analyzes the metadata of the uploaded photos to extract location information (latitude and longitude) and timestamps.

[0504] The extracted location information and timestamps are then sent to a server, which automatically compiles the user's route, determines the order of stops and the duration of stays, and creates a trip log containing details of the stops and associated photos, which is then stored in the user's account.

[0505] The server then provides the generated travel record to an AI analysis system, which uses a database of highly rated restaurants and tourist attractions to generate a new travel route that includes places the user has not yet visited, and the server notifies the user of this new route.

[0506] Furthermore, the emotion engine extracts emotional data from the user's photo captions and text input. The main software used is TextBlob. Based on this emotional data, the server suggests a route that matches the user's emotions and preferences. For example, it can recognize positive emotions from the caption "The sushi was delicious!" and recommend sushi restaurants to the user.

[0507] The user plans and executes the trip again based on the proposed new travel route. After retaking the trip, the user uploads new photos and notifies the server of the results of the new travel route. The published new travel route is rated by other users, and the results are compiled by the server as a popularity ranking. Users who receive high ratings are rewarded.

[0508] As a concrete example, suppose User A visits a new restaurant over the weekend, takes photos of the food with his smartphone, and uploads them all to the app with the caption "The sushi was delicious!" The app automatically acquires location information and timestamps and sends them to the server. The server then aggregates User A's visited route based on this information and suggests a new travel route based on the analysis results of the AI ​​system and emotion engine. User A can plan a new trip based on the proposed route, then reflect the results in the system again and receive rewards by receiving a high rating.

[0509] Example prompts for generative AI models

[0510] Please extract the location and timestamp information from the following travel photos and suggest the best food delivery route based on that information. Also, please analyze the user's sentiment from the captions corresponding to each photo and reflect it in the delivery route.

[0511] Photo 1: / path / to / photo1.jpg

[0512] Caption 1: Delicious pizza!

[0513] Photo 2: / path / to / photo2.jpg

[0514] Caption 2: Burger with the best toppings!

[0515] As a result, suggest new restaurants or delivery routes for users to visit.

[0516] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0517] Step 1:

[0518] Bulk upload of photos taken by users during their trip

[0519] Users use their smartphones or PCs to upload multiple photos taken during their trip to the application in bulk. The input is the photo files taken during the trip, and the output is the photo data sent to the server. This allows users to easily share multiple photos at once to the system.

[0520] Step 2:

[0521] The device parses the metadata of the uploaded photo

[0522] The device extracts Exif information from uploaded photos to obtain location information (latitude and longitude) and timestamp. The input is the photo file, and the output is the extracted location information and timestamp. The specific operation is to read the metadata using the Exifread library and parse it.

[0523] Step 3:

[0524] The server automatically calculates the route traveled based on location information and timestamps.

[0525] The server receives location information and timestamp data sent from the device and automatically compiles the user's travel route based on this. The input is location information and timestamp, and the output is travel route information including details of the places visited and the order in which they were visited. This effectively records the user's travel route.

[0526] Step 4:

[0527] The server generates a trip record from the aggregated results.

[0528] The server automatically generates a user's travel record based on the collected travel route information and related photos. The input is travel route information and photos, and the output is the user's travel record. Specifically, the server organizes photos and information for each place visited and creates a travel record in an easy-to-view format.

[0529] Step 5:

[0530] The server provides the generated travel records to the AI ​​analysis system.

[0531] The server provides the automatically generated travel log to an AI analysis system, which compares it with a database of highly rated restaurants and tourist attractions. The input is the travel log, and the output is a proposal for an optimal new travel route. This identifies new places to visit that are beneficial to the user.

[0532] Step 6:

[0533] The server performs sentiment analysis and personalizes the new route.

[0534] The server uses sentiment analysis tools such as TextBlob to extract emotional data from the user's photo captions. The input is the photo captions, and the output is the extracted emotional data. Based on the emotional data, the AI ​​analysis system generates a personalized new travel route and suggests it to the user.

[0535] Step 7:

[0536] The user revisits the proposed new travel route and publishes the results to the server.

[0537] Users plan their trips based on the proposed new travel routes and upload the results of their visit back to the server. The input is new travel records and photos, and the output is new travel route information that is made public. The value of the information increases as users' travel experiences are shared with other users.

[0538] Step 8:

[0539] The server collects publicly available travel routes and generates ratings and rankings.

[0540] The server collects the published new travel routes along with rating data from other users and generates a popularity ranking. The input is new travel route information and rating data, and the output is a ranking. This makes highly rated travel routes clear and promotes information sharing within the user community.

[0541] Step 9:

[0542] The server will reward highly rated travel route creators.

[0543] The server provides rewards to creators of highly rated travel routes based on the evaluation data. The input is ranking data, and the output is providing rewards. This gives users an incentive to share high-quality information.

[0544] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0545] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0546] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0547] [Second embodiment]

[0548] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0549] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0550] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0551] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0552] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0553] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0554] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0555] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0556] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0557] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0558] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0559] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0560] overview

[0561] This system automatically compiles travel routes based on the location information and timestamps of photos taken by travelers and creates a "travel diary." Furthermore, AI analyzes this "travel diary" to suggest optimal travel routes, shares the results with other travelers, and based on their ratings, forms a popularity ranking and offers rewards to those with the highest ratings.

[0562] System configuration

[0563] 1. User device: A device such as a smartphone or PC that uploads photos in bulk.

[0564] 2. Server: Collects the received data, generates and stores the travel diary, and uses AI to suggest new travel routes and collect evaluations.

[0565] 3. AI system: Analyzes travel records and suggests new travel routes.

[0566] 4. Database: Storage for high-rated spots and rating data.

[0567] Program processing

[0568] Bulk photo posting and location analysis

[0569] When a user returns from their trip, they upload all the photos they took to a dedicated application. The device reads the metadata of the photos, extracts the location information (latitude and longitude) and timestamp, and sends them to the server.

[0570] Automatically compile travel routes and create travel diaries

[0571] The server automatically compiles the user's route based on the received location information and timestamps, determining the places the user visited, the order in which they visited them, and the duration of their stay. The server then uses this information to generate a "travel diary" containing details of the places visited and relevant photos, which is then saved and made available to the user's account.

[0572] AI-powered new travel route suggestions

[0573] The server provides the generated "travel diary" to the AI ​​analysis system. The AI ​​refers to the "travel diary" and generates a new travel route that includes places the user has not visited, based on a database of highly rated restaurants and tourist attractions. The server notifies the user of this new travel route.

[0574] Revisiting and Publishing New Courses

[0575] The user plans and executes another trip using the proposed new travel route. After the trip, the user uploads photos again and notifies the server that the new travel route has been executed. The server then publishes the results of the new travel route to other users and collects their ratings.

[0576] Creating popularity rankings and offering rewards

[0577] Other users rate the new travel routes that are published. The server aggregates the rating data and generates a popularity ranking. Users who create and publish highly rated travel routes are rewarded.

[0578] Specific examples

[0579] After returning from a week-long trip to Europe, User A uploads all the photos he or she has taken with their smartphone to a dedicated app. The device extracts location information and timestamps from the photos and sends them to the server.

[0580] Based on this, the server automatically calculates the order in which User A visited Paris, London, and Berlin, and the duration of their stay, and generates a "travel diary." This "travel diary" is saved in the user's account and can be viewed.

[0581] The AI ​​system then analyzes the travel diary and suggests a new travel route in Paris that includes highly rated cafes and tourist attractions that have not yet been visited, and the server notifies User A. User A then visits Paris again based on the proposed new route and enjoys the new tourist spots. Afterwards, the system uploads photos again and notifies the server of the results.

[0582] Other users rate User A's new travel route, and the server generates a popularity ranking. Finally, User A who receives a high rating is rewarded. This process allows many travelers to plan their trips efficiently and have a fulfilling travel experience.

[0583] The processing flow will be explained below.

[0584] Specific explanation of program processing

[0585] Bulk photo posting and location analysis

[0586] Step 1: Upload a photo

[0587] After the trip, users upload all the photos they took to a dedicated application.

[0588] (Specific operation) Click the "Upload photos" button within the app, select multiple photo files from the file selection screen, and press the "Upload" button.

[0589] Step 2: Extract location and timestamp

[0590] The device extracts location information (GPS coordinates) and timestamp from the metadata of each uploaded photo.

[0591] (Specific operation) Analyzes files on the device and obtains the latitude, longitude, and shooting date and time from the EXIF ​​information of each photo.

[0592] Step 3: Send data to the server

[0593] The terminal transmits the extracted data, including the location information and timestamp, to the server.

[0594] (Specific operation) The location information and timestamp data are packaged in JSON format and sent to the server's API endpoint as an HTTP POST request.

[0595] Automatically compile travel routes and create travel diaries

[0596] Step 4: Aggregating travel routes

[0597] Based on the received location information and timestamp, the server automatically compiles the user's travel route by chronologically listing the places they visited.

[0598] (Specific operation) Based on location information, the locations to be visited are plotted on a map, and the order of visits is determined by an algorithm based on timestamps.

[0599] Step 5: Generate a travel diary

[0600] The server generates a "travel diary" based on the collected travel routes.

[0601] (Specific Actions) Create a travel record document by combining text and images, including the names of places visited, the date and time of visit, the duration of stay, and key photos.

[0602] Step 6: Save to your user account

[0603] The server saves the generated "travel diary" in the user's account and makes it available for viewing.

[0604] (Specific operation) The travel diary data is associated with the user's account information and saved in a database, and a link to the new travel diary is displayed on the user's dashboard.

[0605] AI-powered new travel route suggestions

[0606] Step 7: Provide to the AI ​​analysis system

[0607] The server provides the generated "travel diary" to the AI ​​analysis system.

[0608] (Specific operation) Send the travel diary data to the analysis endpoint of the AI ​​server.

[0609] Step 8: Create a new travel route

[0610] The AI ​​refers to the travel diary and a database of highly rated restaurants and tourist attractions to generate new travel routes that include places the user has not yet visited.

[0611] (Specific operation) Highly rated spots are retrieved from the database, and a new route is generated by analyzing the order of visits and travel efficiency using an algorithm.

[0612] Step 9: Proposal Notification

[0613] The server notifies the user of the new travel route created by the AI.

[0614] (Specific behavior) Based on the user's notification settings, the app will send new routes via in-app notification or email.

[0615] Revisiting and Publishing New Courses

[0616] Step 10: Performing a revisit

[0617] Users can use the proposed new travel routes to plan and carry out their trip again.

[0618] (Specific actions) Make travel reservations and schedules based on the suggested route.

[0619] Step 11: Re-upload your photos

[0620] The user revisits the new route and uploads the photos again, notifying the server that the AI's suggestions have been implemented.

[0621] (Specific operation) Upload new travel photos to the app in bulk, check that they correspond to the new route based on the metadata at the time of upload, and notify the server.

[0622] Step 12: Publish

[0623] The server publishes the results of the new travel route to other users.

[0624] (Specific operation) Group photos and testimonials of new routes and create a public page that can be viewed by other users.

[0625] Creating popularity rankings and offering rewards

[0626] Step 13: User Rating

[0627] Other users rate the new travel routes that are published.

[0628] (Specific operation) Press the "Rate" button on the public page and post a star rating and comment.

[0629] Step 14: Generate Popularity Rankings

[0630] The server aggregates user ratings and generates a popularity ranking.

[0631] (Specific operation) Obtain evaluation data, calculate scores, and update the ranking page.

[0632] Step 15: Offer Rewards

[0633] The server rewards users who create and publish highly rated travel routes.

[0634] (Specific operations) Check the account information of highly rated users, transfer rewards, send reward acquisition notifications within the app, and record them in the reward history.

[0635] This process allows users to share their travel experiences and allows other users to use them to make better travel plans.

[0636] Example 1

[0637] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0638] It is difficult to efficiently organize the large number of photos taken by travelers after their trip and keep track of the places they visited and the order in which they did so. It is also complicated to suggest new travel routes based on past travel records. Furthermore, there is no consistent system for sharing recommended sightseeing spots and dining places with other travelers, receiving ratings, or providing rankings and rewards based on ratings.

[0639] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0640] In this invention, the server includes: a means for uploading photos taken by travelers in bulk; a means for extracting location information and timestamps from uploaded photos; a means for automatically aggregating travel routes based on the extracted data; a means for generating a travel log from the aggregation results; a means for analyzing the generated travel log and proposing new travel routes based on highly rated locations; a means for notifying users of the proposed new travel routes; a means for users to actually visit the new travel routes and publish the results to other users; a means for evaluating the published travel routes; a means for creating a popularity ranking based on the evaluation data; and a means for providing rewards to creators of highly rated travel routes. This allows travelers to easily organize their travel logs and create high-quality travel plans. Furthermore, by sharing information with other travelers and receiving ratings, travel information useful to many people is accumulated.

[0641] A "traveler" is a person who travels, takes photos at travel destinations, and leaves records of the photos in the system.

[0642] A "bulk photo uploading means" is a function or device that a traveler uses to upload multiple photos taken during a trip to the system at one time.

[0643] "Location information" is the geographic coordinate data of the location where the photo was taken, expressed in the form of latitude and longitude.

[0644] A "timestamp" is data that records the date and time a photo was taken.

[0645] The "extracting means" is a function or device for obtaining location information and timestamps from uploaded photos.

[0646] The "means for automatically compiling travel routes" is a function or device that automatically compiles the order of a traveler's visits and travel routes based on location information and timestamps.

[0647] The "means for generating a travel record" is a function or device that creates a detailed travel record based on the collected travel routes.

[0648] The "analysis means" is a function or device for analyzing the generated travel record and proposing a new travel route based on highly rated places.

[0649] "Means for suggesting new travel routes" refers to a function or device that provides users with travel routes that include new tourist spots and restaurants based on the analysis results.

[0650] "Notification means" refers to a function or device for informing the user of a proposed new travel route.

[0651] A "means for publishing" is a function or device that provides the results of new travel routes visited by a user in a manner that is visible to other users.

[0652] The "means for evaluating" is a function or device that collects feedback from other users regarding the published travel route.

[0653] The "means for forming a popularity ranking" is a function or device that determines the ranking of highly rated travel routes based on the collected evaluation data.

[0654] The "means for providing rewards" is a function or device that provides rewards to users who create highly rated travel routes.

[0655] The system for implementing this invention is composed of multiple components: a user terminal, a server, an AI system, and a database. The specific processing and operation of this system will be described below.

[0656] Bulk photo posting and location analysis

[0657] After returning home from their trip, users can upload all the photos they have taken to a dedicated application using their smartphone, PC, or other device. This application requires an internet connection.

[0658] The device retrieves the Exif data from the uploaded photos, extracting the location (latitude and longitude) and timestamp, and temporarily stores this metadata in a local database.

[0659] The device uses an HTTP POST request to send the extracted metadata to the server in JSON format.

[0660] Automatically compile travel routes and create travel diaries

[0661] The server analyzes the received metadata, which involves matching it with previously stored location information to determine which places a traveler visited and in what order. A distributed processing system such as Hadoop may be used to sort the data chronologically and calculate dwell times.

[0662] The server then generates a "travel diary" containing details of the places visited and related photos based on the travel data collected, which is then stored in the user's individual account and can be viewed through a dedicated application.

[0663] AI-powered new travel route suggestions

[0664] The server uses a REST API to provide the generated travel diary to the AI ​​analysis system.

[0665] The AI ​​system analyzes the travel diary and consults a database of highly rated restaurants and attractions, often using a relational database management system such as PostgreSQL.

[0666] The AI ​​system generates a new route and returns this information to the server via API, which then notifies the user of the newly generated route via push notification or email.

[0667] Revisiting and Publishing New Courses

[0668] When the user actually visits the proposed new route, they take photos again and upload them to the app, following the same process as the first time.

[0669] The server receives the results of new route executions and makes them available to other users via a web interface or an in-app feed.

[0670] Other users rate the new travel routes that are published, and these ratings are collected using buttons and comment functions.

[0671] Creating popularity rankings and offering rewards

[0672] The server aggregates the rating data and generates a popularity ranking, which may involve the use of data analysis tools such as BigQuery.

[0673] The server rewards users who create highly rated travel routes, typically in the form of gift cards or points added to the user's account.

[0674] Examples of concrete examples and prompts

[0675] For example, after returning from a week-long trip to Europe, User A uploads all the photos he or she took to a dedicated app. The device extracts location information and timestamps from the photos and sends them to the server. Based on this information, the server automatically calculates the order of visits to Paris, London, and Berlin, as well as the duration of stay, and generates a "travel diary." This travel diary is saved in the user's account and can be viewed.

[0676] The AI ​​system analyzes the generated "travel diary" and suggests a new travel route that includes highly rated cafes and tourist attractions, which the server notifies User A of. User A visits Paris again based on the new route and uploads the results with photos. The results of the new travel route are made public to other users and are rated. The server compiles the rating data, generates a popularity ranking, and provides rewards to User A who receives high ratings.

[0677] Prompt Sentence Examples

[0678] Please explain a program that automatically compiles the route traveled that day based on the metadata (location information and timestamp) of photos uploaded by users and creates a travel diary. Also, please explain the function of an AI system that analyzes the travel diary and suggests new routes.

[0679] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0680] Step 1:

[0681] After the trip is over, users upload all of the photos they took to a dedicated application.

[0682] Input: Photos taken during travel (image files)

[0683] Output: Upload queue in the application

[0684] How it works: The user launches the app and selects multiple photos from the gallery. The selected photos are added to a queue for uploading within the app. This operation is performed via a GUI.

[0685] Step 2:

[0686] The device retrieves the Exif data from the uploaded photo and extracts the location information (latitude and longitude) and timestamp.

[0687] Input: Uploaded photo (image file)

[0688] Output: Metadata including location and timestamp

[0689] What it does: The application uses the Exif library to extract metadata from each image file, storing the extracted data as a JSON object containing latitude and longitude coordinates and a timestamp.

[0690] Step 3:

[0691] The terminal transmits the extracted metadata to the server.

[0692] Input: Metadata including location and timestamp (JSON format)

[0693] Output: Data sent to the server

[0694] What it does: Your application sends the extracted metadata to the server using an HTTP POST request, to a pre-configured API endpoint.

[0695] Step 4:

[0696] The server analyzes the received metadata to determine the order in which the user visited each location and the duration of their stay.

[0697] Input: Metadata including location and timestamp (JSON format)

[0698] Output: Order of visits and duration

[0699] Specific operation: The server stores the received data in a database and sorts it chronologically. The difference between the previous and next timestamps is used to calculate the stay time. A distributed processing system (e.g., Hadoop) is used to efficiently process large amounts of data.

[0700] Step 5:

[0701] The server generates a "travel diary" containing details of the places visited and associated photos and stores it in the user's account.

[0702] Input: Order of visits and duration, location information and timestamp

[0703] Output: "Travel diary" saved in user account

[0704] Specific operation: The server retrieves detailed information about each visited place (e.g., place name, tourist attraction name) from the database based on the location information. It then combines the retrieved information with related photos and the duration of stay to create a "travel diary" and save it in the user's account.

[0705] Step 6:

[0706] The server provides the generated "travel diary" to an AI analysis system and suggests new travel routes.

[0707] Input: Generated "Travel Diary"

[0708] Output: New travel route proposal

[0709] How it works: The server uses a REST API to send the "travel diary" to the AI ​​analysis system, which analyzes the travel diary and suggests new travel routes including highly rated tourist spots and restaurants based on the database.

[0710] Step 7:

[0711] The server notifies the user of the proposed new travel route.

[0712] Input: New travel route proposal

[0713] Output: User notification

[0714] What happens: The server notifies the user of the new route via push notification or email, including details about the new route and recommended places to visit.

[0715] Step 8:

[0716] The user tries a new route and uploads the results again.

[0717] Enter: New travel photos

[0718] Output: Upload queue and metadata including location and timestamp

[0719] How it works: The user plans a new trip based on a new route, and uploads the photos they take after the trip to the dedicated application. Just like the first time, location information and timestamps are extracted from the photos and sent to the server.

[0720] Step 9:

[0721] The server publishes the results of the new travel route to other users and collects their ratings.

[0722] Input: Execution result of new travel route

[0723] Output: Published execution results and evaluations

[0724] What it does: The server publishes the results of new route executions in a web interface or in-app feed, where other users can view the information and provide feedback using the rating buttons and comment functions.

[0725] Step 10:

[0726] The server aggregates the rating data, generates popularity rankings, and provides rewards.

[0727] Input: Collected assessment data

[0728] Output: Popularity ranking and rewards

[0729] How it works: The server aggregates the rating data using data analysis tools such as BigQuery. Users who create highly rated travel routes are rewarded with gift cards or points.

[0730] (Application example 1)

[0731] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0732] Modern travelers take many photos and videos while traveling, but organizing this data and creating travel routes and travel records is time-consuming. Finding the best route for each destination requires collecting a lot of information and planning on your own. Furthermore, there is no system for sharing these travel records with other travelers and receiving their ratings, making it difficult to improve the quality of the travel experience.

[0733] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0734] In this invention, the server includes: a means for uploading photos and videos taken by travelers in bulk; a means for extracting location information and timestamps from the uploaded photos and videos; a means for automatically aggregating travel routes based on the extracted data; a means for generating a travel log from the aggregation results; a means for analyzing the generated travel log and proposing new travel routes; a means for displaying the proposed new travel routes in a virtual environment; a means for notifying the user; a means for the user to actually revisit the new travel route and publish the results to other users; a means for evaluating the published travel routes and forming a popularity ranking; and a means for providing rewards to creators of highly rated travel routes. This enables the automatic creation of travel logs, the proposal of optimal travel routes, and the sharing of information with other travelers and feedback through evaluations.

[0735] "Tourist" means a person who undertakes a trip.

[0736] "Photos and videos" refers to digital content containing visual information captured using a camera or other photographic device.

[0737] "Bulk upload" refers to the process of transferring multiple photos and videos simultaneously to an online server.

[0738] "Location information" refers to geographic coordinate data such as the latitude and longitude of a location.

[0739] A "timestamp" refers to information that records the date and time that digital data was acquired.

[0740] "Travel route" refers to information showing the places a traveler has visited and the route they took.

[0741] A "travel log" refers to a detailed record of the places a traveler visited, the order in which they visited, and the duration of their stay.

[0742] "Analysis" refers to the process of examining data in detail to reveal its structure and relationships.

[0743] "New travel routes" refer to new routes proposed to improve existing visitor routes.

[0744] "Virtual environment" refers to a virtual space generated by a computer.

[0745] "Notification" refers to the process of conveying information to a recipient.

[0746] "Revisiting" refers to the act of returning to a place that you have visited before.

[0747] "Publishing" refers to the process of making specific information available for others to view.

[0748] "Evaluation" refers to the act of expressing an opinion or value about a particular piece of information or action.

[0749] "Popularity ranking" refers to a list of items arranged in order of highest ratings.

[0750] "Reward" refers to the consideration or prize offered for some action or achievement.

[0751] "Artificial intelligence" refers to the technology that enables computers to judge and process data like humans.

[0752] A "prompt sentence" refers to an instruction sentence to be input into a generative AI model.

[0753] This invention is a system that automatically compiles travel routes based on the location information and timestamps of photos and videos taken by travelers and creates a "travel log." Furthermore, this "travel log" is analyzed, and artificial intelligence suggests new optimal travel routes and notifies the user, supporting efficient travel planning and offering rewards based on evaluations.

[0754] System Configuration

[0755] Hardware

[0756] 1. Smart Glasses:

[0757] Examples: Google Glass, Oculus Quest 2

[0758] Role: Taking photos and videos, recording location and timestamp

[0759] 2. Server:

[0760] Examples: AWS, Google Cloud

[0761] Role: Data collection, processing, analysis, and storage

[0762] 3. PC or smartphone:

[0763] Role: Bulk upload of data, providing user interface

[0764] software

[0765] 1. Application development environment:

[0766] Example: Unity, ARKit / ARCore

[0767] Role: Creating virtual environments and providing virtual tours

[0768] 2. AI analysis system:

[0769] Examples: TensorFlow, PyTorch

[0770] Role: Data analysis, suggesting optimal new travel routes, generating prompts

[0771] 3. Database:

[0772] Examples: Firebase, MongoDB

[0773] Role: Storing highly rated spots and rating data

[0774] Processing steps

[0775] 1. Bulk upload of photos and videos:

[0776] Users can upload photos and videos taken with the smart glasses to a cloud server via their smartphones while traveling. The uploading application is installed on the user's PC or smartphone.

[0777] 2. Location and timestamp analysis:

[0778] The server analyzes the metadata of uploaded photos and videos to extract location information (latitude and longitude) and timestamps.

[0779] 3. Automatically track your travel route and create a trip record:

[0780] The server automatically compiles the traveler's route based on the extracted location information and timestamps, thereby determining the places the user visited, the order in which they visited them, and the duration of their stay, creating a detailed "travel record."

[0781] 4. AI-powered new travel route suggestions:

[0782] The generated "travel log" is provided to an AI analysis system, which then uses a database of highly rated restaurants and tourist attractions to suggest new optimal travel routes. The new routes are then notified to the user.

[0783] 5. Route display in virtual environment:

[0784] The proposed new travel route is displayed in a virtual environment, and users can take part in this virtual tour through smart glasses.

[0785] 6. Revisiting new routes and publishing results:

[0786] Users can plan and carry out their trips again based on the proposed new travel routes and share the results with other users.

[0787] 7. Evaluation and ranking formation:

[0788] Other travelers rate the new travel routes that are published, and the server aggregates the rating data to form a popularity ranking.

[0789] 8. Offering Rewards:

[0790] Users who create and publish highly rated travel routes will be rewarded.

[0791] Specific examples

[0792] After returning from a week-long trip, the user uploads all the photos and videos taken with the smart glasses to a dedicated application. The server extracts location information (latitude and longitude) and timestamps from this data and automatically compiles the user's travel route. This identifies the places the user visited, the order in which they visited them, and the duration of their stay, creating a detailed "travel record."

[0793] The AI ​​system then analyzes the "travel log" and proposes a new travel route that includes highly rated spots that the user has not yet visited. The server notifies the user. The user can then check this new travel route in a virtual environment and plan and carry out the trip again. The results can then be made public to other users and evaluated.

[0794] Prompt Sentence Examples

[0795] "Photos and videos taken with smart glasses while traveling are uploaded to the cloud, and the metadata is analyzed to automatically generate a virtual travel route. Based on data on the places visited, AI proposes new, optimal travel routes and provides them in the form of a virtual tour. These tours can be shared with other users, and rewards are provided for those who rate them highly."

[0796] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0797] Step 1:

[0798] The user takes photos and videos while traveling with the smart glasses and uploads them to a cloud server via their smartphone. Specifically, the user uses a dedicated application to execute the bulk upload function. The input is the photographed data, and the output is the data stored on the cloud server.

[0799] Step 2:

[0800] The server analyzes the metadata of uploaded photos and videos to extract location information (latitude and longitude) and timestamp. Specifically, it uses an image processing library to obtain metadata and analyze location information and timestamp. The input is the uploaded photos and videos, and the output is a list of location information and timestamp.

[0801] Step 3:

[0802] The server automatically compiles the travel route of the traveler based on the extracted location information and timestamps. Specifically, it organizes the location information and timestamp data in chronological order and runs an algorithm to identify the places visited and the order in which they were visited. The input is a list of location information and timestamps, and the output is travel route information.

[0803] Step 4:

[0804] The server generates a detailed "travel record" based on the travel route information. Specifically, it associates photos and videos with each visited location and creates a travel record in a format that records the traveler's activities in detail. The input is the travel route information, and the output is the travel record data.

[0805] Step 5:

[0806] The server uses artificial intelligence to analyze the generated travel log and propose a new optimal travel route. Specifically, it uses an AI analysis system to reference a database of highly rated spots and generate a new route that includes unvisited but highly rated locations. The input is the travel log data, and the output is the proposed new travel route.

[0807] Step 6:

[0808] The server displays the proposed new travel route in a virtual environment. Specifically, it generates a virtual tour using Unity or ARKit / ARCore and creates data to display on smart glasses. The input is the new travel route, and the output is the virtual tour data.

[0809] Step 7:

[0810] The server notifies the user, specifically, by sending the details of the new travel route and a link to the virtual tour to the user through a smartphone application. The input is the virtual tour data, and the output is the notification received by the user.

[0811] Step 8:

[0812] The user plans and executes the trip again based on the proposed new travel route, and uploads the results to the server again. Specifically, the photos and videos taken at the newly visited locations are uploaded to the cloud server. The input is the revisited trip data, and the output is the revisited data stored on the server.

[0813] Step 9:

[0814] The server publishes the revisited data to other users, specifically through a web portal or a sharing function within the application, making the data available for viewing by other users. The input is the revisited data, and the output is the published data.

[0815] Step 10:

[0816] Other travelers rate the published new travel routes. The rating results are compiled on the server, and a popularity ranking is formed. Specifically, user ratings are compiled as points and displayed in a ranking format. The input is the rating data, and the output is a popularity ranking.

[0817] Step 11:

[0818] The server provides rewards to users who create and publish highly rated travel routes. Specifically, it runs a system that provides rewards such as points, cash, and coupons to users who rank highly. The input is the ranking data, and the output is the reward provided.

[0819] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0820] overview

[0821] This system automatically compiles travel routes based on the location information and timestamps of photos taken by travelers and creates a "travel diary." Furthermore, the AI ​​analyzes this "travel diary" to suggest optimal travel routes, and the results are shared with other travelers. Based on their ratings, a popularity ranking is created and rewards are offered to those with the highest ratings. Furthermore, by combining it with an emotion engine that recognizes user emotions, more personalized suggestions can be made.

[0822] System configuration

[0823] 1. User device: A device such as a smartphone or PC that uploads photos in bulk.

[0824] 2. Server: Collects the received data, generates and stores the travel diary, and uses AI to suggest new travel routes and collect evaluations.

[0825] 3. AI system: Analyzes travel records and suggests new travel routes.

[0826] 4. Emotion engine: Recognizes user emotions and optimizes travel routes based on that data.

[0827] 5. Database: Storage for high-rated spots and rating data.

[0828] Program processing

[0829] Bulk photo posting and location analysis

[0830] When a user returns from their trip, they upload all the photos they took to a dedicated application. The device reads the metadata of the photos, extracts the location information (latitude and longitude) and timestamp, and sends them to the server.

[0831] Automatically compile travel routes and create travel diaries

[0832] The server automatically compiles the user's route based on the received location information and timestamps, determining the places the user visited, the order in which they visited them, and the duration of their stay. The server then uses this information to generate a "travel diary" containing details of the places visited and relevant photos, which is then saved and made available to the user's account.

[0833] AI-powered new travel route suggestions

[0834] The server provides the generated "travel diary" to the AI ​​analysis system. The AI ​​refers to the "travel diary" and generates a new travel route that includes places the user has not visited, based on a database of highly rated restaurants and tourist attractions. The server notifies the user of this new travel route.

[0835] Combining Emotion Engines

[0836] The emotion engine extracts emotional data from users' photos and text inputs. The extracted emotional data is sent to the server, which then takes this emotional data into account to optimize the route based on the user's mood and preferences, providing a more personalized experience.

[0837] Revisiting and Publishing New Courses

[0838] The user plans and executes another trip using the proposed new travel route. After the trip, the user uploads photos again and notifies the server that the new travel route has been executed. The server then publishes the results of the new travel route to other users and collects their ratings.

[0839] Creating popularity rankings and offering rewards

[0840] Other users rate the new travel routes that are published. The server aggregates the rating data and generates a popularity ranking. Users who create and publish highly rated travel routes are rewarded.

[0841] Specific examples

[0842] After returning from a week-long trip to Europe, User A uploads all the photos he or she has taken with their smartphone to a dedicated app. The device extracts location information and timestamps from the photos and sends them to the server.

[0843] Based on this, the server automatically calculates the order in which User A visited Paris, London, and Berlin, and the duration of their stay, and generates a "travel diary." This "travel diary" is saved in the user's account and can be viewed.

[0844] The AI ​​system then analyzes the travel diary and suggests a new travel route that includes highly rated cafes and tourist attractions that User A has not yet visited in Paris, and the server notifies User A. Furthermore, the emotion engine analyzes User A's emotional data during the trip and incorporates tourist spots and events that match the user's mood into the new travel route.

[0845] User A visits Paris again based on the proposed new route, enjoys new tourist spots, and then uploads photos again and notifies the server of the results.

[0846] Other users rate User A's new travel route, and the server generates a popularity ranking. Ultimately, User A, who receives a high rating, is rewarded. This process allows many travelers to plan their trips efficiently and have a fulfilling travel experience. Furthermore, by combining it with an emotion engine, it is possible to provide a personalized travel experience that matches each individual user's mood and preferences.

[0847] The processing flow will be explained below.

[0848] Explanation of the specific processing of the program

[0849] Bulk photo posting and location analysis

[0850] Step 1: Upload a photo

[0851] After the trip, users upload all the photos they took to a dedicated application.

[0852] (Specific operation) Click the "Upload photos" button within the app, select multiple photo files from the file selection screen, and press the "Upload" button.

[0853] Step 2: Extract location and timestamp

[0854] The device extracts location information (GPS coordinates) and timestamp from the metadata of each uploaded photo.

[0855] (Specific operation) Analyzes files on the device and obtains the latitude, longitude, and shooting date and time from the EXIF ​​information of each photo.

[0856] Step 3: Send data to the server

[0857] The terminal transmits the extracted data, including the location information and timestamp, to the server.

[0858] (Specific operation) The location information and timestamp data are packaged in JSON format and sent to the server's API endpoint as an HTTP POST request.

[0859] Automatically compile travel routes and create travel diaries

[0860] Step 4: Aggregating travel routes

[0861] Based on the received location information and timestamp, the server automatically compiles the user's travel route by chronologically listing the places they visited.

[0862] (Specific operation) Based on location information, the locations to be visited are plotted on a map, and the order of visits is determined by an algorithm based on timestamps.

[0863] Step 5: Generate a travel diary

[0864] The server generates a "travel diary" based on the collected travel routes.

[0865] (Specific Actions) Create a travel record document by combining text and images, including the names of places visited, the date and time of visit, the duration of stay, and key photos.

[0866] Step 6: Save to your user account

[0867] The server saves the generated "travel diary" in the user's account and makes it available for viewing.

[0868] (Specific operation) The travel diary data is associated with the user's account information and saved in a database, and a link to the new travel diary is displayed on the user's dashboard.

[0869] AI-powered new travel route suggestions

[0870] Step 7: Provide to the AI ​​analysis system

[0871] The server provides the generated "travel diary" to the AI ​​analysis system.

[0872] (Specific operation) Send the travel diary data to the analysis endpoint of the AI ​​server.

[0873] Step 8: Create a new travel route

[0874] The AI ​​refers to the travel diary and a database of highly rated restaurants and tourist attractions to generate new travel routes that include places the user has not yet visited.

[0875] (Specific operation) Highly rated spots are retrieved from the database, and a new route is generated by analyzing the order of visits and travel efficiency using an algorithm.

[0876] Step 9: Proposal Notification

[0877] The server notifies the user of the new travel route created by the AI.

[0878] (Specific behavior) Based on the user's notification settings, the app will send new routes via in-app notification or email.

[0879] Combining Emotion Engines

[0880] Step 10: Extracting Emotion Data

[0881] The emotion engine extracts emotional data from user photos and text inputs.

[0882] (Specific operation) Uploaded photos and text are analyzed, and emotional data (e.g., joy, surprise, sadness, etc.) is extracted using an emotion recognition algorithm.

[0883] Step 11: Sending Emotion Data

[0884] The emotion engine transmits the extracted emotion data to the server.

[0885] (Specific operation) The analysis results are sent in JSON format to the server's API endpoint.

[0886] Step 12: Optimize travel routes

[0887] Based on the received emotional data, the server optimizes the travel route to match the user's mood and preferences.

[0888] (Specific operation) Taking into account the emotional data, the most suitable location is selected from among the highly rated spots, and the optimal travel route is regenerated.

[0889] Revisiting and Publishing New Courses

[0890] Step 13: Performing a revisit

[0891] Users can use the proposed new travel routes to plan and carry out their trip again.

[0892] (Specific actions) Make travel reservations and schedules based on the suggested route.

[0893] Step 14: Re-upload photos

[0894] The user revisits the new route and uploads the photos again, notifying the server that the AI's suggestions have been implemented.

[0895] (Specific operation) Upload new travel photos to the app in bulk, check that they correspond to the new route based on the metadata at the time of upload, and notify the server.

[0896] Step 15: Publish the new travel route

[0897] The server publishes the results of the new travel route to other users.

[0898] (Specific operation) Group photos and testimonials of new routes and create a public page that can be viewed by other users.

[0899] Creating popularity rankings and offering rewards

[0900] Step 16: User Rating

[0901] Other users rate the new travel routes that are published.

[0902] (Specific operation) Press the "Rate" button on the public page and post a star rating and comment.

[0903] Step 17: Generate Popularity Rankings

[0904] The server aggregates user ratings and generates a popularity ranking.

[0905] (Specific operation) Obtain evaluation data, calculate scores, and update the ranking page.

[0906] Step 18: Offer Rewards

[0907] The server rewards users who create and publish highly rated travel routes.

[0908] (Specific operations) Check the account information of highly rated users, transfer rewards, send reward acquisition notifications within the app, and record them in the reward history.

[0909] This process allows users to share their travel experiences, allowing other users to use them to plan more fulfilling trips. Furthermore, by combining it with an emotion engine, it is possible to provide a personalized travel experience that matches each user's mood and preferences.

[0910] Example 2

[0911] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0912] It is difficult to efficiently organize and analyze the vast amount of photo data taken by travelers, and to optimize travel routes based on emotional data during travel. Furthermore, systems that allow users to share their travel experiences with other users and receive feedback through ratings to further enhance travel plans are currently inadequate. Conventional systems struggle to plan trips that take into account travelers' emotions and interests, limiting their ability to provide personalized travel recommendations.

[0913] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for uploading photos taken by travelers in bulk, a means for extracting location information and timestamps from the uploaded photos, a means for automatically aggregating travel routes based on the extracted data, a means for generating a travel log from the aggregation results, a means for analyzing the generated travel log and proposing an optimal travel route, a means for extracting emotional data and optimizing the travel route according to the user's mood and preferences, a means for notifying the user of the proposed new travel route, a means for the user to actually revisit the new travel route and publish the results to other users, a means for evaluating the published travel route and forming a popularity ranking, and a means for providing rewards to creators of highly rated travel routes. This enables efficient organization and analysis of travelers' photo data and personalized travel suggestions. Furthermore, by sharing and rating travel experiences, it is possible to create comprehensive travel plans that can be used as reference by other travelers.

[0914] "Tourist" means an individual or group undertaking a trip.

[0915] "Photographs" refer to digital image data taken by travelers during their trip.

[0916] "Bulk upload" refers to sending multiple photo data to a server over the Internet at once.

[0917] "Location information" refers to the location where the photo was taken, indicated by GPS coordinates.

[0918] A "timestamp" refers to information that indicates the date and time a photo was taken.

[0919] "Extraction" refers to extracting specific metadata (location information and timestamp) from the photo data.

[0920] A "travel route" refers to the order of the places visited by a traveler and the route taken by them.

[0921] "Automatically calculated" means that the program calculates the locations visited, their order, and the duration of stay based on location information and timestamps.

[0922] A "travel log" refers to a digital document that compiles a traveler's route and associated photographs and descriptions.

[0923] "Analysis" refers to the act of examining data in detail to find meaning, particularly using AI and machine learning.

[0924] "Optimal travel route" refers to a route that suggests the most efficient and satisfying travel plan based on the user's preferences and rating data.

[0925] "Emotional data" refers to data that indicates the emotional state of a traveler extracted from their photos and text.

[0926] "Optimization" refers to using emotional data to adjust travel routes and recommendations to best suit a traveler's mood and preferences.

[0927] "Notification" refers to the server sending information to the traveler, especially suggestions for new travel routes.

[0928] "Public" means making the results and photos of the route a user has taken visible to other users.

[0929] "Rating" refers to the act of other users reviewing publicly available travel records and travel routes and judging their quality.

[0930] "Popularity ranking" refers to a list that ranks and displays travel records and travel routes that have received particularly high ratings based on evaluation data.

[0931] "Rewards" refers to incentives (e.g., points, gift cards, cash, etc.) offered to users who receive high ratings.

[0932] System Overview

[0933] This invention is a system that automatically compiles travel routes based on the location information and timestamps of photos taken by travelers and generates a "travel log." Furthermore, it analyzes this "travel log" and uses AI to suggest optimal travel routes. It also includes a mechanism for sharing the results with other users, obtaining ratings, forming a popularity ranking, and offering rewards to those with the highest ratings. Furthermore, by combining it with an emotion engine that recognizes user emotions, it is possible to make more personalized suggestions.

[0934] Hardware and Software Configuration

[0935] The system consists of the following major hardware and software components:

[0936] 1. User device: A device such as a smartphone or PC that uploads photos in bulk.

[0937] 2. Server: Collects the received data, generates and stores travel records, and uses AI to suggest new travel routes and compile evaluations.

[0938] 3. AI system: Analyzes travel records and suggests new travel routes.

[0939] 4. Emotion engine: Recognizes user emotions and optimizes travel routes based on that data.

[0940] 5. Database: Storage for high-rated spots and rating data.

[0941] Operation flow

[0942] When a user returns from a trip, they upload all the photos they took during the trip to a dedicated application. The device analyzes the photo metadata, extracts location information (latitude and longitude) and timestamps, and sends them to the server. The server receives this information, calculates the order of the places the user visited and the duration of their stay, and automatically compiles the user's travel route. The server then generates a travel record from the compilation results, saves it in the user's account, and makes it available for viewing.

[0943] The server then provides the generated travel record to an AI analysis system, which then generates a new travel route that includes places the user has not yet visited, and notifies the user of the new travel route.

[0944] Furthermore, the emotion engine extracts emotional data from the user's photos and text inputs, and this emotional data is sent to the server, which then takes the emotional data into account to optimize the travel route according to the user's mood and preferences, thus enabling more personalized travel suggestions.

[0945] When a user plans and executes a repeat trip using a new travel route and uploads photos again, the server receives the results of the new travel route. This allows the route to be published to other users and ratings to be collected. Finally, the server aggregates the rating data and generates a popularity ranking. Users who create and publish highly rated travel routes can receive rewards.

[0946] Specific examples

[0947] After User A returns from a week-long trip to Europe, he uploads all the photos he took with his smartphone to a dedicated app. The device extracts location information and timestamps from the photos and sends them to the server. Based on this, the server automatically tallys up the order in which User A visited Paris, London, and Berlin, as well as the length of time spent there, and generates a "travel log." This "travel log" is saved in the user's account and can be viewed.

[0948] The AI ​​system then analyzes the "travel log" and proposes a new travel route that includes highly rated cafes and tourist attractions that have not yet been visited in Paris, and the server notifies User A. Furthermore, the emotion engine analyzes User A's emotional data during the trip and reflects tourist attractions and events that match the user's mood in the new travel route. User A then visits Paris again based on the proposed new route and enjoys the new tourist attractions. He then uploads photos again and notifies the server of the execution results. Other users rate User A's new travel route, and the server generates a popularity ranking. Finally, User A, who received a high rating, is provided with a reward.

[0949] Prompt Sentence Examples

[0950] "After returning from a week-long trip to Europe, you upload the photos you took to a dedicated app, sending location information and timestamps to a server. The AI ​​system then suggests a new travel route. This new route includes places you've already visited as well as highly rated spots you haven't yet visited."

[0951] The system allows travelers to share their experiences and receive rewards for positive reviews from other travelers, and leverages AI and an emotion engine to provide more personalized travel plans.

[0952] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0953] Step 1: Bulk upload photos

[0954] When a user returns from a trip, they upload all the photos they took during their trip to a dedicated application by opening the app, tapping the "Upload Photos" button, and selecting the desired photo folder.

[0955] Input: Multiple photos taken by the user.

[0956] Output: Photo data waiting to be uploaded.

[0957] Step 2: Extract location and timestamp

[0958] The device analyzes the metadata (EXIF information) from uploaded photos to extract location information (latitude and longitude) and timestamp. This operation is performed automatically by an internal process on the device. Specifically, the device reads the photo file and reads the latitude, longitude, and timestamp tags.

[0959] Input: Uploaded photo data.

[0960] Output: Location and timestamp data for each photo.

[0961] Step 3: Aggregating travel routes

[0962] The server receives the location information and timestamp sent from the device. Based on this information, the server calculates the order of the locations the user visited and the duration of their stay, and automatically aggregates the user's travel route. Specifically, the server sorts the location information along the time axis and calculates the transition time.

[0963] Input: Location and timestamp data.

[0964] Output: Travel route data including order of visits and duration of visits.

[0965] Step 4: Generate a trip record

[0966] The server generates a "trip log" based on the aggregated travel route information, including details of the places visited and related photos. This "trip log" is saved in the user's account and can be viewed. Specifically, the server saves the travel route information and photos in a database and generates an HTML page to display in the web application.

[0967] Input: Travel route data and photo data.

[0968] Output: Trip log data and HTML page.

[0969] Step 5: AI proposes new travel routes

[0970] The server provides the generated "travel log" to an AI analysis system. The AI ​​system analyzes the "travel log" and generates a new travel route that includes places the user has not yet visited based on a database of highly rated restaurants and tourist attractions. Specifically, it uses a machine learning model to analyze similar traveler patterns and recommend the optimal route.

[0971] Input: Travel record data, restaurant and tourist attraction rating database.

[0972] Output: New travel route data.

[0973] Step 6: Extracting emotion data

[0974] The emotion engine extracts emotional data from photos uploaded by users and the accompanying text input. This process is carried out using image analysis technology and natural language processing (NLP). Specifically, facial recognition technology is used to read emotions from photos, and text analysis is used to extract emotions from sentences.

[0975] Input: Photo data, text input data.

[0976] Output: Extracted emotion data.

[0977] Step 7: Optimize the new travel route

[0978] The server optimizes the travel route based on the user's mood and preferences based on the emotional data sent from the emotion engine. Specifically, it uses the emotional data to re-evaluate the priority of destinations and add tourist spots and events that match the user's emotions to the schedule.

[0979] Input: New travel route data, emotion data.

[0980] Output: Optimized travel route data.

[0981] Step 8: Notification of new travel route

[0982] The server notifies the user of the new optimized route via a dedicated app or email. Specific notification procedures include sending a push notification to the user's app or contacting them by email.

[0983] Input: Optimized travel route data.

[0984] Output: A notification message to the user.

[0985] Step 9: Plan and execute a revisit

[0986] The user plans and executes the trip again, referring to the new travel route notified by the server. Specifically, the user checks the new route in the travel diary app and makes a travel plan. After that, the user actually visits the destination.

[0987] Input: Notification of new travel route.

[0988] Output: User revisit plan and execution.

[0989] Step 10: Notification of the results of the new route

[0990] After returning from the trip, the user uploads the photos again and notifies the server of the results of the new travel route. The device again obtains the location information and timestamp and sends them to the server.

[0991] Input: Newly taken photo data.

[0992] Output: The data resulting from the new travel route.

[0993] Step 11: Collecting ratings and generating popularity rankings

[0994] Other users can rate the new routes published, using dedicated applications or websites, for example, through star rating systems or comments, and the server aggregates these rating data to generate a popularity ranking.

[0995] Input: Rating data from other users.

[0996] Output: Updated popularity rankings.

[0997] Step 12: Offer Rewards

[0998] The server will reward users who have received high ratings based on the popularity rankings. Specific forms of rewards can include a points system, gift cards, cash, etc. To provide rewards, the server may add points to the user's account or send a gift card via email.

[0999] Input: Popularity Rankings.

[1000] Output: Providing a reward to the user.

[1001] Through these steps, the system can provide travelers with an efficient and fulfilling travel experience and achieve more personalized service.

[1002] (Application example 2)

[1003] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1004] Conventional travel record and travel diary systems have difficulty efficiently recording and analyzing travelers' visit history and ratings, and lack the functionality to make personalized suggestions that reflect the emotions and preferences of individual users. Furthermore, there is an inadequate mechanism for users to share and rate information based on this data, making it difficult to create attractive travel itineraries or maximize the value of the information.

[1005] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1006] In this invention, the server includes: a means for uploading photos taken by travelers in bulk; a means for extracting location information and timestamps from the uploaded photos; a means for automatically aggregating travel routes based on the extracted data; a means for generating a travel log from the aggregation result; a means for analyzing the generated travel log and proposing new travel routes; a means for notifying users of the proposed new travel routes; a means for users to actually revisit the new travel routes and publish the results to other users; a means for evaluating the published travel routes and forming a popularity ranking; a means for providing rewards to creators of highly rated travel routes; an emotion analysis means for extracting emotion data from captions of the uploaded photos in bulk; and a means for personalizing the proposed travel routes based on the emotion data. This makes it possible to provide personalized travel suggestions that reflect the individual preferences and emotions of users and to efficiently share ratings among users.

[1007] - "Bulk Upload" refers to a User submitting multiple photos to the system at once.

[1008] "Location information" is data that indicates the latitude and longitude of the location where the photo was taken.

[1009] A "timestamp" is data that indicates the date and time a photo was taken.

[1010] A "travel route" is information that indicates the order and route of the places visited by the user.

[1011] A "travel log" is a document that includes a user's travel route and corresponding photos and comments.

[1012] "Analysis" is the process of examining collected data and extracting new information and insights.

[1013] "Suggestions" are actions or routes that the system recommends to users based on the analysis results.

[1014] "Emotional data" is information about a user's emotions extracted from the user's photo captions and text.

[1015] "Personalization" refers to tailoring content to each individual user based on their preferences and emotions.

[1016] "Publishing" refers to making user-generated information or data available for other users to view.

[1017] "Rating" refers to other users' feedback or ratings on published information or data.

[1018] The "popularity ranking" is a list of items that have received particularly high ratings, created based on the evaluation results.

[1019] A "reward" is a monetary or material incentive offered for a certain behavior or outcome.

[1020] "Sentiment analysis" is the process of extracting and analyzing emotional information from text and photo captions entered by users.

[1021] The embodiment of this invention is a system that utilizes photos taken by users and proposes optimal travel routes through emotion analysis and AI analysis. The specific configuration of this system is shown below.

[1022] First, users upload all the photos they took during their trip using a device such as a smartphone or PC. The software used for this is typically Python, Geopy, Exifread, TextBlob, etc. The device then analyzes the metadata of the uploaded photos to extract location information (latitude and longitude) and timestamps.

[1023] The extracted location information and timestamps are then sent to a server, which automatically compiles the user's route, determines the order of stops and the duration of stays, and creates a trip log containing details of the stops and associated photos, which is then stored in the user's account.

[1024] The server then provides the generated travel record to an AI analysis system, which uses a database of highly rated restaurants and tourist attractions to generate a new travel route that includes places the user has not yet visited, and the server notifies the user of this new route.

[1025] Furthermore, the emotion engine extracts emotional data from the user's photo captions and text input. The main software used is TextBlob. Based on this emotional data, the server suggests a route that matches the user's emotions and preferences. For example, it can recognize positive emotions from the caption "The sushi was delicious!" and recommend sushi restaurants to the user.

[1026] The user plans and executes the trip again based on the proposed new travel route. After retaking the trip, the user uploads new photos and notifies the server of the results of the new travel route. The published new travel route is rated by other users, and the results are compiled by the server as a popularity ranking. Users who receive high ratings are rewarded.

[1027] As a concrete example, suppose User A visits a new restaurant over the weekend, takes photos of the food with his smartphone, and uploads them all to the app with the caption "The sushi was delicious!" The app automatically acquires location information and timestamps and sends them to the server. The server then aggregates User A's visited route based on this information and suggests a new travel route based on the analysis results of the AI ​​system and emotion engine. User A can plan a new trip based on the proposed route, then reflect the results in the system again and receive rewards by receiving a high rating.

[1028] Example prompts for generative AI models

[1029] Please extract the location and timestamp information from the following travel photos and suggest the best food delivery route based on that information. Also, please analyze the user's sentiment from the captions corresponding to each photo and reflect it in the delivery route.

[1030] Photo 1: / path / to / photo1.jpg

[1031] Caption 1: Delicious pizza!

[1032] Photo 2: / path / to / photo2.jpg

[1033] Caption 2: Burger with the best toppings!

[1034] As a result, suggest new restaurants or delivery routes for users to visit.

[1035] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1036] Step 1:

[1037] Bulk upload of photos taken by users during their trip

[1038] Users use their smartphones or PCs to upload multiple photos taken during their trip to the application in bulk. The input is the photo files taken during the trip, and the output is the photo data sent to the server. This allows users to easily share multiple photos at once to the system.

[1039] Step 2:

[1040] The device parses the metadata of the uploaded photo

[1041] The device extracts Exif information from uploaded photos to obtain location information (latitude and longitude) and timestamp. The input is the photo file, and the output is the extracted location information and timestamp. The specific operation is to read the metadata using the Exifread library and parse it.

[1042] Step 3:

[1043] The server automatically calculates the route traveled based on location information and timestamps.

[1044] The server receives location information and timestamp data sent from the device and automatically compiles the user's travel route based on this. The input is location information and timestamp, and the output is travel route information including details of the places visited and the order in which they were visited. This effectively records the user's travel route.

[1045] Step 4:

[1046] The server generates a trip record from the aggregated results.

[1047] The server automatically generates a user's travel record based on the collected travel route information and related photos. The input is travel route information and photos, and the output is the user's travel record. Specifically, the server organizes photos and information for each place visited and creates a travel record in an easy-to-view format.

[1048] Step 5:

[1049] The server provides the generated travel records to the AI ​​analysis system.

[1050] The server provides the automatically generated travel log to an AI analysis system, which compares it with a database of highly rated restaurants and tourist attractions. The input is the travel log, and the output is a proposal for an optimal new travel route. This identifies new places to visit that are beneficial to the user.

[1051] Step 6:

[1052] The server performs sentiment analysis and personalizes the new route.

[1053] The server uses sentiment analysis tools such as TextBlob to extract emotional data from the user's photo captions. The input is the photo captions, and the output is the extracted emotional data. Based on the emotional data, the AI ​​analysis system generates a personalized new travel route and suggests it to the user.

[1054] Step 7:

[1055] The user revisits the proposed new travel route and publishes the results to the server.

[1056] Users plan their trips based on the proposed new travel routes and upload the results of their visit back to the server. The input is new travel records and photos, and the output is new travel route information that is made public. The value of the information increases as users' travel experiences are shared with other users.

[1057] Step 8:

[1058] The server collects publicly available travel routes and generates ratings and rankings.

[1059] The server collects the published new travel routes along with rating data from other users and generates a popularity ranking. The input is new travel route information and rating data, and the output is a ranking. This makes highly rated travel routes clear and promotes information sharing within the user community.

[1060] Step 9:

[1061] The server will reward highly rated travel route creators.

[1062] The server provides rewards to creators of highly rated travel routes based on the evaluation data. The input is ranking data, and the output is providing rewards. This gives users an incentive to share high-quality information.

[1063] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1064] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1065] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1066] [Third embodiment]

[1067] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1068] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1069] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1070] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1071] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1072] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1073] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1074] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1075] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1076] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1077] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1078] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1079] overview

[1080] This system automatically compiles travel routes based on the location information and timestamps of photos taken by travelers and creates a "travel diary." Furthermore, AI analyzes this "travel diary" to suggest optimal travel routes, shares the results with other travelers, and based on their ratings, forms a popularity ranking and offers rewards to those with the highest ratings.

[1081] System configuration

[1082] 1. User device: A device such as a smartphone or PC that uploads photos in bulk.

[1083] 2. Server: Collects the received data, generates and stores the travel diary, and uses AI to suggest new travel routes and collect evaluations.

[1084] 3. AI system: Analyzes travel records and suggests new travel routes.

[1085] 4. Database: Storage for high-rated spots and rating data.

[1086] Program processing

[1087] Bulk photo posting and location analysis

[1088] When a user returns from their trip, they upload all the photos they took to a dedicated application. The device reads the metadata of the photos, extracts the location information (latitude and longitude) and timestamp, and sends them to the server.

[1089] Automatically compile travel routes and create travel diaries

[1090] The server automatically compiles the user's route based on the received location information and timestamps, determining the places the user visited, the order in which they visited them, and the duration of their stay. The server then uses this information to generate a "travel diary" containing details of the places visited and relevant photos, which is then saved and made available to the user's account.

[1091] AI-powered new travel route suggestions

[1092] The server provides the generated "travel diary" to the AI ​​analysis system. The AI ​​refers to the "travel diary" and generates a new travel route that includes places the user has not visited, based on a database of highly rated restaurants and tourist attractions. The server notifies the user of this new travel route.

[1093] Revisiting and Publishing New Courses

[1094] The user plans and executes another trip using the proposed new travel route. After the trip, the user uploads photos again and notifies the server that the new travel route has been executed. The server then publishes the results of the new travel route to other users and collects their ratings.

[1095] Creating popularity rankings and offering rewards

[1096] Other users rate the new travel routes that are published. The server aggregates the rating data and generates a popularity ranking. Users who create and publish highly rated travel routes are rewarded.

[1097] Specific examples

[1098] After returning from a week-long trip to Europe, User A uploads all the photos he or she has taken with their smartphone to a dedicated app. The device extracts location information and timestamps from the photos and sends them to the server.

[1099] Based on this, the server automatically calculates the order in which User A visited Paris, London, and Berlin, and the duration of their stay, and generates a "travel diary." This "travel diary" is saved in the user's account and can be viewed.

[1100] The AI ​​system then analyzes the travel diary and suggests a new travel route in Paris that includes highly rated cafes and tourist attractions that have not yet been visited, and the server notifies User A. User A then visits Paris again based on the proposed new route and enjoys the new tourist spots. Afterwards, the system uploads photos again and notifies the server of the results.

[1101] Other users rate User A's new travel route, and the server generates a popularity ranking. Finally, User A who receives a high rating is rewarded. This process allows many travelers to plan their trips efficiently and have a fulfilling travel experience.

[1102] The processing flow will be explained below.

[1103] Specific explanation of program processing

[1104] Bulk photo posting and location analysis

[1105] Step 1: Upload a photo

[1106] After the trip, users upload all the photos they took to a dedicated application.

[1107] (Specific operation) Click the "Upload photos" button within the app, select multiple photo files from the file selection screen, and press the "Upload" button.

[1108] Step 2: Extract location and timestamp

[1109] The device extracts location information (GPS coordinates) and timestamp from the metadata of each uploaded photo.

[1110] (Specific operation) Analyzes files on the device and obtains the latitude, longitude, and shooting date and time from the EXIF ​​information of each photo.

[1111] Step 3: Send data to the server

[1112] The terminal transmits the extracted data, including the location information and timestamp, to the server.

[1113] (Specific operation) The location information and timestamp data are packaged in JSON format and sent to the server's API endpoint as an HTTP POST request.

[1114] Automatically compile travel routes and create travel diaries

[1115] Step 4: Aggregating travel routes

[1116] Based on the received location information and timestamp, the server automatically compiles the user's travel route by chronologically listing the places they visited.

[1117] (Specific operation) Based on location information, the locations to be visited are plotted on a map, and the order of visits is determined by an algorithm based on timestamps.

[1118] Step 5: Generate a travel diary

[1119] The server generates a "travel diary" based on the collected travel routes.

[1120] (Specific Actions) Create a travel record document by combining text and images, including the names of places visited, the date and time of visit, the duration of stay, and key photos.

[1121] Step 6: Save to your user account

[1122] The server saves the generated "travel diary" in the user's account and makes it available for viewing.

[1123] (Specific operation) The travel diary data is associated with the user's account information and saved in a database, and a link to the new travel diary is displayed on the user's dashboard.

[1124] AI-powered new travel route suggestions

[1125] Step 7: Provide to the AI ​​analysis system

[1126] The server provides the generated "travel diary" to the AI ​​analysis system.

[1127] (Specific operation) Send the travel diary data to the analysis endpoint of the AI ​​server.

[1128] Step 8: Create a new travel route

[1129] The AI ​​refers to the travel diary and a database of highly rated restaurants and tourist attractions to generate new travel routes that include places the user has not yet visited.

[1130] (Specific operation) Highly rated spots are retrieved from the database, and a new route is generated by analyzing the order of visits and travel efficiency using an algorithm.

[1131] Step 9: Proposal Notification

[1132] The server notifies the user of the new travel route created by the AI.

[1133] (Specific behavior) Based on the user's notification settings, the app will send new routes via in-app notification or email.

[1134] Revisiting and Publishing New Courses

[1135] Step 10: Performing a revisit

[1136] Users can use the proposed new travel routes to plan and carry out their trip again.

[1137] (Specific actions) Make travel reservations and schedules based on the suggested route.

[1138] Step 11: Re-upload your photos

[1139] The user revisits the new route and uploads the photos again, notifying the server that the AI's suggestions have been implemented.

[1140] (Specific operation) Upload new travel photos to the app in bulk, check that they correspond to the new route based on the metadata at the time of upload, and notify the server.

[1141] Step 12: Publish

[1142] The server publishes the results of the new travel route to other users.

[1143] (Specific operation) Group photos and testimonials of new routes and create a public page that can be viewed by other users.

[1144] Creating popularity rankings and offering rewards

[1145] Step 13: User Rating

[1146] Other users rate the new travel routes that are published.

[1147] (Specific operation) Press the "Rate" button on the public page and post a star rating and comment.

[1148] Step 14: Generate Popularity Rankings

[1149] The server aggregates user ratings and generates a popularity ranking.

[1150] (Specific operation) Obtain evaluation data, calculate scores, and update the ranking page.

[1151] Step 15: Offer Rewards

[1152] The server rewards users who create and publish highly rated travel routes.

[1153] (Specific operations) Check the account information of highly rated users, transfer rewards, send reward acquisition notifications within the app, and record them in the reward history.

[1154] This process allows users to share their travel experiences and allows other users to use them to make better travel plans.

[1155] Example 1

[1156] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1157] It is difficult to efficiently organize the large number of photos taken by travelers after their trip and keep track of the places they visited and the order in which they did so. It is also complicated to suggest new travel routes based on past travel records. Furthermore, there is no consistent system for sharing recommended sightseeing spots and dining places with other travelers, receiving ratings, or providing rankings and rewards based on ratings.

[1158] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1159] In this invention, the server includes: a means for uploading photos taken by travelers in bulk; a means for extracting location information and timestamps from uploaded photos; a means for automatically aggregating travel routes based on the extracted data; a means for generating a travel log from the aggregation results; a means for analyzing the generated travel log and proposing new travel routes based on highly rated locations; a means for notifying users of the proposed new travel routes; a means for users to actually visit the new travel routes and publish the results to other users; a means for evaluating the published travel routes; a means for creating a popularity ranking based on the evaluation data; and a means for providing rewards to creators of highly rated travel routes. This allows travelers to easily organize their travel logs and create high-quality travel plans. Furthermore, by sharing information with other travelers and receiving ratings, travel information useful to many people is accumulated.

[1160] A "traveler" is a person who travels, takes photos at travel destinations, and leaves records of the photos in the system.

[1161] A "bulk photo uploading means" is a function or device that a traveler uses to upload multiple photos taken during a trip to the system at one time.

[1162] "Location information" is the geographic coordinate data of the location where the photo was taken, expressed in the form of latitude and longitude.

[1163] A "timestamp" is data that records the date and time a photo was taken.

[1164] The "extracting means" is a function or device for obtaining location information and timestamps from uploaded photos.

[1165] The "means for automatically compiling travel routes" is a function or device that automatically compiles the order of a traveler's visits and travel routes based on location information and timestamps.

[1166] The "means for generating a travel record" is a function or device that creates a detailed travel record based on the collected travel routes.

[1167] The "analysis means" is a function or device for analyzing the generated travel record and proposing a new travel route based on highly rated places.

[1168] "Means for suggesting new travel routes" refers to a function or device that provides users with travel routes that include new tourist spots and restaurants based on the analysis results.

[1169] "Notification means" refers to a function or device for informing the user of a proposed new travel route.

[1170] A "means for publishing" is a function or device that provides the results of new travel routes visited by a user in a manner that is visible to other users.

[1171] The "means for evaluating" is a function or device that collects feedback from other users regarding the published travel route.

[1172] The "means for forming a popularity ranking" is a function or device that determines the ranking of highly rated travel routes based on the collected evaluation data.

[1173] The "means for providing rewards" is a function or device that provides rewards to users who create highly rated travel routes.

[1174] The system for implementing this invention is composed of multiple components: a user terminal, a server, an AI system, and a database. The specific processing and operation of this system will be described below.

[1175] Bulk photo posting and location analysis

[1176] After returning home from their trip, users can upload all the photos they have taken to a dedicated application using their smartphone, PC, or other device. This application requires an internet connection.

[1177] The device retrieves the Exif data from the uploaded photos, extracting the location (latitude and longitude) and timestamp, and temporarily stores this metadata in a local database.

[1178] The device uses an HTTP POST request to send the extracted metadata to the server in JSON format.

[1179] Automatically compile travel routes and create travel diaries

[1180] The server analyzes the received metadata, which involves matching it with previously stored location information to determine which places a traveler visited and in what order. A distributed processing system such as Hadoop may be used to sort the data chronologically and calculate dwell times.

[1181] The server then generates a "travel diary" containing details of the places visited and related photos based on the travel data collected, which is then stored in the user's individual account and can be viewed through a dedicated application.

[1182] AI-powered new travel route suggestions

[1183] The server uses a REST API to provide the generated travel diary to the AI ​​analysis system.

[1184] The AI ​​system analyzes the travel diary and consults a database of highly rated restaurants and attractions, often using a relational database management system such as PostgreSQL.

[1185] The AI ​​system generates a new route and returns this information to the server via API, which then notifies the user of the newly generated route via push notification or email.

[1186] Revisiting and Publishing New Courses

[1187] When the user actually visits the proposed new route, they take photos again and upload them to the app, following the same process as the first time.

[1188] The server receives the results of new route executions and makes them available to other users via a web interface or an in-app feed.

[1189] Other users rate the new travel routes that are published, and these ratings are collected using buttons and comment functions.

[1190] Creating popularity rankings and offering rewards

[1191] The server aggregates the rating data and generates a popularity ranking, which may involve the use of data analysis tools such as BigQuery.

[1192] The server rewards users who create highly rated travel routes, typically in the form of gift cards or points added to the user's account.

[1193] Examples of concrete examples and prompts

[1194] For example, after returning from a week-long trip to Europe, User A uploads all the photos he or she took to a dedicated app. The device extracts location information and timestamps from the photos and sends them to the server. Based on this information, the server automatically calculates the order of visits to Paris, London, and Berlin, as well as the duration of stay, and generates a "travel diary." This travel diary is saved in the user's account and can be viewed.

[1195] The AI ​​system analyzes the generated "travel diary" and suggests a new travel route that includes highly rated cafes and tourist attractions, which the server notifies User A of. User A visits Paris again based on the new route and uploads the results with photos. The results of the new travel route are made public to other users and are rated. The server compiles the rating data, generates a popularity ranking, and provides rewards to User A who receives high ratings.

[1196] Prompt Sentence Examples

[1197] Please explain a program that automatically compiles the route traveled that day based on the metadata (location information and timestamp) of photos uploaded by users and creates a travel diary. Also, please explain the function of an AI system that analyzes the travel diary and suggests new routes.

[1198] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1199] Step 1:

[1200] After the trip is over, users upload all of the photos they took to a dedicated application.

[1201] Input: Photos taken during travel (image files)

[1202] Output: Upload queue in the application

[1203] How it works: The user launches the app and selects multiple photos from the gallery. The selected photos are added to a queue for uploading within the app. This operation is performed via a GUI.

[1204] Step 2:

[1205] The device retrieves the Exif data from the uploaded photo and extracts the location information (latitude and longitude) and timestamp.

[1206] Input: Uploaded photo (image file)

[1207] Output: Metadata including location and timestamp

[1208] What it does: The application uses the Exif library to extract metadata from each image file, storing the extracted data as a JSON object containing latitude and longitude coordinates and a timestamp.

[1209] Step 3:

[1210] The terminal transmits the extracted metadata to the server.

[1211] Input: Metadata including location and timestamp (JSON format)

[1212] Output: Data sent to the server

[1213] What it does: Your application sends the extracted metadata to the server using an HTTP POST request, to a pre-configured API endpoint.

[1214] Step 4:

[1215] The server analyzes the received metadata to determine the order in which the user visited each location and the duration of their stay.

[1216] Input: Metadata including location and timestamp (JSON format)

[1217] Output: Order of visits and duration

[1218] Specific operation: The server stores the received data in a database and sorts it chronologically. The difference between the previous and next timestamps is used to calculate the stay time. A distributed processing system (e.g., Hadoop) is used to efficiently process large amounts of data.

[1219] Step 5:

[1220] The server generates a "travel diary" containing details of the places visited and associated photos and stores it in the user's account.

[1221] Input: Order of visits and duration, location information and timestamp

[1222] Output: "Travel diary" saved in user account

[1223] Specific operation: The server retrieves detailed information about each visited place (e.g., place name, tourist attraction name) from the database based on the location information. It then combines the retrieved information with related photos and the duration of stay to create a "travel diary" and save it in the user's account.

[1224] Step 6:

[1225] The server provides the generated "travel diary" to an AI analysis system and suggests new travel routes.

[1226] Input: Generated "Travel Diary"

[1227] Output: New travel route proposal

[1228] How it works: The server uses a REST API to send the "travel diary" to the AI ​​analysis system, which analyzes the travel diary and suggests new travel routes including highly rated tourist spots and restaurants based on the database.

[1229] Step 7:

[1230] The server notifies the user of the proposed new travel route.

[1231] Input: New travel route proposal

[1232] Output: User notification

[1233] What happens: The server notifies the user of the new route via push notification or email, including details about the new route and recommended places to visit.

[1234] Step 8:

[1235] The user tries a new route and uploads the results again.

[1236] Enter: New travel photos

[1237] Output: Upload queue and metadata including location and timestamp

[1238] How it works: The user plans a new trip based on a new route, and uploads the photos they take after the trip to the dedicated application. Just like the first time, location information and timestamps are extracted from the photos and sent to the server.

[1239] Step 9:

[1240] The server publishes the results of the new travel route to other users and collects their ratings.

[1241] Input: Execution result of new travel route

[1242] Output: Published execution results and evaluations

[1243] What it does: The server publishes the results of new route executions in a web interface or in-app feed, where other users can view the information and provide feedback using the rating buttons and comment functions.

[1244] Step 10:

[1245] The server aggregates the rating data, generates popularity rankings, and provides rewards.

[1246] Input: Collected assessment data

[1247] Output: Popularity ranking and rewards

[1248] How it works: The server aggregates the rating data using data analysis tools such as BigQuery. Users who create highly rated travel routes are rewarded with gift cards or points.

[1249] (Application example 1)

[1250] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1251] Modern travelers take many photos and videos while traveling, but organizing this data and creating travel routes and travel records is time-consuming. Finding the best route for each destination requires collecting a lot of information and planning on your own. Furthermore, there is no system for sharing these travel records with other travelers and receiving their ratings, making it difficult to improve the quality of the travel experience.

[1252] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1253] In this invention, the server includes: a means for uploading photos and videos taken by travelers in bulk; a means for extracting location information and timestamps from the uploaded photos and videos; a means for automatically aggregating travel routes based on the extracted data; a means for generating a travel log from the aggregation results; a means for analyzing the generated travel log and proposing new travel routes; a means for displaying the proposed new travel routes in a virtual environment; a means for notifying the user; a means for the user to actually revisit the new travel route and publish the results to other users; a means for evaluating the published travel routes and forming a popularity ranking; and a means for providing rewards to creators of highly rated travel routes. This enables the automatic creation of travel logs, the proposal of optimal travel routes, and the sharing of information with other travelers and feedback through evaluations.

[1254] "Tourist" means a person who undertakes a trip.

[1255] "Photos and videos" refers to digital content containing visual information captured using a camera or other photographic device.

[1256] "Bulk upload" refers to the process of transferring multiple photos and videos simultaneously to an online server.

[1257] "Location information" refers to geographic coordinate data such as the latitude and longitude of a location.

[1258] A "timestamp" refers to information that records the date and time that digital data was acquired.

[1259] "Travel route" refers to information showing the places a traveler has visited and the route they took.

[1260] A "travel log" refers to a detailed record of the places a traveler visited, the order in which they visited, and the duration of their stay.

[1261] "Analysis" refers to the process of examining data in detail to reveal its structure and relationships.

[1262] "New travel routes" refer to new routes proposed to improve existing visitor routes.

[1263] "Virtual environment" refers to a virtual space generated by a computer.

[1264] "Notification" refers to the process of conveying information to a recipient.

[1265] "Revisiting" refers to the act of returning to a place that you have visited before.

[1266] "Publishing" refers to the process of making specific information available for others to view.

[1267] "Evaluation" refers to the act of expressing an opinion or value about a particular piece of information or action.

[1268] "Popularity ranking" refers to a list of items arranged in order of highest ratings.

[1269] "Reward" refers to the consideration or prize offered for some action or achievement.

[1270] "Artificial intelligence" refers to the technology that enables computers to judge and process data like humans.

[1271] A "prompt sentence" refers to an instruction sentence to be input into a generative AI model.

[1272] This invention is a system that automatically compiles travel routes based on the location information and timestamps of photos and videos taken by travelers and creates a "travel log." Furthermore, this "travel log" is analyzed, and artificial intelligence suggests new optimal travel routes and notifies the user, supporting efficient travel planning and offering rewards based on evaluations.

[1273] System Configuration

[1274] Hardware

[1275] 1. Smart Glasses:

[1276] Examples: Google Glass, Oculus Quest 2

[1277] Role: Taking photos and videos, recording location and timestamp

[1278] 2. Server:

[1279] Examples: AWS, Google Cloud

[1280] Role: Data collection, processing, analysis, and storage

[1281] 3. PC or smartphone:

[1282] Role: Bulk upload of data, providing user interface

[1283] software

[1284] 1. Application development environment:

[1285] Example: Unity, ARKit / ARCore

[1286] Role: Creating virtual environments and providing virtual tours

[1287] 2. AI analysis system:

[1288] Examples: TensorFlow, PyTorch

[1289] Role: Data analysis, suggesting optimal new travel routes, generating prompts

[1290] 3. Database:

[1291] Examples: Firebase, MongoDB

[1292] Role: Storing highly rated spots and rating data

[1293] Processing steps

[1294] 1. Bulk upload of photos and videos:

[1295] Users can upload photos and videos taken with the smart glasses to a cloud server via their smartphones while traveling. The uploading application is installed on the user's PC or smartphone.

[1296] 2. Location and timestamp analysis:

[1297] The server analyzes the metadata of uploaded photos and videos to extract location information (latitude and longitude) and timestamps.

[1298] 3. Automatically track your travel route and create a trip record:

[1299] The server automatically compiles the traveler's route based on the extracted location information and timestamps, thereby determining the places the user visited, the order in which they visited them, and the duration of their stay, creating a detailed "travel record."

[1300] 4. AI-powered new travel route suggestions:

[1301] The generated "travel log" is provided to an AI analysis system, which then uses a database of highly rated restaurants and tourist attractions to suggest new optimal travel routes. The new routes are then notified to the user.

[1302] 5. Route display in virtual environment:

[1303] The proposed new travel route is displayed in a virtual environment, and users can take part in this virtual tour through smart glasses.

[1304] 6. Revisiting new routes and publishing results:

[1305] Users can plan and carry out their trips again based on the proposed new travel routes and share the results with other users.

[1306] 7. Evaluation and ranking formation:

[1307] Other travelers rate the new travel routes that are published, and the server aggregates the rating data to form a popularity ranking.

[1308] 8. Offering Rewards:

[1309] Users who create and publish highly rated travel routes will be rewarded.

[1310] Specific examples

[1311] After returning from a week-long trip, the user uploads all the photos and videos taken with the smart glasses to a dedicated application. The server extracts location information (latitude and longitude) and timestamps from this data and automatically compiles the user's travel route. This identifies the places the user visited, the order in which they visited them, and the duration of their stay, creating a detailed "travel record."

[1312] The AI ​​system then analyzes the "travel log" and proposes a new travel route that includes highly rated spots that the user has not yet visited. The server notifies the user. The user can then check this new travel route in a virtual environment and plan and carry out the trip again. The results can then be made public to other users and evaluated.

[1313] Prompt Sentence Examples

[1314] "Photos and videos taken with smart glasses while traveling are uploaded to the cloud, and the metadata is analyzed to automatically generate a virtual travel route. Based on data on the places visited, AI proposes new, optimal travel routes and provides them in the form of a virtual tour. These tours can be shared with other users, and rewards are provided for those who rate them highly."

[1315] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1316] Step 1:

[1317] The user takes photos and videos while traveling with the smart glasses and uploads them to a cloud server via their smartphone. Specifically, the user uses a dedicated application to execute the bulk upload function. The input is the photographed data, and the output is the data stored on the cloud server.

[1318] Step 2:

[1319] The server analyzes the metadata of uploaded photos and videos to extract location information (latitude and longitude) and timestamp. Specifically, it uses an image processing library to obtain metadata and analyze location information and timestamp. The input is the uploaded photos and videos, and the output is a list of location information and timestamp.

[1320] Step 3:

[1321] The server automatically compiles the travel route of the traveler based on the extracted location information and timestamps. Specifically, it organizes the location information and timestamp data in chronological order and runs an algorithm to identify the places visited and the order in which they were visited. The input is a list of location information and timestamps, and the output is travel route information.

[1322] Step 4:

[1323] The server generates a detailed "travel record" based on the travel route information. Specifically, it associates photos and videos with each visited location and creates a travel record in a format that records the traveler's activities in detail. The input is the travel route information, and the output is the travel record data.

[1324] Step 5:

[1325] The server uses artificial intelligence to analyze the generated travel log and propose a new optimal travel route. Specifically, it uses an AI analysis system to reference a database of highly rated spots and generate a new route that includes unvisited but highly rated locations. The input is the travel log data, and the output is the proposed new travel route.

[1326] Step 6:

[1327] The server displays the proposed new travel route in a virtual environment. Specifically, it generates a virtual tour using Unity or ARKit / ARCore and creates data to display on smart glasses. The input is the new travel route, and the output is the virtual tour data.

[1328] Step 7:

[1329] The server notifies the user, specifically, by sending the details of the new travel route and a link to the virtual tour to the user through a smartphone application. The input is the virtual tour data, and the output is the notification received by the user.

[1330] Step 8:

[1331] The user plans and executes the trip again based on the proposed new travel route, and uploads the results to the server again. Specifically, the photos and videos taken at the newly visited locations are uploaded to the cloud server. The input is the revisited trip data, and the output is the revisited data stored on the server.

[1332] Step 9:

[1333] The server publishes the revisited data to other users, specifically through a web portal or a sharing function within the application, making the data available for viewing by other users. The input is the revisited data, and the output is the published data.

[1334] Step 10:

[1335] Other travelers rate the published new travel routes. The rating results are compiled on the server, and a popularity ranking is formed. Specifically, user ratings are compiled as points and displayed in a ranking format. The input is the rating data, and the output is a popularity ranking.

[1336] Step 11:

[1337] The server provides rewards to users who create and publish highly rated travel routes. Specifically, it runs a system that provides rewards such as points, cash, and coupons to users who rank highly. The input is the ranking data, and the output is the reward provided.

[1338] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1339] overview

[1340] This system automatically compiles travel routes based on the location information and timestamps of photos taken by travelers and creates a "travel diary." Furthermore, the AI ​​analyzes this "travel diary" to suggest optimal travel routes, and the results are shared with other travelers. Based on their ratings, a popularity ranking is created and rewards are offered to those with the highest ratings. Furthermore, by combining it with an emotion engine that recognizes user emotions, more personalized suggestions can be made.

[1341] System configuration

[1342] 1. User device: A device such as a smartphone or PC that uploads photos in bulk.

[1343] 2. Server: Collects the received data, generates and stores the travel diary, and uses AI to suggest new travel routes and collect evaluations.

[1344] 3. AI system: Analyzes travel records and suggests new travel routes.

[1345] 4. Emotion engine: Recognizes user emotions and optimizes travel routes based on that data.

[1346] 5. Database: Storage for high-rated spots and rating data.

[1347] Program processing

[1348] Bulk photo posting and location analysis

[1349] When a user returns from their trip, they upload all the photos they took to a dedicated application. The device reads the metadata of the photos, extracts the location information (latitude and longitude) and timestamp, and sends them to the server.

[1350] Automatically compile travel routes and create travel diaries

[1351] The server automatically compiles the user's route based on the received location information and timestamps, determining the places the user visited, the order in which they visited them, and the duration of their stay. The server then uses this information to generate a "travel diary" containing details of the places visited and relevant photos, which is then saved and made available to the user's account.

[1352] AI-powered new travel route suggestions

[1353] The server provides the generated "travel diary" to the AI ​​analysis system. The AI ​​refers to the "travel diary" and generates a new travel route that includes places the user has not visited, based on a database of highly rated restaurants and tourist attractions. The server notifies the user of this new travel route.

[1354] Combining Emotion Engines

[1355] The emotion engine extracts emotional data from users' photos and text inputs. The extracted emotional data is sent to the server, which then takes this emotional data into account to optimize the route based on the user's mood and preferences, providing a more personalized experience.

[1356] Revisiting and Publishing New Courses

[1357] The user plans and executes another trip using the proposed new travel route. After the trip, the user uploads photos again and notifies the server that the new travel route has been executed. The server then publishes the results of the new travel route to other users and collects their ratings.

[1358] Creating popularity rankings and offering rewards

[1359] Other users rate the new travel routes that are published. The server aggregates the rating data and generates a popularity ranking. Users who create and publish highly rated travel routes are rewarded.

[1360] Specific examples

[1361] After returning from a week-long trip to Europe, User A uploads all the photos he or she has taken with their smartphone to a dedicated app. The device extracts location information and timestamps from the photos and sends them to the server.

[1362] Based on this, the server automatically calculates the order in which User A visited Paris, London, and Berlin, and the duration of their stay, and generates a "travel diary." This "travel diary" is saved in the user's account and can be viewed.

[1363] The AI ​​system then analyzes the travel diary and suggests a new travel route that includes highly rated cafes and tourist attractions that User A has not yet visited in Paris, and the server notifies User A. Furthermore, the emotion engine analyzes User A's emotional data during the trip and incorporates tourist spots and events that match the user's mood into the new travel route.

[1364] User A visits Paris again based on the proposed new route, enjoys new tourist spots, and then uploads photos again and notifies the server of the results.

[1365] Other users rate User A's new travel route, and the server generates a popularity ranking. Ultimately, User A, who receives a high rating, is rewarded. This process allows many travelers to plan their trips efficiently and have a fulfilling travel experience. Furthermore, by combining it with an emotion engine, it is possible to provide a personalized travel experience that matches each individual user's mood and preferences.

[1366] The processing flow will be explained below.

[1367] Explanation of the specific processing of the program

[1368] Bulk photo posting and location analysis

[1369] Step 1: Upload a photo

[1370] After the trip, users upload all the photos they took to a dedicated application.

[1371] (Specific operation) Click the "Upload photos" button within the app, select multiple photo files from the file selection screen, and press the "Upload" button.

[1372] Step 2: Extract location and timestamp

[1373] The device extracts location information (GPS coordinates) and timestamp from the metadata of each uploaded photo.

[1374] (Specific operation) Analyzes files on the device and obtains the latitude, longitude, and shooting date and time from the EXIF ​​information of each photo.

[1375] Step 3: Send data to the server

[1376] The terminal transmits the extracted data, including the location information and timestamp, to the server.

[1377] (Specific operation) The location information and timestamp data are packaged in JSON format and sent to the server's API endpoint as an HTTP POST request.

[1378] Automatically compile travel routes and create travel diaries

[1379] Step 4: Aggregating travel routes

[1380] Based on the received location information and timestamp, the server automatically compiles the user's travel route by chronologically listing the places they visited.

[1381] (Specific operation) Based on location information, the locations to be visited are plotted on a map, and the order of visits is determined by an algorithm based on timestamps.

[1382] Step 5: Generate a travel diary

[1383] The server generates a "travel diary" based on the collected travel routes.

[1384] (Specific Actions) Create a travel record document by combining text and images, including the names of places visited, the date and time of visit, the duration of stay, and key photos.

[1385] Step 6: Save to your user account

[1386] The server saves the generated "travel diary" in the user's account and makes it available for viewing.

[1387] (Specific operation) The travel diary data is associated with the user's account information and saved in a database, and a link to the new travel diary is displayed on the user's dashboard.

[1388] AI-powered new travel route suggestions

[1389] Step 7: Provide to the AI ​​analysis system

[1390] The server provides the generated "travel diary" to the AI ​​analysis system.

[1391] (Specific operation) Send the travel diary data to the analysis endpoint of the AI ​​server.

[1392] Step 8: Create a new travel route

[1393] The AI ​​refers to the travel diary and a database of highly rated restaurants and tourist attractions to generate new travel routes that include places the user has not yet visited.

[1394] (Specific operation) Highly rated spots are retrieved from the database, and a new route is generated by analyzing the order of visits and travel efficiency using an algorithm.

[1395] Step 9: Proposal Notification

[1396] The server notifies the user of the new travel route created by the AI.

[1397] (Specific behavior) Based on the user's notification settings, the app will send new routes via in-app notification or email.

[1398] Combining Emotion Engines

[1399] Step 10: Extracting Emotion Data

[1400] The emotion engine extracts emotional data from user photos and text inputs.

[1401] (Specific operation) Uploaded photos and text are analyzed, and emotional data (e.g., joy, surprise, sadness, etc.) is extracted using an emotion recognition algorithm.

[1402] Step 11: Sending Emotion Data

[1403] The emotion engine transmits the extracted emotion data to the server.

[1404] (Specific operation) The analysis results are sent in JSON format to the server's API endpoint.

[1405] Step 12: Optimize travel routes

[1406] Based on the received emotional data, the server optimizes the travel route to match the user's mood and preferences.

[1407] (Specific operation) Taking into account the emotional data, the most suitable location is selected from among the highly rated spots, and the optimal travel route is regenerated.

[1408] Revisiting and Publishing New Courses

[1409] Step 13: Performing a revisit

[1410] Users can use the proposed new travel routes to plan and carry out their trip again.

[1411] (Specific actions) Make travel reservations and schedules based on the suggested route.

[1412] Step 14: Re-upload photos

[1413] The user revisits the new route and uploads the photos again, notifying the server that the AI's suggestions have been implemented.

[1414] (Specific operation) Upload new travel photos to the app in bulk, check that they correspond to the new route based on the metadata at the time of upload, and notify the server.

[1415] Step 15: Publish the new travel route

[1416] The server publishes the results of the new travel route to other users.

[1417] (Specific operation) Group photos and testimonials of new routes and create a public page that can be viewed by other users.

[1418] Creating popularity rankings and offering rewards

[1419] Step 16: User Rating

[1420] Other users rate the new travel routes that are published.

[1421] (Specific operation) Press the "Rate" button on the public page and post a star rating and comment.

[1422] Step 17: Generate Popularity Rankings

[1423] The server aggregates user ratings and generates a popularity ranking.

[1424] (Specific operation) Obtain evaluation data, calculate scores, and update the ranking page.

[1425] Step 18: Offer Rewards

[1426] The server rewards users who create and publish highly rated travel routes.

[1427] (Specific operations) Check the account information of highly rated users, transfer rewards, send reward acquisition notifications within the app, and record them in the reward history.

[1428] This process allows users to share their travel experiences, allowing other users to use them to plan more fulfilling trips. Furthermore, by combining it with an emotion engine, it is possible to provide a personalized travel experience that matches each user's mood and preferences.

[1429] Example 2

[1430] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1431] It is difficult to efficiently organize and analyze the vast amount of photo data taken by travelers, and to optimize travel routes based on emotional data during travel. Furthermore, systems that allow users to share their travel experiences with other users and receive feedback through ratings to further enhance travel plans are currently inadequate. Conventional systems struggle to plan trips that take into account travelers' emotions and interests, limiting their ability to provide personalized travel recommendations.

[1432] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for uploading photos taken by travelers in bulk, a means for extracting location information and timestamps from the uploaded photos, a means for automatically aggregating travel routes based on the extracted data, a means for generating a travel log from the aggregation results, a means for analyzing the generated travel log and proposing an optimal travel route, a means for extracting emotional data and optimizing the travel route according to the user's mood and preferences, a means for notifying the user of the proposed new travel route, a means for the user to actually revisit the new travel route and publish the results to other users, a means for evaluating the published travel route and forming a popularity ranking, and a means for providing rewards to creators of highly rated travel routes. This enables efficient organization and analysis of travelers' photo data and personalized travel suggestions. Furthermore, by sharing and rating travel experiences, it is possible to create comprehensive travel plans that can be used as reference by other travelers.

[1433] "Tourist" means an individual or group undertaking a trip.

[1434] "Photographs" refer to digital image data taken by travelers during their trip.

[1435] "Bulk upload" refers to sending multiple photo data to a server over the Internet at once.

[1436] "Location information" refers to the location where the photo was taken, indicated by GPS coordinates.

[1437] A "timestamp" refers to information that indicates the date and time a photo was taken.

[1438] "Extraction" refers to extracting specific metadata (location information and timestamp) from the photo data.

[1439] A "travel route" refers to the order of the places visited by a traveler and the route taken by them.

[1440] "Automatically calculated" means that the program calculates the locations visited, their order, and the duration of stay based on location information and timestamps.

[1441] A "travel log" refers to a digital document that compiles a traveler's route and associated photographs and descriptions.

[1442] "Analysis" refers to the act of examining data in detail to find meaning, particularly using AI and machine learning.

[1443] "Optimal travel route" refers to a route that suggests the most efficient and satisfying travel plan based on the user's preferences and rating data.

[1444] "Emotional data" refers to data that indicates the emotional state of a traveler extracted from their photos and text.

[1445] "Optimization" refers to using emotional data to adjust travel routes and recommendations to best suit a traveler's mood and preferences.

[1446] "Notification" refers to the server sending information to the traveler, especially suggestions for new travel routes.

[1447] "Public" means making the results and photos of the route a user has taken visible to other users.

[1448] "Rating" refers to the act of other users reviewing publicly available travel records and travel routes and judging their quality.

[1449] "Popularity ranking" refers to a list that ranks and displays travel records and travel routes that have received particularly high ratings based on evaluation data.

[1450] "Rewards" refers to incentives (e.g., points, gift cards, cash, etc.) offered to users who receive high ratings.

[1451] System Overview

[1452] This invention is a system that automatically compiles travel routes based on the location information and timestamps of photos taken by travelers and generates a "travel log." Furthermore, it analyzes this "travel log" and uses AI to suggest optimal travel routes. It also includes a mechanism for sharing the results with other users, obtaining ratings, forming a popularity ranking, and offering rewards to those with the highest ratings. Furthermore, by combining it with an emotion engine that recognizes user emotions, it is possible to make more personalized suggestions.

[1453] Hardware and Software Configuration

[1454] The system consists of the following major hardware and software components:

[1455] 1. User device: A device such as a smartphone or PC that uploads photos in bulk.

[1456] 2. Server: Collects the received data, generates and stores travel records, and uses AI to suggest new travel routes and compile evaluations.

[1457] 3. AI system: Analyzes travel records and suggests new travel routes.

[1458] 4. Emotion engine: Recognizes user emotions and optimizes travel routes based on that data.

[1459] 5. Database: Storage for high-rated spots and rating data.

[1460] Operation flow

[1461] When a user returns from a trip, they upload all the photos they took during the trip to a dedicated application. The device analyzes the photo metadata, extracts location information (latitude and longitude) and timestamps, and sends them to the server. The server receives this information, calculates the order of the places the user visited and the duration of their stay, and automatically compiles the user's travel route. The server then generates a travel record from the compilation results, saves it in the user's account, and makes it available for viewing.

[1462] The server then provides the generated travel record to an AI analysis system, which then generates a new travel route that includes places the user has not yet visited, and notifies the user of the new travel route.

[1463] Furthermore, the emotion engine extracts emotional data from the user's photos and text inputs, and this emotional data is sent to the server, which then takes the emotional data into account to optimize the travel route according to the user's mood and preferences, thus enabling more personalized travel suggestions.

[1464] When a user plans and executes a repeat trip using a new travel route and uploads photos again, the server receives the results of the new travel route. This allows the route to be published to other users and ratings to be collected. Finally, the server aggregates the rating data and generates a popularity ranking. Users who create and publish highly rated travel routes can receive rewards.

[1465] Specific examples

[1466] After User A returns from a week-long trip to Europe, he uploads all the photos he took with his smartphone to a dedicated app. The device extracts location information and timestamps from the photos and sends them to the server. Based on this, the server automatically tallys up the order in which User A visited Paris, London, and Berlin, as well as the length of time spent there, and generates a "travel log." This "travel log" is saved in the user's account and can be viewed.

[1467] The AI ​​system then analyzes the "travel log" and proposes a new travel route that includes highly rated cafes and tourist attractions that have not yet been visited in Paris, and the server notifies User A. Furthermore, the emotion engine analyzes User A's emotional data during the trip and reflects tourist attractions and events that match the user's mood in the new travel route. User A then visits Paris again based on the proposed new route and enjoys the new tourist attractions. He then uploads photos again and notifies the server of the execution results. Other users rate User A's new travel route, and the server generates a popularity ranking. Finally, User A, who received a high rating, is provided with a reward.

[1468] Prompt Sentence Examples

[1469] "After returning from a week-long trip to Europe, you upload the photos you took to a dedicated app, sending location information and timestamps to a server. The AI ​​system then suggests a new travel route. This new route includes places you've already visited as well as highly rated spots you haven't yet visited."

[1470] The system allows travelers to share their experiences and receive rewards for positive reviews from other travelers, and leverages AI and an emotion engine to provide more personalized travel plans.

[1471] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1472] Step 1: Bulk upload photos

[1473] When a user returns from a trip, they upload all the photos they took during their trip to a dedicated application by opening the app, tapping the "Upload Photos" button, and selecting the desired photo folder.

[1474] Input: Multiple photos taken by the user.

[1475] Output: Photo data waiting to be uploaded.

[1476] Step 2: Extract location and timestamp

[1477] The device analyzes the metadata (EXIF information) from uploaded photos to extract location information (latitude and longitude) and timestamp. This operation is performed automatically by an internal process on the device. Specifically, the device reads the photo file and reads the latitude, longitude, and timestamp tags.

[1478] Input: Uploaded photo data.

[1479] Output: Location and timestamp data for each photo.

[1480] Step 3: Aggregating travel routes

[1481] The server receives the location information and timestamp sent from the device. Based on this information, the server calculates the order of the locations the user visited and the duration of their stay, and automatically aggregates the user's travel route. Specifically, the server sorts the location information along the time axis and calculates the transition time.

[1482] Input: Location and timestamp data.

[1483] Output: Travel route data including order of visits and duration of visits.

[1484] Step 4: Generate a trip record

[1485] The server generates a "trip log" based on the aggregated travel route information, including details of the places visited and related photos. This "trip log" is saved in the user's account and can be viewed. Specifically, the server saves the travel route information and photos in a database and generates an HTML page to display in the web application.

[1486] Input: Travel route data and photo data.

[1487] Output: Trip log data and HTML page.

[1488] Step 5: AI proposes new travel routes

[1489] The server provides the generated "travel log" to an AI analysis system. The AI ​​system analyzes the "travel log" and generates a new travel route that includes places the user has not yet visited based on a database of highly rated restaurants and tourist attractions. Specifically, it uses a machine learning model to analyze similar traveler patterns and recommend the optimal route.

[1490] Input: Travel record data, restaurant and tourist attraction rating database.

[1491] Output: New travel route data.

[1492] Step 6: Extracting emotion data

[1493] The emotion engine extracts emotional data from photos uploaded by users and the accompanying text input. This process is carried out using image analysis technology and natural language processing (NLP). Specifically, facial recognition technology is used to read emotions from photos, and text analysis is used to extract emotions from sentences.

[1494] Input: Photo data, text input data.

[1495] Output: Extracted emotion data.

[1496] Step 7: Optimize the new travel route

[1497] The server optimizes the travel route based on the user's mood and preferences based on the emotional data sent from the emotion engine. Specifically, it uses the emotional data to re-evaluate the priority of destinations and add tourist spots and events that match the user's emotions to the schedule.

[1498] Input: New travel route data, emotion data.

[1499] Output: Optimized travel route data.

[1500] Step 8: Notification of new travel route

[1501] The server notifies the user of the new optimized route via a dedicated app or email. Specific notification procedures include sending a push notification to the user's app or contacting them by email.

[1502] Input: Optimized travel route data.

[1503] Output: A notification message to the user.

[1504] Step 9: Plan and execute a revisit

[1505] The user plans and executes the trip again, referring to the new travel route notified by the server. Specifically, the user checks the new route in the travel diary app and makes a travel plan. After that, the user actually visits the destination.

[1506] Input: Notification of new travel route.

[1507] Output: User revisit plan and execution.

[1508] Step 10: Notification of the results of the new route

[1509] After returning from the trip, the user uploads the photos again and notifies the server of the results of the new travel route. The device again obtains the location information and timestamp and sends them to the server.

[1510] Input: Newly taken photo data.

[1511] Output: The data resulting from the new travel route.

[1512] Step 11: Collecting ratings and generating popularity rankings

[1513] Other users can rate the new routes published, using dedicated applications or websites, for example, through star rating systems or comments, and the server aggregates these rating data to generate a popularity ranking.

[1514] Input: Rating data from other users.

[1515] Output: Updated popularity rankings.

[1516] Step 12: Offer Rewards

[1517] The server will reward users who have received high ratings based on the popularity rankings. Specific forms of rewards can include a points system, gift cards, cash, etc. To provide rewards, the server may add points to the user's account or send a gift card via email.

[1518] Input: Popularity Rankings.

[1519] Output: Providing a reward to the user.

[1520] Through these steps, the system can provide travelers with an efficient and fulfilling travel experience and achieve more personalized service.

[1521] (Application example 2)

[1522] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1523] Conventional travel record and travel diary systems have difficulty efficiently recording and analyzing travelers' visit history and ratings, and lack the functionality to make personalized suggestions that reflect the emotions and preferences of individual users. Furthermore, there is an inadequate mechanism for users to share and rate information based on this data, making it difficult to create attractive travel itineraries or maximize the value of the information.

[1524] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1525] In this invention, the server includes: a means for uploading photos taken by travelers in bulk; a means for extracting location information and timestamps from the uploaded photos; a means for automatically aggregating travel routes based on the extracted data; a means for generating a travel log from the aggregation result; a means for analyzing the generated travel log and proposing new travel routes; a means for notifying users of the proposed new travel routes; a means for users to actually revisit the new travel routes and publish the results to other users; a means for evaluating the published travel routes and forming a popularity ranking; a means for providing rewards to creators of highly rated travel routes; an emotion analysis means for extracting emotion data from captions of the uploaded photos in bulk; and a means for personalizing the proposed travel routes based on the emotion data. This makes it possible to provide personalized travel suggestions that reflect the individual preferences and emotions of users and to efficiently share ratings among users.

[1526] - "Bulk Upload" refers to a User submitting multiple photos to the system at once.

[1527] "Location information" is data that indicates the latitude and longitude of the location where the photo was taken.

[1528] A "timestamp" is data that indicates the date and time a photo was taken.

[1529] A "travel route" is information that indicates the order and route of the places visited by the user.

[1530] A "travel log" is a document that includes a user's travel route and corresponding photos and comments.

[1531] "Analysis" is the process of examining collected data and extracting new information and insights.

[1532] "Suggestions" are actions or routes that the system recommends to users based on the analysis results.

[1533] "Emotional data" is information about a user's emotions extracted from the user's photo captions and text.

[1534] "Personalization" refers to tailoring content to each individual user based on their preferences and emotions.

[1535] "Publishing" refers to making user-generated information or data available for other users to view.

[1536] "Rating" refers to other users' feedback or ratings on published information or data.

[1537] The "popularity ranking" is a list of items that have received particularly high ratings, created based on the evaluation results.

[1538] A "reward" is a monetary or material incentive offered for a certain behavior or outcome.

[1539] "Sentiment analysis" is the process of extracting and analyzing emotional information from text and photo captions entered by users.

[1540] The embodiment of this invention is a system that utilizes photos taken by users and proposes optimal travel routes through emotion analysis and AI analysis. The specific configuration of this system is shown below.

[1541] First, users upload all the photos they took during their trip using a device such as a smartphone or PC. The software used for this is typically Python, Geopy, Exifread, TextBlob, etc. The device then analyzes the metadata of the uploaded photos to extract location information (latitude and longitude) and timestamps.

[1542] The extracted location information and timestamps are then sent to a server, which automatically compiles the user's route, determines the order of stops and the duration of stays, and creates a trip log containing details of the stops and associated photos, which is then stored in the user's account.

[1543] The server then provides the generated travel record to an AI analysis system, which uses a database of highly rated restaurants and tourist attractions to generate a new travel route that includes places the user has not yet visited, and the server notifies the user of this new route.

[1544] Furthermore, the emotion engine extracts emotional data from the user's photo captions and text input. The main software used is TextBlob. Based on this emotional data, the server suggests a route that matches the user's emotions and preferences. For example, it can recognize positive emotions from the caption "The sushi was delicious!" and recommend sushi restaurants to the user.

[1545] The user plans and executes the trip again based on the proposed new travel route. After retaking the trip, the user uploads new photos and notifies the server of the results of the new travel route. The published new travel route is rated by other users, and the results are compiled by the server as a popularity ranking. Users who receive high ratings are rewarded.

[1546] As a concrete example, suppose User A visits a new restaurant over the weekend, takes photos of the food with his smartphone, and uploads them all to the app with the caption "The sushi was delicious!" The app automatically acquires location information and timestamps and sends them to the server. The server then aggregates User A's visited route based on this information and suggests a new travel route based on the analysis results of the AI ​​system and emotion engine. User A can plan a new trip based on the proposed route, then reflect the results in the system again and receive rewards by receiving a high rating.

[1547] Example prompts for generative AI models

[1548] Please extract the location and timestamp information from the following travel photos and suggest the best food delivery route based on that information. Also, please analyze the user's sentiment from the captions corresponding to each photo and reflect it in the delivery route.

[1549] Photo 1: / path / to / photo1.jpg

[1550] Caption 1: Delicious pizza!

[1551] Photo 2: / path / to / photo2.jpg

[1552] Caption 2: Burger with the best toppings!

[1553] As a result, suggest new restaurants or delivery routes for users to visit.

[1554] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1555] Step 1:

[1556] Bulk upload of photos taken by users during their trip

[1557] Users use their smartphones or PCs to upload multiple photos taken during their trip to the application in bulk. The input is the photo files taken during the trip, and the output is the photo data sent to the server. This allows users to easily share multiple photos at once to the system.

[1558] Step 2:

[1559] The device parses the metadata of the uploaded photo

[1560] The device extracts Exif information from uploaded photos to obtain location information (latitude and longitude) and timestamp. The input is the photo file, and the output is the extracted location information and timestamp. The specific operation is to read the metadata using the Exifread library and parse it.

[1561] Step 3:

[1562] The server automatically calculates the route traveled based on location information and timestamps.

[1563] The server receives location information and timestamp data sent from the device and automatically compiles the user's travel route based on this. The input is location information and timestamp, and the output is travel route information including details of the places visited and the order in which they were visited. This effectively records the user's travel route.

[1564] Step 4:

[1565] The server generates a trip record from the aggregated results.

[1566] The server automatically generates a user's travel record based on the collected travel route information and related photos. The input is travel route information and photos, and the output is the user's travel record. Specifically, the server organizes photos and information for each place visited and creates a travel record in an easy-to-view format.

[1567] Step 5:

[1568] The server provides the generated travel records to the AI ​​analysis system.

[1569] The server provides the automatically generated travel log to an AI analysis system, which compares it with a database of highly rated restaurants and tourist attractions. The input is the travel log, and the output is a proposal for an optimal new travel route. This identifies new places to visit that are beneficial to the user.

[1570] Step 6:

[1571] The server performs sentiment analysis and personalizes the new route.

[1572] The server uses sentiment analysis tools such as TextBlob to extract emotional data from the user's photo captions. The input is the photo captions, and the output is the extracted emotional data. Based on the emotional data, the AI ​​analysis system generates a personalized new travel route and suggests it to the user.

[1573] Step 7:

[1574] The user revisits the proposed new travel route and publishes the results to the server.

[1575] Users plan their trips based on the proposed new travel routes and upload the results of their visit back to the server. The input is new travel records and photos, and the output is new travel route information that is made public. The value of the information increases as users' travel experiences are shared with other users.

[1576] Step 8:

[1577] The server collects publicly available travel routes and generates ratings and rankings.

[1578] The server collects the published new travel routes along with rating data from other users and generates a popularity ranking. The input is new travel route information and rating data, and the output is a ranking. This makes highly rated travel routes clear and promotes information sharing within the user community.

[1579] Step 9:

[1580] The server will reward highly rated travel route creators.

[1581] The server provides rewards to creators of highly rated travel routes based on the evaluation data. The input is ranking data, and the output is providing rewards. This gives users an incentive to share high-quality information.

[1582] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1583] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1584] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1585] [Fourth embodiment]

[1586] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1587] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1588] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1589] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1590] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1591] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1592] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1593] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1594] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1595] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1596] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1597] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1598] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1599] overview

[1600] This system automatically compiles travel routes based on the location information and timestamps of photos taken by travelers and creates a "travel diary." Furthermore, AI analyzes this "travel diary" to suggest optimal travel routes, shares the results with other travelers, and based on their ratings, forms a popularity ranking and offers rewards to those with the highest ratings.

[1601] System configuration

[1602] 1. User device: A device such as a smartphone or PC that uploads photos in bulk.

[1603] 2. Server: Collects the received data, generates and stores the travel diary, and uses AI to suggest new travel routes and collect evaluations.

[1604] 3. AI system: Analyzes travel records and suggests new travel routes.

[1605] 4. Database: Storage for high-rated spots and rating data.

[1606] Program processing

[1607] Bulk photo posting and location analysis

[1608] When a user returns from their trip, they upload all the photos they took to a dedicated application. The device reads the metadata of the photos, extracts the location information (latitude and longitude) and timestamp, and sends them to the server.

[1609] Automatically compile travel routes and create travel diaries

[1610] The server automatically compiles the user's route based on the received location information and timestamps, determining the places the user visited, the order in which they visited them, and the duration of their stay. The server then uses this information to generate a "travel diary" containing details of the places visited and relevant photos, which is then saved and made available to the user's account.

[1611] AI-powered new travel route suggestions

[1612] The server provides the generated "travel diary" to the AI ​​analysis system. The AI ​​refers to the "travel diary" and generates a new travel route that includes places the user has not visited, based on a database of highly rated restaurants and tourist attractions. The server notifies the user of this new travel route.

[1613] Revisiting and Publishing New Courses

[1614] The user plans and executes another trip using the proposed new travel route. After the trip, the user uploads photos again and notifies the server that the new travel route has been executed. The server then publishes the results of the new travel route to other users and collects their ratings.

[1615] Creating popularity rankings and offering rewards

[1616] Other users rate the new travel routes that are published. The server aggregates the rating data and generates a popularity ranking. Users who create and publish highly rated travel routes are rewarded.

[1617] Specific examples

[1618] After returning from a week-long trip to Europe, User A uploads all the photos he or she has taken with their smartphone to a dedicated app. The device extracts location information and timestamps from the photos and sends them to the server.

[1619] Based on this, the server automatically calculates the order in which User A visited Paris, London, and Berlin, and the duration of their stay, and generates a "travel diary." This "travel diary" is saved in the user's account and can be viewed.

[1620] The AI ​​system then analyzes the travel diary and suggests a new travel route in Paris that includes highly rated cafes and tourist attractions that have not yet been visited, and the server notifies User A. User A then visits Paris again based on the proposed new route and enjoys the new tourist spots. Afterwards, the system uploads photos again and notifies the server of the results.

[1621] Other users rate User A's new travel route, and the server generates a popularity ranking. Finally, User A who receives a high rating is rewarded. This process allows many travelers to plan their trips efficiently and have a fulfilling travel experience.

[1622] The processing flow will be explained below.

[1623] Specific explanation of program processing

[1624] Bulk photo posting and location analysis

[1625] Step 1: Upload a photo

[1626] After the trip, users upload all the photos they took to a dedicated application.

[1627] (Specific operation) Click the "Upload photos" button within the app, select multiple photo files from the file selection screen, and press the "Upload" button.

[1628] Step 2: Extract location and timestamp

[1629] The device extracts location information (GPS coordinates) and timestamp from the metadata of each uploaded photo.

[1630] (Specific operation) Analyzes files on the device and obtains the latitude, longitude, and shooting date and time from the EXIF ​​information of each photo.

[1631] Step 3: Send data to the server

[1632] The terminal transmits the extracted data, including the location information and timestamp, to the server.

[1633] (Specific operation) The location information and timestamp data are packaged in JSON format and sent to the server's API endpoint as an HTTP POST request.

[1634] Automatically compile travel routes and create travel diaries

[1635] Step 4: Aggregating travel routes

[1636] Based on the received location information and timestamp, the server automatically compiles the user's travel route by chronologically listing the places they visited.

[1637] (Specific operation) Based on location information, the locations to be visited are plotted on a map, and the order of visits is determined by an algorithm based on timestamps.

[1638] Step 5: Generate a travel diary

[1639] The server generates a "travel diary" based on the collected travel routes.

[1640] (Specific Actions) Create a travel record document by combining text and images, including the names of places visited, the date and time of visit, the duration of stay, and key photos.

[1641] Step 6: Save to your user account

[1642] The server saves the generated "travel diary" in the user's account and makes it available for viewing.

[1643] (Specific operation) The travel diary data is associated with the user's account information and saved in a database, and a link to the new travel diary is displayed on the user's dashboard.

[1644] AI-powered new travel route suggestions

[1645] Step 7: Provide to the AI ​​analysis system

[1646] The server provides the generated "travel diary" to the AI ​​analysis system.

[1647] (Specific operation) Send the travel diary data to the analysis endpoint of the AI ​​server.

[1648] Step 8: Create a new travel route

[1649] The AI ​​refers to the travel diary and a database of highly rated restaurants and tourist attractions to generate new travel routes that include places the user has not yet visited.

[1650] (Specific operation) Highly rated spots are retrieved from the database, and a new route is generated by analyzing the order of visits and travel efficiency using an algorithm.

[1651] Step 9: Proposal Notification

[1652] The server notifies the user of the new travel route created by the AI.

[1653] (Specific behavior) Based on the user's notification settings, the app will send new routes via in-app notification or email.

[1654] Revisiting and Publishing New Courses

[1655] Step 10: Performing a revisit

[1656] Users can use the proposed new travel routes to plan and carry out their trip again.

[1657] (Specific actions) Make travel reservations and schedules based on the suggested route.

[1658] Step 11: Re-upload your photos

[1659] The user revisits the new route and uploads the photos again, notifying the server that the AI's suggestions have been implemented.

[1660] (Specific operation) Upload new travel photos to the app in bulk, check that they correspond to the new route based on the metadata at the time of upload, and notify the server.

[1661] Step 12: Publish

[1662] The server publishes the results of the new travel route to other users.

[1663] (Specific operation) Group photos and testimonials of new routes and create a public page that can be viewed by other users.

[1664] Creating popularity rankings and offering rewards

[1665] Step 13: User Rating

[1666] Other users rate the new travel routes that are published.

[1667] (Specific operation) Press the "Rate" button on the public page and post a star rating and comment.

[1668] Step 14: Generate Popularity Rankings

[1669] The server aggregates user ratings and generates a popularity ranking.

[1670] (Specific operation) Obtain evaluation data, calculate scores, and update the ranking page.

[1671] Step 15: Offer Rewards

[1672] The server rewards users who create and publish highly rated travel routes.

[1673] (Specific operations) Check the account information of highly rated users, transfer rewards, send reward acquisition notifications within the app, and record them in the reward history.

[1674] This process allows users to share their travel experiences and allows other users to use them to make better travel plans.

[1675] Example 1

[1676] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1677] It is difficult to efficiently organize the large number of photos taken by travelers after their trip and keep track of the places they visited and the order in which they did so. It is also complicated to suggest new travel routes based on past travel records. Furthermore, there is no consistent system for sharing recommended sightseeing spots and dining places with other travelers, receiving ratings, or providing rankings and rewards based on ratings.

[1678] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1679] In this invention, the server includes: a means for uploading photos taken by travelers in bulk; a means for extracting location information and timestamps from uploaded photos; a means for automatically aggregating travel routes based on the extracted data; a means for generating a travel log from the aggregation results; a means for analyzing the generated travel log and proposing new travel routes based on highly rated locations; a means for notifying users of the proposed new travel routes; a means for users to actually visit the new travel routes and publish the results to other users; a means for evaluating the published travel routes; a means for creating a popularity ranking based on the evaluation data; and a means for providing rewards to creators of highly rated travel routes. This allows travelers to easily organize their travel logs and create high-quality travel plans. Furthermore, by sharing information with other travelers and receiving ratings, travel information useful to many people is accumulated.

[1680] A "traveler" is a person who travels, takes photos at travel destinations, and leaves records of the photos in the system.

[1681] A "bulk photo uploading means" is a function or device that a traveler uses to upload multiple photos taken during a trip to the system at one time.

[1682] "Location information" is the geographic coordinate data of the location where the photo was taken, expressed in the form of latitude and longitude.

[1683] A "timestamp" is data that records the date and time a photo was taken.

[1684] The "extracting means" is a function or device for obtaining location information and timestamps from uploaded photos.

[1685] The "means for automatically compiling travel routes" is a function or device that automatically compiles the order of a traveler's visits and travel routes based on location information and timestamps.

[1686] The "means for generating a travel record" is a function or device that creates a detailed travel record based on the collected travel routes.

[1687] The "analysis means" is a function or device for analyzing the generated travel record and proposing a new travel route based on highly rated places.

[1688] "Means for suggesting new travel routes" refers to a function or device that provides users with travel routes that include new tourist spots and restaurants based on the analysis results.

[1689] "Notification means" refers to a function or device for informing the user of a proposed new travel route.

[1690] A "means for publishing" is a function or device that provides the results of new travel routes visited by a user in a manner that is visible to other users.

[1691] The "means for evaluating" is a function or device that collects feedback from other users regarding the published travel route.

[1692] The "means for forming a popularity ranking" is a function or device that determines the ranking of highly rated travel routes based on the collected evaluation data.

[1693] The "means for providing rewards" is a function or device that provides rewards to users who create highly rated travel routes.

[1694] The system for implementing this invention is composed of multiple components: a user terminal, a server, an AI system, and a database. The specific processing and operation of this system will be described below.

[1695] Bulk photo posting and location analysis

[1696] After returning home from their trip, users can upload all the photos they have taken to a dedicated application using their smartphone, PC, or other device. This application requires an internet connection.

[1697] The device retrieves the Exif data from the uploaded photos, extracting the location (latitude and longitude) and timestamp, and temporarily stores this metadata in a local database.

[1698] The device uses an HTTP POST request to send the extracted metadata to the server in JSON format.

[1699] Automatically compile travel routes and create travel diaries

[1700] The server analyzes the received metadata, which involves matching it with previously stored location information to determine which places a traveler visited and in what order. A distributed processing system such as Hadoop may be used to sort the data chronologically and calculate dwell times.

[1701] The server then generates a "travel diary" containing details of the places visited and related photos based on the travel data collected, which is then stored in the user's individual account and can be viewed through a dedicated application.

[1702] AI-powered new travel route suggestions

[1703] The server uses a REST API to provide the generated travel diary to the AI ​​analysis system.

[1704] The AI ​​system analyzes the travel diary and consults a database of highly rated restaurants and attractions, often using a relational database management system such as PostgreSQL.

[1705] The AI ​​system generates a new route and returns this information to the server via API, which then notifies the user of the newly generated route via push notification or email.

[1706] Revisiting and Publishing New Courses

[1707] When the user actually visits the proposed new route, they take photos again and upload them to the app, following the same process as the first time.

[1708] The server receives the results of new route executions and makes them available to other users via a web interface or an in-app feed.

[1709] Other users rate the new travel routes that are published, and these ratings are collected using buttons and comment functions.

[1710] Creating popularity rankings and offering rewards

[1711] The server aggregates the rating data and generates a popularity ranking, which may involve the use of data analysis tools such as BigQuery.

[1712] The server rewards users who create highly rated travel routes, typically in the form of gift cards or points added to the user's account.

[1713] Examples of concrete examples and prompts

[1714] For example, after returning from a week-long trip to Europe, User A uploads all the photos he or she took to a dedicated app. The device extracts location information and timestamps from the photos and sends them to the server. Based on this information, the server automatically calculates the order of visits to Paris, London, and Berlin, as well as the duration of stay, and generates a "travel diary." This travel diary is saved in the user's account and can be viewed.

[1715] The AI ​​system analyzes the generated "travel diary" and suggests a new travel route that includes highly rated cafes and tourist attractions, which the server notifies User A of. User A visits Paris again based on the new route and uploads the results with photos. The results of the new travel route are made public to other users and are rated. The server compiles the rating data, generates a popularity ranking, and provides rewards to User A who receives high ratings.

[1716] Prompt Sentence Examples

[1717] Please explain a program that automatically compiles the route traveled that day based on the metadata (location information and timestamp) of photos uploaded by users and creates a travel diary. Also, please explain the function of an AI system that analyzes the travel diary and suggests new routes.

[1718] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1719] Step 1:

[1720] After the trip is over, users upload all of the photos they took to a dedicated application.

[1721] Input: Photos taken during travel (image files)

[1722] Output: Upload queue in the application

[1723] How it works: The user launches the app and selects multiple photos from the gallery. The selected photos are added to a queue for uploading within the app. This operation is performed via a GUI.

[1724] Step 2:

[1725] The device retrieves the Exif data from the uploaded photo and extracts the location information (latitude and longitude) and timestamp.

[1726] Input: Uploaded photo (image file)

[1727] Output: Metadata including location and timestamp

[1728] What it does: The application uses the Exif library to extract metadata from each image file, storing the extracted data as a JSON object containing latitude and longitude coordinates and a timestamp.

[1729] Step 3:

[1730] The terminal transmits the extracted metadata to the server.

[1731] Input: Metadata including location and timestamp (JSON format)

[1732] Output: Data sent to the server

[1733] What it does: Your application sends the extracted metadata to the server using an HTTP POST request, to a pre-configured API endpoint.

[1734] Step 4:

[1735] The server analyzes the received metadata to determine the order in which the user visited each location and the duration of their stay.

[1736] Input: Metadata including location and timestamp (JSON format)

[1737] Output: Order of visits and duration

[1738] Specific operation: The server stores the received data in a database and sorts it chronologically. The difference between the previous and next timestamps is used to calculate the stay time. A distributed processing system (e.g., Hadoop) is used to efficiently process large amounts of data.

[1739] Step 5:

[1740] The server generates a "travel diary" containing details of the places visited and associated photos and stores it in the user's account.

[1741] Input: Order of visits and duration, location information and timestamp

[1742] Output: "Travel diary" saved in user account

[1743] Specific operation: The server retrieves detailed information about each visited place (e.g., place name, tourist attraction name) from the database based on the location information. It then combines the retrieved information with related photos and the duration of stay to create a "travel diary" and save it in the user's account.

[1744] Step 6:

[1745] The server provides the generated "travel diary" to an AI analysis system and suggests new travel routes.

[1746] Input: Generated "Travel Diary"

[1747] Output: New travel route proposal

[1748] How it works: The server uses a REST API to send the "travel diary" to the AI ​​analysis system, which analyzes the travel diary and suggests new travel routes including highly rated tourist spots and restaurants based on the database.

[1749] Step 7:

[1750] The server notifies the user of the proposed new travel route.

[1751] Input: New travel route proposal

[1752] Output: User notification

[1753] What happens: The server notifies the user of the new route via push notification or email, including details about the new route and recommended places to visit.

[1754] Step 8:

[1755] The user tries a new route and uploads the results again.

[1756] Enter: New travel photos

[1757] Output: Upload queue and metadata including location and timestamp

[1758] How it works: The user plans a new trip based on a new route, and uploads the photos they take after the trip to the dedicated application. Just like the first time, location information and timestamps are extracted from the photos and sent to the server.

[1759] Step 9:

[1760] The server publishes the results of the new travel route to other users and collects their ratings.

[1761] Input: Execution result of new travel route

[1762] Output: Published execution results and evaluations

[1763] What it does: The server publishes the results of new route executions in a web interface or in-app feed, where other users can view the information and provide feedback using the rating buttons and comment functions.

[1764] Step 10:

[1765] The server aggregates the rating data, generates popularity rankings, and provides rewards.

[1766] Input: Collected assessment data

[1767] Output: Popularity ranking and rewards

[1768] How it works: The server aggregates the rating data using data analysis tools such as BigQuery. Users who create highly rated travel routes are rewarded with gift cards or points.

[1769] (Application example 1)

[1770] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1771] Modern travelers take many photos and videos while traveling, but organizing this data and creating travel routes and travel records is time-consuming. Finding the best route for each destination requires collecting a lot of information and planning on your own. Furthermore, there is no system for sharing these travel records with other travelers and receiving their ratings, making it difficult to improve the quality of the travel experience.

[1772] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1773] In this invention, the server includes: a means for uploading photos and videos taken by travelers in bulk; a means for extracting location information and timestamps from the uploaded photos and videos; a means for automatically aggregating travel routes based on the extracted data; a means for generating a travel log from the aggregation results; a means for analyzing the generated travel log and proposing new travel routes; a means for displaying the proposed new travel routes in a virtual environment; a means for notifying the user; a means for the user to actually revisit the new travel route and publish the results to other users; a means for evaluating the published travel routes and forming a popularity ranking; and a means for providing rewards to creators of highly rated travel routes. This enables the automatic creation of travel logs, the proposal of optimal travel routes, and the sharing of information with other travelers and feedback through evaluations.

[1774] "Tourist" means a person who undertakes a trip.

[1775] "Photos and videos" refers to digital content containing visual information captured using a camera or other photographic device.

[1776] "Bulk upload" refers to the process of transferring multiple photos and videos simultaneously to an online server.

[1777] "Location information" refers to geographic coordinate data such as the latitude and longitude of a location.

[1778] A "timestamp" refers to information that records the date and time that digital data was acquired.

[1779] "Travel route" refers to information showing the places a traveler has visited and the route they took.

[1780] A "travel log" refers to a detailed record of the places a traveler visited, the order in which they visited, and the duration of their stay.

[1781] "Analysis" refers to the process of examining data in detail to reveal its structure and relationships.

[1782] "New travel routes" refer to new routes proposed to improve existing visitor routes.

[1783] "Virtual environment" refers to a virtual space generated by a computer.

[1784] "Notification" refers to the process of conveying information to a recipient.

[1785] "Revisiting" refers to the act of returning to a place that you have visited before.

[1786] "Publishing" refers to the process of making specific information available for others to view.

[1787] "Evaluation" refers to the act of expressing an opinion or value about a particular piece of information or action.

[1788] "Popularity ranking" refers to a list of items arranged in order of highest ratings.

[1789] "Reward" refers to the consideration or prize offered for some action or achievement.

[1790] "Artificial intelligence" refers to the technology that enables computers to judge and process data like humans.

[1791] A "prompt sentence" refers to an instruction sentence to be input into a generative AI model.

[1792] This invention is a system that automatically compiles travel routes based on the location information and timestamps of photos and videos taken by travelers and creates a "travel log." Furthermore, this "travel log" is analyzed, and artificial intelligence suggests new optimal travel routes and notifies the user, supporting efficient travel planning and offering rewards based on evaluations.

[1793] System Configuration

[1794] Hardware

[1795] 1. Smart Glasses:

[1796] Examples: Google Glass, Oculus Quest 2

[1797] Role: Taking photos and videos, recording location and timestamp

[1798] 2. Server:

[1799] Examples: AWS, Google Cloud

[1800] Role: Data collection, processing, analysis, and storage

[1801] 3. PC or smartphone:

[1802] Role: Bulk upload of data, providing user interface

[1803] software

[1804] 1. Application development environment:

[1805] Example: Unity, ARKit / ARCore

[1806] Role: Creating virtual environments and providing virtual tours

[1807] 2. AI analysis system:

[1808] Examples: TensorFlow, PyTorch

[1809] Role: Data analysis, suggesting optimal new travel routes, generating prompts

[1810] 3. Database:

[1811] Examples: Firebase, MongoDB

[1812] Role: Storing highly rated spots and rating data

[1813] Processing steps

[1814] 1. Bulk upload of photos and videos:

[1815] Users can upload photos and videos taken with the smart glasses to a cloud server via their smartphones while traveling. The uploading application is installed on the user's PC or smartphone.

[1816] 2. Location and timestamp analysis:

[1817] The server analyzes the metadata of uploaded photos and videos to extract location information (latitude and longitude) and timestamps.

[1818] 3. Automatically track your travel route and create a trip record:

[1819] The server automatically compiles the traveler's route based on the extracted location information and timestamps, thereby determining the places the user visited, the order in which they visited them, and the duration of their stay, creating a detailed "travel record."

[1820] 4. AI-powered new travel route suggestions:

[1821] The generated "travel log" is provided to an AI analysis system, which then uses a database of highly rated restaurants and tourist attractions to suggest new optimal travel routes. The new routes are then notified to the user.

[1822] 5. Route display in virtual environment:

[1823] The proposed new travel route is displayed in a virtual environment, and users can take part in this virtual tour through smart glasses.

[1824] 6. Revisiting new routes and publishing results:

[1825] Users can plan and carry out their trips again based on the proposed new travel routes and share the results with other users.

[1826] 7. Evaluation and ranking formation:

[1827] Other travelers rate the new travel routes that are published, and the server aggregates the rating data to form a popularity ranking.

[1828] 8. Offering Rewards:

[1829] Users who create and publish highly rated travel routes will be rewarded.

[1830] Specific examples

[1831] After returning from a week-long trip, the user uploads all the photos and videos taken with the smart glasses to a dedicated application. The server extracts location information (latitude and longitude) and timestamps from this data and automatically compiles the user's travel route. This identifies the places the user visited, the order in which they visited them, and the duration of their stay, creating a detailed "travel record."

[1832] The AI ​​system then analyzes the "travel log" and proposes a new travel route that includes highly rated spots that the user has not yet visited. The server notifies the user. The user can then check this new travel route in a virtual environment and plan and carry out the trip again. The results can then be made public to other users and evaluated.

[1833] Prompt Sentence Examples

[1834] "Photos and videos taken with smart glasses while traveling are uploaded to the cloud, and the metadata is analyzed to automatically generate a virtual travel route. Based on data on the places visited, AI proposes new, optimal travel routes and provides them in the form of a virtual tour. These tours can be shared with other users, and rewards are provided for those who rate them highly."

[1835] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1836] Step 1:

[1837] The user takes photos and videos while traveling with the smart glasses and uploads them to a cloud server via their smartphone. Specifically, the user uses a dedicated application to execute the bulk upload function. The input is the photographed data, and the output is the data stored on the cloud server.

[1838] Step 2:

[1839] The server analyzes the metadata of uploaded photos and videos to extract location information (latitude and longitude) and timestamp. Specifically, it uses an image processing library to obtain metadata and analyze location information and timestamp. The input is the uploaded photos and videos, and the output is a list of location information and timestamp.

[1840] Step 3:

[1841] The server automatically compiles the travel route of the traveler based on the extracted location information and timestamps. Specifically, it organizes the location information and timestamp data in chronological order and runs an algorithm to identify the places visited and the order in which they were visited. The input is a list of location information and timestamps, and the output is travel route information.

[1842] Step 4:

[1843] The server generates a detailed "travel record" based on the travel route information. Specifically, it associates photos and videos with each visited location and creates a travel record in a format that records the traveler's activities in detail. The input is the travel route information, and the output is the travel record data.

[1844] Step 5:

[1845] The server uses artificial intelligence to analyze the generated travel log and propose a new optimal travel route. Specifically, it uses an AI analysis system to reference a database of highly rated spots and generate a new route that includes unvisited but highly rated locations. The input is the travel log data, and the output is the proposed new travel route.

[1846] Step 6:

[1847] The server displays the proposed new travel route in a virtual environment. Specifically, it generates a virtual tour using Unity or ARKit / ARCore and creates data to display on smart glasses. The input is the new travel route, and the output is the virtual tour data.

[1848] Step 7:

[1849] The server notifies the user, specifically, by sending the details of the new travel route and a link to the virtual tour to the user through a smartphone application. The input is the virtual tour data, and the output is the notification received by the user.

[1850] Step 8:

[1851] The user plans and executes the trip again based on the proposed new travel route, and uploads the results to the server again. Specifically, the photos and videos taken at the newly visited locations are uploaded to the cloud server. The input is the revisited trip data, and the output is the revisited data stored on the server.

[1852] Step 9:

[1853] The server publishes the revisited data to other users, specifically through a web portal or a sharing function within the application, making the data available for viewing by other users. The input is the revisited data, and the output is the published data.

[1854] Step 10:

[1855] Other travelers rate the published new travel routes. The rating results are compiled on the server, and a popularity ranking is formed. Specifically, user ratings are compiled as points and displayed in a ranking format. The input is the rating data, and the output is a popularity ranking.

[1856] Step 11:

[1857] The server provides rewards to users who create and publish highly rated travel routes. Specifically, it runs a system that provides rewards such as points, cash, and coupons to users who rank highly. The input is the ranking data, and the output is the reward provided.

[1858] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1859] overview

[1860] This system automatically compiles travel routes based on the location information and timestamps of photos taken by travelers and creates a "travel diary." Furthermore, the AI ​​analyzes this "travel diary" to suggest optimal travel routes, and the results are shared with other travelers. Based on their ratings, a popularity ranking is created and rewards are offered to those with the highest ratings. Furthermore, by combining it with an emotion engine that recognizes user emotions, more personalized suggestions can be made.

[1861] System configuration

[1862] 1. User device: A device such as a smartphone or PC that uploads photos in bulk.

[1863] 2. Server: Collects the received data, generates and stores the travel diary, and uses AI to suggest new travel routes and collect evaluations.

[1864] 3. AI system: Analyzes travel records and suggests new travel routes.

[1865] 4. Emotion engine: Recognizes user emotions and optimizes travel routes based on that data.

[1866] 5. Database: Storage for high-rated spots and rating data.

[1867] Program processing

[1868] Bulk photo posting and location analysis

[1869] When a user returns from their trip, they upload all the photos they took to a dedicated application. The device reads the metadata of the photos, extracts the location information (latitude and longitude) and timestamp, and sends them to the server.

[1870] Automatically compile travel routes and create travel diaries

[1871] The server automatically compiles the user's route based on the received location information and timestamps, determining the places the user visited, the order in which they visited them, and the duration of their stay. The server then uses this information to generate a "travel diary" containing details of the places visited and relevant photos, which is then saved and made available to the user's account.

[1872] AI-powered new travel route suggestions

[1873] The server provides the generated "travel diary" to the AI ​​analysis system. The AI ​​refers to the "travel diary" and generates a new travel route that includes places the user has not visited, based on a database of highly rated restaurants and tourist attractions. The server notifies the user of this new travel route.

[1874] Combining Emotion Engines

[1875] The emotion engine extracts emotional data from users' photos and text inputs. The extracted emotional data is sent to the server, which then takes this emotional data into account to optimize the route based on the user's mood and preferences, providing a more personalized experience.

[1876] Revisiting and Publishing New Courses

[1877] The user plans and executes another trip using the proposed new travel route. After the trip, the user uploads photos again and notifies the server that the new travel route has been executed. The server then publishes the results of the new travel route to other users and collects their ratings.

[1878] Creating popularity rankings and offering rewards

[1879] Other users rate the new travel routes that are published. The server aggregates the rating data and generates a popularity ranking. Users who create and publish highly rated travel routes are rewarded.

[1880] Specific examples

[1881] After returning from a week-long trip to Europe, User A uploads all the photos he or she has taken with their smartphone to a dedicated app. The device extracts location information and timestamps from the photos and sends them to the server.

[1882] Based on this, the server automatically calculates the order in which User A visited Paris, London, and Berlin, and the duration of their stay, and generates a "travel diary." This "travel diary" is saved in the user's account and can be viewed.

[1883] The AI ​​system then analyzes the travel diary and suggests a new travel route that includes highly rated cafes and tourist attractions that User A has not yet visited in Paris, and the server notifies User A. Furthermore, the emotion engine analyzes User A's emotional data during the trip and incorporates tourist spots and events that match the user's mood into the new travel route.

[1884] User A visits Paris again based on the proposed new route, enjoys new tourist spots, and then uploads photos again and notifies the server of the results.

[1885] Other users rate User A's new travel route, and the server generates a popularity ranking. Ultimately, User A, who receives a high rating, is rewarded. This process allows many travelers to plan their trips efficiently and have a fulfilling travel experience. Furthermore, by combining it with an emotion engine, it is possible to provide a personalized travel experience that matches each individual user's mood and preferences.

[1886] The processing flow will be explained below.

[1887] Explanation of the specific processing of the program

[1888] Bulk photo posting and location analysis

[1889] Step 1: Upload a photo

[1890] After the trip, users upload all the photos they took to a dedicated application.

[1891] (Specific operation) Click the "Upload photos" button within the app, select multiple photo files from the file selection screen, and press the "Upload" button.

[1892] Step 2: Extract location and timestamp

[1893] The device extracts location information (GPS coordinates) and timestamp from the metadata of each uploaded photo.

[1894] (Specific operation) Analyzes files on the device and obtains the latitude, longitude, and shooting date and time from the EXIF ​​information of each photo.

[1895] Step 3: Send data to the server

[1896] The terminal transmits the extracted data, including the location information and timestamp, to the server.

[1897] (Specific operation) The location information and timestamp data are packaged in JSON format and sent to the server's API endpoint as an HTTP POST request.

[1898] Automatically compile travel routes and create travel diaries

[1899] Step 4: Aggregating travel routes

[1900] Based on the received location information and timestamp, the server automatically compiles the user's travel route by chronologically listing the places they visited.

[1901] (Specific operation) Based on location information, the locations to be visited are plotted on a map, and the order of visits is determined by an algorithm based on timestamps.

[1902] Step 5: Generate a travel diary

[1903] The server generates a "travel diary" based on the collected travel routes.

[1904] (Specific Actions) Create a travel record document by combining text and images, including the names of places visited, the date and time of visit, the duration of stay, and key photos.

[1905] Step 6: Save to your user account

[1906] The server saves the generated "travel diary" in the user's account and makes it available for viewing.

[1907] (Specific operation) The travel diary data is associated with the user's account information and saved in a database, and a link to the new travel diary is displayed on the user's dashboard.

[1908] AI-powered new travel route suggestions

[1909] Step 7: Provide to the AI ​​analysis system

[1910] The server provides the generated "travel diary" to the AI ​​analysis system.

[1911] (Specific operation) Send the travel diary data to the analysis endpoint of the AI ​​server.

[1912] Step 8: Create a new travel route

[1913] The AI ​​refers to the travel diary and a database of highly rated restaurants and tourist attractions to generate new travel routes that include places the user has not yet visited.

[1914] (Specific operation) Highly rated spots are retrieved from the database, and a new route is generated by analyzing the order of visits and travel efficiency using an algorithm.

[1915] Step 9: Proposal Notification

[1916] The server notifies the user of the new travel route created by the AI.

[1917] (Specific behavior) Based on the user's notification settings, the app will send new routes via in-app notification or email.

[1918] Combining Emotion Engines

[1919] Step 10: Extracting Emotion Data

[1920] The emotion engine extracts emotional data from user photos and text inputs.

[1921] (Specific operation) Uploaded photos and text are analyzed, and emotional data (e.g., joy, surprise, sadness, etc.) is extracted using an emotion recognition algorithm.

[1922] Step 11: Sending Emotion Data

[1923] The emotion engine transmits the extracted emotion data to the server.

[1924] (Specific operation) The analysis results are sent in JSON format to the server's API endpoint.

[1925] Step 12: Optimize travel routes

[1926] Based on the received emotional data, the server optimizes the travel route to match the user's mood and preferences.

[1927] (Specific operation) Taking into account the emotional data, the most suitable location is selected from among the highly rated spots, and the optimal travel route is regenerated.

[1928] Revisiting and Publishing New Courses

[1929] Step 13: Performing a revisit

[1930] Users can use the proposed new travel routes to plan and carry out their trip again.

[1931] (Specific actions) Make travel reservations and schedules based on the suggested route.

[1932] Step 14: Re-upload photos

[1933] The user revisits the new route and uploads the photos again, notifying the server that the AI's suggestions have been implemented.

[1934] (Specific operation) Upload new travel photos to the app in bulk, check that they correspond to the new route based on the metadata at the time of upload, and notify the server.

[1935] Step 15: Publish the new travel route

[1936] The server publishes the results of the new travel route to other users.

[1937] (Specific operation) Group photos and testimonials of new routes and create a public page that can be viewed by other users.

[1938] Creating popularity rankings and offering rewards

[1939] Step 16: User Rating

[1940] Other users rate the new travel routes that are published.

[1941] (Specific operation) Press the "Rate" button on the public page and post a star rating and comment.

[1942] Step 17: Generate Popularity Rankings

[1943] The server aggregates user ratings and generates a popularity ranking.

[1944] (Specific operation) Obtain evaluation data, calculate scores, and update the ranking page.

[1945] Step 18: Offer Rewards

[1946] The server rewards users who create and publish highly rated travel routes.

[1947] (Specific operations) Check the account information of highly rated users, transfer rewards, send reward acquisition notifications within the app, and record them in the reward history.

[1948] This process allows users to share their travel experiences, allowing other users to use them to plan more fulfilling trips. Furthermore, by combining it with an emotion engine, it is possible to provide a personalized travel experience that matches each user's mood and preferences.

[1949] Example 2

[1950] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1951] It is difficult to efficiently organize and analyze the vast amount of photo data taken by travelers, and to optimize travel routes based on emotional data during travel. Furthermore, systems that allow users to share their travel experiences with other users and receive feedback through ratings to further enhance travel plans are currently inadequate. Conventional systems struggle to plan trips that take into account travelers' emotions and interests, limiting their ability to provide personalized travel recommendations.

[1952] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for uploading photos taken by travelers in bulk, a means for extracting location information and timestamps from the uploaded photos, a means for automatically aggregating travel routes based on the extracted data, a means for generating a travel log from the aggregation results, a means for analyzing the generated travel log and proposing an optimal travel route, a means for extracting emotional data and optimizing the travel route according to the user's mood and preferences, a means for notifying the user of the proposed new travel route, a means for the user to actually revisit the new travel route and publish the results to other users, a means for evaluating the published travel route and forming a popularity ranking, and a means for providing rewards to creators of highly rated travel routes. This enables efficient organization and analysis of travelers' photo data and personalized travel suggestions. Furthermore, by sharing and rating travel experiences, it is possible to create comprehensive travel plans that can be used as reference by other travelers.

[1953] "Tourist" means an individual or group undertaking a trip.

[1954] "Photographs" refer to digital image data taken by travelers during their trip.

[1955] "Bulk upload" refers to sending multiple photo data to a server over the Internet at once.

[1956] "Location information" refers to the location where the photo was taken, indicated by GPS coordinates.

[1957] A "timestamp" refers to information t...

Claims

1. A way for travelers to upload photos taken in bulk, means for extracting location information and timestamps of uploaded photos; A means to automatically compile travel routes based on the extracted data, and means for generating a travel record from the aggregated results; A means to analyze the generated travel records and propose new travel routes; a means of informing the user of proposed new travel routes; A means for users to actually revisit new travel routes and share their results with other users; a means for evaluating published travel routes and forming popularity rankings; A means for providing rewards to creators of highly rated travel routes; A system including:

2. The system according to claim 1, wherein the system automatically determines the places to be visited and the order in which they are to be visited based on the location information and timestamps of the photos.

3. The system according to claim 1, wherein AI proposes optimal new travel routes based on travel records.

Images