System

Abstract

A system is provided.SOLUTION: This system is provided with a means for allowing a user to input his or her own attribute, taste and an image close to a desired scene, a means for allowing a server to analyze the inputted information and image, and to automatically generate a plurality of itineraries matched with the user, and a means for providing the plurality of itineraries generated by the server with catchy titles, and for allowing the user to change a part of the itineraries. The system includes a means for updating the change contents in real time, a means for collecting the information of the moving means, the accommodation facilities and the sightseeing spots by the server and reflecting it on the itinerary, a means for providing the local information and the trivia to the user by the server before the travel, and a means for proposing the next travel plan based on the evaluation of the user by the server after the travel.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional travel planning requires a great deal of time and effort if users conduct detailed research and plan their trips themselves without contacting a travel agency. Furthermore, when using a travel agency, the cost is high and it is difficult to obtain a trip plan that suits your preferences. Furthermore, there is no flexible system that can adapt to frequent changes and adjustments to travel plans in real time. There is a need for a system that can solve these issues and allow users to efficiently and effectively obtain customized trip plans. [Means for solving the problem]

[0005] The present invention provides a system that includes the following means: a means for a user to input their attributes, preferences, and images that are similar to the scenery they wish to see, a means for a server to analyze the input information and images and automatically generate multiple itineraries tailored to the user, a means for the server to present the multiple itineraries generated by the server with catchy titles, a means for the user to change parts of the itinerary and update those changes in real time, a means for the server to collect information on transportation, accommodations, and tourist spots and reflect it in the itinerary, a means for the server to provide the user with local information and trivia before the trip, and a means for the server to suggest the next itinerary based on the user's ratings after the trip, which allows the user to easily create detailed travel plans and receive consistent support from before the trip to after the trip.

[0006] "User" means an individual who uses the System to create, adjust, and review travel plans.

[0007] "Server" refers to the central computing unit that receives input information from users, performs analysis, and automatically generates and coordinates multiple itineraries.

[0008] "Attributes" refers to personal information such as a user's age, gender, and preferences, and is information used to personalize travel plans.

[0009] "Preferences" refer to the interests and themes that users prioritize while traveling, such as nature, culture, and adventure.

[0010] "Image" refers to a data file that is used to upload a visual that is close to the scenery the user wants to see to the app.

[0011] "Analysis" refers to the process in which the server calculates and evaluates the meaning and relevance of information and images entered by the user.

[0012] "Itinerary" means a list that specifically outlines the User's planned activities and schedule during the User's trip.

[0013] The term "catchy title" refers to a title given to the generated itinerary that is attractive and intriguing to users.

[0014] "Real-time" refers to the ability to instantly reflect user input and changes and provide near-immediate results.

[0015] "Transportation" refers to the means of transportation used by a User during their trip, such as airplanes, trains, buses, taxis, etc.

[0016] "Accommodation" refers to a place where a user stays during their trip, such as a hotel, inn, or guesthouse.

[0017] "Attractions" refers to tourist attractions and places of interest that a user plans to visit.

[0018] "Information" refers to knowledge including local culture, history, precautions, trivia, etc. that the server provides to users.

[0019] "Rating" refers to the feedback given by users after completing a trip regarding their satisfaction with and impressions of the trip.

[0020] "Plan" refers to a detailed travel plan designed based on the user's attributes, preferences, and travel period. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] Initial Setup

[0043] A user downloads and installs the app. When the user launches the app, they are presented with a new user registration page where they enter their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.).

[0044] Examples:

[0045] The user inputs that they prefer "natural."

[0046] The server stores the entered information in a database.

[0047] Image acquisition and analysis

[0048] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage, and the server retrieves and analyzes the image. Based on the analysis results, a list of recommended tourist spots is generated.

[0049] Examples:

[0050] A user uploads a photo of Mount Fuji.

[0051] The server analyzes the image and suggests Japan, where Mount Fuji is located.

[0052] Generate a provisional itinerary

[0053] The server generates multiple provisional itineraries based on the user's information and the results of image analysis, gives them catchy titles, and automatically suggests transportation and accommodation options.

[0054] Examples:

[0055] The server titles the itinerary it generates as "7 days to enjoy Japanese culture and nature."

[0056] The server displays the itinerary to the user.

[0057] Day 1: Arrival at Haneda Airport, sightseeing in Tokyo, accommodation (Hotel ABC)

[0058] Day 2: Sightseeing in Akihabara, Ueno Zoo, and lodging at Hotel XYZ

[0059] Day 3: Travel to Hakone, hot spring experience, and accommodation (hot spring inn DEF)

[0060] Itinerary amendments and real-time updates

[0061] If the user checks the proposed itinerary and decides to change some of the tourist attractions or accommodations, they can make the changes through the app, and the server will update the changes in real time.

[0062] Examples:

[0063] The user enters that they would like to change their accommodation for the third day to a different hot spring inn.

[0064] The server generates a new itinerary and sends it to the device.

[0065] New plan:

[0066] Day 3: Travel to Hakone, hot spring experience, and accommodation (Hot Spring Inn GHI)

[0067] Providing a list of necessary items and what to bring

[0068] Once the travel plan is finalized, the server will send the necessary information and a list of what to bring to the user's email, and a real-time notification will be sent to the user's device.

[0069] Examples:

[0070] The server automatically generates an inventory list.

[0071] Send email:

[0072] passport

[0073] VISA (Japan)

[0074] JPY

[0075] Clothing (7 days' worth)

[0076] Terminal notification: "Travel in 5 days. Please check your passport and VISA."

[0077] Providing local information

[0078] Before the trip, the server collects information on local culture, trivia, precautions, etc. and notifies the user.

[0079] Examples:

[0080] The server collects information about hot spring culture.

[0081] Device notification: "Learn about Japanese hot spring culture: How to use towels and etiquette"

[0082] Post-trip evaluation and suggestions for next time

[0083] After the trip, users input their satisfaction and impressions of the trip into the app, and the server uses this information to improve the next trip plan and make new suggestions.

[0084] Examples:

[0085] The user rates their trip satisfaction as "very satisfied."

[0086] Server's new suggestion: "Why not plan your next trip to Japan? We'll suggest some new tourist spots."

[0087] As described above, the system of the present invention provides users with efficient, customized travel plans and consistently supports them from planning to execution and even next proposals.

[0088] The processing flow will be explained below.

[0089] Step 1:

[0090] The user downloads and installs the app. The device launches the app and displays a new user registration page. The user enters their name, age, gender, travel preferences, etc. The device sends the information to the server. The server stores the user information in a database.

[0091] Step 2:

[0092] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage. The server retrieves the image from cloud storage and sends it to the image analysis module. The server's image analysis module analyzes the image and generates a list of recommended tourist spots based on the analysis results.

[0093] Step 3:

[0094] The server generates multiple provisional itineraries based on the user's preferences and the results of image analysis. The server gives the generated itineraries catchy titles. The server collects information on transportation and accommodations and reflects this in the itinerary. The server sends the multiple provisional itineraries it has generated to the user's terminal for provision.

[0095] Step 4:

[0096] The user reviews the proposed itinerary and inputs changes to some of the attractions and accommodations. The device sends the changes to the server. The server receives the change request and updates the travel plan with the new elements in real time. The server sends the updated plan to the device for presentation to the user.

[0097] Step 5:

[0098] The server generates the necessary information and a list of items to bring based on the confirmed travel plan. The server sends the generated information to the user's email and also sends a notification to the device. The device displays the notification to the user.

[0099] Step 6:

[0100] The server collects information about the destination's culture, trivia, and things to be aware of. The server then sends the collected information to the user's device. The device then notifies the user of the local information.

[0101] Step 7:

[0102] After a trip, the user enters their satisfaction and impressions of the trip into the app. The device sends the input information to the server. The server stores the user's rating in a database. The server uses the rating results to improve the next travel plan and send new suggestions to the user.

[0103] Example 1

[0104] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0105] Conventional travel planning systems require users to collect a large amount of information and create plans manually, which requires time and effort. Furthermore, it is difficult to change plans once they have been created, and information cannot be updated in real time, making it difficult to flexibly respond to various situations during a trip. Furthermore, they lack the functionality to provide useful information before a trip or to reflect impressions after returning home in the next plan.

[0106] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0107] In this invention, the server includes: means for a user to input images that are similar to the user's attributes, preferences, and scenery they wish to see; means for the server to analyze the input information and images and automatically generate multiple travel plans tailored to the user; means for presenting the multiple travel plans generated by the server with catchy titles; means for the user to change parts of the travel plan and updating the changes in real time; means for the server to collect information on transportation, accommodations, and tourist spots and reflect it in the travel plan; means for the server to provide the user with local information and trivia before the trip; means for the server to suggest the next travel plan based on the user's evaluation after the trip; and means for automatically generating a list of items to pack for the trip based on the information the user registered in the system and notifying the user in real time. This enables the user to efficiently obtain a customized travel plan and receive consistent support from planning to execution and even next suggestions.

[0108] "User" means any person or entity that uses the System to make travel plans.

[0109] "Attributes" refers to personal information such as a user's name, age, and gender.

[0110] "Preferences" refers to information related to the type of travel and destinations that a user prefers.

[0111] "Scenery" refers to images related to specific views or locations that users want to see.

[0112] "Means for inputting images" refers to an interface that allows users to upload images that resemble the scenery they want to see into the system.

[0113] "Analysis" is the process by which the server processes the images uploaded by users to derive relevant tourist information.

[0114] "Travel Plan" refers to the schedule and activity details generated by the server based on the user's attributes, preferences, and image analysis results.

[0115] A "catchy title" refers to an attractive name or phrase that grabs the user's attention.

[0116] "Modifiable" means that users can make adjustments or modifications to existing travel plans.

[0117] "Means of real-time updates" refers to a mechanism whereby when a user inputs changes, the server immediately reflects those changes.

[0118] "Transportation" refers to the means of transportation used during a trip.

[0119] "Accommodation" refers to hotels, inns, etc. where users stay during their travel plans.

[0120] "Tourist Spots" refers to famous places and tourist destinations that users can visit.

[0121] "Local information and trivia" refers to cultural knowledge and points of caution about the travel destination.

[0122] "Rating" refers to the satisfaction and impressions that users enter after their trip.

[0123] A "packing list" is an automated list of items you will need for your trip.

[0124] "Means of notification" refers to the way in which the system notifies the user of important information.

[0125] The purpose of this system is to enable users to efficiently create customized travel plans and to consistently support the process from planning to execution and even future proposals. An embodiment of the system will be described below.

[0126] Initial Setup

[0127] user

[0128] First, a user downloads and installs the application. When the user launches the app, they are presented with a new user registration page where they enter their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.). Based on this information, the server creates a user profile and stores it in a database.

[0129] Specific examples

[0130] If the user indicates that they prefer "natural," the server stores this information in a database.

[0131] Image acquisition and analysis

[0132] user

[0133] Users upload an image that resembles the scenery they want to see to the app. For example, a user can upload a photo of Mt. Fuji.

[0134] Terminal

[0135] The device uploads the image data to cloud storage.

[0136] server

[0137] The server retrieves images from cloud storage and analyzes them using an AI image analysis module. Based on the analysis results, a list of recommended tourist spots is generated.

[0138] Specific examples

[0139] If a user uploads a photo of Mount Fuji, the server analyzes the image and suggests "Japan," where Mount Fuji is located.

[0140] Generate a provisional itinerary

[0141] server

[0142] The server generates multiple provisional travel plans based on the user's attribute information and the results of image analysis. Each plan is given a catchy title. Information on transportation, accommodation, and tourist spots is automatically added to the generated travel plans.

[0143] Specific examples

[0144] The tentative itinerary is titled "Seven Days to Enjoy Japanese Culture and Nature" and includes the following detailed schedule:

[0145] Day 1: Arrival at Haneda Airport, sightseeing in Tokyo, accommodation (Hotel ABC)

[0146] Day 2: Sightseeing in Akihabara, Ueno Zoo, and lodging at Hotel XYZ

[0147] Day 3: Travel to Hakone, hot spring experience, and accommodation (hot spring inn DEF)

[0148] Itinerary amendments and real-time updates

[0149] user

[0150] Users can review the proposed itinerary and, if they wish to change some of the tourist attractions or accommodations, enter the changes via the app.

[0151] server

[0152] The server receives the changes entered in real time, generates a new itinerary, and sends it to the terminal.

[0153] Specific examples

[0154] If the user requests to change their accommodation on the third day to a different hot spring inn, the server will generate a new plan and make the following changes:

[0155] Day 3: Travel to Hakone, hot spring experience, and accommodation (Hot Spring Inn GHI)

[0156] Providing a list of necessary items and what to bring

[0157] server

[0158] After the travel plan is finalized, the server automatically generates the necessary information and a list of items to bring, and sends this information to the user's email address and device in real time.

[0159] Specific examples

[0160] The server will automatically generate an inventory list and email it to the user with the following:

[0161] passport

[0162] Visa (Japan)

[0163] JPY

[0164] Clothing (7 days' worth)

[0165] Device notification: "Travel in 5 days. Please check your passport and visa."

[0166] Providing local information

[0167] server

[0168] Before the trip, the server collects information on local culture, trivia, precautions, etc. and notifies the user's device.

[0169] Specific examples

[0170] The server collects information about hot spring culture and notifies the device of the following:

[0171] "Learn about Japanese hot spring culture: How to use towels and etiquette"

[0172] Post-trip evaluation and suggestions for next time

[0173] user

[0174] After the trip, users enter their satisfaction and impressions of the trip into the app.

[0175] server

[0176] The server will use the user's ratings to improve the next travel plan and send new suggestions to the user's device.

[0177] Specific examples

[0178] If the user enters "very satisfied" as their satisfaction with the trip, the server will notify them, "Why not plan another trip to Japan? We'll suggest some new tourist spots for you."

[0179] The system of the present invention allows users to efficiently create customized travel plans and provides consistent support from execution to next proposal.

[0180] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0181] Step 1:

[0182] user

[0183] A user downloads and installs the app, then launches it for the first time after installation is complete.

[0184] Specific actions

[0185] Search for, download, and install the app from the app store.

[0186] input

[0187] Your device information and internet connection.

[0188] output

[0189] Installed apps.

[0190] Step 2:

[0191] user

[0192] A user launches the app and visits the new user registration page, where they enter their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.).

[0193] Specific actions

[0194] Enter the name "Taro Tanaka", age "30", gender "male", and travel preference "nature".

[0195] input

[0196] Your personal information and travel preferences.

[0197] output

[0198] New user registration data including the information entered.

[0199] Step 3:

[0200] server

[0201] The server receives the entered user information and stores it in a database.

[0202] Specific actions

[0203] On the server side, establish a database connection and execute a query to insert the user information.

[0204] input

[0205] New User Registration Data.

[0206] output

[0207] User information stored in a database.

[0208] Step 4:

[0209] user

[0210] Users upload images that are similar to the scenery they want to see to the app.

[0211] Specific actions

[0212] Select the image file on your device and press the "Upload" button.

[0213] input

[0214] An image file selected by the user.

[0215] output

[0216] Uploaded image data.

[0217] Step 5:

[0218] Terminal

[0219] The terminal transfers the uploaded image data to cloud storage.

[0220] Specific actions

[0221] Call the cloud storage API to upload image data.

[0222] input

[0223] Image data selected by the user.

[0224] output

[0225] The URL of the image stored in cloud storage.

[0226] Step 6:

[0227] server

[0228] The server retrieves images from cloud storage and sends them to the AI ​​image analysis module, which generates a list of recommended tourist spots based on the analysis results.

[0229] Specific actions

[0230] Call the image analysis API and obtain the analysis results.

[0231] input

[0232] Image URL in cloud storage.

[0233] output

[0234] A list of tourist destinations as a result of the analysis.

[0235] Step 7:

[0236] server

[0237] The server generates multiple provisional travel plans based on the user's attribute information and the results of image analysis, and gives each plan a catchy title.

[0238] Specific actions

[0239] A title generation algorithm is executed to give a title to the travel plan.

[0240] input

[0241] User attribute information and image analysis results.

[0242] output

[0243] Tentative travel plan list.

[0244] Step 8:

[0245] server

[0246] The server adds information about transportation, accommodation, and tourist spots to the tentative travel plan and presents it to the user.

[0247] Specific actions

[0248] Run a query to add transportation and accommodation information for each itinerary.

[0249] input

[0250] Tentative travel plan list.

[0251] output

[0252] The completed travel itinerary list.

[0253] Step 9:

[0254] user

[0255] Users can review the proposed travel plan and, if they wish to make any changes, enter the changes through the app.

[0256] Specific actions

[0257] On the travel plan screen, select the item you want to change and enter the new information.

[0258] input

[0259] User modification requests.

[0260] output

[0261] Change request data.

[0262] Step 10:

[0263] server

[0264] The server receives the changes in real time, generates a new travel plan, and sends it to the terminal.

[0265] Specific actions

[0266] It runs an algorithm to regenerate a new itinerary based on the changes and sends the updated data to the device.

[0267] input

[0268] The user's modification request data.

[0269] output

[0270] Updated travel planning data.

[0271] Step 11:

[0272] server

[0273] After the server confirms the travel plan, it automatically generates the necessary information and a list of items to bring, and sends them to the user's email address. It also sends a real-time notification to the user's device.

[0274] Specific actions

[0275] Run inventory generation algorithms and send emails and notifications.

[0276] input

[0277] Confirmed travel planning data.

[0278] output

[0279] Inventory list and notification data.

[0280] Step 12:

[0281] server

[0282] Before a trip, the server collects local information and trivia and notifies the user's device.

[0283] Specific actions

[0284] It calls the local information API to collect information and sends push notifications to the user's device.

[0285] input

[0286] Local information API data.

[0287] output

[0288] Local information notification data.

[0289] Step 13:

[0290] user

[0291] After the trip, users enter their satisfaction and impressions of the trip into the app.

[0292] Specific actions

[0293] Fill out the evaluation form in the app and press the submit button.

[0294] input

[0295] User satisfaction and opinion data.

[0296] output

[0297] Submitted evaluation data.

[0298] Step 14:

[0299] server

[0300] The server uses the user's ratings to refine the next travel plan and send new suggestions to the user.

[0301] Specific actions

[0302] The next itinerary generation algorithm is run to generate new proposals and send them to the user's device.

[0303] input

[0304] Submitted evaluation data.

[0305] output

[0306] New travel planning data.

[0307] (Application example 1)

[0308] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0309] Conventional online shopping lacks the ability to offer products tailored to individual user preferences and needs, resulting in a poor quality shopping experience. In particular, it is difficult for users to find the products and designs they specifically desire, creating a need for a hassle-free way to select a set of products that suits them.

[0310] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0311] In this invention, the server includes means for the user to input their attributes, preferences, and images similar to the products they wish to see, means for the server to analyze the input information and images and automatically generate multiple product sets tailored to the user, means for presenting the multiple product sets generated by the server with catchy titles, means for the user to change parts of the product sets and means for updating the changes in real time, and means for the server to suggest the next shopping plan based on the user's ratings. This enables product suggestions to be individually customized for the user, improving the quality of online shopping.

[0312] "User" refers to an individual who uses the system to select and purchase products.

[0313] "Attributes" refers to personal information such as a user's name, age, gender, and preferences.

[0314] "Preferences" refer to a user's tendency to be particularly interested in a particular style or category.

[0315] "Image" refers to visual data entered by the user that shows the desired product or design.

[0316] "Server" refers to the main computer system that processes user input and image data and generates product sets based on the analysis results.

[0317] "Analyzing images" refers to analyzing uploaded image data through a program and extracting relevant information based on its content.

[0318] "Means for automatically generating product sets" refers to the process of automatically selecting and combining multiple related products based on user attributes and the results of image analysis.

[0319] A "catchy title" refers to giving the generated product set a name that will attract the user's interest.

[0320] The "presentation means" refers to a method or function for visually presenting the generated product set to the user.

[0321] "Modifiable" means that users can modify parts of the suggested product set to suit their preferences.

[0322] "Means of real-time updates" refers to methods and functions that allow changes made by users to be reflected immediately on the system.

[0323] "Means to suggest next shopping plans" refers to a system that improves future shopping experiences and suggests new products based on user ratings and feedback.

[0324] A "checklist" is a list of items or tasks that a user needs to complete after their shopping plan is finalized.

[0325] "Means of notification" refers to the methods and functions by which the system notifies the user of information.

[0326] "Collecting satisfaction and feedback" refers to collecting feedback and comments provided by users as data.

[0327] "Means for improvement" refers to methods and functions for effectively improving the next proposal or the system itself based on the collected feedback.

[0328] The invention begins with the user entering their attributes, preferences, and images of products similar to the one they want to see. The user first downloads and installs the application. When the application is launched, a new user registration page appears, where the user enters their name, age, gender, and shopping preferences (e.g., fashion, home appliances, outdoor gear, etc.). This information is stored in the cloud and used to customize the shopping experience.

[0329] Next, the user uploads an image of the product they want to see to the app. The device uploads the image data to cloud storage, and the server retrieves and analyzes the image. This image analysis uses OpenCV and KMeans clustering. For example, if a user uploads an image of a modern living room, the app analyzes its color hues and suggests furniture and decoration items in a similar style.

[0330] The server generates a set of related products based on the user's information and image analysis results, and gives it a catchy title such as "Modern Living Room Set." The generated product set is displayed in the app, and the user can easily add it to their shopping list. If the user reviews the suggested product set and wants to change some of the products, the changes can be reflected in real time. For example, if the user wants to change the color of a sofa, the server will re-suggest a product list including the sofa in the new color.

[0331] Once the shopping list is finalized, the server automatically generates the necessary checklist and notifies the user, including items and tasks to be checked before purchasing. The app also provides additional information about the products, such as their background and use cases.

[0332] After making a purchase, users can enter their satisfaction and feedback on their experience into the app, which the server uses to improve their next shopping plan and make new suggestions.

[0333] The system uses the following main hardware and software: OpenCV and the KMeans clustering algorithm for image analysis, the Flask framework on the server side, MongoDB and MySQL for the database, and smtplib for sending emails.

[0334] An example of a prompt is as follows:

[0335] Example prompt sentence:

[0336] Upload an image of a modern living room to represent the product users want to see.

[0337] As a result of the analysis, the server suggests related modern style furniture (e.g. sofa, table, carpet).

[0338] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0339] Step 1:

[0340] The user downloads and installs the application. After launching it, a new user registration page appears, where the user enters their name, age, gender, and shopping preferences. This information is sent from the device to the cloud server and stored in a database. The input in this process is user information, and the output is the storage of information in the cloud database.

[0341] Step 2:

[0342] The user selects an image that is similar to the product they want to see and uploads it to the app. The device uploads the image data to cloud storage, and the server retrieves the image data. In this step, the input is the image data, and the output is saving it to cloud storage and retrieving it by the server.

[0343] Step 3:

[0344] The server analyzes the uploaded image data using OpenCV and KMeans clustering. Specifically, it extracts the hue information of the image and classifies it into multiple clusters. The input is the image data, and the output is a color palette classified by the hue information.

[0345] Step 4:

[0346] The server generates a related product set based on the color palette obtained as a result of the analysis. Using a generative AI model, it inputs the user's preference information and color palette and proposes the optimal product set. The input in this step is the analysis result and the user's preference information, and the output is the proposed product set. This product set is given a catchy title such as "Modern Living Room Set."

[0347] Step 5:

[0348] The server presents the generated product set to the user through the application. The user can review the proposal and modify parts of the product set if necessary. The modifications are sent to the server in real time, and the server generates a corresponding new product set and presents it to the user again. The input in this step is the user's modification request, and the output is the updated product set.

[0349] Step 6:

[0350] Once the user has finalized the product set, the server automatically generates a checklist, which includes items and tasks to be checked before purchasing. This checklist is sent to the user's device. The input is the finalized product set, and the output is the generated checklist.

[0351] Step 7:

[0352] After a purchase, the user enters feedback about their satisfaction and experience into the application. The server collects this feedback and uses it to improve the next shopping plan. In this step, the input is the user's feedback and the output is the improved next shopping plan.

[0353] Example prompt sentence:

[0354] Step 1: Register as a new user by entering your name, age, gender and shopping preferences.

[0355] Step 2: Upload an image that resembles the product you want to see.

[0356] Step 3: Generate a color palette based on the results of image analysis.

[0357] Step 4: Propose a product set based on the color palette and user preference information.

[0358] Step 5: User reviews the proposed product set and makes changes if necessary.

[0359] Step 6: Automatically generate a checklist after the product set is confirmed.

[0360] Step 7: Collect post-purchase feedback and refine your next plan.

[0361] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0362] Initial Setup

[0363] The user downloads and installs the app. The device launches the app and displays a new user registration page. The user enters their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.). The app also performs initial setup to recognize the user's emotions.

[0364] Examples:

[0365] The user inputs that they prefer "natural."

[0366] The server stores the entered information in a database.

[0367] Image input and emotion analysis

[0368] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage, and the server retrieves the image and analyzes the image and the user's input information using an emotion engine. Based on the analysis results, a list of recommended tourist spots is generated and the user's emotional state is recorded.

[0369] Examples:

[0370] A user uploads a photo of Mount Fuji.

[0371] The server analyzes the image and suggests Japan.

[0372] The emotion engine recognizes the user's emotion of joy.

[0373] Generate a provisional itinerary

[0374] The server generates multiple provisional itineraries based on the user's information, image analysis, and sentiment analysis results. It then gives each itinerary a catchy title and automatically suggests transportation and accommodation options.

[0375] Examples:

[0376] The server titles the itinerary it generates as "7 days to enjoy Japanese culture and nature."

[0377] The server displays the itinerary to the user.

[0378] Day 1: Arrival at Haneda Airport, sightseeing in Tokyo, accommodation (Hotel ABC)

[0379] Day 2: Sightseeing in Akihabara, Ueno Zoo, and lodging at Hotel XYZ

[0380] Day 3: Travel to Hakone, hot spring experience, and accommodation (hot spring inn DEF)

[0381] Itinerary amendments and real-time updates

[0382] If a user reviews the proposed itinerary and decides to change some of the tourist attractions or accommodations, they can make the changes through the app, and the server will update the changes in real time.In addition, the emotion engine recognizes the user's real-time emotional state and automatically adjusts the itinerary accordingly.

[0383] Examples:

[0384] The user enters that they would like to change their accommodation for the third day to a different hot spring inn.

[0385] The server generates a new itinerary and sends it to the device.

[0386] The emotion engine recognizes the user's emotions and proposes the optimal plan.

[0387] New plan:

[0388] Day 3: Travel to Hakone, hot spring experience, and accommodation (Hot Spring Inn GHI)

[0389] Providing a list of necessary items and what to bring

[0390] Once the travel plan is finalized, the server will send the necessary information and a list of what to bring to the user's email, and a real-time notification will be sent to the user's device.

[0391] Examples:

[0392] The server automatically generates an inventory list.

[0393] Send email:

[0394] passport

[0395] VISA (Japan)

[0396] JPY

[0397] Clothing (7 days' worth)

[0398] Terminal notification: "Travel in 5 days. Please check your passport and VISA."

[0399] Providing local information

[0400] Before the trip, the server collects information on local culture, trivia, precautions, etc. and notifies the user.

[0401] Examples:

[0402] The server collects information about hot spring culture.

[0403] Device notification: "Learn about Japanese hot spring culture: How to use towels and etiquette"

[0404] Emotion monitoring and optimization during travel

[0405] During the trip, the emotion engine continuously monitors the user's emotions and optimizes the travel experience based on the emotional data. If the user is enjoying themselves, it will make suggestions to maintain that state, and if they are feeling stressed, it will suggest relaxing activities.

[0406] Examples:

[0407] The emotion engine recognizes the user's level of enjoyment.

[0408] Device notification: "There's a recommended cafe nearby. Would you like to relax?"

[0409] Post-trip evaluation and suggestions for next time

[0410] After a trip, users input their satisfaction and impressions of the trip into the app. The server uses this information to improve the next trip plan and make new suggestions. The emotion engine also utilizes emotional data from the trip to make more personalized suggestions.

[0411] Examples:

[0412] The user rates their trip satisfaction as "very satisfied."

[0413] The server makes the next suggestion based on the emotion data.

[0414] New suggestion: "Why not plan your next trip to Japan? We'll suggest some new tourist spots."

[0415] As described above, the system of the present invention efficiently acquires customized travel plans for users and provides consistent support from planning to execution and even next-time suggestions. By combining it with an emotion engine, it is possible to provide an optimal travel experience that reflects the user's real-time emotional state.

[0416] The processing flow will be explained below.

[0417] Step 1:

[0418] The user downloads and installs the app. The device launches the app and displays the new user registration page. The user enters their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.) to initialize the emotion engine. The device sends the input information to the server. The server stores the user information in a database.

[0419] Step 2:

[0420] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage. The server retrieves the image from cloud storage and sends it to the image analysis module and emotion engine. The server's image analysis module analyzes the image, and the emotion engine recognizes the user's emotional state. A list of recommended tourist spots is generated based on the analysis results and emotion data.

[0421] Step 3:

[0422] The server generates multiple provisional itineraries based on user information, image analysis, and sentiment analysis. It then gives each itinerary a catchy title. The server then collects information on transportation and accommodations and incorporates this information into the itinerary. The multiple provisional itineraries generated by the server are then sent to the user's device for provision.

[0423] Step 4:

[0424] The user reviews the proposed itinerary and inputs changes to some of the tourist attractions and accommodations. The device sends the changes to the server. The server receives the change request and updates the travel plan with the new elements in real time. The emotion engine recognizes the user's real-time emotional state and automatically adjusts the itinerary accordingly. The server sends the updated plan to the device for presentation to the user.

[0425] Step 5:

[0426] The server generates the necessary information and a list of items to bring based on the confirmed travel plan. The server sends the generated information to the user's email and also sends a notification to the device. The device displays the notification to the user.

[0427] Step 6:

[0428] The server collects information about the destination's culture, trivia, and things to be aware of. The server then sends the collected information to the user's device. The device then notifies the user of the local information.

[0429] Step 7:

[0430] During the trip, the emotion engine continuously monitors the user's emotions. The server makes suggestions to optimize the travel experience based on the emotion data. The device then notifies the user of the optimized information.

[0431] Step 8:

[0432] After a trip, the user enters their satisfaction and impressions of the trip into the app. The device sends the input information to the server. The server stores the user's ratings in a database. The emotion engine utilizes the emotional data from the trip, and the server sends the user new suggestions to improve their next travel plan.

[0433] Example 2

[0434] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0435] Conventional travel planning support systems have difficulty providing travel plans that fully reflect the user's preferences and emotions. It is also difficult to adjust plans in response to real-time emotional changes during travel. This has resulted in the issue of users' travel experiences not being optimized.

[0436] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0437] In this invention, the server includes a means for a user to input images that are similar to their attributes, preferences, and scenery they want to see, a means for analyzing the input information and images using a generative AI model and automatically generating multiple itineraries tailored to the user, and a means for monitoring the user's emotional state during the trip using an emotion engine to optimize the travel experience. This makes it possible to provide a personalized travel plan that reflects the user's preferences and emotions, and to adjust the plan to suit real-time emotional changes during the trip.

[0438] "User" means an individual who intends to use the System to create a travel plan.

[0439] "Attributes" refers to personal information such as a user's name, age, and gender.

[0440] "Preferences" refers to the user's preferred travel style and purpose, such as nature, culture, adventure, etc.

[0441] "Images similar to the scenery you want to see" are images that express the image of scenery or places you want to see as a travel destination.

[0442] A "server" is a computer system that analyzes data and generates travel plans based on information provided by users.

[0443] A "generative AI model" is an algorithm that uses artificial intelligence to analyze input information and images and generate the optimal travel plan for the user.

[0444] An "emotion engine" is a technology that recognizes the user's emotional state and uses it as data for analysis.

[0445] An "itinerary" is a detailed plan that lists the travel schedule and places to visit.

[0446] "Title" is a catchphrase or description that is given to the generated itinerary.

[0447] "Transportation" refers to the means of transportation used during a trip (e.g., plane, train, bus).

[0448] "Accommodation" refers to the place where you stay during your trip, such as a hotel or inn.

[0449] "Tourist attractions" refer to places of interest and activities that you can visit while traveling.

[0450] A "packing list" is a list of items you will need while traveling.

[0451] "Local information" refers to information about the culture, customs, and precautions of the destination.

[0452] "User rating" refers to the satisfaction and impressions that users enter into the system after their trip.

[0453] "Next Travel Plan" is a plan for your next trip that is suggested based on the user's past ratings and emotional data.

[0454] This invention is a system that provides a personalized travel plan that reflects the user's preferences and emotions, and adjusts the plan optimally in response to real-time changes in emotions during the trip. This system operates in cooperation with the user, terminal, and server.

[0455] Initial Setup

[0456] First, the user downloads and installs the application. The device launches the application and displays a new user registration page. The user enters their name, age, gender, and travel preferences (nature, culture, adventure, etc.). They also enter data for emotion recognition as part of the initial setup.

[0457] Image input and emotion analysis

[0458] The user uploads an image that is close to the scenery they want to see to the application. The device then uploads this image data to cloud storage (e.g., AWS S3). The server retrieves the image from cloud storage and uses a generative AI model to analyze the image and the user's input information. For example, if a user uploads a photo of Mt. Fuji, the server uses a generative AI model (e.g., AWS Rekognition) to recommend "Japan," and the emotion engine recognizes the user's emotion of joy.

[0459] Generate a provisional itinerary

[0460] The server generates multiple provisional itineraries based on user information, image analysis, and sentiment analysis. It also gives each itinerary a catchy title and automatically suggests transportation and accommodation options. For example, the server might title the itinerary "Seven days to fully enjoy Japanese culture and nature" and suggest a specific itinerary.

[0461] Itinerary amendments and real-time updates

[0462] If a user reviews the proposed itinerary and wants to change some of the tourist attractions or accommodations, they can input the changes through the application, and the server will update the changes in real time. The emotion engine also recognizes the user's real-time emotional state and automatically adjusts the itinerary accordingly.

[0463] Providing a list of necessary items and what to bring

[0464] Once the travel plan is finalized, the server automatically generates a list of items to bring and sends it to the user's email. A real-time notification is also sent to the device. For example, the server generates a list of items to bring, such as "passport, VISA, Japanese yen, and clothing (for 7 days)," and notifies the user.

[0465] Providing local information

[0466] Before a trip, the server collects information about local culture, customs, and precautions, and notifies the user. For example, it provides information such as "Learn about Japanese hot spring culture: how to use towels and etiquette."

[0467] Emotion monitoring and optimization during travel

[0468] During the trip, the emotion engine continuously monitors the user's emotions and optimizes the travel experience based on the emotional data. For example, it can make suggestions such as, "There's a recommended cafe nearby. Why not relax there?" to optimize the user's travel experience.

[0469] Post-trip evaluation and suggestions for next time

[0470] After the trip, the user enters their satisfaction and impressions of the trip into the application. Based on this, the server will improve the next travel plan and make new suggestions. For example, it may say, "Why not plan another trip to Japan next time? We'll suggest some new tourist spots."

[0471] This system allows users to efficiently obtain customized travel plans and provides consistent support from planning to execution, as well as future proposals. By utilizing an emotion engine, it is possible to provide an optimal travel experience that reflects the user's real-time emotional state.

[0472] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0473] Step 1: Download and install the app

[0474] The user downloads and installs the application.

[0475] Input: Download link from the app store.

[0476] Output: Application installed on the device.

[0477] Specific behavior:

[0478] A user opens an app store and searches for an application.

[0479] Click on the application to begin the download.

[0480] The device installs the application.

[0481] Step 2: First launch and enter user information

[0482] The device launches the installed application and displays the new user registration page.

[0483] Input: User information (name, age, gender, travel preferences).

[0484] Output: Sending user information to the server.

[0485] Specific behavior:

[0486] The device launches the application.

[0487] Users enter their name, age, gender, and travel preferences.

[0488] The terminal transmits the input information to the server.

[0489] Step 3: Save user information

[0490] The server stores the entered user information in a database.

[0491] Input: User information (name, age, gender, travel preferences).

[0492] Output: User information stored in the database.

[0493] Specific behavior:

[0494] The server analyzes the received user information.

[0495] The server stores the analyzed information in a database.

[0496] Step 4: Upload an image

[0497] Users upload an image that is close to the view they want to see to the application.

[0498] Input: Image file.

[0499] Output: Image files saved in cloud storage.

[0500] Specific behavior:

[0501] The user selects an image of the scenery they want to see.

[0502] The device uploads the selected images to cloud storage.

[0503] Step 5: Image acquisition and analysis

[0504] The server retrieves the images from cloud storage and analyzes them using a generative AI model.

[0505] Input: Image files in cloud storage, user information.

[0506] Output: Analyzed image information and tourist spot suggestions.

[0507] Specific behavior:

[0508] The server downloads the image from the cloud storage.

[0509] The server analyzes the image using a generative AI model (e.g., AWS Rekognition).

[0510] Based on the analysis results, appropriate tourist destinations are suggested.

[0511] Step 6: Sentiment Analysis

[0512] The emotion engine recognizes the user's emotions based on the analysis results.

[0513] Input: Analyzed image information, user information.

[0514] Output: Recognized user emotion data.

[0515] Specific behavior:

[0516] The server transmits the analysis results to the emotion engine.

[0517] The emotion engine analyzes the user's emotions.

[0518] Record your emotional state in a database.

[0519] Step 7: Generate a provisional itinerary

[0520] The server generates multiple provisional itineraries based on user information, image analysis, and emotion analysis results.

[0521] Input: User information, analysis results, emotion data.

[0522] Output: Multiple provisional itineraries generated.

[0523] Specific behavior:

[0524] The server generates multiple provisional itineraries based on the input data.

[0525] Give the generated itinerary a catchy title.

[0526] Step 8: View your itinerary

[0527] The server generates an itinerary and displays it to the user.

[0528] Input: Multiple generated provisional itineraries.

[0529] Output: The itinerary displayed on the terminal.

[0530] Specific behavior:

[0531] The server sends the itinerary details to the terminal and displays them to the user.

[0532] The user reviews the displayed itinerary.

[0533] Step 9: Itinerary Modifications and Real-Time Updates

[0534] The user reviews the proposed itinerary and changes some of the attractions and accommodations.

[0535] Input: The user's correction request.

[0536] Output: The revised itinerary.

[0537] Specific behavior:

[0538] A user enters a request to change travel plans.

[0539] The server analyzes the changes and generates a new itinerary in real time.

[0540] Step 10: Provide a packing list

[0541] The server automatically generates an inventory list and sends it to the user by email. It also notifies the user on their device.

[0542] Input: Confirmed travel plans.

[0543] Output: Inventory list.

[0544] Specific behavior:

[0545] The server generates a list of necessary items to bring based on the travel plan.

[0546] The list of belongings will be sent via email and notified to the device.

[0547] Step 11: Provide local information

[0548] The server collects information about the culture of the destination, trivia, and things to be aware of, and notifies the user.

[0549] Input: Travel destination information.

[0550] Output: Local information notification.

[0551] Specific behavior:

[0552] The server collects local information from the Internet.

[0553] The collected information is sent to the user's device.

[0554] Step 12: Emotional monitoring and optimization during the trip

[0555] The emotion engine continuously monitors user emotions to optimize the travel experience.

[0556] Input: Real-time emotion data.

[0557] Output: Optimized trip plan.

[0558] Specific behavior:

[0559] The emotion engine uses smartphone sensor data to monitor emotions.

[0560] The server analyzes the emotional data and optimizes the travel experience in real time.

[0561] Your device will display notifications suggesting the best activities for you.

[0562] Step 13: Post-trip evaluation and suggestions for next time

[0563] The user enters their satisfaction and impressions of the trip into the app, and the server suggests plans for the next trip.

[0564] Input: Travel satisfaction and impressions.

[0565] Output: Next trip suggestions.

[0566] Specific behavior:

[0567] Users enter their impressions after the trip.

[0568] The server analyzes the data and makes suggestions for your next trip.

[0569] (Application example 2)

[0570] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0571] Conventional food delivery services struggled to provide optimal recommendations based on users' emotions and preferences, forcing them to spend time selecting menu items. Furthermore, there were limited ways to obtain detailed information about dishes or specific restaurants in advance, making it difficult for users to place highly satisfying orders. Furthermore, there was no system in place to utilize feedback after an order to personalize the next order.

[0572] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for the user to input their attributes, preferences, and images similar to the food they want to eat, a means for the server to analyze the input information and images and automatically generate multiple restaurants and menus tailored to the user, and a means for automatically generating and notifying the user of an information provision list required after the user confirms their order. This makes it possible to make suggestions optimized to the user's emotions and preferences.

[0573] "User" means an individual who uses the application to deliver food.

[0574] "Attributes" are a user's age, gender, and other basic profile information.

[0575] "Preferences" refers to the user's food preferences and favorite food categories (e.g., Japanese food, Italian food, etc.).

[0576] "Images similar to the food you want to eat" are images that users upload to the app that represent the food they want to eat that day or at that moment.

[0577] A "server" is a computer system that analyzes information obtained from users and generates and manages delivery proposals.

[0578] "Analyzing information" is the process of processing data based on attributes, preferences, and image data obtained from users to generate optimal suggestions.

[0579] "Automatically generating restaurants and menus" means selecting the most suitable restaurants and delivery menus for users based on analyzed information.

[0580] A "catchy title" means giving the proposed restaurant or menu an attention-grabbing title.

[0581] "Suggestion" means presenting a list of restaurants and menus that are deemed most suitable for the user.

[0582] "Modifiable" means that the user has the ability to adjust or change the suggested restaurant or menu item themselves.

[0583] "Real-time updates" means that changes made by the user are reflected immediately and the latest information is provided.

[0584] "Method of food delivery" refers to the method or means by which the food ordered by the user is delivered to the specified location.

[0585] "Information about the food provided" refers to detailed information about the food ordered by the user (e.g., ingredients, calories, allergen information, etc.).

[0586] A "rating" is feedback a user gives about the service or food provided after placing an order.

[0587] "Make next recommendation" means creating a personalized recommendation for your next order based on the collected ratings and other data.

[0588] The "information list" is a list of detailed information (e.g., confirmation items, usage guide, etc.) that the user will need after confirming their order.

[0589] "Notify" means sending important information or reminders to users to make sure they know.

[0590] The system for implementing this invention consists of an application that provides optimal food delivery suggestions based on the user's preferences and emotions. The system uses a smartphone as its primary hardware platform and the following software components:

[0591] Hardware used:

[0592] Smartphone (iOS or Android device)

[0593] Software used:

[0594] Mobile application frameworks (e.g., React Native)

[0595] Cloud storage (AWS S3)

[0596] Sentiment analysis engine (Google Cloud Vision API, Microsoft Azure Face API, etc.)

[0597] Backend server (Node.js, Express)

[0598] Database (MongoDB)

[0599] Detailed process flow:

[0600] User Registration:

[0601] After a user downloads and installs the application, they register as a new user. They enter their name, age, gender, and food preference (e.g., Japanese, Italian, etc.). This user information is collected using a mobile application framework and sent to the backend server, which stores this information in a database (MongoDB).

[0602] Examples:

[0603] If a user inputs that they like "Japanese food," the server stores this information in a database.

[0604] Example prompt sentence:

[0605] "You love Japanese food! Next, upload a photo of the dish you want to eat."

[0606] Image capture and sentiment analysis:

[0607] Users upload an image of a dish they want to eat to the app. The image is stored in cloud storage (AWS S3), and its URL is sent to the backend server. The server uses the Google Cloud Vision API to analyze the image and recognize the user's emotional state.

[0608] Examples:

[0609] If a user uploads a photo of sushi, the server analyzes the image, recognizes the user's emotions, and suggests sushi restaurants.

[0610] Example prompt sentence:

[0611] "Analyzing... Please wait a moment. We've found some recommended sushi restaurants!"

[0612] Auto-generate and suggest restaurants and menus:

[0613] The server then generates multiple restaurant and menu options based on the analysis results, taking into account the user's preferences and emotional data. The options are then presented to the user with catchy titles.

[0614] Examples:

[0615] The server will recommend restaurants to users under titles such as "Three Places Where You Can Enjoy Authentic Sushi," and each restaurant will display its features and recommended menu items.

[0616] Example prompt sentence:

[0617] "We've picked out three places where you can enjoy authentic sushi. Check them out!"

[0618] Order confirmation and real-time changes:

[0619] Users can review the suggested restaurants and menu items and make any changes they want to make through the app, which will then update the server in real time and present the latest suggestions.

[0620] Examples:

[0621] If the user indicates that they would like to change the second restaurant, the server generates a new suggestion and sends it to the device.

[0622] Example prompt sentence:

[0623] "We've updated our settings. Here are our new suggestions."

[0624] Pre-order information and notices:

[0625] After the order is confirmed, the server automatically generates and notifies the user of necessary information (e.g., reservation confirmation, usage guide, etc.), including important information and reminders.

[0626] Examples:

[0627] The server automatically generates a list of items to bring and usage guidelines, and sends them to the user via email or device notification.

[0628] Example prompt sentence:

[0629] "Reservation confirmation: Sushi Dai 7:00 PM~" "15-minute notice: Your reservation time for Sushi Dai is approaching."

[0630] Gathering feedback and suggestions for next time:

[0631] After the order is completed, the server collects the user's satisfaction and feedback, and based on this, generates personalized suggestions for the next order.

[0632] Examples:

[0633] If the user rates the trip as "very satisfied," the server will refine the next travel plan and make new suggestions.

[0634] Example prompt sentence:

[0635] "Would you like to enjoy sushi again next time? We'll suggest a nearby sushi restaurant."

[0636] This allows users to have a food delivery experience that is optimized to their emotions and preferences.

[0637] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0638] Step 1:

[0639] A user downloads the application and registers as a new user. They enter their name, age, gender, and food preferences. This user information is collected through an input form on the device and sent to the backend server. The server receives this information and stores it in a database (MongoDB).

[0640] Input: User's name, age, gender, food preferences

[0641] Output: User information stored in the database

[0642] Specific operation: After entering user information on the terminal and pressing the "Submit" button, the information is sent to the backend server, which receives it and stores it in the database.

[0643] Step 2:

[0644] The user uploads an image of a dish they want to eat to the app. The device then uploads the image to cloud storage (AWS S3) and sends the image URL to the backend server. The server receives the URL and uses the Google Cloud Vision API to analyze the image and recognize the user's emotion.

[0645] Input: Image of the food you want to eat

[0646] Output: Parsed emotion data

[0647] Specific operation: Press the "Upload Image" button on the device, select and upload an image. The URL of the uploaded image is sent to the backend server and passed to the Google Cloud Vision API. The API analyzes the image and returns the results to the server.

[0648] Step 3:

[0649] The server then generates multiple restaurant and menu options based on the analysis results, using data from existing restaurant databases and online sources. Taking into account the user's preferences and emotional data, the server selects the most suitable restaurant and menu options and suggests them to the user with catchy titles.

[0650] Input: User preference data, emotion analysis results

[0651] Output: A list of multiple restaurants and their menus

[0652] Specific operation: Query the database on the server to obtain relevant restaurant and menu information. Based on the obtained data, generate suitable suggestions for the user and send them back to the application.

[0653] Step 4:

[0654] If the user wants to review the suggested restaurants and menus and make any changes, they can do so through the app. These changes are sent in real time from the device to the server, which immediately updates the suggestions and presents updated suggestions again.

[0655] Input: User change request

[0656] Output: Updated list of restaurants and menus

[0657] Specific operation: Modify the proposal content on the terminal and press the "Submit changes" button. The changes are sent to the server, which updates the database and generates and returns a new proposal list.

[0658] Step 5:

[0659] After the order is confirmed, the server automatically generates the necessary information for the user (reservation confirmation, usage guide, etc.) and sends it to the user via email or device notification, including important information and reminders.

[0660] Input: Order confirmation data

[0661] Output: Auto-generated informational list and notifications

[0662] Specific operation: Using a template engine on the server, text information such as reservation confirmations and reminders is generated and sent to the user's email address or device.

[0663] Step 6:

[0664] When a user enters their satisfaction and feedback after placing an order, the server collects and stores this information in a database. This feedback is used to generate more personalized ordering suggestions for the next time.

[0665] Input: User satisfaction rating, impressions

[0666] Output: personalized suggestions for next time

[0667] What happens: You enter your satisfaction rating and feedback on your device and press the "Submit" button. The information is sent to the server and stored in a database. This feedback is used as a reference when generating the next recommendation.

[0668] The above are the specific processing steps of the system.

[0669] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0670] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0671] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0672] [Second embodiment]

[0673] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0674] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0675] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0676] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0677] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0678] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0679] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0680] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0681] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0682] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0683] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0684] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0685] Initial Setup

[0686] A user downloads and installs the app. When the user launches the app, they are presented with a new user registration page where they enter their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.).

[0687] Examples:

[0688] The user inputs that they prefer "natural."

[0689] The server stores the entered information in a database.

[0690] Image acquisition and analysis

[0691] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage, and the server retrieves and analyzes the image. Based on the analysis results, a list of recommended tourist spots is generated.

[0692] Examples:

[0693] A user uploads a photo of Mount Fuji.

[0694] The server analyzes the image and suggests Japan, where Mount Fuji is located.

[0695] Generate a provisional itinerary

[0696] The server generates multiple provisional itineraries based on the user's information and the results of image analysis, gives them catchy titles, and automatically suggests transportation and accommodation options.

[0697] Examples:

[0698] The server titles the itinerary it generates as "7 days to enjoy Japanese culture and nature."

[0699] The server displays the itinerary to the user.

[0700] Day 1: Arrival at Haneda Airport, sightseeing in Tokyo, accommodation (Hotel ABC)

[0701] Day 2: Sightseeing in Akihabara, Ueno Zoo, and lodging at Hotel XYZ

[0702] Day 3: Travel to Hakone, hot spring experience, and accommodation (hot spring inn DEF)

[0703] Itinerary amendments and real-time updates

[0704] If the user checks the proposed itinerary and decides to change some of the tourist attractions or accommodations, they can make the changes through the app, and the server will update the changes in real time.

[0705] Examples:

[0706] The user enters that they would like to change their accommodation for the third day to a different hot spring inn.

[0707] The server generates a new itinerary and sends it to the device.

[0708] New plan:

[0709] Day 3: Travel to Hakone, hot spring experience, and accommodation (Hot Spring Inn GHI)

[0710] Providing a list of necessary items and what to bring

[0711] Once the travel plan is finalized, the server will send the necessary information and a list of what to bring to the user's email, and a real-time notification will be sent to the user's device.

[0712] Examples:

[0713] The server automatically generates an inventory list.

[0714] Send email:

[0715] passport

[0716] VISA (Japan)

[0717] JPY

[0718] Clothing (7 days' worth)

[0719] Terminal notification: "Travel in 5 days. Please check your passport and VISA."

[0720] Providing local information

[0721] Before the trip, the server collects information on local culture, trivia, precautions, etc. and notifies the user.

[0722] Examples:

[0723] The server collects information about hot spring culture.

[0724] Device notification: "Learn about Japanese hot spring culture: How to use towels and etiquette"

[0725] Post-trip evaluation and suggestions for next time

[0726] After the trip, users input their satisfaction and impressions of the trip into the app, and the server uses this information to improve the next trip plan and make new suggestions.

[0727] Examples:

[0728] The user rates their trip satisfaction as "very satisfied."

[0729] Server's new suggestion: "Why not plan your next trip to Japan? We'll suggest some new tourist spots."

[0730] As described above, the system of the present invention provides users with efficient, customized travel plans and consistently supports them from planning to execution and even next proposals.

[0731] The processing flow will be explained below.

[0732] Step 1:

[0733] The user downloads and installs the app. The device launches the app and displays a new user registration page. The user enters their name, age, gender, travel preferences, etc. The device sends the information to the server. The server stores the user information in a database.

[0734] Step 2:

[0735] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage. The server retrieves the image from cloud storage and sends it to the image analysis module. The server's image analysis module analyzes the image and generates a list of recommended tourist spots based on the analysis results.

[0736] Step 3:

[0737] The server generates multiple provisional itineraries based on the user's preferences and the results of image analysis. The server gives the generated itineraries catchy titles. The server collects information on transportation and accommodations and reflects this in the itinerary. The server sends the multiple provisional itineraries it has generated to the user's terminal for provision.

[0738] Step 4:

[0739] The user reviews the proposed itinerary and inputs changes to some of the attractions and accommodations. The device sends the changes to the server. The server receives the change request and updates the travel plan with the new elements in real time. The server sends the updated plan to the device for presentation to the user.

[0740] Step 5:

[0741] The server generates the necessary information and a list of items to bring based on the confirmed travel plan. The server sends the generated information to the user's email and also sends a notification to the device. The device displays the notification to the user.

[0742] Step 6:

[0743] The server collects information about the destination's culture, trivia, and things to be aware of. The server then sends the collected information to the user's device. The device then notifies the user of the local information.

[0744] Step 7:

[0745] After a trip, the user enters their satisfaction and impressions of the trip into the app. The device sends the input information to the server. The server stores the user's rating in a database. The server uses the rating results to improve the next travel plan and send new suggestions to the user.

[0746] Example 1

[0747] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0748] Conventional travel planning systems require users to collect a large amount of information and create plans manually, which requires time and effort. Furthermore, it is difficult to change plans once they have been created, and information cannot be updated in real time, making it difficult to flexibly respond to various situations during a trip. Furthermore, they lack the functionality to provide useful information before a trip or to reflect impressions after returning home in the next plan.

[0749] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0750] In this invention, the server includes: means for a user to input images that are similar to the user's attributes, preferences, and scenery they wish to see; means for the server to analyze the input information and images and automatically generate multiple travel plans tailored to the user; means for presenting the multiple travel plans generated by the server with catchy titles; means for the user to change parts of the travel plan and updating the changes in real time; means for the server to collect information on transportation, accommodations, and tourist spots and reflect it in the travel plan; means for the server to provide the user with local information and trivia before the trip; means for the server to suggest the next travel plan based on the user's evaluation after the trip; and means for automatically generating a list of items to pack for the trip based on the information the user registered in the system and notifying the user in real time. This enables the user to efficiently obtain a customized travel plan and receive consistent support from planning to execution and even next suggestions.

[0751] "User" means any person or entity that uses the System to make travel plans.

[0752] "Attributes" refers to personal information such as a user's name, age, and gender.

[0753] "Preferences" refers to information related to the type of travel and destinations that a user prefers.

[0754] "Scenery" refers to images related to specific views or locations that users want to see.

[0755] "Means for inputting images" refers to an interface that allows users to upload images that resemble the scenery they want to see into the system.

[0756] "Analysis" is the process by which the server processes the images uploaded by users to derive relevant tourist information.

[0757] "Travel Plan" refers to the schedule and activity details generated by the server based on the user's attributes, preferences, and image analysis results.

[0758] A "catchy title" refers to an attractive name or phrase that grabs the user's attention.

[0759] "Modifiable" means that users can make adjustments or modifications to existing travel plans.

[0760] "Means of real-time updates" refers to a mechanism whereby when a user inputs changes, the server immediately reflects those changes.

[0761] "Transportation" refers to the means of transportation used during a trip.

[0762] "Accommodation" refers to hotels, inns, etc. where users stay during their travel plans.

[0763] "Tourist Spots" refers to famous places and tourist destinations that users can visit.

[0764] "Local information and trivia" refers to cultural knowledge and points of caution about the travel destination.

[0765] "Rating" refers to the satisfaction and impressions that users enter after their trip.

[0766] A "packing list" is an automated list of items you will need for your trip.

[0767] "Means of notification" refers to the way in which the system notifies the user of important information.

[0768] The purpose of this system is to enable users to efficiently create customized travel plans and to consistently support the process from planning to execution and even future proposals. An embodiment of the system will be described below.

[0769] Initial Setup

[0770] user

[0771] First, a user downloads and installs the application. When the user launches the app, they are presented with a new user registration page where they enter their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.). Based on this information, the server creates a user profile and stores it in a database.

[0772] Specific examples

[0773] If the user indicates that they prefer "natural," the server stores this information in a database.

[0774] Image acquisition and analysis

[0775] user

[0776] Users upload an image that resembles the scenery they want to see to the app. For example, a user can upload a photo of Mt. Fuji.

[0777] Terminal

[0778] The device uploads the image data to cloud storage.

[0779] server

[0780] The server retrieves images from cloud storage and analyzes them using an AI image analysis module. Based on the analysis results, a list of recommended tourist spots is generated.

[0781] Specific examples

[0782] If a user uploads a photo of Mount Fuji, the server analyzes the image and suggests "Japan," where Mount Fuji is located.

[0783] Generate a provisional itinerary

[0784] server

[0785] The server generates multiple provisional travel plans based on the user's attribute information and the results of image analysis. Each plan is given a catchy title. Information on transportation, accommodation, and tourist spots is automatically added to the generated travel plans.

[0786] Specific examples

[0787] The tentative itinerary is titled "Seven Days to Enjoy Japanese Culture and Nature" and includes the following detailed schedule:

[0788] Day 1: Arrival at Haneda Airport, sightseeing in Tokyo, accommodation (Hotel ABC)

[0789] Day 2: Sightseeing in Akihabara, Ueno Zoo, and lodging at Hotel XYZ

[0790] Day 3: Travel to Hakone, hot spring experience, and accommodation (hot spring inn DEF)

[0791] Itinerary amendments and real-time updates

[0792] user

[0793] Users can review the proposed itinerary and, if they wish to change some of the tourist attractions or accommodations, enter the changes via the app.

[0794] server

[0795] The server receives the changes entered in real time, generates a new itinerary, and sends it to the terminal.

[0796] Specific examples

[0797] If the user requests to change their accommodation on the third day to a different hot spring inn, the server will generate a new plan and make the following changes:

[0798] Day 3: Travel to Hakone, hot spring experience, and accommodation (Hot Spring Inn GHI)

[0799] Providing a list of necessary items and what to bring

[0800] server

[0801] After the travel plan is finalized, the server automatically generates the necessary information and a list of items to bring, and sends this information to the user's email address and device in real time.

[0802] Specific examples

[0803] The server will automatically generate an inventory list and email it to the user with the following:

[0804] passport

[0805] Visa (Japan)

[0806] JPY

[0807] Clothing (7 days' worth)

[0808] Device notification: "Travel in 5 days. Please check your passport and visa."

[0809] Providing local information

[0810] server

[0811] Before the trip, the server collects information on local culture, trivia, precautions, etc. and notifies the user's device.

[0812] Specific examples

[0813] The server collects information about hot spring culture and notifies the device of the following:

[0814] "Learn about Japanese hot spring culture: How to use towels and etiquette"

[0815] Post-trip evaluation and suggestions for next time

[0816] user

[0817] After the trip, users enter their satisfaction and impressions of the trip into the app.

[0818] server

[0819] The server will use the user's ratings to improve the next travel plan and send new suggestions to the user's device.

[0820] Specific examples

[0821] If the user enters "very satisfied" as their satisfaction with the trip, the server will notify them, "Why not plan another trip to Japan? We'll suggest some new tourist spots for you."

[0822] The system of the present invention allows users to efficiently create customized travel plans and provides consistent support from execution to next proposal.

[0823] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0824] Step 1:

[0825] user

[0826] A user downloads and installs the app, then launches it for the first time after installation is complete.

[0827] Specific actions

[0828] Search for, download, and install the app from the app store.

[0829] input

[0830] Your device information and internet connection.

[0831] output

[0832] Installed apps.

[0833] Step 2:

[0834] user

[0835] A user launches the app and visits the new user registration page, where they enter their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.).

[0836] Specific actions

[0837] Enter the name "Taro Tanaka", age "30", gender "male", and travel preference "nature".

[0838] input

[0839] Your personal information and travel preferences.

[0840] output

[0841] New user registration data including the information entered.

[0842] Step 3:

[0843] server

[0844] The server receives the entered user information and stores it in a database.

[0845] Specific actions

[0846] On the server side, establish a database connection and execute a query to insert the user information.

[0847] input

[0848] New User Registration Data.

[0849] output

[0850] User information stored in a database.

[0851] Step 4:

[0852] user

[0853] Users upload images that are similar to the scenery they want to see to the app.

[0854] Specific actions

[0855] Select the image file on your device and press the "Upload" button.

[0856] input

[0857] An image file selected by the user.

[0858] output

[0859] Uploaded image data.

[0860] Step 5:

[0861] Terminal

[0862] The terminal transfers the uploaded image data to cloud storage.

[0863] Specific actions

[0864] Call the cloud storage API to upload image data.

[0865] input

[0866] Image data selected by the user.

[0867] output

[0868] The URL of the image stored in cloud storage.

[0869] Step 6:

[0870] server

[0871] The server retrieves images from cloud storage and sends them to the AI ​​image analysis module, which generates a list of recommended tourist spots based on the analysis results.

[0872] Specific actions

[0873] Call the image analysis API and obtain the analysis results.

[0874] input

[0875] Image URL in cloud storage.

[0876] output

[0877] A list of tourist destinations as a result of the analysis.

[0878] Step 7:

[0879] server

[0880] The server generates multiple provisional travel plans based on the user's attribute information and the results of image analysis, and gives each plan a catchy title.

[0881] Specific actions

[0882] A title generation algorithm is executed to give a title to the travel plan.

[0883] input

[0884] User attribute information and image analysis results.

[0885] output

[0886] Tentative travel plan list.

[0887] Step 8:

[0888] server

[0889] The server adds information about transportation, accommodation, and tourist spots to the tentative travel plan and presents it to the user.

[0890] Specific actions

[0891] Run a query to add transportation and accommodation information for each itinerary.

[0892] input

[0893] Tentative travel plan list.

[0894] output

[0895] The completed travel itinerary list.

[0896] Step 9:

[0897] user

[0898] Users can review the proposed travel plan and, if they wish to make any changes, enter the changes through the app.

[0899] Specific actions

[0900] On the travel plan screen, select the item you want to change and enter the new information.

[0901] input

[0902] User modification requests.

[0903] output

[0904] Change request data.

[0905] Step 10:

[0906] server

[0907] The server receives the changes in real time, generates a new travel plan, and sends it to the terminal.

[0908] Specific actions

[0909] It runs an algorithm to regenerate a new itinerary based on the changes and sends the updated data to the device.

[0910] input

[0911] The user's modification request data.

[0912] output

[0913] Updated travel planning data.

[0914] Step 11:

[0915] server

[0916] After the server confirms the travel plan, it automatically generates the necessary information and a list of items to bring, and sends them to the user's email address. It also sends a real-time notification to the user's device.

[0917] Specific actions

[0918] Run inventory generation algorithms and send emails and notifications.

[0919] input

[0920] Confirmed travel planning data.

[0921] output

[0922] Inventory list and notification data.

[0923] Step 12:

[0924] server

[0925] Before a trip, the server collects local information and trivia and notifies the user's device.

[0926] Specific actions

[0927] It calls the local information API to collect information and sends push notifications to the user's device.

[0928] input

[0929] Local information API data.

[0930] output

[0931] Local information notification data.

[0932] Step 13:

[0933] user

[0934] After the trip, users enter their satisfaction and impressions of the trip into the app.

[0935] Specific actions

[0936] Fill out the evaluation form in the app and press the submit button.

[0937] input

[0938] User satisfaction and opinion data.

[0939] output

[0940] Submitted evaluation data.

[0941] Step 14:

[0942] server

[0943] The server uses the user's ratings to refine the next travel plan and send new suggestions to the user.

[0944] Specific actions

[0945] The next itinerary generation algorithm is run to generate new proposals and send them to the user's device.

[0946] input

[0947] Submitted evaluation data.

[0948] output

[0949] New travel planning data.

[0950] (Application example 1)

[0951] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0952] Conventional online shopping lacks the ability to offer products tailored to individual user preferences and needs, resulting in a poor quality shopping experience. In particular, it is difficult for users to find the products and designs they specifically desire, creating a need for a hassle-free way to select a set of products that suits them.

[0953] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0954] In this invention, the server includes means for the user to input their attributes, preferences, and images similar to the products they wish to see, means for the server to analyze the input information and images and automatically generate multiple product sets tailored to the user, means for presenting the multiple product sets generated by the server with catchy titles, means for the user to change parts of the product sets and means for updating the changes in real time, and means for the server to suggest the next shopping plan based on the user's ratings. This enables product suggestions to be individually customized for the user, improving the quality of online shopping.

[0955] "User" refers to an individual who uses the system to select and purchase products.

[0956] "Attributes" refers to personal information such as a user's name, age, gender, and preferences.

[0957] "Preferences" refer to a user's tendency to be particularly interested in a particular style or category.

[0958] "Image" refers to visual data entered by the user that shows the desired product or design.

[0959] "Server" refers to the main computer system that processes user input and image data and generates product sets based on the analysis results.

[0960] "Analyzing images" refers to analyzing uploaded image data through a program and extracting relevant information based on its content.

[0961] "Means for automatically generating product sets" refers to the process of automatically selecting and combining multiple related products based on user attributes and the results of image analysis.

[0962] A "catchy title" refers to giving the generated product set a name that will attract the user's interest.

[0963] The "presentation means" refers to a method or function for visually presenting the generated product set to the user.

[0964] "Modifiable" means that users can modify parts of the suggested product set to suit their preferences.

[0965] "Means of real-time updates" refers to methods and functions that allow changes made by users to be reflected immediately on the system.

[0966] "Means to suggest next shopping plans" refers to a system that improves future shopping experiences and suggests new products based on user ratings and feedback.

[0967] A "checklist" is a list of items or tasks that a user needs to complete after their shopping plan is finalized.

[0968] "Means of notification" refers to the methods and functions by which the system notifies the user of information.

[0969] "Collecting satisfaction and feedback" refers to collecting feedback and comments provided by users as data.

[0970] "Means for improvement" refers to methods and functions for effectively improving the next proposal or the system itself based on the collected feedback.

[0971] The invention begins with the user entering their attributes, preferences, and images of products similar to the one they want to see. The user first downloads and installs the application. When the application is launched, a new user registration page appears, where the user enters their name, age, gender, and shopping preferences (e.g., fashion, home appliances, outdoor gear, etc.). This information is stored in the cloud and used to customize the shopping experience.

[0972] Next, the user uploads an image of the product they want to see to the app. The device uploads the image data to cloud storage, and the server retrieves and analyzes the image. This image analysis uses OpenCV and KMeans clustering. For example, if a user uploads an image of a modern living room, the app analyzes its color hues and suggests furniture and decoration items in a similar style.

[0973] The server generates a set of related products based on the user's information and image analysis results, and gives it a catchy title such as "Modern Living Room Set." The generated product set is displayed in the app, and the user can easily add it to their shopping list. If the user reviews the suggested product set and wants to change some of the products, the changes can be reflected in real time. For example, if the user wants to change the color of a sofa, the server will re-suggest a product list including the sofa in the new color.

[0974] Once the shopping list is finalized, the server automatically generates the necessary checklist and notifies the user, including items and tasks to be checked before purchasing. The app also provides additional information about the products, such as their background and use cases.

[0975] After making a purchase, users can enter their satisfaction and feedback on their experience into the app, which the server uses to improve their next shopping plan and make new suggestions.

[0976] The system uses the following main hardware and software: OpenCV and the KMeans clustering algorithm for image analysis, the Flask framework on the server side, MongoDB and MySQL for the database, and smtplib for sending emails.

[0977] An example of a prompt is as follows:

[0978] Example prompt sentence:

[0979] Upload an image of a modern living room to represent the product users want to see.

[0980] As a result of the analysis, the server suggests related modern style furniture (e.g. sofa, table, carpet).

[0981] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0982] Step 1:

[0983] The user downloads and installs the application. After launching it, a new user registration page appears, where the user enters their name, age, gender, and shopping preferences. This information is sent from the device to the cloud server and stored in a database. The input in this process is user information, and the output is the storage of information in the cloud database.

[0984] Step 2:

[0985] The user selects an image that is similar to the product they want to see and uploads it to the app. The device uploads the image data to cloud storage, and the server retrieves the image data. In this step, the input is the image data, and the output is saving it to cloud storage and retrieving it by the server.

[0986] Step 3:

[0987] The server analyzes the uploaded image data using OpenCV and KMeans clustering. Specifically, it extracts the hue information of the image and classifies it into multiple clusters. The input is the image data, and the output is a color palette classified by the hue information.

[0988] Step 4:

[0989] The server generates a related product set based on the color palette obtained as a result of the analysis. Using a generative AI model, it inputs the user's preference information and color palette and proposes the optimal product set. The input in this step is the analysis result and the user's preference information, and the output is the proposed product set. This product set is given a catchy title such as "Modern Living Room Set."

[0990] Step 5:

[0991] The server presents the generated product set to the user through the application. The user can review the proposal and modify parts of the product set if necessary. The modifications are sent to the server in real time, and the server generates a corresponding new product set and presents it to the user again. The input in this step is the user's modification request, and the output is the updated product set.

[0992] Step 6:

[0993] Once the user has finalized the product set, the server automatically generates a checklist, which includes items and tasks to be checked before purchasing. This checklist is sent to the user's device. The input is the finalized product set, and the output is the generated checklist.

[0994] Step 7:

[0995] After a purchase, the user enters feedback about their satisfaction and experience into the application. The server collects this feedback and uses it to improve the next shopping plan. In this step, the input is the user's feedback and the output is the improved next shopping plan.

[0996] Example prompt sentence:

[0997] Step 1: Register as a new user by entering your name, age, gender and shopping preferences.

[0998] Step 2: Upload an image that resembles the product you want to see.

[0999] Step 3: Generate a color palette based on the results of image analysis.

[1000] Step 4: Propose a product set based on the color palette and user preference information.

[1001] Step 5: User reviews the proposed product set and makes changes if necessary.

[1002] Step 6: Automatically generate a checklist after the product set is confirmed.

[1003] Step 7: Collect post-purchase feedback and refine your next plan.

[1004] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1005] Initial Setup

[1006] The user downloads and installs the app. The device launches the app and displays a new user registration page. The user enters their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.). The app also performs initial setup to recognize the user's emotions.

[1007] Examples:

[1008] The user inputs that they prefer "natural."

[1009] The server stores the entered information in a database.

[1010] Image input and emotion analysis

[1011] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage, and the server retrieves the image and analyzes the image and the user's input information using an emotion engine. Based on the analysis results, a list of recommended tourist spots is generated and the user's emotional state is recorded.

[1012] Examples:

[1013] A user uploads a photo of Mount Fuji.

[1014] The server analyzes the image and suggests Japan.

[1015] The emotion engine recognizes the user's emotion of joy.

[1016] Generate a provisional itinerary

[1017] The server generates multiple provisional itineraries based on the user's information, image analysis, and sentiment analysis results. It then gives each itinerary a catchy title and automatically suggests transportation and accommodation options.

[1018] Examples:

[1019] The server titles the itinerary it generates as "7 days to enjoy Japanese culture and nature."

[1020] The server displays the itinerary to the user.

[1021] Day 1: Arrival at Haneda Airport, sightseeing in Tokyo, accommodation (Hotel ABC)

[1022] Day 2: Sightseeing in Akihabara, Ueno Zoo, and lodging at Hotel XYZ

[1023] Day 3: Travel to Hakone, hot spring experience, and accommodation (hot spring inn DEF)

[1024] Itinerary amendments and real-time updates

[1025] If a user reviews the proposed itinerary and decides to change some of the tourist attractions or accommodations, they can make the changes through the app, and the server will update the changes in real time.In addition, the emotion engine recognizes the user's real-time emotional state and automatically adjusts the itinerary accordingly.

[1026] Examples:

[1027] The user enters that they would like to change their accommodation for the third day to a different hot spring inn.

[1028] The server generates a new itinerary and sends it to the device.

[1029] The emotion engine recognizes the user's emotions and proposes the optimal plan.

[1030] New plan:

[1031] Day 3: Travel to Hakone, hot spring experience, and accommodation (Hot Spring Inn GHI)

[1032] Providing a list of necessary items and what to bring

[1033] Once the travel plan is finalized, the server will send the necessary information and a list of what to bring to the user's email, and a real-time notification will be sent to the user's device.

[1034] Examples:

[1035] The server automatically generates an inventory list.

[1036] Send email:

[1037] passport

[1038] VISA (Japan)

[1039] JPY

[1040] Clothing (7 days' worth)

[1041] Terminal notification: "Travel in 5 days. Please check your passport and VISA."

[1042] Providing local information

[1043] Before the trip, the server collects information on local culture, trivia, precautions, etc. and notifies the user.

[1044] Examples:

[1045] The server collects information about hot spring culture.

[1046] Device notification: "Learn about Japanese hot spring culture: How to use towels and etiquette"

[1047] Emotion monitoring and optimization during travel

[1048] During the trip, the emotion engine continuously monitors the user's emotions and optimizes the travel experience based on the emotional data. If the user is enjoying themselves, it will make suggestions to maintain that state, and if they are feeling stressed, it will suggest relaxing activities.

[1049] Examples:

[1050] The emotion engine recognizes the user's level of enjoyment.

[1051] Device notification: "There's a recommended cafe nearby. Would you like to relax?"

[1052] Post-trip evaluation and suggestions for next time

[1053] After a trip, users input their satisfaction and impressions of the trip into the app. The server uses this information to improve the next trip plan and make new suggestions. The emotion engine also utilizes emotional data from the trip to make more personalized suggestions.

[1054] Examples:

[1055] The user rates their trip satisfaction as "very satisfied."

[1056] The server makes the next suggestion based on the emotion data.

[1057] New suggestion: "Why not plan your next trip to Japan? We'll suggest some new tourist spots."

[1058] As described above, the system of the present invention efficiently acquires customized travel plans for users and provides consistent support from planning to execution and even next-time suggestions. By combining it with an emotion engine, it is possible to provide an optimal travel experience that reflects the user's real-time emotional state.

[1059] The processing flow will be explained below.

[1060] Step 1:

[1061] The user downloads and installs the app. The device launches the app and displays the new user registration page. The user enters their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.) to initialize the emotion engine. The device sends the input information to the server. The server stores the user information in a database.

[1062] Step 2:

[1063] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage. The server retrieves the image from cloud storage and sends it to the image analysis module and emotion engine. The server's image analysis module analyzes the image, and the emotion engine recognizes the user's emotional state. A list of recommended tourist spots is generated based on the analysis results and emotion data.

[1064] Step 3:

[1065] The server generates multiple provisional itineraries based on user information, image analysis, and sentiment analysis. It then gives each itinerary a catchy title. The server then collects information on transportation and accommodations and incorporates this information into the itinerary. The multiple provisional itineraries generated by the server are then sent to the user's device for provision.

[1066] Step 4:

[1067] The user reviews the proposed itinerary and inputs changes to some of the tourist attractions and accommodations. The device sends the changes to the server. The server receives the change request and updates the travel plan with the new elements in real time. The emotion engine recognizes the user's real-time emotional state and automatically adjusts the itinerary accordingly. The server sends the updated plan to the device for presentation to the user.

[1068] Step 5:

[1069] The server generates the necessary information and a list of items to bring based on the confirmed travel plan. The server sends the generated information to the user's email and also sends a notification to the device. The device displays the notification to the user.

[1070] Step 6:

[1071] The server collects information about the destination's culture, trivia, and things to be aware of. The server then sends the collected information to the user's device. The device then notifies the user of the local information.

[1072] Step 7:

[1073] During the trip, the emotion engine continuously monitors the user's emotions. The server makes suggestions to optimize the travel experience based on the emotion data. The device then notifies the user of the optimized information.

[1074] Step 8:

[1075] After a trip, the user enters their satisfaction and impressions of the trip into the app. The device sends the input information to the server. The server stores the user's ratings in a database. The emotion engine utilizes the emotional data from the trip, and the server sends the user new suggestions to improve their next travel plan.

[1076] Example 2

[1077] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1078] Conventional travel planning support systems have difficulty providing travel plans that fully reflect the user's preferences and emotions. It is also difficult to adjust plans in response to real-time emotional changes during travel. This has resulted in the issue of users' travel experiences not being optimized.

[1079] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1080] In this invention, the server includes a means for a user to input images that are similar to their attributes, preferences, and scenery they want to see, a means for analyzing the input information and images using a generative AI model and automatically generating multiple itineraries tailored to the user, and a means for monitoring the user's emotional state during the trip using an emotion engine to optimize the travel experience. This makes it possible to provide a personalized travel plan that reflects the user's preferences and emotions, and to adjust the plan to suit real-time emotional changes during the trip.

[1081] "User" means an individual who intends to use the System to create a travel plan.

[1082] "Attributes" refers to personal information such as a user's name, age, and gender.

[1083] "Preferences" refers to the user's preferred travel style and purpose, such as nature, culture, adventure, etc.

[1084] "Images similar to the scenery you want to see" are images that express the image of scenery or places you want to see as a travel destination.

[1085] A "server" is a computer system that analyzes data and generates travel plans based on information provided by users.

[1086] A "generative AI model" is an algorithm that uses artificial intelligence to analyze input information and images and generate the optimal travel plan for the user.

[1087] An "emotion engine" is a technology that recognizes the user's emotional state and uses it as data for analysis.

[1088] An "itinerary" is a detailed plan that lists the travel schedule and places to visit.

[1089] "Title" is a catchphrase or description that is given to the generated itinerary.

[1090] "Transportation" refers to the means of transportation used during a trip (e.g., plane, train, bus).

[1091] "Accommodation" refers to the place where you stay during your trip, such as a hotel or inn.

[1092] "Tourist attractions" refer to places of interest and activities that you can visit while traveling.

[1093] A "packing list" is a list of items you will need while traveling.

[1094] "Local information" refers to information about the culture, customs, and precautions of the destination.

[1095] "User rating" refers to the satisfaction and impressions that users enter into the system after their trip.

[1096] "Next Travel Plan" is a plan for your next trip that is suggested based on the user's past ratings and emotional data.

[1097] This invention is a system that provides a personalized travel plan that reflects the user's preferences and emotions, and adjusts the plan optimally in response to real-time changes in emotions during the trip. This system operates in cooperation with the user, terminal, and server.

[1098] Initial Setup

[1099] First, the user downloads and installs the application. The device launches the application and displays a new user registration page. The user enters their name, age, gender, and travel preferences (nature, culture, adventure, etc.). They also enter data for emotion recognition as part of the initial setup.

[1100] Image input and emotion analysis

[1101] The user uploads an image that is close to the scenery they want to see to the application. The device then uploads this image data to cloud storage (e.g., AWS S3). The server retrieves the image from cloud storage and uses a generative AI model to analyze the image and the user's input information. For example, if a user uploads a photo of Mt. Fuji, the server uses a generative AI model (e.g., AWS Rekognition) to recommend "Japan," and the emotion engine recognizes the user's emotion of joy.

[1102] Generate a provisional itinerary

[1103] The server generates multiple provisional itineraries based on user information, image analysis, and sentiment analysis. It also gives each itinerary a catchy title and automatically suggests transportation and accommodation options. For example, the server might title the itinerary "Seven days to fully enjoy Japanese culture and nature" and suggest a specific itinerary.

[1104] Itinerary amendments and real-time updates

[1105] If a user reviews the proposed itinerary and wants to change some of the tourist attractions or accommodations, they can input the changes through the application, and the server will update the changes in real time. The emotion engine also recognizes the user's real-time emotional state and automatically adjusts the itinerary accordingly.

[1106] Providing a list of necessary items and what to bring

[1107] Once the travel plan is finalized, the server automatically generates a list of items to bring and sends it to the user's email. A real-time notification is also sent to the device. For example, the server generates a list of items to bring, such as "passport, VISA, Japanese yen, and clothing (for 7 days)," and notifies the user.

[1108] Providing local information

[1109] Before a trip, the server collects information about local culture, customs, and precautions, and notifies the user. For example, it provides information such as "Learn about Japanese hot spring culture: how to use towels and etiquette."

[1110] Emotion monitoring and optimization during travel

[1111] During the trip, the emotion engine continuously monitors the user's emotions and optimizes the travel experience based on the emotional data. For example, it can make suggestions such as, "There's a recommended cafe nearby. Why not relax there?" to optimize the user's travel experience.

[1112] Post-trip evaluation and suggestions for next time

[1113] After the trip, the user enters their satisfaction and impressions of the trip into the application. Based on this, the server will improve the next travel plan and make new suggestions. For example, it may say, "Why not plan another trip to Japan next time? We'll suggest some new tourist spots."

[1114] This system allows users to efficiently obtain customized travel plans and provides consistent support from planning to execution, as well as future proposals. By utilizing an emotion engine, it is possible to provide an optimal travel experience that reflects the user's real-time emotional state.

[1115] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1116] Step 1: Download and install the app

[1117] The user downloads and installs the application.

[1118] Input: Download link from the app store.

[1119] Output: Application installed on the device.

[1120] Specific behavior:

[1121] A user opens an app store and searches for an application.

[1122] Click on the application to begin the download.

[1123] The device installs the application.

[1124] Step 2: First launch and enter user information

[1125] The device launches the installed application and displays the new user registration page.

[1126] Input: User information (name, age, gender, travel preferences).

[1127] Output: Sending user information to the server.

[1128] Specific behavior:

[1129] The device launches the application.

[1130] Users enter their name, age, gender, and travel preferences.

[1131] The terminal transmits the input information to the server.

[1132] Step 3: Save user information

[1133] The server stores the entered user information in a database.

[1134] Input: User information (name, age, gender, travel preferences).

[1135] Output: User information stored in the database.

[1136] Specific behavior:

[1137] The server analyzes the received user information.

[1138] The server stores the analyzed information in a database.

[1139] Step 4: Upload an image

[1140] Users upload an image that is close to the view they want to see to the application.

[1141] Input: Image file.

[1142] Output: Image files saved in cloud storage.

[1143] Specific behavior:

[1144] The user selects an image of the scenery they want to see.

[1145] The device uploads the selected images to cloud storage.

[1146] Step 5: Image acquisition and analysis

[1147] The server retrieves the images from cloud storage and analyzes them using a generative AI model.

[1148] Input: Image files in cloud storage, user information.

[1149] Output: Analyzed image information and tourist spot suggestions.

[1150] Specific behavior:

[1151] The server downloads the image from the cloud storage.

[1152] The server analyzes the image using a generative AI model (e.g., AWS Rekognition).

[1153] Based on the analysis results, appropriate tourist destinations are suggested.

[1154] Step 6: Sentiment Analysis

[1155] The emotion engine recognizes the user's emotions based on the analysis results.

[1156] Input: Analyzed image information, user information.

[1157] Output: Recognized user emotion data.

[1158] Specific behavior:

[1159] The server transmits the analysis results to the emotion engine.

[1160] The emotion engine analyzes the user's emotions.

[1161] Record your emotional state in a database.

[1162] Step 7: Generate a provisional itinerary

[1163] The server generates multiple provisional itineraries based on user information, image analysis, and emotion analysis results.

[1164] Input: User information, analysis results, emotion data.

[1165] Output: Multiple provisional itineraries generated.

[1166] Specific behavior:

[1167] The server generates multiple provisional itineraries based on the input data.

[1168] Give the generated itinerary a catchy title.

[1169] Step 8: View your itinerary

[1170] The server generates an itinerary and displays it to the user.

[1171] Input: Multiple generated provisional itineraries.

[1172] Output: The itinerary displayed on the terminal.

[1173] Specific behavior:

[1174] The server sends the itinerary details to the terminal and displays them to the user.

[1175] The user reviews the displayed itinerary.

[1176] Step 9: Itinerary Modifications and Real-Time Updates

[1177] The user reviews the proposed itinerary and changes some of the attractions and accommodations.

[1178] Input: The user's correction request.

[1179] Output: The revised itinerary.

[1180] Specific behavior:

[1181] A user enters a request to change travel plans.

[1182] The server analyzes the changes and generates a new itinerary in real time.

[1183] Step 10: Provide a packing list

[1184] The server automatically generates an inventory list and sends it to the user by email. It also notifies the user on their device.

[1185] Input: Confirmed travel plans.

[1186] Output: Inventory list.

[1187] Specific behavior:

[1188] The server generates a list of necessary items to bring based on the travel plan.

[1189] The list of belongings will be sent via email and notified to the device.

[1190] Step 11: Provide local information

[1191] The server collects information about the culture of the destination, trivia, and things to be aware of, and notifies the user.

[1192] Input: Travel destination information.

[1193] Output: Local information notification.

[1194] Specific behavior:

[1195] The server collects local information from the Internet.

[1196] The collected information is sent to the user's device.

[1197] Step 12: Emotional monitoring and optimization during the trip

[1198] The emotion engine continuously monitors user emotions to optimize the travel experience.

[1199] Input: Real-time emotion data.

[1200] Output: Optimized trip plan.

[1201] Specific behavior:

[1202] The emotion engine uses smartphone sensor data to monitor emotions.

[1203] The server analyzes the emotional data and optimizes the travel experience in real time.

[1204] Your device will display notifications suggesting the best activities for you.

[1205] Step 13: Post-trip evaluation and suggestions for next time

[1206] The user enters their satisfaction and impressions of the trip into the app, and the server suggests plans for the next trip.

[1207] Input: Travel satisfaction and impressions.

[1208] Output: Next trip suggestions.

[1209] Specific behavior:

[1210] Users enter their impressions after the trip.

[1211] The server analyzes the data and makes suggestions for your next trip.

[1212] (Application example 2)

[1213] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1214] Conventional food delivery services struggled to provide optimal recommendations based on users' emotions and preferences, forcing them to spend time selecting menu items. Furthermore, there were limited ways to obtain detailed information about dishes or specific restaurants in advance, making it difficult for users to place highly satisfying orders. Furthermore, there was no system in place to utilize feedback after an order to personalize the next order.

[1215] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for the user to input their attributes, preferences, and images similar to the food they want to eat, a means for the server to analyze the input information and images and automatically generate multiple restaurants and menus tailored to the user, and a means for automatically generating and notifying the user of an information provision list required after the user confirms their order. This makes it possible to make suggestions optimized to the user's emotions and preferences.

[1216] "User" means an individual who uses the application to deliver food.

[1217] "Attributes" are a user's age, gender, and other basic profile information.

[1218] "Preferences" refers to the user's food preferences and favorite food categories (e.g., Japanese food, Italian food, etc.).

[1219] "Images similar to the food you want to eat" are images that users upload to the app that represent the food they want to eat that day or at that moment.

[1220] A "server" is a computer system that analyzes information obtained from users and generates and manages delivery proposals.

[1221] "Analyzing information" is the process of processing data based on attributes, preferences, and image data obtained from users to generate optimal suggestions.

[1222] "Automatically generating restaurants and menus" means selecting the most suitable restaurants and delivery menus for users based on analyzed information.

[1223] A "catchy title" means giving the proposed restaurant or menu an attention-grabbing title.

[1224] "Suggestion" means presenting a list of restaurants and menus that are deemed most suitable for the user.

[1225] "Modifiable" means that the user has the ability to adjust or change the suggested restaurant or menu item themselves.

[1226] "Real-time updates" means that changes made by the user are reflected immediately and the latest information is provided.

[1227] "Method of food delivery" refers to the method or means by which the food ordered by the user is delivered to the specified location.

[1228] "Information about the food provided" refers to detailed information about the food ordered by the user (e.g., ingredients, calories, allergen information, etc.).

[1229] A "rating" is feedback a user gives about the service or food provided after placing an order.

[1230] "Make next recommendation" means creating a personalized recommendation for your next order based on the collected ratings and other data.

[1231] The "information list" is a list of detailed information (e.g., confirmation items, usage guide, etc.) that the user will need after confirming their order.

[1232] "Notify" means sending important information or reminders to users to make sure they know.

[1233] The system for implementing this invention consists of an application that provides optimal food delivery suggestions based on the user's preferences and emotions. The system uses a smartphone as its primary hardware platform and the following software components:

[1234] Hardware used:

[1235] Smartphone (iOS or Android device)

[1236] Software used:

[1237] Mobile application frameworks (e.g., React Native)

[1238] Cloud storage (AWS S3)

[1239] Sentiment analysis engine (Google Cloud Vision API, Microsoft Azure Face API, etc.)

[1240] Backend server (Node.js, Express)

[1241] Database (MongoDB)

[1242] Detailed process flow:

[1243] User Registration:

[1244] After a user downloads and installs the application, they register as a new user. They enter their name, age, gender, and food preference (e.g., Japanese, Italian, etc.). This user information is collected using a mobile application framework and sent to the backend server, which stores this information in a database (MongoDB).

[1245] Examples:

[1246] If a user inputs that they like "Japanese food," the server stores this information in a database.

[1247] Example prompt sentence:

[1248] "You love Japanese food! Next, upload a photo of the dish you want to eat."

[1249] Image capture and sentiment analysis:

[1250] Users upload an image of a dish they want to eat to the app. The image is stored in cloud storage (AWS S3), and its URL is sent to the backend server. The server uses the Google Cloud Vision API to analyze the image and recognize the user's emotional state.

[1251] Examples:

[1252] If a user uploads a photo of sushi, the server analyzes the image, recognizes the user's emotions, and suggests sushi restaurants.

[1253] Example prompt sentence:

[1254] "Analyzing... Please wait a moment. We've found some recommended sushi restaurants!"

[1255] Auto-generate and suggest restaurants and menus:

[1256] The server then generates multiple restaurant and menu options based on the analysis results, taking into account the user's preferences and emotional data. The options are then presented to the user with catchy titles.

[1257] Examples:

[1258] The server will recommend restaurants to users under titles such as "Three Places Where You Can Enjoy Authentic Sushi," and each restaurant will display its features and recommended menu items.

[1259] Example prompt sentence:

[1260] "We've picked out three places where you can enjoy authentic sushi. Check them out!"

[1261] Order confirmation and real-time changes:

[1262] Users can review the suggested restaurants and menu items and make any changes they want to make through the app, which will then update the server in real time and present the latest suggestions.

[1263] Examples:

[1264] If the user indicates that they would like to change the second restaurant, the server generates a new suggestion and sends it to the device.

[1265] Example prompt sentence:

[1266] "We've updated our settings. Here are our new suggestions."

[1267] Pre-order information and notices:

[1268] After the order is confirmed, the server automatically generates and notifies the user of necessary information (e.g., reservation confirmation, usage guide, etc.), including important information and reminders.

[1269] Examples:

[1270] The server automatically generates a list of items to bring and usage guidelines, and sends them to the user via email or device notification.

[1271] Example prompt sentence:

[1272] "Reservation confirmation: Sushi Dai 7:00 PM~" "15-minute notice: Your reservation time for Sushi Dai is approaching."

[1273] Gathering feedback and suggestions for next time:

[1274] After the order is completed, the server collects the user's satisfaction and feedback, and based on this, generates personalized suggestions for the next order.

[1275] Examples:

[1276] If the user rates the trip as "very satisfied," the server will refine the next travel plan and make new suggestions.

[1277] Example prompt sentence:

[1278] "Would you like to enjoy sushi again next time? We'll suggest a nearby sushi restaurant."

[1279] This allows users to have a food delivery experience that is optimized to their emotions and preferences.

[1280] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1281] Step 1:

[1282] A user downloads the application and registers as a new user. They enter their name, age, gender, and food preferences. This user information is collected through an input form on the device and sent to the backend server. The server receives this information and stores it in a database (MongoDB).

[1283] Input: User's name, age, gender, food preferences

[1284] Output: User information stored in the database

[1285] Specific operation: After entering user information on the terminal and pressing the "Submit" button, the information is sent to the backend server, which receives it and stores it in the database.

[1286] Step 2:

[1287] The user uploads an image of a dish they want to eat to the app. The device then uploads the image to cloud storage (AWS S3) and sends the image URL to the backend server. The server receives the URL and uses the Google Cloud Vision API to analyze the image and recognize the user's emotion.

[1288] Input: Image of the food you want to eat

[1289] Output: Parsed emotion data

[1290] Specific operation: Press the "Upload Image" button on the device, select and upload an image. The URL of the uploaded image is sent to the backend server and passed to the Google Cloud Vision API. The API analyzes the image and returns the results to the server.

[1291] Step 3:

[1292] The server then generates multiple restaurant and menu options based on the analysis results, using data from existing restaurant databases and online sources. Taking into account the user's preferences and emotional data, the server selects the most suitable restaurant and menu options and suggests them to the user with catchy titles.

[1293] Input: User preference data, emotion analysis results

[1294] Output: A list of multiple restaurants and their menus

[1295] Specific operation: Query the database on the server to obtain relevant restaurant and menu information. Based on the obtained data, generate suitable suggestions for the user and send them back to the application.

[1296] Step 4:

[1297] If the user wants to review the suggested restaurants and menus and make any changes, they can do so through the app. These changes are sent in real time from the device to the server, which immediately updates the suggestions and presents updated suggestions again.

[1298] Input: User change request

[1299] Output: Updated list of restaurants and menus

[1300] Specific operation: Modify the proposal content on the terminal and press the "Submit changes" button. The changes are sent to the server, which updates the database and generates and returns a new proposal list.

[1301] Step 5:

[1302] After the order is confirmed, the server automatically generates the necessary information for the user (reservation confirmation, usage guide, etc.) and sends it to the user via email or device notification, including important information and reminders.

[1303] Input: Order confirmation data

[1304] Output: Auto-generated informational list and notifications

[1305] Specific operation: Using a template engine on the server, text information such as reservation confirmations and reminders is generated and sent to the user's email address or device.

[1306] Step 6:

[1307] When a user enters their satisfaction and feedback after placing an order, the server collects and stores this information in a database. This feedback is used to generate more personalized ordering suggestions for the next time.

[1308] Input: User satisfaction rating, impressions

[1309] Output: personalized suggestions for next time

[1310] What happens: You enter your satisfaction rating and feedback on your device and press the "Submit" button. The information is sent to the server and stored in a database. This feedback is used as a reference when generating the next recommendation.

[1311] The above are the specific processing steps of the system.

[1312] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1313] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1314] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1315] [Third embodiment]

[1316] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1317] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1318] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1319] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1320] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1321] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1322] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1323] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1324] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1325] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1326] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1327] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1328] Initial Setup

[1329] A user downloads and installs the app. When the user launches the app, they are presented with a new user registration page where they enter their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.).

[1330] Examples:

[1331] The user inputs that they prefer "natural."

[1332] The server stores the entered information in a database.

[1333] Image acquisition and analysis

[1334] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage, and the server retrieves and analyzes the image. Based on the analysis results, a list of recommended tourist spots is generated.

[1335] Examples:

[1336] A user uploads a photo of Mount Fuji.

[1337] The server analyzes the image and suggests Japan, where Mount Fuji is located.

[1338] Generate a provisional itinerary

[1339] The server generates multiple provisional itineraries based on the user's information and the results of image analysis, gives them catchy titles, and automatically suggests transportation and accommodation options.

[1340] Examples:

[1341] The server titles the itinerary it generates as "7 days to enjoy Japanese culture and nature."

[1342] The server displays the itinerary to the user.

[1343] Day 1: Arrival at Haneda Airport, sightseeing in Tokyo, accommodation (Hotel ABC)

[1344] Day 2: Sightseeing in Akihabara, Ueno Zoo, and lodging at Hotel XYZ

[1345] Day 3: Travel to Hakone, hot spring experience, and accommodation (hot spring inn DEF)

[1346] Itinerary amendments and real-time updates

[1347] If the user checks the proposed itinerary and decides to change some of the tourist attractions or accommodations, they can make the changes through the app, and the server will update the changes in real time.

[1348] Examples:

[1349] The user enters that they would like to change their accommodation for the third day to a different hot spring inn.

[1350] The server generates a new itinerary and sends it to the device.

[1351] New plan:

[1352] Day 3: Travel to Hakone, hot spring experience, and accommodation (Hot Spring Inn GHI)

[1353] Providing a list of necessary items and what to bring

[1354] Once the travel plan is finalized, the server will send the necessary information and a list of what to bring to the user's email, and a real-time notification will be sent to the user's device.

[1355] Examples:

[1356] The server automatically generates an inventory list.

[1357] Send email:

[1358] passport

[1359] VISA (Japan)

[1360] JPY

[1361] Clothing (7 days' worth)

[1362] Terminal notification: "Travel in 5 days. Please check your passport and VISA."

[1363] Providing local information

[1364] Before the trip, the server collects information on local culture, trivia, precautions, etc. and notifies the user.

[1365] Examples:

[1366] The server collects information about hot spring culture.

[1367] Device notification: "Learn about Japanese hot spring culture: How to use towels and etiquette"

[1368] Post-trip evaluation and suggestions for next time

[1369] After the trip, users input their satisfaction and impressions of the trip into the app, and the server uses this information to improve the next trip plan and make new suggestions.

[1370] Examples:

[1371] The user rates their trip satisfaction as "very satisfied."

[1372] Server's new suggestion: "Why not plan your next trip to Japan? We'll suggest some new tourist spots."

[1373] As described above, the system of the present invention provides users with efficient, customized travel plans and consistently supports them from planning to execution and even next proposals.

[1374] The processing flow will be explained below.

[1375] Step 1:

[1376] The user downloads and installs the app. The device launches the app and displays a new user registration page. The user enters their name, age, gender, travel preferences, etc. The device sends the information to the server. The server stores the user information in a database.

[1377] Step 2:

[1378] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage. The server retrieves the image from cloud storage and sends it to the image analysis module. The server's image analysis module analyzes the image and generates a list of recommended tourist spots based on the analysis results.

[1379] Step 3:

[1380] The server generates multiple provisional itineraries based on the user's preferences and the results of image analysis. The server gives the generated itineraries catchy titles. The server collects information on transportation and accommodations and reflects this in the itinerary. The server sends the multiple provisional itineraries it has generated to the user's terminal for provision.

[1381] Step 4:

[1382] The user reviews the proposed itinerary and inputs changes to some of the attractions and accommodations. The device sends the changes to the server. The server receives the change request and updates the travel plan with the new elements in real time. The server sends the updated plan to the device for presentation to the user.

[1383] Step 5:

[1384] The server generates the necessary information and a list of items to bring based on the confirmed travel plan. The server sends the generated information to the user's email and also sends a notification to the device. The device displays the notification to the user.

[1385] Step 6:

[1386] The server collects information about the destination's culture, trivia, and things to be aware of. The server then sends the collected information to the user's device. The device then notifies the user of the local information.

[1387] Step 7:

[1388] After a trip, the user enters their satisfaction and impressions of the trip into the app. The device sends the input information to the server. The server stores the user's rating in a database. The server uses the rating results to improve the next travel plan and send new suggestions to the user.

[1389] Example 1

[1390] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1391] Conventional travel planning systems require users to collect a large amount of information and create plans manually, which requires time and effort. Furthermore, it is difficult to change plans once they have been created, and information cannot be updated in real time, making it difficult to flexibly respond to various situations during a trip. Furthermore, they lack the functionality to provide useful information before a trip or to reflect impressions after returning home in the next plan.

[1392] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1393] In this invention, the server includes: means for a user to input images that are similar to the user's attributes, preferences, and scenery they wish to see; means for the server to analyze the input information and images and automatically generate multiple travel plans tailored to the user; means for presenting the multiple travel plans generated by the server with catchy titles; means for the user to change parts of the travel plan and updating the changes in real time; means for the server to collect information on transportation, accommodations, and tourist spots and reflect it in the travel plan; means for the server to provide the user with local information and trivia before the trip; means for the server to suggest the next travel plan based on the user's evaluation after the trip; and means for automatically generating a list of items to pack for the trip based on the information the user registered in the system and notifying the user in real time. This enables the user to efficiently obtain a customized travel plan and receive consistent support from planning to execution and even next suggestions.

[1394] "User" means any person or entity that uses the System to make travel plans.

[1395] "Attributes" refers to personal information such as a user's name, age, and gender.

[1396] "Preferences" refers to information related to the type of travel and destinations that a user prefers.

[1397] "Scenery" refers to images related to specific views or locations that users want to see.

[1398] "Means for inputting images" refers to an interface that allows users to upload images that resemble the scenery they want to see into the system.

[1399] "Analysis" is the process by which the server processes the images uploaded by users to derive relevant tourist information.

[1400] "Travel Plan" refers to the schedule and activity details generated by the server based on the user's attributes, preferences, and image analysis results.

[1401] A "catchy title" refers to an attractive name or phrase that grabs the user's attention.

[1402] "Modifiable" means that users can make adjustments or modifications to existing travel plans.

[1403] "Means of real-time updates" refers to a mechanism whereby when a user inputs changes, the server immediately reflects those changes.

[1404] "Transportation" refers to the means of transportation used during a trip.

[1405] "Accommodation" refers to hotels, inns, etc. where users stay during their travel plans.

[1406] "Tourist Spots" refers to famous places and tourist destinations that users can visit.

[1407] "Local information and trivia" refers to cultural knowledge and points of caution about the travel destination.

[1408] "Rating" refers to the satisfaction and impressions that users enter after their trip.

[1409] A "packing list" is an automated list of items you will need for your trip.

[1410] "Means of notification" refers to the way in which the system notifies the user of important information.

[1411] The purpose of this system is to enable users to efficiently create customized travel plans and to consistently support the process from planning to execution and even future proposals. An embodiment of the system will be described below.

[1412] Initial Setup

[1413] user

[1414] First, a user downloads and installs the application. When the user launches the app, they are presented with a new user registration page where they enter their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.). Based on this information, the server creates a user profile and stores it in a database.

[1415] Specific examples

[1416] If the user indicates that they prefer "natural," the server stores this information in a database.

[1417] Image acquisition and analysis

[1418] user

[1419] Users upload an image that resembles the scenery they want to see to the app. For example, a user can upload a photo of Mt. Fuji.

[1420] Terminal

[1421] The device uploads the image data to cloud storage.

[1422] server

[1423] The server retrieves images from cloud storage and analyzes them using an AI image analysis module. Based on the analysis results, a list of recommended tourist spots is generated.

[1424] Specific examples

[1425] If a user uploads a photo of Mount Fuji, the server analyzes the image and suggests "Japan," where Mount Fuji is located.

[1426] Generate a provisional itinerary

[1427] server

[1428] The server generates multiple tentative travel plans based on the user's attribute information and the results of image analysis. Each plan is given a catchy title. Information on transportation, accommodation, and tourist spots is automatically added to the generated travel plans.

[1429] Specific examples

[1430] The tentative itinerary is titled "Seven Days to Enjoy Japanese Culture and Nature" and includes the following detailed schedule:

[1431] Day 1: Arrival at Haneda Airport, sightseeing in Tokyo, accommodation (Hotel ABC)

[1432] Day 2: Sightseeing in Akihabara, Ueno Zoo, and lodging at Hotel XYZ

[1433] Day 3: Travel to Hakone, hot spring experience, and accommodation (hot spring inn DEF)

[1434] Itinerary amendments and real-time updates

[1435] user

[1436] Users can review the proposed itinerary and, if they wish to change some of the tourist attractions or accommodations, enter the changes via the app.

[1437] server

[1438] The server receives the changes entered in real time, generates a new itinerary, and sends it to the terminal.

[1439] Specific examples

[1440] If the user requests to change their accommodation on the third day to a different hot spring inn, the server will generate a new plan and make the following changes:

[1441] Day 3: Travel to Hakone, hot spring experience, and accommodation (Hot Spring Inn GHI)

[1442] Providing a list of necessary items and what to bring

[1443] server

[1444] After the travel plan is finalized, the server automatically generates the necessary information and a list of items to bring, and sends this information to the user's email address and device in real time.

[1445] Specific examples

[1446] The server will automatically generate an inventory list and email it to the user with the following:

[1447] passport

[1448] Visa (Japan)

[1449] JPY

[1450] Clothing (7 days' worth)

[1451] Device notification: "Travel in 5 days. Please check your passport and visa."

[1452] Providing local information

[1453] server

[1454] Before the trip, the server collects information on local culture, trivia, precautions, etc. and notifies the user's device.

[1455] Specific examples

[1456] The server collects information about hot spring culture and notifies the device of the following:

[1457] "Learn about Japanese hot spring culture: How to use towels and etiquette"

[1458] Post-trip evaluation and suggestions for next time

[1459] user

[1460] After the trip, users enter their satisfaction and impressions of the trip into the app.

[1461] server

[1462] The server will use the user's ratings to improve the next travel plan and send new suggestions to the user's device.

[1463] Specific examples

[1464] If the user enters "very satisfied" as their satisfaction with the trip, the server will notify them, "Why not plan another trip to Japan? We'll suggest some new tourist spots for you."

[1465] The system of the present invention allows users to efficiently create customized travel plans and provides consistent support from execution to next proposal.

[1466] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1467] Step 1:

[1468] user

[1469] A user downloads and installs the app, then launches it for the first time after installation is complete.

[1470] Specific actions

[1471] Search for, download, and install the app from the app store.

[1472] input

[1473] Your device information and internet connection.

[1474] output

[1475] Installed apps.

[1476] Step 2:

[1477] user

[1478] A user launches the app and visits the new user registration page, where they enter their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.).

[1479] Specific actions

[1480] Enter the name "Taro Tanaka", age "30", gender "male", and travel preference "nature".

[1481] input

[1482] Your personal information and travel preferences.

[1483] output

[1484] New user registration data including the information entered.

[1485] Step 3:

[1486] server

[1487] The server receives the entered user information and stores it in a database.

[1488] Specific actions

[1489] On the server side, establish a database connection and execute a query to insert the user information.

[1490] input

[1491] New User Registration Data.

[1492] output

[1493] User information stored in a database.

[1494] Step 4:

[1495] user

[1496] Users upload images that are similar to the scenery they want to see to the app.

[1497] Specific actions

[1498] Select the image file on your device and press the "Upload" button.

[1499] input

[1500] An image file selected by the user.

[1501] output

[1502] Uploaded image data.

[1503] Step 5:

[1504] Terminal

[1505] The terminal transfers the uploaded image data to cloud storage.

[1506] Specific actions

[1507] Call the cloud storage API to upload image data.

[1508] input

[1509] Image data selected by the user.

[1510] output

[1511] The URL of the image stored in cloud storage.

[1512] Step 6:

[1513] server

[1514] The server retrieves images from cloud storage and sends them to the AI ​​image analysis module, which generates a list of recommended tourist spots based on the analysis results.

[1515] Specific actions

[1516] Call the image analysis API and obtain the analysis results.

[1517] input

[1518] Image URL in cloud storage.

[1519] output

[1520] A list of tourist destinations as a result of the analysis.

[1521] Step 7:

[1522] server

[1523] The server generates multiple provisional travel plans based on the user's attribute information and the results of image analysis, and gives each plan a catchy title.

[1524] Specific actions

[1525] A title generation algorithm is executed to give a title to the travel plan.

[1526] input

[1527] User attribute information and image analysis results.

[1528] output

[1529] Tentative travel plan list.

[1530] Step 8:

[1531] server

[1532] The server adds information about transportation, accommodation, and tourist spots to the tentative travel plan and presents it to the user.

[1533] Specific actions

[1534] Run a query to add transportation and accommodation information for each itinerary.

[1535] input

[1536] Tentative travel plan list.

[1537] output

[1538] The completed travel itinerary list.

[1539] Step 9:

[1540] user

[1541] Users can review the proposed travel plan and, if they wish to make any changes, enter the changes through the app.

[1542] Specific actions

[1543] On the travel plan screen, select the item you want to change and enter the new information.

[1544] input

[1545] User modification requests.

[1546] output

[1547] Change request data.

[1548] Step 10:

[1549] server

[1550] The server receives the changes in real time, generates a new travel plan, and sends it to the terminal.

[1551] Specific actions

[1552] It runs an algorithm to regenerate a new itinerary based on the changes and sends the updated data to the device.

[1553] input

[1554] The user's modification request data.

[1555] output

[1556] Updated travel planning data.

[1557] Step 11:

[1558] server

[1559] After the server confirms the travel plan, it automatically generates the necessary information and a list of items to bring, and sends them to the user's email address. It also sends a real-time notification to the user's device.

[1560] Specific actions

[1561] Run inventory generation algorithms and send emails and notifications.

[1562] input

[1563] Confirmed travel planning data.

[1564] output

[1565] Inventory list and notification data.

[1566] Step 12:

[1567] server

[1568] Before a trip, the server collects local information and trivia and notifies the user's device.

[1569] Specific actions

[1570] It calls the local information API to collect information and sends push notifications to the user's device.

[1571] input

[1572] Local information API data.

[1573] output

[1574] Local information notification data.

[1575] Step 13:

[1576] user

[1577] After the trip, users enter their satisfaction and impressions of the trip into the app.

[1578] Specific actions

[1579] Fill out the evaluation form in the app and press the submit button.

[1580] input

[1581] User satisfaction and opinion data.

[1582] output

[1583] Submitted evaluation data.

[1584] Step 14:

[1585] server

[1586] The server uses the user's ratings to refine the next travel plan and send new suggestions to the user.

[1587] Specific actions

[1588] The next itinerary generation algorithm is run to generate new proposals and send them to the user's device.

[1589] input

[1590] Submitted evaluation data.

[1591] output

[1592] New travel planning data.

[1593] (Application example 1)

[1594] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1595] Conventional online shopping lacks the ability to offer products tailored to individual user preferences and needs, resulting in a poor quality shopping experience. In particular, it is difficult for users to find the products and designs they specifically desire, creating a need for a hassle-free way to select a set of products that suits them.

[1596] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1597] In this invention, the server includes means for the user to input their attributes, preferences, and images similar to the products they wish to see, means for the server to analyze the input information and images and automatically generate multiple product sets tailored to the user, means for presenting the multiple product sets generated by the server with catchy titles, means for the user to change parts of the product sets and means for updating the changes in real time, and means for the server to suggest the next shopping plan based on the user's ratings. This enables product suggestions to be individually customized for the user, improving the quality of online shopping.

[1598] "User" refers to an individual who uses the system to select and purchase products.

[1599] "Attributes" refers to personal information such as a user's name, age, gender, and preferences.

[1600] "Preferences" refer to a user's tendency to be particularly interested in a particular style or category.

[1601] "Image" refers to visual data entered by the user that shows the desired product or design.

[1602] "Server" refers to the main computer system that processes user input and image data and generates product sets based on the analysis results.

[1603] "Analyzing images" refers to analyzing uploaded image data through a program and extracting relevant information based on its content.

[1604] "Means for automatically generating product sets" refers to the process of automatically selecting and combining multiple related products based on user attributes and the results of image analysis.

[1605] A "catchy title" refers to giving the generated product set a name that will attract the user's interest.

[1606] The "presentation means" refers to a method or function for visually presenting the generated product set to the user.

[1607] "Modifiable" means that users can modify parts of the suggested product set to suit their preferences.

[1608] "Means of real-time updates" refers to methods and functions that allow changes made by users to be reflected immediately on the system.

[1609] "Means to suggest next shopping plans" refers to a system that improves future shopping experiences and suggests new products based on user ratings and feedback.

[1610] A "checklist" is a list of items or tasks that a user needs to complete after their shopping plan is finalized.

[1611] "Means of notification" refers to the methods and functions by which the system notifies the user of information.

[1612] "Collecting satisfaction and feedback" refers to collecting feedback and comments provided by users as data.

[1613] "Means for improvement" refers to methods and functions for effectively improving the next proposal or the system itself based on the collected feedback.

[1614] The invention begins with the user entering their attributes, preferences, and images of products similar to the one they want to see. The user first downloads and installs the application. When the application is launched, a new user registration page appears, where the user enters their name, age, gender, and shopping preferences (e.g., fashion, home appliances, outdoor gear, etc.). This information is stored in the cloud and used to customize the shopping experience.

[1615] Next, the user uploads an image of the product they want to see to the app. The device uploads the image data to cloud storage, and the server retrieves and analyzes the image. This image analysis uses OpenCV and KMeans clustering. For example, if a user uploads an image of a modern living room, the app analyzes its color hues and suggests furniture and decoration items in a similar style.

[1616] The server generates a set of related products based on the user's information and image analysis results, and gives it a catchy title such as "Modern Living Room Set." The generated product set is displayed in the app, and the user can easily add it to their shopping list. If the user reviews the suggested product set and wants to change some of the products, the changes can be reflected in real time. For example, if the user wants to change the color of a sofa, the server will re-suggest a product list including the sofa in the new color.

[1617] Once the shopping list is finalized, the server automatically generates the necessary checklist and notifies the user, including items and tasks to be checked before purchasing. The app also provides additional information about the products, such as their background and use cases.

[1618] After making a purchase, users can enter their satisfaction and feedback on their experience into the app, which the server uses to improve their next shopping plan and make new suggestions.

[1619] The system uses the following main hardware and software: OpenCV and the KMeans clustering algorithm for image analysis, the Flask framework on the server side, MongoDB and MySQL for the database, and smtplib for sending emails.

[1620] An example of a prompt is as follows:

[1621] Example prompt sentence:

[1622] Upload an image of a modern living room to represent the product users want to see.

[1623] As a result of the analysis, the server suggests related modern style furniture (e.g. sofa, table, carpet).

[1624] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1625] Step 1:

[1626] The user downloads and installs the application. After launching it, a new user registration page appears, where the user enters their name, age, gender, and shopping preferences. This information is sent from the device to the cloud server and stored in a database. The input in this process is user information, and the output is the storage of information in the cloud database.

[1627] Step 2:

[1628] The user selects an image that is similar to the product they want to see and uploads it to the app. The device uploads the image data to cloud storage, and the server retrieves the image data. In this step, the input is the image data, and the output is saving it to cloud storage and retrieving it by the server.

[1629] Step 3:

[1630] The server analyzes the uploaded image data using OpenCV and KMeans clustering. Specifically, it extracts the hue information of the image and classifies it into multiple clusters. The input is the image data, and the output is a color palette classified by the hue information.

[1631] Step 4:

[1632] The server generates a related product set based on the color palette obtained as a result of the analysis. Using a generative AI model, it inputs the user's preference information and color palette and proposes the optimal product set. The input in this step is the analysis result and the user's preference information, and the output is the proposed product set. This product set is given a catchy title such as "Modern Living Room Set."

[1633] Step 5:

[1634] The server presents the generated product set to the user through the application. The user can review the proposal and modify parts of the product set if necessary. The modifications are sent to the server in real time, and the server generates a corresponding new product set and presents it to the user again. The input in this step is the user's modification request, and the output is the updated product set.

[1635] Step 6:

[1636] Once the user has finalized the product set, the server automatically generates a checklist, which includes items and tasks to be checked before purchasing. This checklist is sent to the user's device. The input is the finalized product set, and the output is the generated checklist.

[1637] Step 7:

[1638] After a purchase, the user enters feedback about their satisfaction and experience into the application. The server collects this feedback and uses it to improve the next shopping plan. In this step, the input is the user's feedback and the output is the improved next shopping plan.

[1639] Example prompt sentence:

[1640] Step 1: Register as a new user by entering your name, age, gender and shopping preferences.

[1641] Step 2: Upload an image that resembles the product you want to see.

[1642] Step 3: Generate a color palette based on the results of image analysis.

[1643] Step 4: Propose a product set based on the color palette and user preference information.

[1644] Step 5: User reviews the proposed product set and makes changes if necessary.

[1645] Step 6: Automatically generate a checklist after the product set is confirmed.

[1646] Step 7: Collect post-purchase feedback and refine your next plan.

[1647] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1648] Initial Setup

[1649] The user downloads and installs the app. The device launches the app and displays a new user registration page. The user enters their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.). The app also performs initial setup to recognize the user's emotions.

[1650] Examples:

[1651] The user inputs that they prefer "natural."

[1652] The server stores the entered information in a database.

[1653] Image input and emotion analysis

[1654] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage, and the server retrieves the image and analyzes the image and the user's input information using an emotion engine. Based on the analysis results, a list of recommended tourist spots is generated and the user's emotional state is recorded.

[1655] Examples:

[1656] A user uploads a photo of Mount Fuji.

[1657] The server analyzes the image and suggests Japan.

[1658] The emotion engine recognizes the user's emotion of joy.

[1659] Generate a provisional itinerary

[1660] The server generates multiple provisional itineraries based on the user's information, image analysis, and sentiment analysis results. It then gives each itinerary a catchy title and automatically suggests transportation and accommodation options.

[1661] Examples:

[1662] The server titles the itinerary it generates as "7 days to enjoy Japanese culture and nature."

[1663] The server displays the itinerary to the user.

[1664] Day 1: Arrival at Haneda Airport, sightseeing in Tokyo, accommodation (Hotel ABC)

[1665] Day 2: Sightseeing in Akihabara, Ueno Zoo, and lodging at Hotel XYZ

[1666] Day 3: Travel to Hakone, hot spring experience, and accommodation (hot spring inn DEF)

[1667] Itinerary amendments and real-time updates

[1668] If a user reviews the proposed itinerary and decides to change some of the tourist attractions or accommodations, they can make the changes through the app, and the server will update the changes in real time.In addition, the emotion engine recognizes the user's real-time emotional state and automatically adjusts the itinerary accordingly.

[1669] Examples:

[1670] The user enters that they would like to change their accommodation for the third day to a different hot spring inn.

[1671] The server generates a new itinerary and sends it to the device.

[1672] The emotion engine recognizes the user's emotions and proposes the optimal plan.

[1673] New plan:

[1674] Day 3: Travel to Hakone, hot spring experience, and accommodation (Hot Spring Inn GHI)

[1675] Providing a list of necessary items and what to bring

[1676] Once the travel plan is finalized, the server will send the necessary information and a list of what to bring to the user's email, and a real-time notification will be sent to the user's device.

[1677] Examples:

[1678] The server automatically generates an inventory list.

[1679] Send email:

[1680] passport

[1681] VISA (Japan)

[1682] JPY

[1683] Clothing (7 days' worth)

[1684] Terminal notification: "Travel in 5 days. Please check your passport and VISA."

[1685] Providing local information

[1686] Before the trip, the server collects information on local culture, trivia, precautions, etc. and notifies the user.

[1687] Examples:

[1688] The server collects information about hot spring culture.

[1689] Device notification: "Learn about Japanese hot spring culture: How to use towels and etiquette"

[1690] Emotion monitoring and optimization during travel

[1691] During the trip, the emotion engine continuously monitors the user's emotions and optimizes the travel experience based on the emotional data. If the user is enjoying themselves, it will make suggestions to maintain that state, and if they are feeling stressed, it will suggest relaxing activities.

[1692] Examples:

[1693] The emotion engine recognizes the user's level of enjoyment.

[1694] Device notification: "There's a recommended cafe nearby. Would you like to relax?"

[1695] Post-trip evaluation and suggestions for next time

[1696] After a trip, users input their satisfaction and impressions of the trip into the app. The server uses this information to improve the next trip plan and make new suggestions. The emotion engine also utilizes emotional data from the trip to make more personalized suggestions.

[1697] Examples:

[1698] The user rates their trip satisfaction as "very satisfied."

[1699] The server makes the next suggestion based on the emotion data.

[1700] New suggestion: "Why not plan your next trip to Japan? We'll suggest some new tourist spots."

[1701] As described above, the system of the present invention efficiently acquires customized travel plans for users and provides consistent support from planning to execution and even next-time suggestions. By combining it with an emotion engine, it is possible to provide an optimal travel experience that reflects the user's real-time emotional state.

[1702] The processing flow will be explained below.

[1703] Step 1:

[1704] The user downloads and installs the app. The device launches the app and displays the new user registration page. The user enters their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.) to initialize the emotion engine. The device sends the input information to the server. The server stores the user information in a database.

[1705] Step 2:

[1706] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage. The server retrieves the image from cloud storage and sends it to the image analysis module and emotion engine. The server's image analysis module analyzes the image, and the emotion engine recognizes the user's emotional state. A list of recommended tourist spots is generated based on the analysis results and emotion data.

[1707] Step 3:

[1708] The server generates multiple provisional itineraries based on user information, image analysis, and sentiment analysis. It then gives each itinerary a catchy title. The server then collects information on transportation and accommodations and incorporates this information into the itinerary. The multiple provisional itineraries generated by the server are then sent to the user's device for provision.

[1709] Step 4:

[1710] The user reviews the proposed itinerary and inputs changes to some of the tourist attractions and accommodations. The device sends the changes to the server. The server receives the change request and updates the travel plan with the new elements in real time. The emotion engine recognizes the user's real-time emotional state and automatically adjusts the itinerary accordingly. The server sends the updated plan to the device for presentation to the user.

[1711] Step 5:

[1712] The server generates the necessary information and a list of items to bring based on the confirmed travel plan. The server sends the generated information to the user's email and also sends a notification to the device. The device displays the notification to the user.

[1713] Step 6:

[1714] The server collects information about the destination's culture, trivia, and things to be aware of. The server then sends the collected information to the user's device. The device then notifies the user of the local information.

[1715] Step 7:

[1716] During the trip, the emotion engine continuously monitors the user's emotions. The server makes suggestions to optimize the travel experience based on the emotion data. The device then notifies the user of the optimized information.

[1717] Step 8:

[1718] After a trip, the user enters their satisfaction and impressions of the trip into the app. The device sends the input information to the server. The server stores the user's ratings in a database. The emotion engine utilizes the emotional data from the trip, and the server sends the user new suggestions to improve their next travel plan.

[1719] Example 2

[1720] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1721] Conventional travel planning support systems have difficulty providing travel plans that fully reflect the user's preferences and emotions. It is also difficult to adjust plans in response to real-time emotional changes during travel. This has resulted in the issue of users' travel experiences not being optimized.

[1722] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1723] In this invention, the server includes a means for a user to input images that are similar to their attributes, preferences, and scenery they want to see, a means for analyzing the input information and images using a generative AI model and automatically generating multiple itineraries tailored to the user, and a means for monitoring the user's emotional state during the trip using an emotion engine to optimize the travel experience. This makes it possible to provide a personalized travel plan that reflects the user's preferences and emotions, and to adjust the plan to suit real-time emotional changes during the trip.

[1724] "User" means an individual who intends to use the System to create a travel plan.

[1725] "Attributes" refers to personal information such as a user's name, age, and gender.

[1726] "Preferences" refers to the user's preferred travel style and purpose, such as nature, culture, adventure, etc.

[1727] "Images similar to the scenery you want to see" are images that express the image of scenery or places you want to see as a travel destination.

[1728] A "server" is a computer system that analyzes data and generates travel plans based on information provided by users.

[1729] A "generative AI model" is an algorithm that uses artificial intelligence to analyze input information and images and generate the optimal travel plan for the user.

[1730] An "emotion engine" is a technology that recognizes the user's emotional state and uses it as data for analysis.

[1731] An "itinerary" is a detailed plan that lists the travel schedule and places to visit.

[1732] "Title" is a catchphrase or description that is given to the generated itinerary.

[1733] "Transportation" refers to the means of transportation used during a trip (e.g., plane, train, bus).

[1734] "Accommodation" refers to the place where you stay during your trip, such as a hotel or inn.

[1735] "Tourist attractions" refer to places of interest and activities that you can visit while traveling.

[1736] A "packing list" is a list of items you will need while traveling.

[1737] "Local information" refers to information about the culture, customs, and precautions of the destination.

[1738] "User rating" refers to the satisfaction and impressions that users enter into the system after their trip.

[1739] "Next Travel Plan" is a plan for your next trip that is suggested based on the user's past ratings and emotional data.

[1740] This invention is a system that provides a personalized travel plan that reflects the user's preferences and emotions, and adjusts the plan optimally in response to real-time changes in emotions during the trip. This system operates in cooperation with the user, terminal, and server.

[1741] Initial Setup

[1742] First, the user downloads and installs the application. The device launches the application and displays a new user registration page. The user enters their name, age, gender, and travel preferences (nature, culture, adventure, etc.). They also enter data for emotion recognition as part of the initial setup.

[1743] Image input and emotion analysis

[1744] The user uploads an image that is close to the scenery they want to see to the application. The device then uploads this image data to cloud storage (e.g., AWS S3). The server retrieves the image from cloud storage and uses a generative AI model to analyze the image and the user's input information. For example, if a user uploads a photo of Mt. Fuji, the server uses a generative AI model (e.g., AWS Rekognition) to recommend "Japan," and the emotion engine recognizes the user's emotion of joy.

[1745] Generate a provisional itinerary

[1746] The server generates multiple provisional itineraries based on user information, image analysis, and sentiment analysis. It also gives each itinerary a catchy title and automatically suggests transportation and accommodation options. For example, the server might title the itinerary "Seven days to fully enjoy Japanese culture and nature" and suggest a specific itinerary.

[1747] Itinerary amendments and real-time updates

[1748] If a user reviews the proposed itinerary and wants to change some of the tourist attractions or accommodations, they can input the changes through the application, and the server will update the changes in real time. The emotion engine also recognizes the user's real-time emotional state and automatically adjusts the itinerary accordingly.

[1749] Providing a list of necessary items and what to bring

[1750] Once the travel plan is finalized, the server automatically generates a list of items to bring and sends it to the user's email. A real-time notification is also sent to the device. For example, the server generates a list of items to bring, such as "passport, VISA, Japanese yen, and clothing (for 7 days)," and notifies the user.

[1751] Providing local information

[1752] Before a trip, the server collects information about local culture, customs, and precautions, and notifies the user. For example, it provides information such as "Learn about Japanese hot spring culture: how to use towels and etiquette."

[1753] Emotion monitoring and optimization during travel

[1754] During the trip, the emotion engine continuously monitors the user's emotions and optimizes the travel experience based on the emotional data. For example, it can make suggestions such as, "There's a recommended cafe nearby. Why not relax there?" to optimize the user's travel experience.

[1755] Post-trip evaluation and suggestions for next time

[1756] After the trip, the user enters their satisfaction and impressions of the trip into the application. Based on this, the server will improve the next travel plan and make new suggestions. For example, it may say, "Why not plan another trip to Japan next time? We'll suggest some new tourist spots."

[1757] This system allows users to efficiently obtain customized travel plans and provides consistent support from planning to execution, as well as future proposals. By utilizing an emotion engine, it is possible to provide an optimal travel experience that reflects the user's real-time emotional state.

[1758] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1759] Step 1: Download and install the app

[1760] The user downloads and installs the application.

[1761] Input: Download link from the app store.

[1762] Output: Application installed on the device.

[1763] Specific behavior:

[1764] A user opens an app store and searches for an application.

[1765] Click on the application to begin the download.

[1766] The device installs the application.

[1767] Step 2: First launch and enter user information

[1768] The device launches the installed application and displays the new user registration page.

[1769] Input: User information (name, age, gender, travel preferences).

[1770] Output: Sending user information to the server.

[1771] Specific behavior:

[1772] The device launches the application.

[1773] Users enter their name, age, gender, and travel preferences.

[1774] The terminal transmits the input information to the server.

[1775] Step 3: Save user information

[1776] The server stores the entered user information in a database.

[1777] Input: User information (name, age, gender, travel preferences).

[1778] Output: User information stored in the database.

[1779] Specific behavior:

[1780] The server analyzes the received user information.

[1781] The server stores the analyzed information in a database.

[1782] Step 4: Upload an image

[1783] Users upload an image that is close to the view they want to see to the application.

[1784] Input: Image file.

[1785] Output: Image files saved in cloud storage.

[1786] Specific behavior:

[1787] The user selects an image of the scenery they want to see.

[1788] The device uploads the selected images to cloud storage.

[1789] Step 5: Image acquisition and analysis

[1790] The server retrieves the images from cloud storage and analyzes them using a generative AI model.

[1791] Input: Image files in cloud storage, user information.

[1792] Output: Analyzed image information and tourist spot suggestions.

[1793] Specific behavior:

[1794] The server downloads the image from the cloud storage.

[1795] The server analyzes the image using a generative AI model (e.g., AWS Rekognition).

[1796] Based on the analysis results, appropriate tourist destinations are suggested.

[1797] Step 6: Sentiment Analysis

[1798] The emotion engine recognizes the user's emotions based on the analysis results.

[1799] Input: Analyzed image information, user information.

[1800] Output: Recognized user emotion data.

[1801] Specific behavior:

[1802] The server transmits the analysis results to the emotion engine.

[1803] The emotion engine analyzes the user's emotions.

[1804] Record your emotional state in a database.

[1805] Step 7: Generate a provisional itinerary

[1806] The server generates multiple provisional itineraries based on user information, image analysis, and emotion analysis results.

[1807] Input: User information, analysis results, emotion data.

[1808] Output: Multiple provisional itineraries generated.

[1809] Specific behavior:

[1810] The server generates multiple provisional itineraries based on the input data.

[1811] Give the generated itinerary a catchy title.

[1812] Step 8: View your itinerary

[1813] The server generates an itinerary and displays it to the user.

[1814] Input: Multiple generated provisional itineraries.

[1815] Output: The itinerary displayed on the terminal.

[1816] Specific behavior:

[1817] The server sends the itinerary details to the terminal and displays them to the user.

[1818] The user reviews the displayed itinerary.

[1819] Step 9: Itinerary Modifications and Real-Time Updates

[1820] The user reviews the proposed itinerary and changes some of the attractions and accommodations.

[1821] Input: The user's correction request.

[1822] Output: The revised itinerary.

[1823] Specific behavior:

[1824] A user enters a request to change travel plans.

[1825] The server analyzes the changes and generates a new itinerary in real time.

[1826] Step 10: Provide a packing list

[1827] The server automatically generates an inventory list and sends it to the user by email. It also notifies the user on their device.

[1828] Input: Confirmed travel plans.

[1829] Output: Inventory list.

[1830] Specific behavior:

[1831] The server generates a list of necessary items to bring based on the travel plan.

[1832] The list of belongings will be sent via email and notified to the device.

[1833] Step 11: Provide local information

[1834] The server collects information about the culture of the destination, trivia, and things to be aware of, and notifies the user.

[1835] Input: Travel destination information.

[1836] Output: Local information notification.

[1837] Specific behavior:

[1838] The server collects local information from the Internet.

[1839] The collected information is sent to the user's device.

[1840] Step 12: Emotional monitoring and optimization during the trip

[1841] The emotion engine continuously monitors user emotions to optimize the travel experience.

[1842] Input: Real-time emotion data.

[1843] Output: Optimized trip plan.

[1844] Specific behavior:

[1845] The emotion engine uses smartphone sensor data to monitor emotions.

[1846] The server analyzes the emotional data and optimizes the travel experience in real time.

[1847] Your device will display notifications suggesting the best activities for you.

[1848] Step 13: Post-trip evaluation and suggestions for next time

[1849] The user enters their satisfaction and impressions of the trip into the app, and the server suggests plans for the next trip.

[1850] Input: Travel satisfaction and impressions.

[1851] Output: Next trip suggestions.

[1852] Specific behavior:

[1853] Users enter their impressions after the trip.

[1854] The server analyzes the data and makes suggestions for your next trip.

[1855] (Application example 2)

[1856] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1857] Conventional food delivery services struggled to provide optimal recommendations based on users' emotions and preferences, forcing them to spend time selecting menu items. Furthermore, there were limited ways to obtain detailed information about dishes or specific restaurants in advance, making it difficult for users to place highly satisfying orders. Furthermore, there was no system in place to utilize feedback after an order to personalize the next order.

[1858] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for the user to input their attributes, preferences, and images similar to the food they want to eat, a means for the server to analyze the input information and images and automatically generate multiple restaurants and menus tailored to the user, and a means for automatically generating and notifying the user of an information provision list required after the user confirms their order. This makes it possible to make suggestions optimized to the user's emotions and preferences.

[1859] "User" means an individual who uses the application to deliver food.

[1860] "Attributes" are a user's age, gender, and other basic profile information.

[1861] "Preferences" refers to the user's food preferences and favorite food categories (e.g., Japanese food, Italian food, etc.).

[1862] "Images similar to the food you want to eat" are images that users upload to the app that represent the food they want to eat that day or at that moment.

[1863] A "server" is a computer system that analyzes information obtained from users and generates and manages delivery proposals.

[1864] "Analyzing information" is the process of processing data based on attributes, preferences, and image data obtained from users to generate optimal suggestions.

[1865] "Automatically generating restaurants and menus" means selecting the most suitable restaurants and delivery menus for users based on analyzed information.

[1866] A "catchy title" means giving the proposed restaurant or menu an attention-grabbing title.

[1867] "Suggestion" means presenting a list of restaurants and menus that are deemed most suitable for the user.

[1868] "Modifiable" means that the user has the ability to adjust or change the suggested restaurant or menu item themselves.

[1869] "Real-time updates" means that changes made by the user are reflected immediately and the latest information is provided.

[1870] "Method of food delivery" refers to the method or means by which the food ordered by the user is delivered to the specified location.

[1871] "Information about the food provided" refers to detailed information about the food ordered by the user (e.g., ingredients, calories, allergen information, etc.).

[1872] A "rating" is feedback a user gives about the service or food provided after placing an order.

[1873] "Make next recommendation" means creating a personalized recommendation for your next order based on the collected ratings and other data.

[1874] The "information list" is a list of detailed information (e.g., confirmation items, usage guide, etc.) that the user will need after confirming their order.

[1875] "Notify" means sending important information or reminders to users to make sure they know.

[1876] The system for implementing this invention consists of an application that provides optimal food delivery suggestions based on the user's preferences and emotions. The system uses a smartphone as its primary hardware platform and the following software components:

[1877] Hardware used:

[1878] Smartphone (iOS or Android device)

[1879] Software used:

[1880] Mobile application frameworks (e.g., React Native)

[1881] Cloud storage (AWS S3)

[1882] Sentiment analysis engine (Google Cloud Vision API, Microsoft Azure Face API, etc.)

[1883] Backend server (Node.js, Express)

[1884] Database (MongoDB)

[1885] Detailed process flow:

[1886] User Registration:

[1887] After a user downloads and installs the application, they register as a new user. They enter their name, age, gender, and food preference (e.g., Japanese, Italian, etc.). This user information is collected using a mobile application framework and sent to the backend server, which stores this information in a database (MongoDB).

[1888] Examples:

[1889] If a user inputs that they like "Japanese food," the server stores this information in a database.

[1890] Example prompt sentence:

[1891] "You love Japanese food! Next, upload a photo of the dish you want to eat."

[1892] Image capture and sentiment analysis:

[1893] Users upload an image of a dish they want to eat to the app. The image is stored in cloud storage (AWS S3), and its URL is sent to the backend server. The server uses the Google Cloud Vision API to analyze the image and recognize the user's emotional state.

[1894] Examples:

[1895] If a user uploads a photo of sushi, the server analyzes the image, recognizes the user's emotions, and suggests sushi restaurants.

[1896] Example prompt sentence:

[1897] "Analyzing... Please wait a moment. We've found some recommended sushi restaurants!"

[1898] Auto-generate and suggest restaurants and menus:

[1899] The server then generates multiple restaurant and menu options based on the analysis results, taking into account the user's preferences and emotional data. The options are then presented to the user with catchy titles.

[1900] Examples:

[1901] The server will recommend restaurants to users under titles such as "Three Places Where You Can Enjoy Authentic Sushi," and each restaurant will display its features and recommended menu items.

[1902] Example prompt sentence:

[1903] "We've picked out three places where you can enjoy authentic sushi. Check them out!"

[1904] Order confirmation and real-time changes:

[1905] Users can review the suggested restaurants and menu items and make any changes they want to make through the app, which will then update the server in real time and present the latest suggestions.

[1906] Examples:

[1907] If the user indicates that they would like to change the second restaurant, the server generates a new suggestion and sends it to the device.

[1908] Example prompt sentence:

[1909] "We've updated our settings. Here are our new suggestions."

[1910] Pre-order information and notices:

[1911] After the order is confirmed, the server automatically generates and notifies the user of necessary information (e.g., reservation confirmation, usage guide, etc.), including important information and reminders.

[1912] Examples:

[1913] The server automatically generates a list of items to bring and usage guidelines, and sends them to the user via email or device notification.

[1914] Example prompt sentence:

[1915] "Reservation confirmation: Sushi Dai 7:00 PM~" "15-minute notice: Your reservation time for Sushi Dai is approaching."

[1916] Gathering feedback and suggestions for next time:

[1917] After the order is completed, the server collects the user's satisfaction and feedback, and based on this, generates personalized suggestions for the next order.

[1918] Examples:

[1919] If the user rates the trip as "very satisfied," the server will refine the next travel plan and make new suggestions.

[1920] Example prompt sentence:

[1921] "Would you like to enjoy sushi again next time? We'll suggest a nearby sushi restaurant."

[1922] This allows users to have a food delivery experience that is optimized to their emotions and preferences.

[1923] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1924] Step 1:

[1925] A user downloads the application and registers as a new user. They enter their name, age, gender, and food preferences. This user information is collected through an input form on the device and sent to the backend server. The server receives this information and stores it in a database (MongoDB).

[1926] Input: User's name, age, gender, food preferences

[1927] Output: User information stored in the database

[1928] Specific operation: After entering user information on the terminal and pressing the "Submit" button, the information is sent to the backend server, which receives it and stores it in the database.

[1929] Step 2:

[1930] The user uploads an image of a dish they want to eat to the app. The device then uploads the image to cloud storage (AWS S3) and sends the image URL to the backend server. The server receives the URL and uses the Google Cloud Vision API to analyze the image and recognize the user's emotion.

[1931] Input: Image of the food you want to eat

[1932] Output: Parsed emotion data

[1933] Specific operation: Press the "Upload Image" button on the device, select and upload an image. The URL of the uploaded image is sent to the backend server and passed to the Google Cloud Vision API. The API analyzes the image and returns the results to the server.

[1934] Step 3:

[1935] The server then generates multiple restaurant and menu options based on the analysis results, using data from existing restaurant databases and online sources. Taking into account the user's preferences and emotional data, the server selects the most suitable restaurant and menu options and suggests them to the user with catchy titles.

[1936] Input: User preference data, emotion analysis results

[1937] Output: A list of multiple restaurants and their menus

[1938] Specific operation: Query the database on the server to obtain relevant restaurant and menu information. Based on the obtained data, generate suitable suggestions for the user and send them back to the application.

[1939] Step 4:

[1940] If the user wants to review the suggested restaurants and menus and make any changes, they can do so through the app. These changes are sent in real time from the device to the server, which immediately updates the suggestions and presents updated suggestions again.

[1941] Input: User change request

[1942] Output: Updated list of restaurants and menus

[1943] Specific operation: Modify the proposal content on the terminal and press the "Submit changes" button. The changes are sent to the server, which updates the database and generates and returns a new proposal list.

[1944] Step 5:

[1945] After the order is confirmed, the server automatically generates the necessary information for the user (reservation confirmation, usage guide, etc.) and sends it to the user via email or device notification, including important information and reminders.

[1946] Input: Order confirmation data

[1947] Output: Auto-generated informational list and notifications

[1948] Specific operation: Using a template engine on the server, text information such as reservation confirmations and reminders is generated and sent to the user's email address or device.

[1949] Step 6:

[1950] When a user enters their satisfaction and feedback after placing an order, the server collects and stores this information in a database. This feedback is used to generate more personalized ordering suggestions for the next time.

[1951] Input: User satisfaction rating, impressions

[1952] Output: personalized suggestions for next time

[1953] What happens: You enter your satisfaction rating and feedback on your device and press the "Submit" button. The information is sent to the server and stored in a database. This feedback is used as a reference when generating the next recommendation.

[1954] The above are the specific processing steps of the system.

[1955] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1956] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1957] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1958] [Fourth embodiment]

[1959] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1960] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1961] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1962] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1963] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1964] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1965] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1966] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1967] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1968] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1969] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1970] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1971] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1972] Initial Setup

[1973] A user downloads and installs the app. When the user launches the app, they are presented with a new user registration page where they enter their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.).

[1974] Examples:

[1975] The user inputs that they prefer "natural."

[1976] The server stores the entered information in a database.

[1977] Image acquisition and analysis

[1978] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage, and the server retrieves and analyzes the image. Based on the analysis results, a list of recommended tourist spots is generated.

[1979] Examples:

[1980] A user uploads a photo of Mount Fuji.

[1981] The server analyzes the image and suggests Japan, where Mount Fuji is located.

[1982] Generate a provisional itinerary

[1983] The server generates multiple provisional itineraries based on the user's information and the results of image analysis, gives them catchy titles, and automatically suggests transportation and accommodation options.

[1984] Examples:

[1985] The server titles the itinerary it generates as "7 days to enjoy Japanese culture and nature."

[1986] The server displays the itinerary to the user.

[1987] Day 1: Arrival at Haneda Airport, sightseeing in Tokyo, accommodation (Hotel ABC)

[1988] Day 2: Sightseeing in Akihabara, Ueno Zoo, and lodging at Hotel XYZ

[1989] Day 3: Travel to Hakone, hot spring experience, and accommodation (hot spring inn DEF)

[1990] Itinerary amendments and real-time updates

[1991] If the user checks the proposed itinerary and decides to change some of the tourist attractions or accommodations, they can make the changes through the app, and the server will update the changes in real time.

[1992] Examples:

[1993] The user enters that they would like to change their accommodation for the third day to a different hot spring inn.

[1994] The server generates a new itinerary and sends it to the device.

[1995] New plan:

[1996] Day 3: Travel to Hakone, hot spring experience, and accommodation (Hot Spring Inn GHI)

[1997] Providing a list of necessary items and what to bring

[1998] Once the travel plan is finalized, the server will send the necessary information and a list of what to bring to the user's email, and a real-time notification will be sent to the user's device.

[1999] Examples:

[2000] The server automatically generates an inventory list.

[2001] Send email:

[2002] passport

[2003] VISA (Japan)

[2004] JPY

[2005] Clothing (7 days' worth)

[2006] Terminal notification: "Travel in 5 days. Please check your passport and VISA."

[2007] Providing local information

[2008] Before the trip, the server collects information on local culture, trivia, precautions, etc. and notifies the user.

[2009] Examples:

[2010] The server collects information about hot spring culture.

[2011] Device notification: "Learn about Japanese hot spring culture: How to use towels and etiquette"

[2012] Post-trip evaluation and suggestions for next time

[2013] After the trip, users input their satisfaction and impressions of the trip into the app, and the server uses this information to improve the next trip plan and make new suggestions.

[2014] Examples:

[2015] The user rates their trip satisfaction as "very satisfied."

[2016] Server's new suggestion: "Why not plan your next trip to Japan? We'll suggest some new tourist spots."

[2017] As described above, the system of the present invention provides users with efficient, customized travel plans and consistently supports them from planning to execution and even next proposals.

[2018] The processing flow will be explained below.

[2019] Step 1:

[2020] The user downloads and installs the app. The device launches the app and displays a new user registration page. The user enters their name, age, gender, travel preferences, etc. The device sends the information to the server. The server stores the user information in a database.

[2021] Step 2:

[2022] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage. The server retrieves the image from cloud storage and sends it to the image analysis module. The server's image analysis module analyzes the image and generates a list of recommended tourist spots based on the analysis results.

[2023] Step 3:

[2024] The server generates multiple provisional itineraries based on the user's preferences and the results of image analysis. The server gives the generated itineraries catchy titles. The server collects information on transportation and accommodations and reflects this in the itinerary. The server sends the multiple provisional itineraries it has generated to the user's terminal for provision.

[2025] Step 4:

[2026] The user reviews the proposed itinerary and inputs changes to some of the attractions and accommodations. The device sends the changes to the server. The server receives the change request and updates the travel plan with the new elements in real time. The server sends the updated plan to the device for presentation to the user.

[2027] Step 5:

[2028] The server generates the necessary information and a list of items to bring based on the confirmed travel plan. The server sends the generated information to the user's email and also sends a notification to the device. The device displays the notification to the user.

[2029] Step 6:

[2030] The server collects information about the destination's culture, trivia, and things to be aware of. The server then sends the collected information to the user's device. The device then notifies the user of the local information.

[2031] Step 7:

[2032] After a trip, the user enters their satisfaction and impressions of the trip into the app. The device sends the input information to the server. The server stores the user's rating in a database. The server uses the rating results to improve the next travel plan and send new suggestions to the user.

[2033] Example 1

[2034] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2035] Conventional travel planning systems require users to collect a large amount of information and create plans manually, which requires time and effort. Furthermore, it is difficult to change plans once they have been created, and information cannot be updated in real time, making it difficult to flexibly respond to various situations during a trip. Furthermore, they lack the functionality to provide useful information before a trip or to reflect impressions after returning home in the next plan.

[2036] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2037] In this invention, the server includes: means for a user to input images that are similar to the user's attributes, preferences, and scenery they wish to see; means for the server to analyze the input information and images and automatically generate multiple travel plans tailored to the user; means for presenting the multiple travel plans generated by the server with catchy titles; means for the user to change parts of the travel plan and updating the changes in real time; means for the server to collect information on transportation, accommodations, and tourist spots and reflect it in the travel plan; means for the server to provide the user with local information and trivia before the trip; means for the server to suggest the next travel plan based on the user's evaluation after the trip; and means for automatically generating a list of items to pack for the trip based on the information the user registered in the system and notifying the user in real time. This enables the user to efficiently obtain a customized travel plan and receive consistent support from planning to execution and even next suggestions.

[2038] "User" means any person or entity that uses the System to make travel plans.

[2039] "Attributes" refers to personal information such as a user's name, age, and gender.

[2040] "Preferences" refers to information related to the type of travel and destinations that a user prefers.

[2041] "Scenery" refers to images related to specific views or locations that users want to see.

[2042] "Means for inputting images" refers to an interface that allows users to upload images that resemble the scenery they want to see into the system.

[2043] "Analysis" is the process by which the server processes the images uploaded by users to derive relevant tourist information.

[2044] "Travel Plan" refers to the schedule and activity details generated by the server based on the user's attributes, preferences, and image analysis results.

[2045] A "catchy title" refers to an attractive name or phrase that grabs the user's attention.

[2046] "Modifiable" means that users can make adjustments or modifications to existing travel plans.

[2047] "Means of real-time updates" refers to a mechanism whereby when a user inputs changes, the server immediately reflects those changes.

[2048] "Transportation" refers to the means of transportation used during a trip.

[2049] "Accommodation" refers to hotels, inns, etc. where users stay during their travel plans.

[2050] "Tourist Spots" refers to famous places and tourist destinations that users can visit.

[2051] "Local information and trivia" refers to cultural knowledge and points of caution about the travel destination.

[2052] "Rating" refers to the satisfaction and impressions that users enter after their trip.

[2053] A "packing list" is an automated list of items you will need for your trip.

[2054] "Means of notification" refers to the way in which the system notifies the user of important information.

[2055] The purpose of this system is to enable users to efficiently create customized travel plans and to consistently support the process from planning to execution and even future proposals. An embodiment of the system will be described below.

[2056] Initial Setup

[2057] user

[2058] First, a user downloads and installs the application. When the user launches the app, they are presented with a new user registration page where they enter their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.). Based on this information, the server creates a user profile and stores it in a database.

[2059] Specific examples

[2060] If the user indicates that they prefer "natural," the server stores this information in a database.

[2061] Image acquisition and analysis

[2062] user

[2063] Users upload an image that resembles the scenery they want to see to the app. For example, a user can upload a photo of Mt. Fuji.

[2064] Terminal

[2065] The device uploads the image data to cloud storage.

[2066] server

[2067] The server retrieves images from cloud storage and analyzes them using an AI image analysis module. Based on the analysis results, a list of recommended tourist spots is generated.

[2068] Specific examples

[2069] If a user uploads a photo of Mount Fuji, the server analyzes the image and suggests "Japan," where Mount Fuji is located.

[2070] Generate a provisional itinerary

[2071] server

[2072] The server generates multiple provisional travel plans based on the user's attribute information and the results of image analysis. Each plan is given a catchy title. Information on transportation, accommodation, and tourist spots is automatically added to the generated travel plans.

[2073] Specific examples

[2074] The tentative itinerary is titled "Seven Days to Enjoy Japanese Culture and Nature" and includes the following detailed schedule:

[2075] Day 1: Arrival at Haneda Airport, sightseeing in Tokyo, accommodation (Hotel ABC)

[2076] Day 2: Sightseeing in Akihabara, Ueno Zoo, and lodging at Hotel XYZ

[2077] Day 3: Travel to Hakone, hot spring experience, and accommodation (hot spring inn DEF)

[2078] Itinerary amendments and real-time updates

[2079] user

[2080] Users can review the proposed itinerary and, if they wish to change some of the tourist attractions or accommodations, enter the changes via the app.

[2081] server

[2082] The server receives the changes entered in real time, generates a new itinerary, and sends it to the terminal.

[2083] Specific examples

[2084] If the user requests to change their accommodation on the third day to a different hot spring inn, the server will generate a new plan and make the following changes:

[2085] Day 3: Travel to Hakone, hot spring experience, and accommodation (Hot Spring Inn GHI)

[2086] Providing a list of necessary items and what to bring

[2087] server

[2088] After the travel plan is finalized, the server automatically generates the necessary information and a list of items to bring, and sends this information to the user's email address and device in real time.

[2089] Specific examples

[2090] The server will automatically generate an inventory list and email it to the user with the following:

[2091] passport

[2092] Visa (Japan)

[2093] JPY

[2094] Clothing (7 days' worth)

[2095] Device notification: "Travel in 5 days. Please check your passport and visa."

[2096] Providing local information

[2097] server

[2098] Before the trip, the server collects information on local culture, trivia, precautions, etc. and notifies the user's device.

[2099] Specific examples

[2100] The server collects information about hot spring culture and notifies the device of the following:

[2101] "Learn about Japanese hot spring culture: How to use towels and etiquette"

[2102] Post-trip evaluation and suggestions for next time

[2103] user

[2104] After the trip, users enter their satisfaction and impressions of the trip into the app.

[2105] server

[2106] The server will use the user's ratings to improve the next travel plan and send new suggestions to the user's device.

[2107] Specific examples

[2108] If the user enters "very satisfied" as their satisfaction with the trip, the server will notify them, "Why not plan another trip to Japan? We'll suggest some new tourist spots for you."

[2109] The system of the present invention allows users to efficiently create customized travel plans and provides consistent support from execution to next proposal.

[2110] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2111] Step 1:

[2112] user

[2113] A user downloads and installs the app, then launches it for the first time after installation is complete.

[2114] Specific actions

[2115] Search for, download, and install the app from the app store.

[2116] input

[2117] Your device information and internet connection.

[2118] output

[2119] Installed apps.

[2120] Step 2:

[2121] user

[2122] A user launches the app and visits the new user registration page, where they enter their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.).

[2123] Specific actions

[2124] Enter the name "Taro Tanaka", age "30", gender "male", and travel preference "nature".

[2125] input

[2126] Your personal information and travel preferences.

[2127] output

[2128] New user registration data including the information entered.

[2129] Step 3:

[2130] server

[2131] The server receives the entered user information and stores it in a database.

[2132] Specific actions

[2133] On the server side, establish a database connection and execute a query to insert the user information.

[2134] input

[2135] New User Registration Data.

[2136] output

[2137] User information stored in a database.

[2138] Step 4:

[2139] user

[2140] Users upload images that are similar to the scenery they want to see to the app.

[2141] Specific actions

[2142] Select the image file on your device and press the "Upload" button.

[2143] input

[2144] An image file selected by the user.

[2145] output

[2146] Uploaded image data.

[2147] Step 5:

[2148] Terminal

[2149] The terminal transfers the uploaded image data to cloud storage.

[2150] Specific actions

[2151] Call the cloud storage API to upload image data.

[2152] input

[2153] Image data selected by the user.

[2154] output

[2155] The URL of the image stored in cloud storage.

[2156] Step 6:

[2157] server

[2158] The server retrieves images from cloud storage and sends them to the AI ​​image analysis module, which generates a list of recommended tourist spots based on the analysis results.

[2159] Specific actions

[2160] Call the image analysis API and obtain the analysis results.

[2161] input

[2162] Image URL in cloud storage.

[2163] output

[2164] A list of tourist destinations as a result of the analysis.

[2165] Step 7:

[2166] server

[2167] The server generates multiple provisional travel plans based on the user's attribute information and the results of image analysis, and gives each plan a catchy title.

[2168] Specific actions

[2169] A title generation algorithm is executed to give a title to the travel plan.

[2170] input

[2171] User attribute information and image analysis results.

[2172] output

[2173] Tentative travel plan list.

[2174] Step 8:

[2175] server

[2176] The server adds information about transportation, accommodation, and tourist spots to the tentative travel plan and presents it to the user.

[2177] Specific actions

[2178] Run a query to add transportation and accommodation information for each itinerary.

[2179] input

[2180] Tentative travel plan list.

[2181] output

[2182] The completed travel itinerary list.

[2183] Step 9:

[2184] user

[2185] Users can review the proposed travel plan and, if they wish to make any changes, enter the changes through the app.

[2186] Specific actions

[2187] On the travel plan screen, select the item you want to change and enter the new information.

[2188] input

[2189] User modification requests.

[2190] output

[2191] Change request data.

[2192] Step 10:

[2193] server

[2194] The server receives the changes in real time, generates a new travel plan, and sends it to the terminal.

[2195] Specific actions

[2196] It runs an algorithm to regenerate a new itinerary based on the changes and sends the updated data to the device.

[2197] input

[2198] The user's modification request data.

[2199] output

[2200] Updated travel planning data.

[2201] Step 11:

[2202] server

[2203] After the server confirms the travel plan, it automatically generates the necessary information and a list of items to bring, and sends them to the user's email address. It also sends a real-time notification to the user's device.

[2204] Specific actions

[2205] Run inventory generation algorithms and send emails and notifications.

[2206] input

[2207] Confirmed travel planning data.

[2208] output

[2209] Inventory list and notification data.

[2210] Step 12:

[2211] server

[2212] Before a trip, the server collects local information and trivia and notifies the user's device.

[2213] Specific actions

[2214] It calls the local information API to collect information and sends push notifications to the user's device.

[2215] input

[2216] Local information API data.

[2217] output

[2218] Local information notification data.

[2219] Step 13:

[2220] user

[2221] After the trip, users enter their satisfaction and impressions of the trip into the app.

[2222] Specific actions

[2223] Fill out the evaluation form in the app and press the submit button.

[2224] input

[2225] User satisfaction and opinion data.

[2226] output

[2227] Submitted evaluation data.

[2228] Step 14:

[2229] server

[2230] The server uses the user's ratings to refine the next travel plan and send new suggestions to the user.

[2231] Specific actions

[2232] The next itinerary generation algorithm is run to generate new proposals and send them to the user's device.

[2233] input

[2234] Submitted evaluation data.

[2235] output

[2236] New travel planning data.

[2237] (Application example 1)

[2238] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2239] Conventional online shopping lacks the ability to offer products tailored to individual user preferences and needs, resulting in a poor quality shopping experience. In particular, it is difficult for users to find the products and designs they specifically desire, creating a need for a hassle-free way to select a set of products that suits them.

[2240] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2241] In this invention, the server includes means for the user to input their attributes, preferences, and images similar to the products they wish to see, means for the server to analyze the input information and images and automatically generate multiple product sets tailored to the user, means for presenting the multiple product sets generated by the server with catchy titles, means for the user to change parts of the product sets and means for updating the changes in real time, and means for the server to suggest the next shopping plan based on the user's ratings. This enables product suggestions to be individually customized for the user, improving the quality of online shopping.

[2242] "User" refers to an individual who uses the system to select and purchase products.

[2243] "Attributes" refers to personal information such as a user's name, age, gender, and preferences.

[2244] "Preferences" refer to a user's tendency to be particularly interested in a particular style or category.

[2245] "Image" refers to visual data entered by the user that shows the desired product or design.

[2246] "Server" refers to the main computer system that processes user input and image data and generates product sets based on the analysis results.

[2247] "Analyzing images" refers to analyzing uploaded image data through a program and extracting relevant information based on its content.

[2248] "Means for automatically generating product sets" refers to the process of automatically selecting and combining multiple related products based on user attributes and the results of image analysis.

[2249] A "catchy title" refers to giving the generated product set a name that will attract the user's interest.

[2250] The "presentation means" refers to a method or function for visually presenting the generated product set to the user.

[2251] "Modifiable" means that users can modify parts of the suggested product set to suit their preferences.

[2252] "Means of real-time updates" refers to methods and functions that allow changes made by users to be reflected immediately on the system.

[2253] "Means to suggest next shopping plans" refers to a system that improves future shopping experiences and suggests new products based on user ratings and feedback.

[2254] A "checklist" is a list of items or tasks that a user needs to complete after their shopping plan is finalized.

[2255] "Means of notification" refers to the methods and functions by which the system notifies the user of information.

[2256] "Collecting satisfaction and feedback" refers to collecting feedback and comments provided by users as data.

[2257] "Means for improvement" refers to methods and functions for effectively improving the next proposal or the system itself based on the collected feedback.

[2258] The invention begins with the user entering their attributes, preferences, and images of products similar to the one they want to see. The user first downloads and installs the application. When the application is launched, a new user registration page appears, where the user enters their name, age, gender, and shopping preferences (e.g., fashion, home appliances, outdoor gear, etc.). This information is stored in the cloud and used to customize the shopping experience.

[2259] Next, the user uploads an image of the product they want to see to the app. The device uploads the image data to cloud storage, and the server retrieves and analyzes the image. This image analysis uses OpenCV and KMeans clustering. For example, if a user uploads an image of a modern living room, the app analyzes its color hues and suggests furniture and decoration items in a similar style.

[2260] The server generates a set of related products based on the user's information and image analysis results, and gives it a catchy title such as "Modern Living Room Set." The generated product set is displayed in the app, and the user can easily add it to their shopping list. If the user reviews the suggested product set and wants to change some of the products, the changes can be reflected in real time. For example, if the user wants to change the color of a sofa, the server will re-suggest a product list including the sofa in the new color.

[2261] Once the shopping list is finalized, the server automatically generates the necessary checklist and notifies the user, including items and tasks to be checked before purchasing. The app also provides additional information about the products, such as their background and use cases.

[2262] After making a purchase, users can enter their satisfaction and feedback on their experience into the app, which the server uses to improve their next shopping plan and make new suggestions.

[2263] The system uses the following main hardware and software: OpenCV and the KMeans clustering algorithm for image analysis, the Flask framework on the server side, MongoDB and MySQL for the database, and smtplib for sending emails.

[2264] An example of a prompt is as follows:

[2265] Example prompt sentence:

[2266] Upload an image of a modern living room to represent the product users want to see.

[2267] As a result of the analysis, the server suggests related modern style furniture (e.g. sofa, table, carpet).

[2268] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2269] Step 1:

[2270] The user downloads and installs the application. After launching it, a new user registration page appears, where the user enters their name, age, gender, and shopping preferences. This information is sent from the device to the cloud server and stored in a database. The input in this process is user information, and the output is the storage of information in the cloud database.

[2271] Step 2:

[2272] The user selects an image that is similar to the product they want to see and uploads it to the app. The device uploads the image data to cloud storage, and the server retrieves the image data. In this step, the input is the image data, and the output is saving it to cloud storage and retrieving it by the server.

[2273] Step 3:

[2274] The server analyzes the uploaded image data using OpenCV and KMeans clustering. Specifically, it extracts the hue information of the image and classifies it into multiple clusters. The input is the image data, and the output is a color palette classified by the hue information.

[2275] Step 4:

[2276] The server generates a related product set based on the color palette obtained as a result of the analysis. Using a generative AI model, it inputs the user's preference information and color palette and proposes the optimal product set. The input in this step is the analysis result and the user's preference information, and the output is the proposed product set. This product set is given a catchy title such as "Modern Living Room Set."

[2277] Step 5:

[2278] The server presents the generated product set to the user through the application. The user can review the proposal and modify parts of the product set if necessary. The modifications are sent to the server in real time, and the server generates a corresponding new product set and presents it to the user again. The input in this step is the user's modification request, and the output is the updated product set.

[2279] Step 6:

[2280] Once the user has finalized the product set, the server automatically generates a checklist, which includes items and tasks to be checked before purchasing. This checklist is sent to the user's device. The input is the finalized product set, and the output is the generated checklist.

[2281] Step 7:

[2282] After a purchase, the user enters feedback about their satisfaction and experience into the application. The server collects this feedback and uses it to improve the next shopping plan. In this step, the input is the user's feedback and the output is the improved next shopping plan.

[2283] Example prompt sentence:

[2284] Step 1: Register as a new user by entering your name, age, gender and shopping preferences.

[2285] Step 2: Upload an image that resembles the product you want to see.

[2286] Step 3: Generate a color palette based on the results of image analysis.

[2287] Step 4: Propose a product set based on the color palette and user preference information.

[2288] Step 5: User reviews the proposed product set and makes changes if necessary.

[2289] Step 6: Automatically generate a checklist after the product set is confirmed.

[2290] Step 7: Collect post-purchase feedback and refine your next plan.

[2291] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2292] Initial Setup

[2293] The user downloads and installs the app. The device launches the app and displays a new user registration page. The user enters their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.). The app also performs initial setup to recognize the user's emotions.

[2294] Examples:

[2295] The user inputs that they prefer "natural."

[2296] The server stores the entered information in a database.

[2297] Image input and emotion analysis

[2298] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage, and the server retrieves the image and analyzes the image and the user's input information using an emotion engine. Based on the analysis results, a list of recommended tourist spots is generated and the user's emotional state is recorded.

[2299] Examples:

[2300] A user uploads a photo of Mount Fuji.

[2301] The server analyzes the image and suggests Japan.

[2302] The emotion engine recognizes the user's emotion of joy.

[2303] Generate a provisional itinerary

[2304] The server generates multiple provisional itineraries based on the user's information, image analysis, and sentiment analysis results. It then gives each itinerary a catchy title and automatically suggests transportation and accommodation options.

[2305] Examples:

[2306] The server titles the itinerary it generates as "7 days to enjoy Japanese culture and nature."

[2307] The server displays the itinerary to the user.

[2308] Day 1: Arrival at Haneda Airport, sightseeing in Tokyo, accommodation (Hotel ABC)

[2309] Day 2: Sightseeing in Akihabara, Ueno Zoo, and lodging at Hotel XYZ

[2310] Day 3: Travel to Hakone, hot spring experience, and accommodation (hot spring inn DEF)

[2311] Itinerary amendments and real-time updates

[2312] If a user reviews the proposed itinerary and decides to change some of the tourist attractions or accommodations, they can make the changes through the app, and the server will update the changes in real time.In addition, the emotion engine recognizes the user's real-time emotional state and automatically adjusts the itinerary accordingly.

[2313] Examples:

[2314] The user enters that they would like to change their accommodation for the third day to a different hot spring inn.

[2315] The server generates a new itinerary and sends it to the device.

[2316] The emotion engine recognizes the user's emotions and proposes the optimal plan.

[2317] New plan:

[2318] Day 3: Travel to Hakone, hot spring experience, and accommodation (Hot Spring Inn GHI)

[2319] Providing a list of necessary items and what to bring

[2320] Once the travel plan is finalized, the server will send the necessary information and a list of what to bring to the user's email, and a real-time notification will be sent to the user's device.

[2321] Examples:

[2322] The server automatically generates an inventory list.

[2323] Send email:

[2324] passport

[2325] VISA (Japan)

[2326] JPY

[2327] Clothing (7 days' worth)

[2328] Terminal notification: "Travel in 5 days. Please check your passport and VISA."

[2329] Providing local information

[2330] Before the trip, the server collects information on local culture, trivia, precautions, etc. and notifies the user.

[2331] Examples:

[2332] The server collects information about hot spring culture.

[2333] Device notification: "Learn about Japanese hot spring culture: How to use towels and etiquette"

[2334] Emotion monitoring and optimization during travel

[2335] During the trip, the emotion engine continuously monitors the user's emotions and optimizes the travel experience based on the emotional data. If the user is enjoying themselves, it will make suggestions to maintain that state, and if they are feeling stressed, it will suggest relaxing activities.

[2336] Examples:

[2337] The emotion engine recognizes the user's level of enjoyment.

[2338] Device notification: "There's a recommended cafe nearby. Would you like to relax?"

[2339] Post-trip evaluation and suggestions for next time

[2340] After a trip, users input their satisfaction and impressions of the trip into the app. The server uses this information to improve the next trip plan and make new suggestions. The emotion engine also utilizes emotional data from the trip to make more personalized suggestions.

[2341] Examples:

[2342] The user rates their trip satisfaction as "very satisfied."

[2343] The server makes the next suggestion based on the emotion data.

[2344] New suggestion: "Why not plan your next trip to Japan? We'll suggest some new tourist spots."

[2345] As described above, the system of the present invention efficiently acquires customized travel plans for users and provides consistent support from planning to execution and even next-time suggestions. By combining it with an emotion engine, it is possible to provide an optimal travel experience that reflects the user's real-time emotional state.

[2346] The processing flow will be explained below.

[2347] Step 1:

[2348] The user downloads and installs the app. The device launches the app and displays the new user registration page. The user enters their name, age, gender, and travel preferences (e.g., nature, culture, adventure, etc.) to initialize the emotion engine. The device sends the input information to the server. The server stores the user information in a database.

[2349] Step 2:

[2350] The user uploads an image that is close to the scenery they want to see to the app. The device uploads the image data to cloud storage. The server retrieves the image from cloud storage and sends it to the image analysis module and emotion engine. The server's image analysis module analyzes the image, and the emotion engine recognizes the user's emotional state. A list of recommended tourist spots is generated based on the analysis results and emotion data.

[2351] Step 3:

[2352] The server generates multiple provisional itineraries based on user information, image analysis, and sentiment analysis. It then gives each itinerary a catchy title. The server then collects information on transportation and accommodations and incorporates this information into the itinerary. The multiple provisional itineraries generated by the server are then sent to the user's device for provision.

[2353] Step 4:

[2354] The user reviews the proposed itinerary and inputs changes to some of the tourist attractions and accommodations. The device sends the changes to the server. The server receives the change request and updates the travel plan with the new elements in real time. The emotion engine recognizes the user's real-time emotional state and automatically adjusts the itinerary accordingly. The server sends the updated plan to the device for presentation to the user.

[2355] Step 5:

[2356] The server generates the necessary information and a list of items to bring based on the confirmed travel plan. The server sends the generated information to the user's email and also sends a notification to the device. The device displays the notification to the user.

[2357] Step 6:

[2358] The server collects information about the destination's culture, trivia, and things to be aware of. The server then sends the collected information to the user's device. The device then notifies the user of the local information.

[2359] Step 7:

[2360] During the trip, the emotion engine continuously monitors the user's emotions. The server makes suggestions to optimize the travel experience based on the emotion data. The device then notifies the user of the optimized information.

[2361] Step 8:

[2362] After a trip, the user enters their satisfaction and impressions of the trip into the app. The device sends the input information to the server. The server stores the user's ratings in a database. The emotion engine utilizes the emotional data from the trip, and the server sends the user new suggestions to improve their next travel plan.

[2363] Example 2

[2364] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2365] Conventional travel planning support systems have difficulty providing travel plans that fully reflect the user's preferences and emotions. It is also difficult to adjust plans in response to real-time emotional changes during travel. This has resulted in the issue of users' travel experiences not being optimized.

[2366] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2367] In this invention, the server includes a means for a user to input images that are similar to their attributes, preferences, and scenery they want to see, a means for analyzing the input information and images using a generative AI model and automatically generating multiple itineraries tailored to the user, and a means for monitoring the user's emotional state during the trip using an emotion engine to optimize the travel experience. This makes it possible to provide a personalized travel plan that reflects the user's preferences and emotions, and to adjust the plan to suit real-time emotional changes during the trip.

[2368] "User" means an individual who intends to use the System to create a travel plan.

[2369] "Attributes" refers to personal information such as a user's name, age, and gender.

[2370] "Preferences" refers to the user's preferred travel style and purpose, such as nature, culture, adventure, etc.

[2371] "Images similar to the scenery you want to see" are images that express the image of scenery or places you want to see as a travel destination.

[2372] A "server" is a computer system that analyzes data and generates travel plans based on information provided by users.

[2373] A "generative AI model" is an algorithm that uses artificial intelligence to analyze input information and images and generate the optimal travel plan for the user.

[2374] An "emotion engine" is a technology that recognizes the user's emotional state and uses it as data for analysis.

[2375] An "itinerary" is a detailed plan that lists the travel schedule and places to visit.

[2376] "Title" is a catchphrase or description that is given to the generated itinerary.

[2377] "Transportation" refers to the means of transportation used during a trip (e.g., plane, train, bus).

[2378] "Accommodation" refers to the place where you stay during your trip, such as a hotel or inn.

[2379] "Tourist attractions" refer to places of interest and activities that you can visit while traveling.

[2380] A "packing list" is a list of items you will need while traveling.

[2381] "Local information" refers to information about the culture, customs, and precautions of the destination.

[2382] "User rating" refers to the satisfaction and impressions that users enter into the system after their trip.

[2383] "Next Travel Plan" is a plan for your next trip that is suggested based on the user's past ratings and emotional data.

[2384] This invention is a system that provides a personalized travel plan that reflects the user's preferences and emotions, and adjusts the plan optimally in response to real-time changes in emotions during the trip. This system operates in cooperation with the user, terminal, and server.

[2385] Initial Setup

[2386] First, the user downloads and installs the application. The device launches the application and displays a new user registration page. The user enters their name, age, gender, and travel preferences (nature, culture, adventure, etc.). They also enter data for emotion recognition as part of the initial setup.

[2387] Image input and emotion analysis

[2388] The user uploads an image that is close to the scenery they want to see to the application. The device then uploads this image data to cloud storage (e.g., AWS S3). The server retrieves the image from cloud storage and uses a generative AI model to analyze the image and the user's input information. For example, if a user uploads a photo of Mt. Fuji, the server uses a generative AI model (e.g., AWS Rekognition) to recommend "Japan," and the emotion engine recognizes the user's emotion of joy.

[2389] Generate a provisional itinerary

[2390] The server generates multiple provisional itineraries based on user information, image analysis, and sentiment analysis. It also gives each itinerary a catchy title and automatically suggests transportation and accommodation options. For example, the server might title the itinerary "Seven days to fully enjoy Japanese culture and nature" and suggest a specific itinerary.

[2391] Itinerary amendments and real-time updates

[2392] If a user reviews the proposed itinerary and wants to change some of the tourist attractions or accommodations, they can input the changes through the application, and the server will update the changes in real time. The emotion engine also recognizes the user's real-time emotional state and automatically adjusts the itinerary accordingly.

[2393] Providing a list of necessary items and what to bring

[2394] Once the travel plan is finalized, the server automatically generates a list of items to bring and sends it to the user's email. A real-time notification is also sent to the device. For example, the server generates a list of items to bring, such as "passport, VISA, Japanese yen, and clothing (for 7 days)," and notifies the user.

[2395] Providing local information

[2396] Before a trip, the server collects information about local culture, customs, and precautions, and notifies the user. For example, it provides information such as "Learn about Japanese hot spring culture: how to use towels and etiquette."

[2397] Emotion monitoring and optimization during travel

[2398] During the trip, the emotion engine continuously monitors the user's emotions and optimizes the travel experience based on the emotional data. For example, it can make suggestions such as, "There's a recommended cafe nearby. Why not relax there?" to optimize the user's travel experience.

[2399] Post-trip evaluation and suggestions for next time

[2400] After the trip, the user enters their satisfaction and impressions of the trip into the application. Based on this, the server will improve the next travel plan and make new suggestions. For example, it may say, "Why not plan another trip to Japan next time? We'll suggest some new tourist spots."

[2401] This system allows users to efficiently obtain customized travel plans and provides consistent support from planning to execution, as well as future proposals. By utilizing an emotion engine, it is possible to provide an optimal travel experience that reflects the user's real-time emotional state.

[2402] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2403] Step 1: Download and install the app

[2404] The user downloads and installs the application.

[2405] Input: Download link from the app store.

[2406] Output: Application installed on the device.

[2407] Specific behavior:

[2408] A user opens an app store and searches for an application.

[2409] Click on the application to begin the download.

[2410] The device installs the application.

[2411] Step 2: First launch and enter user information

[2412] The device launches the installed application and displays the new user registration page.

[2413] Input: User information (name, age, gender, travel preferences).

[2414] Output: Sending user information to the server.

[2415] Specific behavior:

[2416] The device launches the application.

[2417] Users enter their name, age, gender, and travel preferences.

[2418] The terminal transmits the input information to the server.

[2419] Step 3: Save user information

[2420] The server stores the entered user information in a database.

[2421] Input: User information (name, age, gender, travel preferences).

[2422] Output: User information stored in the database.

[2423] Specific behavior:

[2424] The server analyzes the received user information.

[2425] The server stores the analyzed information in a database.

[2426] Step 4: Upload an image

[2427] Users upload an image that is close to the view they want to see to the application.

[2428] Input: Image file.

[2429] Output: Image files saved in cloud storage.

[2430] Specific behavior:

[2431] The user selects an image of the scenery they want to see.

[2432] The device uploads the selected images to cloud storage.

[2433] Step 5: Image acquisition and analysis

[2434] The server retrieves the images from cloud storage and analyzes them using a generative AI model.

[2435] Input: Image files in cloud storage, user information.

[2436] Output: Analyzed image information and tourist spot suggestions.

[2437] Specific behavior:

[2438] The server downloads the image from the cloud storage.

[2439] The server analyzes the image using a generative AI model (e.g., AWS Rekognition).

[2440] Based on the analysis results, appropriate tourist destinations are suggested.

[2441] Step 6: Sentiment Analysis

[2442] The emotion engine recognizes the user's emotions based on the analysis results.

[2443] Input: Analyzed image information, user information.

[2444] Output: Recognized user emotion data.

[2445] Specific behavior:

[2446] The server transmits the analysis results to the emotion engine.

[2447] The emotion engine analyzes the user's emotions.

[2448] Record your emotional state in a database.

[2449] Step 7: Generate a provisional itinerary

[2450] The server generates multiple provisional itineraries based on user information, image analysis, and emotion analysis results.

[2451] Input: User information, analysis results, emotion data.

[2452] Output: Multiple provisional itineraries generated.

[2453] Specific behavior:

[2454] The server generates multiple provisional itineraries based on the input data.

[2455] Give the generated itinerary a catchy title.

[2456] Step 8: View your itinerary

[2457] The server generates an itinerary and displays it to the user.

[2458] Input: Multiple generated provisional itineraries.

[2459] Output: The itinerary displayed on the terminal.

[2460] Specific behavior:

[2461] The server sends the itinerary details to the terminal and displays them to the user.

[2462] The user reviews the displayed itinerary.

[2463] Step 9: Itinerary Modifications and Real-Time Updates

[2464] The user reviews the proposed itinerary and changes some of the attractions and accommodations.

[2465] Input: The user's correction request.

[2466] Output: The revised itinerary.

[2467] Specific behavior:

[2468] A user enters a request to change travel plans.

[2469] The server analyzes the changes and generates a new itinerary in real time.

[2470] Step 10: Provide a packing list

[2471] The server automatically generates an inventory list and sends it to the user by email. It also notifies the user on their device.

[2472] Input: Confirmed travel plans.

[2473] Output: Inventory list.

[2474] Specific behavior:

[2475] The server generates a list of necessary items to bring based on the travel plan.

[2476] The list of belongings will be sent via email and notified to the device.

[2477] Step 11: Provide local information

[2478] The server collects information about the culture of the destination, trivia, and things to be aware of, and notifies the user.

[2479] Input: Travel destination information.

[2480] Output: Local information notification.

[2481] Specific behavior:

[2482] The server collects local information from the Internet.

[2483] The collected information is sent to the user's device.

[2484] Step 12: Emotional monitoring and optimization during the trip

[2485] The emotion engine continuously monitors user emotions to optimize the travel experience.

[2486] Input: Real-time emotion data.

[2487] Output: Optimized trip plan.

[2488] Specific behavior:

[2489] The emotion engine uses smartphone sensor data to monitor emotions.

[2490] The server analyzes the emotional data and optimizes the travel experience in real time.

[2491] Your device will display notifications suggesting the best activities for you.

[2492] Step 13: Post-trip evaluation and suggestions for next time

[2493] The user enters their satisfaction and impressions of the trip into the app, and the server suggests plans for the next trip.

[2494] Input: Travel satisfaction and impressions.

[2495] Output: Next trip suggestions.

[2496] Specific behavior:

[2497] Users enter their impressions after the trip.

[2498] The server analyzes the data and makes suggestions for your next trip.

[2499] (Application example 2)

[2500] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2501] Conventional food delivery services struggled to provide optimal recommendations based on users' emotions and preferences, forcing them to spend time selecting menu items. Furthermore, there were limited ways to obtain detailed information about dishes or specific restaurants in advance, making it difficult for users to place highly satisfying orders. Furthermore, there was no system in place to utilize feedback after an order to personalize the next order.

[2502] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for the user to input their attributes, preferences, and images similar to the food they want to eat, a means for the server to analyze the input information and images and automatically generate multiple restaurants and menus tailored to the user, and a means for automatically generating and notifying the user of an information provision list required after the user confirms their order. This makes it possible to make suggestions optimized to the user's emotions and preferences.

[2503] "User" means an individual who uses the application to deliver food.

[2504] "Attributes" are a user's age, gender, and other basic profile information.

[2505] "Preferences" refers to the user's food preferences and favorite food categories (e.g., Japanese food, Italian food, etc.).

[2506] "Images similar to the food you want to eat" are images that users upload to the app that represent the food they want to eat that day or at that moment.

[2507] A "server" is a computer system that analyzes information obtained from users and generates and manages delivery proposals.

[2508] "Analyzing information" is the process of processing data based on attributes, preferences, and image data obtained from users to generate optimal suggestions.

[2509] "Automatically generating restaurants and menus" means selecting the most suitable restaurants and delivery menus for users based on analyzed information.

[2510] A "catchy title" means giving the proposed restaurant or menu an attention-grabbing title.

[2511] "Suggestion" means presenting a list of restaurants and menus that are deemed most suitable for the user.

[2512] "Modifiable" means that the user has the ability to adjust or change the suggested restaurant or menu item themselves.

[2513] "Real-time updates" means that changes made by the user are reflected immediately and the latest information is provided.

[2514] "Method of food delivery" refers to the method or means by which the food ordered by the user is delivered to the specified location.

[2515] "Information about the food provided" refers to detailed information about the food ordered by the user (e.g., ingredients, calories, allergen information, etc.).

[2516] A "rating" is feedback a user gives about the service or food provided after placing an order.

[2517] "Make next recommendation" means creating a personalized recommendation for your next order based on the collected ratings and other data.

[2518] The "information list" is a list of detailed information (e.g., confirmation items, usage guide, etc.) that the user will need after confirming their order.

[2519] "Notify" means sending important information or reminders to users to make sure they know.

[2520] The system for implementing this invention consists of an application that provides optimal food delivery suggestions based on the user's preferences and emotions. The system uses a smartphone as its primary hardware platform and the following software components:

[2521] Hardware used:

[2522] Smartphone (iOS or Android device)

[2523] Software used:

[2524] Mobile application frameworks (e.g., React Native)

[2525] Cloud storage (AWS S3)

[2526] Sentiment analysis engine (Google Cloud Vision API, Microsoft Azure Face API, etc.)

[2527] Backend server (Node.js, Express)

[2528] Database (MongoDB)

[2529] Detailed process flow:

[2530] User Registration:

[2531] After a user downloads and installs the application, they register as a new user. They enter their name, age, gender, and food preference (e.g., Japanese, Italian, etc.). This user information is collected using a mobile application framework and sent to the backend server, which stores this information in a database (MongoDB).

[2532] Examples:

[2533] If a user inputs that they like "Japanese food," the server stores this information in a database.

[2534] Example prompt sentence:

[2535] "You love Japanese food! Next, upload a photo of the dish you want to eat."

[2536] Image capture and sentiment analysis:

[2537] Users upload an image of a dish they want to eat to the app. The image is stored in cloud storage (AWS S3), and its URL is sent to the backend server. The server uses the Google Cloud Vision API to analyze the image and recognize the user's emotional state.

[2538] Examples:

[2539] If a user uploads a photo of sushi, the server analyzes the image, recognizes the user's emotions, and suggests sushi restaurants.

[2540] Example prompt sentence:

[2541] "Analyzing... Please wait a moment. We've found some recommended sushi restaurants!"

[2542] Auto-generate and suggest restaurants and menus:

[2543] The server then generates multiple restaurant and menu options based on the analysis results, taking into account the user's preferences and emotional data. The options are then presented to the user with catchy titles.

[2544] Examples:

[2545] The server will recommend restaurants to users under titles such as "Three Places Where You Can Enjoy Authentic Sushi," and each restaurant will display its features and recommended menu items.

[2546] Example prompt sentence:

[2547] "We've picked out three places where you can enjoy authentic sushi. Check them out!"

[2548] Order confirmation and real-time changes:

[2549] Users can review the suggested restaurants and menu items and make any changes they want to make through the app, which will then update the server in real time and present the latest suggestions.

[2550] Examples:

[2551] If the user indicates that they would like to change the second restaurant, the server generates a new suggestion and sends it to the device.

[2552] Example prompt sentence:

[2553] "We've updated our settings. Here are our new suggestions."

[2554] Pre-order information and notices:

[2555] After the order is confirmed, the server automatically generates and notifies the user of necessary information (e.g., reservation confirmation, usage guide, etc.), including important information and reminders.

[2556] Examples:

[2557] The server automatically generates a list of items to bring and usage guidelines, and sends them to the user via email or device notification.

[2558] Example prompt sentence:

[2559] "Reservation confirmation: Sushi Dai 7:00 PM~" "15-minute notice: Your reservation time for Sushi Dai is approaching."

[2560] Gathering feedback and suggestions for next time:

[2561] After the order is completed, the server collects the user's satisfaction and feedback, and based on this, generates personalized suggestions for the next order.

[2562] Examples:

[2563] If the user rates the trip as "very satisfied," the server will refine the next travel plan and make new suggestions.

[2564] Example prompt sentence:

[2565] "Would you like to enjoy sushi again next time? We'll suggest a nearby sushi restaurant."

[2566] This allows users to have a food delivery experience that is optimized to their emotions and preferences.

[2567] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2568] Step 1:

[2569] A user downloads the application and registers as a new user. They enter their name, age, gender, and food preferences. This user information is collected through an input form on the device and sent to the backend server. The server receives this information and stores it in a database (MongoDB).

[2570] Input: User's name, age, gender, food preferences

[2571] Output: User information stored in the database

[2572] Specific operation: After entering user information on the terminal and pressing the "Submit" button, the information is sent to the backend server, which receives it and stores it in the database.

[2573] Step 2:

[2574] The user uploads an image of a dish they want to eat to the app. The device then uploads the image to cloud storage (AWS S3) and sends the image URL to the backend server. The server receives the URL and uses the Google Cloud Vision API to analyze the image and recognize the user's emotion.

[2575] Input: Image of the food you want to eat

[2576] Output: Parsed emotion data

[2577] Specific operation: Press the "Upload Image" button on the device, select and upload an image. The URL of the uploaded image is sent to the backend server and passed to the Google Cloud Vision API. The API analyzes the image and returns the results to the server.

[2578] Step 3:

[2579] The server then generates multiple restaurant and menu options based on the analysis results, using data from existing restaurant databases and online sources. Taking into account the user's preferences and emotional data, the server selects the most suitable restaurant and menu options and suggests them to the user with catchy titles.

[2580] Input: User preference data, emotion analysis results

[2581] Output: A list of multiple restaurants and their menus

[2582] Specific operation: Query the database on the server to obtain relevant restaurant and menu information. Based on the obtained data, generate suitable suggestions for the user and send them back to the application.

[2583] Step 4:

[2584] If the user wants to review the suggested restaurants and menus and make any changes, they can do so through the app. These changes are sent in real time from the device to the server, which immediately updates the suggestions and presents updated suggestions again.

[2585] Input: User change request

[2586] Output: Updated list of restaurants and menus

[2587] Specific operation: Modify the proposal content on the terminal and press the "Submit changes" button. The changes are sent to the server, which updates the database and generates and returns a new proposal list.

[2588] Step 5:

[2589] After the order is confirmed, the server automatically generates the necessary information for the user (reservation confirmation, usage guide, etc.) and sends it to the user via email or device notification, including important information and reminders.

[2590] Input: Order confirmation data

[2591] Output: Auto-generated informational list and notifications

[2592] Specific operation: Using a template engine on the server, text information such as reservation confirmations and reminders is generated and sent to the user's email address or device.

[2593] Step 6:

[2594] When a user enters their satisfaction and feedback after placing an order, the server collects and stores this information in a database. This feedback is used to generate more personalized ordering suggestions for the next time.

[2595] Input: User satisfaction rating, impressions

[2596] Output: personalized suggestions for next time

[2597] What happens: You enter your satisfaction rating and feedback on your device and press the "Submit" button. The information is sent to the server and stored in a database. This feedback is used as a reference when generating the next recommendation.

[2598] The above are the specific processing steps of the system.

[2599] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2600] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2601] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2602] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2603] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arr...

Claims

1. A means for the user to input their attributes, preferences, and images that resemble the scenery they want to see; The server analyzes the input information and images and automatically generates multiple itineraries tailored to the user. a means for presenting the plurality of itineraries generated by the server with catchy titles; A means for users to change parts of their itinerary and update those changes in real time; A means for the server to collect information on transportation, accommodation, and tourist spots and reflect it in the itinerary; A means for the server to provide users with local information and trivia before their trip, After the trip, the server will suggest the next travel plan based on the user's rating. A system including:

2. 2. The system according to claim 1, further comprising means for automatically generating and notifying a user of a list of necessary items to be brought after the user has finalized his / her travel plans.

3. 2. The system according to claim 1, further comprising means for the server to collect travel satisfaction and impressions after the trip is completed and to improve the next travel plan based on the collected information.

Images