System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting and storing user profile information; means for generating personalized travel suggestions based on the profile information; means for analyzing natural-language questions from users; means for analyzing photos provided by users and suggesting similar places; and means for collecting feedback from users and using it to train a generative AI model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditionally, planning a holiday required manually gathering information from multiple sources and optimizing it. This required users to spend time and effort, and they were prone to missing out on new experiences. Furthermore, typical recommendation systems were unable to adequately address users' individual preferences, making it difficult to provide consistently personalized suggestions. The present invention aims to solve these problems and help users easily plan their holidays and discover new experiences. [Means for solving the problem]

[0005] The present invention provides a system including means for collecting and storing user profile information, means for generating personalized outing suggestions based on the profile information, means for analyzing natural language questions from users, means for analyzing photos provided by the users and suggesting similar places, and means for collecting user feedback and using it to train a generative AI model. The system also includes means for providing an interactive interface for users to input natural language questions, and means for analyzing photos using image recognition technology to extract features. This allows users to receive personalized outing suggestions with simple operations and efficiently discover new experiences.

[0006] "User Profile Information" means information about a User's interests, past travel history, and preferred activities.

[0007] "Personalized destination suggestions" are destination suggestions that are individually tailored based on the user's profile information.

[0008] A "natural language question" refers to a question or request entered by a user using ordinary natural language.

[0009] "Image recognition technology" is a technology that analyzes images and recognizes the features and patterns within them.

[0010] A "generative AI model" is a machine learning algorithm that generates new suggestions based on given data.

[0011] An "interactive interface" is an interface that allows a user to enter a question or request and receive a response thereto.

[0012] "Feedback" refers to ratings and comments provided by users on suggested trips and experiences.

[0013] "Means used for learning" refers to the process of retraining the generative AI model using collected feedback information. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The present invention relates to a system that provides personalized travel destination suggestions based on a user's profile information. This system operates primarily through interactions between a server, a terminal, and a user. Specific embodiments of the system are described below.

[0036] User Profile Defaults

[0037] First, the device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, and preferred activities. The server receives this information and stores it as a user profile.

[0038] For example, when a user selects "hiking" or "visiting museums" from various travel activities, that information is stored on the server.

[0039] Generate personalized suggestions

[0040] The user then inputs a natural language question about the places they want to go or the activities they want to do through the device's conversational interface. The server sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[0041] For example, if a user asks, "What are some natural places I can go to this weekend?" the server will take into account the user's past profile information and current location to suggest suitable parks or mountains.

[0042] Photo-based recommendations

[0043] The device provides an interface for users to upload photos they have taken, and the user uploads the photo they want. The server passes the uploaded photo to an image recognition module to analyze its features. Based on the analysis results, the server searches a database for similar places and uses a generative AI model to create personalized suggestions. These suggestions are returned to the device and displayed to the user.

[0044] For example, if a user uploads a photo of a lake they visited last month, the server analyzes the lake's characteristics and suggests places with similar natural scenery.

[0045] Learning from user feedback

[0046] The device then provides an interface for users to enter feedback about the places they have visited. Users enter their ratings and impressions of the destinations, and the device sends this to the server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy, allowing future suggestions to better suit the user's needs.

[0047] For example, if a user visits a suggested museum and rates the experience as "very satisfying," this feedback information is stored on the server and used to learn how to further personalize future suggestions.

[0048] The system allows users to easily plan their holidays hassle-free and efficiently discover new places and experiences.

[0049] The processing flow will be explained below.

[0050] Program processing flow

[0051] User Profile Defaults

[0052] Step 1:

[0053] The user operates the terminal to access the system and the user registration screen is displayed.

[0054] Step 2:

[0055] The user enters basic information such as name, email address, and password and presses the registration button.

[0056] Step 3:

[0057] The terminal transmits the input information to the server.

[0058] Step 4:

[0059] The server stores the received information in a database and returns a success message to the terminal.

[0060] Step 5:

[0061] The device will then display a profile setup screen where users can set their interests, past travel history, and preferred activities.

[0062] Step 6:

[0063] The user selects their preference from the options and presses the save button.

[0064] Step 7:

[0065] The terminal transmits the selected information to the server.

[0066] Step 8:

[0067] The server receives this information and stores it in a database as a user profile.

[0068] Generate personalized suggestions

[0069] Step 9:

[0070] The terminal displays an interactive interface that allows the user to enter questions in natural language.

[0071] Step 10:

[0072] Users enter questions about places they want to go and activities they want to do.

[0073] Step 11:

[0074] The terminal transmits the entered question to the server.

[0075] Step 12:

[0076] The server passes the received question to a natural language processing module to analyze the user's intent.

[0077] Step 13:

[0078] The server searches the database for suitable location candidates based on the analysis results.

[0079] Step 14:

[0080] The server uses a generative AI model to generate suggestions based on the user's profile and the questions asked.

[0081] Step 15:

[0082] The server sends the generated proposal back to the terminal.

[0083] Step 16:

[0084] The device displays suggested destinations to the user.

[0085] Photo-based recommendations

[0086] Step 17:

[0087] The device provides an interface that allows users to upload photos.

[0088] Step 18:

[0089] The user selects a photo and presses the upload button.

[0090] Step 19:

[0091] The terminal transmits the uploaded photos to the server.

[0092] Step 20:

[0093] The server passes the received photo to an image recognition module to analyze its features.

[0094] Step 21:

[0095] The server searches a database for similar locations based on the analysis results.

[0096] Step 22:

[0097] The server uses a generative AI model to generate personalized suggestions.

[0098] Step 23:

[0099] The server sends the proposal back to the terminal.

[0100] Step 24:

[0101] The device will display suggested similar locations to the user.

[0102] Learning from user feedback

[0103] Step 25:

[0104] The device displays an interface that allows the user to enter feedback about the places they have visited.

[0105] Step 26:

[0106] The user enters a rating and comment and presses the send button.

[0107] Step 27:

[0108] The terminal transmits the feedback information to the server.

[0109] Step 28:

[0110] The server stores the received feedback information in a database.

[0111] Step 29:

[0112] The server uses the stored feedback information to retrain the generative AI model.

[0113] Step 30:

[0114] The server improves the model's predictive accuracy through re-learning and reflects this in future proposals.

[0115] Example 1

[0116] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0117] Conventional tourist information systems have had challenges in making personalized suggestions based on users' interests and past travel history, and in suggesting appropriate locations in response to questions entered in natural language by users. Furthermore, there were no effective systems for making recommendations based on images provided by users or for retraining generative AI models using user feedback. Thus, a comprehensive system was needed to meet the diverse needs of users and improve their experience.

[0118] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0119] In this invention, the server includes means for collecting and storing user profile information, means for generating personalized outing destination suggestions based on the profile information, means for analyzing questions in natural language from users, means for analyzing images provided by users and suggesting similar places, means for collecting user evaluation information and using it for retraining the generation AI model, means for searching and generating suitable suggestions from a database based on questions entered in natural language, and means for accumulating and utilizing information provided by users. This makes it possible to meet the diverse needs of users and provide outing destination suggestions optimized for each individual user.

[0120] "Profile Information" is the basic data used to generate personalized suggestions, such as a user's basic information, interests, past travel history, and preferred activities.

[0121] "Personalized suggestions" are individually optimized suggestions for places to go based on the user's profile information and current situation.

[0122] A "natural language question" is a natural language question or instruction entered by a user through a dialogue interface.

[0123] "Image analysis" is a technology used to extract features from user-provided images and suggest similar locations.

[0124] "Evaluation information" is feedback data such as satisfaction and impressions provided by users regarding places they have visited or experiences they have had.

[0125] A "generative AI model" is an artificial intelligence model used to generate new suggestions based on collected data.

[0126] A "database" is an information storage system for centrally managing user profile information, past travel history, rating information, etc.

[0127] An "interactive interface" is a user interface that allows a user to enter natural language questions and interact with a system.

[0128] The present invention is a comprehensive system that provides personalized travel destination suggestions based on user profile information. This system operates through the interaction of a server, terminals, and users.

[0129] First, the device displays a user registration screen, where the user enters basic information such as name, email address, and password. This information is sent to the server, validated, and stored in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, preferred activities, etc. This information is also sent to the server and stored as a user profile.

[0130] For example, when a user selects an activity such as "hiking" or "visiting a museum," the server stores this information in a database, thereby accumulating data on the user's individual interests and preferences.

[0131] Next, the user enters a natural language question about the places they want to go or the activities they want to do through the device's conversational interface. The server sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[0132] For example, if a user asks, "Where can I go this weekend to enjoy nature?", the server will take into account the user's past profile information and current location to suggest suitable parks and mountains. This process allows for quick and accurate suggestions to be made in response to the user's question.

[0133] The device also provides an interface for users to upload images they have taken. Users upload their desired images, and the server passes the images to an image recognition module to analyze their features. Based on the analysis results, the server searches for similar places and generates personalized suggestions using a generative AI model. These suggestions are returned to the device and displayed to the user.

[0134] For example, if a user uploads an image of a lake they visited last month, the server will analyze the lake's characteristics and suggest places with similar natural scenery, allowing users to discover new places based on their past experiences.

[0135] Finally, the device provides an interface for users to input their ratings and impressions of the places they visited. The user then inputs this information, and the device sends it to the server. The server then stores the rating information in a database and uses it to retrain the generative AI model, thereby improving the accuracy of future suggestions.

[0136] The system allows users to easily plan their holidays hassle-free and efficiently discover new places and experiences.

[0137] An example prompt for using a generative AI model is:

[0138] "Based on the places you've traveled to in the past year and your experiences, what places would you recommend for our next vacation?"

[0139] "Suggest a new travel destination based on photos of places you've recently visited."

[0140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0141] Step 1:

[0142] The device displays a user registration screen. The user enters basic information such as name, email address, and password. This information is sent by the device to the server. The server receives this basic information and checks the validity of the data. Specifically, it checks whether the email address format is correct and whether the password meets a certain level of strength. After these checks are complete, the server stores the information in a database.

[0143] Step 2:

[0144] The device displays a profile setting screen. The user enters their interests, past travel history, preferred activities, etc. This information is also sent from the device to the server. The server analyzes the received information, creates a user profile, and stores it in a database. For example, if an activity such as "hiking" or "visiting museums" is selected, that information is stored in the database by category.

[0145] Step 3:

[0146] The user inputs a question in natural language about the places they want to go or the activities they want to do through the device's conversational interface. The device then sends the question to the server. The server receives the question and analyzes it using a natural language processing module. Specifically, it extracts keywords from the text and analyzes it to understand its intent. Based on the analysis results, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The generated suggestions are output to the device and displayed to the user.

[0147] For example, if a user asks, "What are some natural places I can go to this weekend?" the server extracts the keyword "natural places" and suggests suitable parks and mountains, taking into account the user's past profile information and current location.

[0148] Step 4:

[0149] The device provides an interface that allows users to upload images they have taken. The user selects and uploads the desired image. This image is then sent from the device to the server. The server passes the received image to an image recognition module, which analyzes its features. Specifically, key objects and landscape features in the image are extracted. The server then uses these features to search a database for similar locations and generates personalized suggestions using a generative AI model. The generated suggestions are output to the device and displayed to the user.

[0150] For example, if a user uploads an image of a lake they visited last month, the server analyzes the lake's characteristics and suggests places with similar natural scenery.

[0151] Step 5:

[0152] The device provides an interface for users to input their ratings and impressions of the places they have visited. The user inputs their ratings and impressions of the places they have visited. This information is sent from the device to the server. The server receives the rating information and stores it in a database. This collected information is also used to retrain the generative AI model. Specifically, the existing model is updated based on the user's rating data, improving the accuracy of future suggestions.

[0153] For example, if a user visits a suggested museum and rates the experience as "very satisfying," this rating information is stored on the server and used as learning data to further personalize future suggestions.

[0154] (Application example 1)

[0155] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0156] Conventional food delivery services have the problem that users must research many menus and restaurants themselves, which makes it time-consuming to make the best choice. Furthermore, simply providing general menu recommendations without fully considering the user's preferences or past ordering history fails to increase user satisfaction. Furthermore, the system lacks a mechanism for utilizing user feedback to make more appropriate suggestions in the future. It also lacks a function to suggest restaurants that serve similar dishes based on a photo of a dish, making it impossible to provide suggestions that meet diverse needs.

[0157] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0158] In this invention, the server includes means for collecting and storing user profile information, means for generating personalized outing suggestions based on the profile information, means for analyzing natural language questions from the user, means for analyzing photos provided by the user and suggesting similar places, means for collecting user feedback and using it to train a generative AI model, means for suggesting restaurants and menus based on the user's food and beverage preferences and order history, and means for receiving questions from the user in real time and providing an interactive interface for suggesting appropriate eating and drinking locations, thereby reducing the burden on the user and enabling personalized and optimal food delivery suggestions.

[0159] "User profile information" refers to individual attributes and behavioral history, including basic information about the user, past order history, preferences, etc.

[0160] "Personalized Suggestions" means suggestions that are individually optimized based on a user's profile information.

[0161] A "natural language question" is a question that the user enters in text format and the system analyzes based on that content.

[0162] "Means for analyzing photos and suggesting similar places" refers to a method for analyzing photos provided by users using image recognition technology and suggesting places with similar characteristics.

[0163] A "generative AI model" is an algorithm that uses machine learning technology to learn from user profile information and feedback, and then generates suggestions based on the results.

[0164] "Feedback" refers to the evaluations and impressions of suggestions provided by users, and is information that is used to improve and learn from the system.

[0165] "Food and beverage preferences and order history" refers to individual preference information for food delivery services, including the dishes a user has ordered in the past, the types of dishes they like, and allergy information.

[0166] A "conversational interface" is a user interface that allows users to enter questions in natural language and receive responses in real time.

[0167] The present invention relates to a system that provides personalized recommendations for places to go and food delivery based on a user's profile information. This system operates primarily through interactions between a server, a terminal, and a user. Specific embodiments of the system are described below.

[0168] User Profile Defaults

[0169] First, the device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, preferred activities, food and beverage preferences, and order history. The server receives this information and stores it as a user profile.

[0170] Generate personalized suggestions

[0171] The user then inputs a natural language question about the places they want to go, the activities they want to do, or the food they want to eat through the device's conversational interface. The server then sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches its database for suitable destinations, restaurants, and menus, and uses a generative AI model to generate personalized suggestions. The suggestions are then sent back to the device and displayed to the user.

[0172] Photo-based recommendations

[0173] The device provides an interface for users to upload photos they have taken, and the user uploads the photo they want. The server passes the uploaded photo to an image recognition module, which analyzes its features. Based on the analysis results, the server searches a database for similar places and dishes and uses a generative AI model to create personalized suggestions. These suggestions are then returned to the device and displayed to the user.

[0174] Learning from user feedback

[0175] The device then provides an interface for users to input feedback about the places they visited and the food they ordered. Users enter their ratings and impressions of the destination and food, and the device sends this to the server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy and allows future suggestions to better adapt to the user's needs.

[0176] Specific examples

[0177] The user enters the following information into the app:

[0178] Name: "Yamada Taro"

[0179] Email address: "yamada.taro@example.com"

[0180] Preferences: Say "I like spicy food"

[0181] After that, when the user asks the app, "What's your recommended lunch today?", the server will suggest the best place to eat and the best food based on the user's preferences. For example, it might suggest a restaurant called "Spicy Corner."

[0182] Prompt Sentence Examples

[0183] "What's your recommended lunch today?"

[0184] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0185] Step 1: Initial User Profile Setup

[0186] The device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The device sends the input data to the server, which receives this information and stores it in a database. The user then uses the device to enter their interests, past orders, and preferences in a profile setting screen. The server also receives this additional information, which is stored in a database.

[0187] Input: User's basic information, interests, past orders, preferences

[0188] Output: User profile information stored in the database

[0189] Step 2: Receiving and parsing the user's question

[0190] The user inputs a question in natural language through the device's dialogue interface. The device then sends the question to the server, which uses a natural language processing module to analyze the question and extract the user's intent. For example, a question might be, "What lunch do you recommend today?"

[0191] Input: User's natural language question

[0192] Output: Parsed question (user intent)

[0193] Step 3: Generate personalized suggestions

[0194] The server searches for suitable candidates from its database based on the analyzed question and user profile information. The server uses a generative AI model to select the most suitable restaurant and menu. For example, if a user's profile states that they "like spicy food," restaurants that serve spicy food will be selected.

[0195] Input: Parsed question content, user profile information

[0196] Output: Personalized suggestions (restaurants and menus)

[0197] Step 4: Sending suggestions to users

[0198] The server sends the generated personalized suggestions to the device, which receives the information and displays the suggestions to the user, for example, a restaurant called "Spicy Corner."

[0199] Input: Personalized suggestions

[0200] Output: Proposal displayed on user's device

[0201] Step 5: Photo-based recommendations

[0202] Users upload photos using the device's interface, which then sends them to a server. The server uses an image recognition module to analyze the photo and extract its features. It then searches a database for places and dishes with similar features and uses a generative AI model to generate suggestions.

[0203] Input: User uploaded photo

[0204] Output: Personalized suggestions for similar places and dishes

[0205] Step 6: Gather user feedback and learn

[0206] Users enter feedback about the places they visited and the food they ordered into the device, which then sends this feedback to the server, which stores it in a database and uses it to retrain the generative AI model, improving the accuracy of future suggestions.

[0207] Input: User feedback

[0208] Output: Feedback stored in a database, trained generative AI model

[0209] Through these steps, the system of the present invention can provide personalized suggestions that match the user's preferences and needs.

[0210] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0211] The present invention combines a system that proposes personalized outing destinations based on a user's profile information with an emotion engine that recognizes the user's emotions. This system operates mainly through interactions between a server, a terminal, and the user. Specific embodiments of the system are described below.

[0212] User Profile Defaults

[0213] First, the device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, and preferred activities. The server receives this information and stores it as a user profile.

[0214] For example, when a user selects "hiking" or "visiting museums" from various travel activities, that information is stored on the server.

[0215] Generate personalized suggestions

[0216] The user then inputs a natural language question about the places they want to go or the activities they want to do through the device's conversational interface. The server sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[0217] For example, if a user asks, "What are some natural places I can go to this weekend?" the server will take into account the user's past profile information and current location to suggest suitable parks or mountains.

[0218] Photo-based recommendations

[0219] The device provides an interface for users to upload photos they have taken, and the user uploads the photo they want. The server passes the uploaded photo to an image recognition module to analyze its features. Based on the analysis results, the server searches a database for similar places and uses a generative AI model to create personalized suggestions. These suggestions are returned to the device and displayed to the user.

[0220] For example, if a user uploads a photo of a lake they visited last month, the server analyzes the lake's characteristics and suggests places with similar natural scenery.

[0221] Learning from user feedback

[0222] The device then provides an interface for users to enter feedback about the places they have visited. Users enter their ratings and impressions of the destinations, and the device sends this to the server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy, allowing future suggestions to better suit the user's needs.

[0223] For example, if a user visits a suggested museum and rates the experience as "very satisfying," this feedback information is stored on the server and used to learn how to further personalize future suggestions.

[0224] Incorporating an emotion engine

[0225] The present invention further incorporates an emotion recognition engine to analyze the user's emotional state and optimize suggestions based on this. The device captures the user's facial expressions, voice, or input text and sends them to the emotion recognition engine. The emotion recognition engine analyzes the emotional state and provides the results to the server. The server then generates more suitable outing suggestions based on this emotional information.

[0226] For example, if the user is perceived as tired, suggestions for relaxing spas and hot springs will be prioritized, while if the user is excited, active activities and events will be suggested.

[0227] In this way, incorporating an emotion engine into this system enables more appropriate suggestions based on the user's current emotional state, further improving the user experience. This configuration allows users to easily plan their holidays without hassle, and efficiently discover new places and experiences.

[0228] The processing flow will be explained below.

[0229] Program processing flow

[0230] User Profile Defaults

[0231] Step 1:

[0232] The user operates the terminal to access the system and the user registration screen is displayed.

[0233] Step 2:

[0234] The user enters basic information such as name, email address, and password and presses the registration button.

[0235] Step 3:

[0236] The terminal transmits the input information to the server.

[0237] Step 4:

[0238] The server stores the received information in a database and returns a success message to the terminal.

[0239] Step 5:

[0240] The device will then display a profile setup screen where users can set their interests, past travel history, and preferred activities.

[0241] Step 6:

[0242] The user selects their preference from the options and presses the save button.

[0243] Step 7:

[0244] The terminal transmits the selected information to the server.

[0245] Step 8:

[0246] The server receives this information and stores it in a database as a user profile.

[0247] Generate personalized suggestions

[0248] Step 9:

[0249] The terminal displays an interactive interface that allows the user to enter questions in natural language.

[0250] Step 10:

[0251] Users enter questions about places they want to go and activities they want to do.

[0252] Step 11:

[0253] The terminal transmits the entered question to the server.

[0254] Step 12:

[0255] The server passes the received question to a natural language processing module to analyze the user's intent.

[0256] Step 13:

[0257] The server searches the database for suitable location candidates based on the analysis results.

[0258] Step 14:

[0259] The server uses a generative AI model to generate suggestions based on the user's profile and the questions asked.

[0260] Step 15:

[0261] The server sends the generated proposal back to the terminal.

[0262] Step 16:

[0263] The device displays suggested destinations to the user.

[0264] Photo-based recommendations

[0265] Step 17:

[0266] The device provides an interface that allows users to upload photos.

[0267] Step 18:

[0268] The user selects a photo and presses the upload button.

[0269] Step 19:

[0270] The terminal transmits the uploaded photos to the server.

[0271] Step 20:

[0272] The server passes the received photo to an image recognition module to analyze its features.

[0273] Step 21:

[0274] The server searches a database for similar locations based on the analysis results.

[0275] Step 22:

[0276] The server uses a generative AI model to generate personalized suggestions.

[0277] Step 23:

[0278] The server sends the proposal back to the terminal.

[0279] Step 24:

[0280] The device will display suggested similar locations to the user.

[0281] Learning from user feedback

[0282] Step 25:

[0283] The device displays an interface that allows the user to enter feedback about the places they have visited.

[0284] Step 26:

[0285] The user enters a rating and comment and presses the send button.

[0286] Step 27:

[0287] The terminal transmits the feedback information to the server.

[0288] Step 28:

[0289] The server stores the received feedback information in a database.

[0290] Step 29:

[0291] The server uses the stored feedback information to retrain the generative AI model.

[0292] Step 30:

[0293] The server improves the model's predictive accuracy through re-learning and reflects this in future proposals.

[0294] Incorporating an emotion engine

[0295] Step 31:

[0296] The device displays an interface that captures the user's facial expressions, voice, or input text.

[0297] Step 32:

[0298] Users input facial expressions and voice through a camera and microphone, and then enter text.

[0299] Step 33:

[0300] The device sends the collected data to an emotion recognition engine.

[0301] Step 34:

[0302] The emotion recognition engine analyzes the user's emotional state and sends the results to the server.

[0303] Step 35:

[0304] The server searches a database for suitable location candidates based on the emotional state.

[0305] Step 36:

[0306] The server uses a generative AI model to generate suggestions based on the user's profile, questions, and emotional state.

[0307] Step 37:

[0308] The server sends the generated proposal back to the terminal.

[0309] Step 38:

[0310] The device displays suggested destinations to the user.

[0311] For example, if the server detects that the user is tired, it will suggest a relaxing spa or hot spring, and if the server detects that the user is excited, it will suggest an active activity or event. This allows the user to receive the most appropriate suggestions based on their emotional state at that time.

[0312] Example 2

[0313] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0314] Conventional personalized recommendation systems make suggestions based solely on the user's profile information and feedback, and therefore are unable to take into account the user's current emotional state. This results in a limited user experience, as the system is unable to provide appropriate suggestions based on the user's current emotions and mood. Furthermore, photo-based recommendation systems are limited to simple similarity analysis, and are therefore unable to provide personalized suggestions that are insufficient to address the user's emotions and interests.

[0315] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0316] In this invention, the server includes means for collecting and storing user profile information, means for generating personalized outing suggestions based on the profile information, means for analyzing natural language questions from the user, means for analyzing photos provided by the user and suggesting similar places, means for collecting user feedback and using it to train the generative AI model, and means for analyzing the user's emotional state and optimizing the suggestions based thereon, thereby enabling more appropriate personalized suggestions according to the user's current emotional state and mood, improving the user experience.

[0317] "User profile information" refers to data such as a user's basic information, interests, past travel history, and preferred activities.

[0318] "Personalized suggestions" are trip suggestions that are individually generated based on the user's profile information and emotional state.

[0319] A "natural language question" is a natural language text question entered by a user through an interactive interface.

[0320] "Image recognition technology" is a technology that uses computer vision to extract and analyze image features.

[0321] A "generative AI model" is a model that uses artificial intelligence to analyze and generate data and provide personalized suggestions to users.

[0322] "User feedback" refers to the ratings and impressions users enter about the places they visit and the experiences they have had.

[0323] "Analyzing emotional state" refers to analyzing and understanding the user's emotions at that time from their facial expressions, voice, or text.

[0324] An "emotion recognition engine" is a software or hardware system for analyzing a user's emotional state.

[0325] The present invention combines a system that proposes personalized outing destinations based on a user's profile information with an emotion engine that recognizes the user's emotions. This system operates mainly through interactions between a server, a terminal, and a user. Specific embodiments of the system are described in detail below.

[0326] User Profile Defaults

[0327] First, the device displays a user registration screen, where the user enters basic information such as name, email address, and password. Next, the server receives the basic information sent from the device and stores it in a database.

[0328] The device then displays a profile setup screen where the user selects their interests, past travel history, and preferred activities. The server receives this information and stores it in a database as a user profile.

[0329] For example, when a user selects "hiking" or "visiting museums" and presses the save button, the server saves this in the database.

[0330] Generate personalized suggestions

[0331] Through the device's conversational interface, users input questions in natural language about places they want to go or activities they want to do. For example, they input a question like, "What are some natural places I can go to this weekend?" The server sends this question to a natural language processing module (e.g., SpaCy or NLTK), which analyzes the question.

[0332] Based on the analysis results, the server searches the database for suitable destinations and generates personalized suggestions using a generative AI model (e.g., GPT-3).The generated suggestions are returned to the device and displayed to the user.

[0333] Photo-based recommendations

[0334] The device provides an interface for users to upload photos they have taken. The user selects the photos they want to upload. Specifically, the user uploads a photo of a lake they visited last month.

[0335] The server passes the uploaded photo to an image recognition module (e.g., TensorFlow or OpenCV) to analyze its features. Based on the analysis results, the server searches a database for similar places and uses a generative AI model to create personalized suggestions. These suggestions are then returned to the device and displayed to the user.

[0336] Learning from user feedback

[0337] The device then provides an interface for users to input their ratings and impressions of the places they visited. The user enters their feedback, and the information is sent from the device to the server.

[0338] The server stores this feedback in a database and uses the collected information to retrain the generative AI model, improving its prediction accuracy and making future suggestions more personalized.

[0339] For example, a user visits a suggested museum and rates the experience as "very satisfying." This feedback information is stored on the server, and the next suggestion is optimized based on it.

[0340] Incorporating an emotion engine

[0341] The present invention further incorporates an emotion recognition engine to analyze the user's emotional state and optimize suggestions based on it. Specifically, the device captures the user's facial expressions, voice, or input text and sends them to an emotion recognition engine (e.g., Affectiva or Microsoft Azure Emotion API). The emotion recognition engine analyzes the emotional state and provides the results to the server.

[0342] The server then uses this emotional information to generate more appropriate recommendations for outings. For example, if the user's emotional state is analyzed as "tired," recommendations for relaxing spas and hot springs will be prioritized. If the user's emotional state is analyzed as "excited," active activities and events will be suggested.

[0343] In this way, by incorporating an emotion recognition engine into this system, it becomes possible to make more appropriate suggestions based on the user's current emotional state, further improving the user experience.

[0344] Examples of prompts include, "What are some natural places I can go to this weekend?" or "Can you tell me some places similar to this photo?"

[0345] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0346] Step 1:

[0347] The device displays a user registration screen, where the user enters basic information such as name, email address, and password.

[0348] Input: The user enters their name, email address, and password.

[0349] Output: Basic information about the user that is sent to the server.

[0350] Specifically, the user enters "Yamada Taro," "yamada@example.com," and "password123," and presses the send button.

[0351] Step 2:

[0352] The server receives the basic information sent from the terminal and stores it in a database.

[0353] Input: Basic user information sent from the device.

[0354] Output: Basic information of the user stored in the database.

[0355] Specifically, the server stores "Yamada Taro", "yamada@example.com", and "password123" in the database.

[0356] Step 3:

[0357] The device will then display a profile setup screen where the user can select their interests, past travel history and preferred activities.

[0358] Input: User-selected interests and activities.

[0359] Output: The profile information sent to the server.

[0360] Specifically, the user selects "Hiking" or "Museum Tour" and presses the save button.

[0361] Step 4:

[0362] The server receives this input information and stores it in a database as a user profile.

[0363] Input: Profile information sent from the device.

[0364] Output: User profile stored in the database.

[0365] Specifically, the server stores information about "hiking" and "visiting museums" in a database.

[0366] Step 5:

[0367] Through the device's interactive interface, users input questions in natural language about places they want to go and activities they want to do.

[0368] Input: The user's natural language question.

[0369] Output: The question sent to the server.

[0370] Specifically, the user types, "What are some natural places I can go to this weekend?"

[0371] Step 6:

[0372] The server sends this question to a natural language processing module (e.g., SpaCy or NLTK) to analyze the question.

[0373] Input: The user's natural language question.

[0374] Output: Analysis result of the question.

[0375] Specifically, the server extracts keywords such as "nature" and "this weekend."

[0376] Step 7:

[0377] Based on the analysis results, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model (e.g., GPT-3).

[0378] Input: Question analysis results and user profile information.

[0379] Output: Personalized suggestions.

[0380] Specifically, the server generates a list of parks and mountains based on the user's profile and the criteria "nature" and "this weekend."

[0381] Step 8:

[0382] The generated suggestions are returned to the device and displayed to the user.

[0383] Enter: personalized suggestions.

[0384] Output: The suggestions displayed on the terminal.

[0385] Specifically, the device will display a list of parks and mountains to the user.

[0386] Step 9:

[0387] The device provides an interface for users to upload photos they have taken, and users can select the photos they want to upload.

[0388] Input: A photo uploaded by the user.

[0389] Output: The photo data sent to the server.

[0390] Specifically, the user uploads a photo of a lake they visited last month.

[0391] Step 10:

[0392] The server passes the uploaded photo to an image recognition module (e.g., TensorFlow or OpenCV) to analyze its features.

[0393] Input: The uploaded photo.

[0394] Output: Photo feature analysis results.

[0395] Specifically, the server extracts the features "lake" and "nature" from the photo.

[0396] Step 11:

[0397] Based on the analysis results, the server searches for similar places in its database and uses a generative AI model to create personalized suggestions.

[0398] Input: Photo feature analysis results and user profile information.

[0399] Output: Personalized suggestions.

[0400] Specifically, the server generates and displays similar place suggestions to the user.

[0401] Step 12:

[0402] The device then provides an interface for users to input their ratings and impressions of the places they visited. The user enters their feedback, and the information is sent from the device to the server.

[0403] Input: Feedback entered by the user.

[0404] Output: Feedback data sent to the server.

[0405] Specifically, the user inputs a rating of "very satisfied."

[0406] Step 13:

[0407] The server stores the feedback in a database and uses the collected information to retrain the generative AI model.

[0408] Input: User feedback.

[0409] Output: Feedback information stored in a database and a retrained generative AI model.

[0410] Specifically, the server stores the response "very satisfied" and reflects it in the next proposal.

[0411] Step 14:

[0412] The device captures the user's facial expressions, voice, or input text and sends it to an emotion recognition engine (e.g., Affectiva or Microsoft Azure Emotion API).

[0413] Input: The user's facial expression, voice, or input text.

[0414] Output: Emotion data sent to the emotion recognition engine.

[0415] Specifically, the device takes a photo of the user's facial expression with its camera.

[0416] Step 15:

[0417] The emotion recognition engine analyzes the emotional state and provides the results to the server.

[0418] Input: Emotion data sent from the device.

[0419] Output: The analyzed emotional state results.

[0420] Specifically, the emotion recognition engine analyzes the emotion "tired."

[0421] Step 16:

[0422] The server uses this emotional information to generate more suitable suggestions for outing destinations.

[0423] Input: Parsed emotional state results and user profile information.

[0424] Output: Personalized suggestions.

[0425] Specifically, if the system analyzes that the user is "tired," it generates and displays suggestions for relaxing spas and hot springs to the user.

[0426] (Application example 2)

[0427] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0428] In modern society, eating out or visiting recreational spots is common to escape from busy daily lives, but the system often fails to provide optimal suggestions based on the user's emotional state. This makes it difficult for users to find a place or meal that suits their mood, and as a result, the selected place or meal may be unsatisfying. Therefore, there is a need for a system that can suggest appropriate places to go or meals based on the user's emotional state.

[0429] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting and storing user profile information, means for recognizing the user's emotional state, means for optimizing meal suggestions based on the user's emotional information, means for analyzing questions in natural language from the user, means for analyzing photos provided by the user and suggesting similar places, and means for collecting feedback from the user and using it to train the generative AI model. This makes it possible to suggest places and meals that match the user's emotional state.

[0430] "User profile information" refers to data such as a user's basic information, interests, past behavior, and preferred activities.

[0431] "Personalized destinations" refer to destinations and activities that are individually optimized based on a user's profile information and emotional state.

[0432] A "natural language question" is a question typed by a user in a normal, everyday conversational format for the system to recognize and interpret.

[0433] "Image recognition technology" refers to the technology of extracting and analyzing features from image data using algorithms such as machine learning and deep learning.

[0434] A "generative AI model" refers to a model that uses artificial intelligence to generate new information based on data, and is primarily used for learning suggestions and feedback.

[0435] "Emotional state recognition" refers to a technology that analyzes a user's facial expressions, voice, input text, etc. to identify their emotions at that time.

[0436] "Meal suggestion optimization" refers to the process of suggesting the most suitable meals and restaurants based on the user's current emotional state and profile information.

[0437] "Feedback collection" refers to the process of collecting user evaluations and opinions on services and proposals they have experienced and using them to improve the system.

[0438] The present invention provides a system that provides personalized outing and dining recommendations based on a user's profile information. It also recognizes the user's emotional state and optimizes the recommendations accordingly. A specific embodiment of the system is described below.

[0439] User Profile Defaults

[0440] The device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. The device then displays a profile setting screen, where the user selects their interests, favorite meals, and past visit history. The server receives this information and stores it as a user profile.

[0441] Emotion Recognition and Personalized Suggestions

[0442] The user then inputs their emotional state through the device's conversational interface using natural language or voice. The server sends this input to an emotion recognition engine, which analyzes the emotional state. Based on the analysis, the server searches a database for suitable meals, restaurants, or outings and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[0443] Feedback and Learning

[0444] The device provides an interface for users to enter feedback about the places they visited and the meals they ate. Users enter their ratings and impressions, and the device sends them to a server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy, allowing future suggestions to better adapt to the user's needs.

[0445] For example, if a user says, "I'm tired today," the server will use an emotion recognition engine to analyze the user's state of fatigue and suggest relaxing cafes and restaurants that serve healthy snacks. If the user also says, "I'm feeling very energetic today," the server will suggest restaurants and events that involve vigorous activity.

[0446] Prompt Sentence Examples

[0447] User Input: I'm feeling a bit down today

[0448] Response: You'll see suggestions for warm, inviting cafes and soup restaurants to soothe your thirst.

[0449] User Input: Feeling adventurous today

[0450] Response: Suggestions include restaurants serving exotic and new cuisines and information about food festivals.

[0451] In this way, the system will be able to provide personalized suggestions based on the user's emotional state, further improving the user experience.

[0452] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0453] Step 1: Register your user profile information

[0454] The server receives the user's basic information (name, email address, password, etc.) sent from the device and stores it in a database. The device then displays a profile setting screen for the user, where the user enters their interests, favorite meals, and past visit history. The server receives this input information and stores it in a database as a user profile.

[0455] Input: User's basic information, interests, favorite foods, past visit history

[0456] Output: User profile information stored in a database

[0457] Step 2: Recognizing your emotional state

[0458] The user inputs their emotional state through the terminal using natural language or voice. The server sends this input to the emotion recognition engine and analyzes the emotional state. The analyzed emotional state information is returned to the server.

[0459] Input: Natural language or voice input about the user's emotional state

[0460] Output: Parsed emotional state information

[0461] Step 3: Generate personalized suggestions

[0462] The server uses the analyzed emotional state information to search a database for suitable meals, restaurants, or outings, and then uses a generative AI model to compile these suggestions into personalized suggestions, which are then sent back to the device and displayed to the user.

[0463] Input: Analyzed emotional state information, user profile information

[0464] Output: Personalized dining, restaurant, and outing suggestions

[0465] Step 4: Gather feedback

[0466] The device provides an interface for users to input feedback about the places they visited and the meals they ate. Users enter their ratings and impressions, and the device sends them to a server, which stores the feedback in a database.

[0467] Input: User ratings and comments

[0468] Output: Feedback information stored in a database

[0469] Step 5: Retraining the generative AI model

[0470] The server retrains the generative AI model based on the collected feedback information, improving the model's prediction accuracy and making future suggestions more tailored to the user's needs.

[0471] Input: Feedback information stored in the database

[0472] Output: Retrained generative AI model

[0473] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0474] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0475] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0476] [Second embodiment]

[0477] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0478] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0479] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0480] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0481] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0482] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0483] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0484] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0485] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0486] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0487] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0488] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0489] The present invention relates to a system that provides personalized travel destination suggestions based on a user's profile information. This system operates primarily through interactions between a server, a terminal, and a user. Specific embodiments of the system are described below.

[0490] User Profile Defaults

[0491] First, the device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, and preferred activities. The server receives this information and stores it as a user profile.

[0492] For example, when a user selects "hiking" or "visiting museums" from various travel activities, that information is stored on the server.

[0493] Generate personalized suggestions

[0494] The user then inputs a natural language question about the places they want to go or the activities they want to do through the device's conversational interface. The server sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[0495] For example, if a user asks, "What are some natural places I can go to this weekend?" the server will take into account the user's past profile information and current location to suggest suitable parks or mountains.

[0496] Photo-based recommendations

[0497] The device provides an interface for users to upload photos they have taken, and the user uploads the photo they want. The server passes the uploaded photo to an image recognition module to analyze its features. Based on the analysis results, the server searches a database for similar places and uses a generative AI model to create personalized suggestions. These suggestions are returned to the device and displayed to the user.

[0498] For example, if a user uploads a photo of a lake they visited last month, the server analyzes the lake's characteristics and suggests places with similar natural scenery.

[0499] Learning from user feedback

[0500] The device then provides an interface for users to enter feedback about the places they have visited. Users enter their ratings and impressions of the destinations, and the device sends this to the server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy, allowing future suggestions to better suit the user's needs.

[0501] For example, if a user visits a suggested museum and rates the experience as "very satisfying," this feedback information is stored on the server and used to learn how to further personalize future suggestions.

[0502] The system allows users to easily plan their holidays hassle-free and efficiently discover new places and experiences.

[0503] The processing flow will be explained below.

[0504] Program processing flow

[0505] User Profile Defaults

[0506] Step 1:

[0507] The user operates the terminal to access the system and the user registration screen is displayed.

[0508] Step 2:

[0509] The user enters basic information such as name, email address, and password and presses the registration button.

[0510] Step 3:

[0511] The terminal transmits the input information to the server.

[0512] Step 4:

[0513] The server stores the received information in a database and returns a success message to the terminal.

[0514] Step 5:

[0515] The device will then display a profile setup screen where users can set their interests, past travel history, and preferred activities.

[0516] Step 6:

[0517] The user selects their preference from the options and presses the save button.

[0518] Step 7:

[0519] The terminal transmits the selected information to the server.

[0520] Step 8:

[0521] The server receives this information and stores it in a database as a user profile.

[0522] Generate personalized suggestions

[0523] Step 9:

[0524] The terminal displays an interactive interface that allows the user to enter questions in natural language.

[0525] Step 10:

[0526] Users enter questions about places they want to go and activities they want to do.

[0527] Step 11:

[0528] The terminal transmits the entered question to the server.

[0529] Step 12:

[0530] The server passes the received question to a natural language processing module to analyze the user's intent.

[0531] Step 13:

[0532] The server searches the database for suitable location candidates based on the analysis results.

[0533] Step 14:

[0534] The server uses a generative AI model to generate suggestions based on the user's profile and the questions asked.

[0535] Step 15:

[0536] The server sends the generated proposal back to the terminal.

[0537] Step 16:

[0538] The device displays suggested destinations to the user.

[0539] Photo-based recommendations

[0540] Step 17:

[0541] The device provides an interface that allows users to upload photos.

[0542] Step 18:

[0543] The user selects a photo and presses the upload button.

[0544] Step 19:

[0545] The terminal transmits the uploaded photos to the server.

[0546] Step 20:

[0547] The server passes the received photo to an image recognition module to analyze its features.

[0548] Step 21:

[0549] The server searches a database for similar locations based on the analysis results.

[0550] Step 22:

[0551] The server uses a generative AI model to generate personalized suggestions.

[0552] Step 23:

[0553] The server sends the proposal back to the terminal.

[0554] Step 24:

[0555] The device will display suggested similar locations to the user.

[0556] Learning from user feedback

[0557] Step 25:

[0558] The device displays an interface that allows the user to enter feedback about the places they have visited.

[0559] Step 26:

[0560] The user enters a rating and comment and presses the send button.

[0561] Step 27:

[0562] The terminal transmits the feedback information to the server.

[0563] Step 28:

[0564] The server stores the received feedback information in a database.

[0565] Step 29:

[0566] The server uses the stored feedback information to retrain the generative AI model.

[0567] Step 30:

[0568] The server improves the model's predictive accuracy through re-learning and reflects this in future proposals.

[0569] Example 1

[0570] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0571] Conventional tourist information systems have had challenges in making personalized suggestions based on users' interests and past travel history, and in suggesting appropriate locations in response to questions entered in natural language by users. Furthermore, there were no effective systems for making recommendations based on images provided by users or for retraining generative AI models using user feedback. Thus, a comprehensive system was needed to meet the diverse needs of users and improve their experience.

[0572] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0573] In this invention, the server includes means for collecting and storing user profile information, means for generating personalized outing destination suggestions based on the profile information, means for analyzing questions in natural language from users, means for analyzing images provided by users and suggesting similar places, means for collecting user evaluation information and using it for retraining the generation AI model, means for searching and generating suitable suggestions from a database based on questions entered in natural language, and means for accumulating and utilizing information provided by users. This makes it possible to meet the diverse needs of users and provide outing destination suggestions optimized for each individual user.

[0574] "Profile Information" is the basic data used to generate personalized suggestions, such as a user's basic information, interests, past travel history, and preferred activities.

[0575] "Personalized suggestions" are individually optimized suggestions for places to go based on the user's profile information and current situation.

[0576] A "natural language question" is a natural language question or instruction entered by a user through a dialogue interface.

[0577] "Image analysis" is a technology used to extract features from user-provided images and suggest similar locations.

[0578] "Evaluation information" is feedback data such as satisfaction and impressions provided by users regarding places they have visited or experiences they have had.

[0579] A "generative AI model" is an artificial intelligence model used to generate new suggestions based on collected data.

[0580] A "database" is an information storage system for centrally managing user profile information, past travel history, rating information, etc.

[0581] An "interactive interface" is a user interface that allows a user to enter natural language questions and interact with a system.

[0582] The present invention is a comprehensive system that provides personalized travel destination suggestions based on user profile information. This system operates through the interaction of a server, terminals, and users.

[0583] First, the device displays a user registration screen, where the user enters basic information such as name, email address, and password. This information is sent to the server, validated, and stored in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, preferred activities, etc. This information is also sent to the server and stored as a user profile.

[0584] For example, when a user selects an activity such as "hiking" or "visiting a museum," the server stores this information in a database, thereby accumulating data on the user's individual interests and preferences.

[0585] Next, the user enters a natural language question about the places they want to go or the activities they want to do through the device's conversational interface. The server sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[0586] For example, if a user asks, "Where can I go this weekend to enjoy nature?", the server will take into account the user's past profile information and current location to suggest suitable parks and mountains. This process allows for quick and accurate suggestions to be made in response to the user's question.

[0587] The device also provides an interface for users to upload images they have taken. Users upload their desired images, and the server passes the images to an image recognition module to analyze their features. Based on the analysis results, the server searches for similar places and generates personalized suggestions using a generative AI model. These suggestions are returned to the device and displayed to the user.

[0588] For example, if a user uploads an image of a lake they visited last month, the server will analyze the lake's characteristics and suggest places with similar natural scenery, allowing users to discover new places based on their past experiences.

[0589] Finally, the device provides an interface for users to input their ratings and impressions of the places they visited. The user then inputs this information, and the device sends it to the server. The server then stores the rating information in a database and uses it to retrain the generative AI model, thereby improving the accuracy of future suggestions.

[0590] The system allows users to easily plan their holidays hassle-free and efficiently discover new places and experiences.

[0591] An example prompt for using a generative AI model is:

[0592] "Based on the places you've traveled to in the past year and your experiences, what places would you recommend for our next vacation?"

[0593] "Suggest a new travel destination based on photos of places you've recently visited."

[0594] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0595] Step 1:

[0596] The device displays a user registration screen. The user enters basic information such as name, email address, and password. This information is sent by the device to the server. The server receives this basic information and checks the validity of the data. Specifically, it checks whether the email address format is correct and whether the password meets a certain level of strength. After these checks are complete, the server stores the information in a database.

[0597] Step 2:

[0598] The device displays a profile setting screen. The user enters their interests, past travel history, preferred activities, etc. This information is also sent from the device to the server. The server analyzes the received information, creates a user profile, and stores it in a database. For example, if an activity such as "hiking" or "visiting museums" is selected, that information is stored in the database by category.

[0599] Step 3:

[0600] The user inputs a question in natural language about the places they want to go or the activities they want to do through the device's conversational interface. The device then sends the question to the server. The server receives the question and analyzes it using a natural language processing module. Specifically, it extracts keywords from the text and analyzes it to understand its intent. Based on the analysis results, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The generated suggestions are output to the device and displayed to the user.

[0601] For example, if a user asks, "What are some natural places I can go to this weekend?" the server extracts the keyword "natural places" and suggests suitable parks and mountains, taking into account the user's past profile information and current location.

[0602] Step 4:

[0603] The device provides an interface that allows users to upload images they have taken. The user selects and uploads the desired image. This image is then sent from the device to the server. The server passes the received image to an image recognition module, which analyzes its features. Specifically, key objects and landscape features in the image are extracted. The server then uses these features to search a database for similar locations and generates personalized suggestions using a generative AI model. The generated suggestions are output to the device and displayed to the user.

[0604] For example, if a user uploads an image of a lake they visited last month, the server analyzes the lake's characteristics and suggests places with similar natural scenery.

[0605] Step 5:

[0606] The device provides an interface for users to input their ratings and impressions of the places they have visited. The user inputs their ratings and impressions of the places they have visited. This information is sent from the device to the server. The server receives the rating information and stores it in a database. This collected information is also used to retrain the generative AI model. Specifically, the existing model is updated based on the user's rating data, improving the accuracy of future suggestions.

[0607] For example, if a user visits a suggested museum and rates the experience as "very satisfying," this rating information is stored on the server and used as learning data to further personalize future suggestions.

[0608] (Application example 1)

[0609] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0610] Conventional food delivery services have the problem that users must research many menus and restaurants themselves, which makes it time-consuming to make the best choice. Furthermore, simply providing general menu recommendations without fully considering the user's preferences or past ordering history fails to increase user satisfaction. Furthermore, the system lacks a mechanism for utilizing user feedback to make more appropriate suggestions in the future. It also lacks a function to suggest restaurants that serve similar dishes based on a photo of a dish, making it impossible to provide suggestions that meet diverse needs.

[0611] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0612] In this invention, the server includes means for collecting and storing user profile information, means for generating personalized outing suggestions based on the profile information, means for analyzing natural language questions from the user, means for analyzing photos provided by the user and suggesting similar places, means for collecting user feedback and using it to train a generative AI model, means for suggesting restaurants and menus based on the user's food and beverage preferences and order history, and means for receiving questions from the user in real time and providing an interactive interface for suggesting appropriate eating and drinking locations, thereby reducing the burden on the user and enabling personalized and optimal food delivery suggestions.

[0613] "User profile information" refers to individual attributes and behavioral history, including basic information about the user, past order history, preferences, etc.

[0614] "Personalized Suggestions" means suggestions that are individually optimized based on a user's profile information.

[0615] A "natural language question" is a question that the user enters in text format and the system analyzes based on that content.

[0616] "Means for analyzing photos and suggesting similar places" refers to a method for analyzing photos provided by users using image recognition technology and suggesting places with similar characteristics.

[0617] A "generative AI model" is an algorithm that uses machine learning technology to learn from user profile information and feedback, and then generates suggestions based on the results.

[0618] "Feedback" refers to the evaluations and impressions of suggestions provided by users, and is information that is used to improve and learn from the system.

[0619] "Food and beverage preferences and order history" refers to individual preference information for food delivery services, including the dishes a user has ordered in the past, the types of dishes they like, and allergy information.

[0620] A "conversational interface" is a user interface that allows users to enter questions in natural language and receive responses in real time.

[0621] The present invention relates to a system that provides personalized recommendations for places to go and food delivery based on a user's profile information. This system operates primarily through interactions between a server, a terminal, and a user. Specific embodiments of the system are described below.

[0622] User Profile Defaults

[0623] First, the device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, preferred activities, food and beverage preferences, and order history. The server receives this information and stores it as a user profile.

[0624] Generate personalized suggestions

[0625] The user then inputs a natural language question about the places they want to go, the activities they want to do, or the food they want to eat through the device's conversational interface. The server then sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches its database for suitable destinations, restaurants, and menus, and uses a generative AI model to generate personalized suggestions. The suggestions are then sent back to the device and displayed to the user.

[0626] Photo-based recommendations

[0627] The device provides an interface for users to upload photos they have taken, and the user uploads the photo they want. The server passes the uploaded photo to an image recognition module, which analyzes its features. Based on the analysis results, the server searches a database for similar places and dishes and uses a generative AI model to create personalized suggestions. These suggestions are then returned to the device and displayed to the user.

[0628] Learning from user feedback

[0629] The device then provides an interface for users to input feedback about the places they visited and the food they ordered. Users enter their ratings and impressions of the destination and food, and the device sends this to the server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy and allows future suggestions to better adapt to the user's needs.

[0630] Specific examples

[0631] The user enters the following information into the app:

[0632] Name: "Yamada Taro"

[0633] Email address: "yamada.taro@example.com"

[0634] Preferences: Say "I like spicy food"

[0635] After that, when the user asks the app, "What's your recommended lunch today?", the server will suggest the best place to eat and the best food based on the user's preferences. For example, it might suggest a restaurant called "Spicy Corner."

[0636] Prompt Sentence Examples

[0637] "What's your recommended lunch today?"

[0638] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0639] Step 1: Initial User Profile Setup

[0640] The device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The device sends the input data to the server, which receives this information and stores it in a database. The user then uses the device to enter their interests, past orders, and preferences in a profile setting screen. The server also receives this additional information, which is stored in a database.

[0641] Input: User's basic information, interests, past orders, preferences

[0642] Output: User profile information stored in the database

[0643] Step 2: Receiving and parsing the user's question

[0644] The user inputs a question in natural language through the device's dialogue interface. The device then sends the question to the server, which uses a natural language processing module to analyze the question and extract the user's intent. For example, a question might be, "What lunch do you recommend today?"

[0645] Input: User's natural language question

[0646] Output: Parsed question (user intent)

[0647] Step 3: Generate personalized suggestions

[0648] The server searches for suitable candidates from its database based on the analyzed question and user profile information. The server uses a generative AI model to select the most suitable restaurant and menu. For example, if a user's profile states that they "like spicy food," restaurants that serve spicy food will be selected.

[0649] Input: Parsed question content, user profile information

[0650] Output: Personalized suggestions (restaurants and menus)

[0651] Step 4: Sending suggestions to users

[0652] The server sends the generated personalized suggestions to the device, which receives the information and displays the suggestions to the user, for example, a restaurant called "Spicy Corner."

[0653] Input: Personalized suggestions

[0654] Output: Proposal displayed on user's device

[0655] Step 5: Photo-based recommendations

[0656] Users upload photos using the device's interface, which then sends them to a server. The server uses an image recognition module to analyze the photo and extract its features. It then searches a database for places and dishes with similar features and uses a generative AI model to generate suggestions.

[0657] Input: User uploaded photo

[0658] Output: Personalized suggestions for similar places and dishes

[0659] Step 6: Gather user feedback and learn

[0660] Users enter feedback about the places they visited and the food they ordered into the device, which then sends this feedback to the server, which stores it in a database and uses it to retrain the generative AI model, improving the accuracy of future suggestions.

[0661] Input: User feedback

[0662] Output: Feedback stored in a database, trained generative AI model

[0663] Through these steps, the system of the present invention can provide personalized suggestions that match the user's preferences and needs.

[0664] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0665] The present invention combines a system that proposes personalized outing destinations based on a user's profile information with an emotion engine that recognizes the user's emotions. This system operates mainly through interactions between a server, a terminal, and the user. Specific embodiments of the system are described below.

[0666] User Profile Defaults

[0667] First, the device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, and preferred activities. The server receives this information and stores it as a user profile.

[0668] For example, when a user selects "hiking" or "visiting museums" from various travel activities, that information is stored on the server.

[0669] Generate personalized suggestions

[0670] The user then inputs a natural language question about the places they want to go or the activities they want to do through the device's conversational interface. The server sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[0671] For example, if a user asks, "What are some natural places I can go to this weekend?" the server will take into account the user's past profile information and current location to suggest suitable parks or mountains.

[0672] Photo-based recommendations

[0673] The device provides an interface for users to upload photos they have taken, and the user uploads the photo they want. The server passes the uploaded photo to an image recognition module to analyze its features. Based on the analysis results, the server searches a database for similar places and uses a generative AI model to create personalized suggestions. These suggestions are returned to the device and displayed to the user.

[0674] For example, if a user uploads a photo of a lake they visited last month, the server analyzes the lake's characteristics and suggests places with similar natural scenery.

[0675] Learning from user feedback

[0676] The device then provides an interface for users to enter feedback about the places they have visited. Users enter their ratings and impressions of the destinations, and the device sends this to the server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy, allowing future suggestions to better suit the user's needs.

[0677] For example, if a user visits a suggested museum and rates the experience as "very satisfying," this feedback information is stored on the server and used to learn how to further personalize future suggestions.

[0678] Incorporating an emotion engine

[0679] The present invention further incorporates an emotion recognition engine to analyze the user's emotional state and optimize suggestions based on this. The device captures the user's facial expressions, voice, or input text and sends them to the emotion recognition engine. The emotion recognition engine analyzes the emotional state and provides the results to the server. The server then generates more suitable outing suggestions based on this emotional information.

[0680] For example, if the user is perceived as tired, suggestions for relaxing spas and hot springs will be prioritized, while if the user is excited, active activities and events will be suggested.

[0681] In this way, incorporating an emotion engine into this system enables more appropriate suggestions based on the user's current emotional state, further improving the user experience. This configuration allows users to easily plan their holidays without hassle, and efficiently discover new places and experiences.

[0682] The processing flow will be explained below.

[0683] Program processing flow

[0684] User Profile Defaults

[0685] Step 1:

[0686] The user operates the terminal to access the system and the user registration screen is displayed.

[0687] Step 2:

[0688] The user enters basic information such as name, email address, and password and presses the registration button.

[0689] Step 3:

[0690] The terminal transmits the input information to the server.

[0691] Step 4:

[0692] The server stores the received information in a database and returns a success message to the terminal.

[0693] Step 5:

[0694] The device will then display a profile setup screen where users can set their interests, past travel history, and preferred activities.

[0695] Step 6:

[0696] The user selects their preference from the options and presses the save button.

[0697] Step 7:

[0698] The terminal transmits the selected information to the server.

[0699] Step 8:

[0700] The server receives this information and stores it in a database as a user profile.

[0701] Generate personalized suggestions

[0702] Step 9:

[0703] The terminal displays an interactive interface that allows the user to enter questions in natural language.

[0704] Step 10:

[0705] Users enter questions about places they want to go and activities they want to do.

[0706] Step 11:

[0707] The terminal transmits the entered question to the server.

[0708] Step 12:

[0709] The server passes the received question to a natural language processing module to analyze the user's intent.

[0710] Step 13:

[0711] The server searches the database for suitable location candidates based on the analysis results.

[0712] Step 14:

[0713] The server uses a generative AI model to generate suggestions based on the user's profile and the questions asked.

[0714] Step 15:

[0715] The server sends the generated proposal back to the terminal.

[0716] Step 16:

[0717] The device displays suggested destinations to the user.

[0718] Photo-based recommendations

[0719] Step 17:

[0720] The device provides an interface that allows users to upload photos.

[0721] Step 18:

[0722] The user selects a photo and presses the upload button.

[0723] Step 19:

[0724] The terminal transmits the uploaded photos to the server.

[0725] Step 20:

[0726] The server passes the received photo to an image recognition module to analyze its features.

[0727] Step 21:

[0728] The server searches a database for similar locations based on the analysis results.

[0729] Step 22:

[0730] The server uses a generative AI model to generate personalized suggestions.

[0731] Step 23:

[0732] The server sends the proposal back to the terminal.

[0733] Step 24:

[0734] The device will display suggested similar locations to the user.

[0735] Learning from user feedback

[0736] Step 25:

[0737] The device displays an interface that allows the user to enter feedback about the places they have visited.

[0738] Step 26:

[0739] The user enters a rating and comment and presses the send button.

[0740] Step 27:

[0741] The terminal transmits the feedback information to the server.

[0742] Step 28:

[0743] The server stores the received feedback information in a database.

[0744] Step 29:

[0745] The server uses the stored feedback information to retrain the generative AI model.

[0746] Step 30:

[0747] The server improves the model's predictive accuracy through re-learning and reflects this in future proposals.

[0748] Incorporating an emotion engine

[0749] Step 31:

[0750] The device displays an interface that captures the user's facial expressions, voice, or input text.

[0751] Step 32:

[0752] Users input facial expressions and voice through a camera and microphone, and then enter text.

[0753] Step 33:

[0754] The device sends the collected data to an emotion recognition engine.

[0755] Step 34:

[0756] The emotion recognition engine analyzes the user's emotional state and sends the results to the server.

[0757] Step 35:

[0758] The server searches a database for suitable location candidates based on the emotional state.

[0759] Step 36:

[0760] The server uses a generative AI model to generate suggestions based on the user's profile, questions, and emotional state.

[0761] Step 37:

[0762] The server sends the generated proposal back to the terminal.

[0763] Step 38:

[0764] The device displays suggested destinations to the user.

[0765] For example, if the server detects that the user is tired, it will suggest a relaxing spa or hot spring, and if the server detects that the user is excited, it will suggest an active activity or event. This allows the user to receive the most appropriate suggestions based on their emotional state at that time.

[0766] Example 2

[0767] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0768] Conventional personalized recommendation systems make suggestions based solely on the user's profile information and feedback, and therefore are unable to take into account the user's current emotional state. This results in a limited user experience, as the system is unable to provide appropriate suggestions based on the user's current emotions and mood. Furthermore, photo-based recommendation systems are limited to simple similarity analysis, and are therefore unable to provide personalized suggestions that are insufficient to address the user's emotions and interests.

[0769] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0770] In this invention, the server includes means for collecting and storing user profile information, means for generating personalized outing suggestions based on the profile information, means for analyzing natural language questions from the user, means for analyzing photos provided by the user and suggesting similar places, means for collecting user feedback and using it to train the generative AI model, and means for analyzing the user's emotional state and optimizing the suggestions based thereon, thereby enabling more appropriate personalized suggestions according to the user's current emotional state and mood, improving the user experience.

[0771] "User profile information" refers to data such as a user's basic information, interests, past travel history, and preferred activities.

[0772] "Personalized suggestions" are trip suggestions that are individually generated based on the user's profile information and emotional state.

[0773] A "natural language question" is a natural language text question entered by a user through an interactive interface.

[0774] "Image recognition technology" is a technology that uses computer vision to extract and analyze image features.

[0775] A "generative AI model" is a model that uses artificial intelligence to analyze and generate data and provide personalized suggestions to users.

[0776] "User feedback" refers to the ratings and impressions users enter about the places they visit and the experiences they have had.

[0777] "Analyzing emotional state" refers to analyzing and understanding the user's emotions at that time from their facial expressions, voice, or text.

[0778] An "emotion recognition engine" is a software or hardware system for analyzing a user's emotional state.

[0779] The present invention combines a system that proposes personalized outing destinations based on a user's profile information with an emotion engine that recognizes the user's emotions. This system operates mainly through interactions between a server, a terminal, and a user. Specific embodiments of the system are described in detail below.

[0780] User Profile Defaults

[0781] First, the device displays a user registration screen, where the user enters basic information such as name, email address, and password. Next, the server receives the basic information sent from the device and stores it in a database.

[0782] The device then displays a profile setup screen where the user selects their interests, past travel history, and preferred activities. The server receives this information and stores it in a database as a user profile.

[0783] For example, when a user selects "hiking" or "visiting museums" and presses the save button, the server saves this in the database.

[0784] Generate personalized suggestions

[0785] Through the device's conversational interface, users input questions in natural language about places they want to go or activities they want to do. For example, they input a question like, "What are some natural places I can go to this weekend?" The server sends this question to a natural language processing module (e.g., SpaCy or NLTK), which analyzes the question.

[0786] Based on the analysis results, the server searches the database for suitable destinations and generates personalized suggestions using a generative AI model (e.g., GPT-3).The generated suggestions are returned to the device and displayed to the user.

[0787] Photo-based recommendations

[0788] The device provides an interface for users to upload photos they have taken. The user selects the photos they want to upload. Specifically, the user uploads a photo of a lake they visited last month.

[0789] The server passes the uploaded photo to an image recognition module (e.g., TensorFlow or OpenCV) to analyze its features. Based on the analysis results, the server searches a database for similar places and uses a generative AI model to create personalized suggestions. These suggestions are then returned to the device and displayed to the user.

[0790] Learning from user feedback

[0791] The device then provides an interface for users to input their ratings and impressions of the places they visited. The user enters their feedback, and the information is sent from the device to the server.

[0792] The server stores this feedback in a database and uses the collected information to retrain the generative AI model, improving its prediction accuracy and making future suggestions more personalized.

[0793] For example, a user can visit a suggested museum and rate their experience as "very satisfied." This feedback information is stored on the server, and the next suggestion is optimized accordingly.

[0794] Incorporating an emotion engine

[0795] The present invention further incorporates an emotion recognition engine to analyze the user's emotional state and optimize suggestions based on it. Specifically, the device captures the user's facial expressions, voice, or input text and sends them to an emotion recognition engine (e.g., Affectiva or Microsoft Azure Emotion API). The emotion recognition engine analyzes the emotional state and provides the results to the server.

[0796] The server then uses this emotional information to generate more appropriate recommendations for outings. For example, if the user's emotional state is analyzed as "tired," recommendations for relaxing spas and hot springs will be prioritized. If the user's emotional state is analyzed as "excited," active activities and events will be suggested.

[0797] In this way, by incorporating an emotion recognition engine into this system, it becomes possible to make more appropriate suggestions based on the user's current emotional state, further improving the user experience.

[0798] Examples of prompts include, "What are some natural places I can go to this weekend?" or "Can you tell me some places similar to this photo?"

[0799] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0800] Step 1:

[0801] The device displays a user registration screen, where the user enters basic information such as name, email address, and password.

[0802] Input: The user enters their name, email address, and password.

[0803] Output: Basic information about the user that is sent to the server.

[0804] Specifically, the user enters "Yamada Taro," "yamada@example.com," and "password123," and presses the send button.

[0805] Step 2:

[0806] The server receives the basic information sent from the terminal and stores it in a database.

[0807] Input: Basic user information sent from the device.

[0808] Output: Basic information of the user stored in the database.

[0809] Specifically, the server stores "Yamada Taro", "yamada@example.com", and "password123" in the database.

[0810] Step 3:

[0811] The device will then display a profile setup screen where the user can select their interests, past travel history and preferred activities.

[0812] Input: User-selected interests and activities.

[0813] Output: The profile information sent to the server.

[0814] Specifically, the user selects "Hiking" or "Museum Tour" and presses the save button.

[0815] Step 4:

[0816] The server receives this input information and stores it in a database as a user profile.

[0817] Input: Profile information sent from the device.

[0818] Output: User profile stored in the database.

[0819] Specifically, the server stores information about "hiking" and "visiting museums" in a database.

[0820] Step 5:

[0821] Through the device's interactive interface, users input questions in natural language about places they want to go and activities they want to do.

[0822] Input: The user's natural language question.

[0823] Output: The question sent to the server.

[0824] Specifically, the user types, "What are some natural places I can go to this weekend?"

[0825] Step 6:

[0826] The server sends this question to a natural language processing module (e.g., SpaCy or NLTK) to analyze the question.

[0827] Input: The user's natural language question.

[0828] Output: Analysis result of the question.

[0829] Specifically, the server extracts keywords such as "nature" and "this weekend."

[0830] Step 7:

[0831] Based on the analysis results, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model (e.g., GPT-3).

[0832] Input: Question analysis results and user profile information.

[0833] Output: Personalized suggestions.

[0834] Specifically, the server generates a list of parks and mountains based on the user's profile and the criteria "nature" and "this weekend."

[0835] Step 8:

[0836] The generated suggestions are returned to the device and displayed to the user.

[0837] Enter: personalized suggestions.

[0838] Output: The suggestions displayed on the terminal.

[0839] Specifically, the device will display a list of parks and mountains to the user.

[0840] Step 9:

[0841] The device provides an interface for users to upload photos they have taken, and users can select the photos they want to upload.

[0842] Input: A photo uploaded by the user.

[0843] Output: The photo data sent to the server.

[0844] Specifically, the user uploads a photo of a lake they visited last month.

[0845] Step 10:

[0846] The server passes the uploaded photo to an image recognition module (e.g., TensorFlow or OpenCV) to analyze its features.

[0847] Input: The uploaded photo.

[0848] Output: Photo feature analysis results.

[0849] Specifically, the server extracts the features "lake" and "nature" from the photo.

[0850] Step 11:

[0851] Based on the analysis results, the server searches for similar places in its database and uses a generative AI model to create personalized suggestions.

[0852] Input: Photo feature analysis results and user profile information.

[0853] Output: Personalized suggestions.

[0854] Specifically, the server generates and displays similar place suggestions to the user.

[0855] Step 12:

[0856] The device then provides an interface for users to input their ratings and impressions of the places they visited. The user enters their feedback, and the information is sent from the device to the server.

[0857] Input: Feedback entered by the user.

[0858] Output: Feedback data sent to the server.

[0859] Specifically, the user inputs a rating of "very satisfied."

[0860] Step 13:

[0861] The server stores the feedback in a database and uses the collected information to retrain the generative AI model.

[0862] Input: User feedback.

[0863] Output: Feedback information stored in a database and a retrained generative AI model.

[0864] Specifically, the server stores the response "very satisfied" and reflects it in the next proposal.

[0865] Step 14:

[0866] The device captures the user's facial expressions, voice, or input text and sends it to an emotion recognition engine (e.g., Affectiva or Microsoft Azure Emotion API).

[0867] Input: The user's facial expression, voice, or input text.

[0868] Output: Emotion data sent to the emotion recognition engine.

[0869] Specifically, the device takes a photo of the user's facial expression with its camera.

[0870] Step 15:

[0871] The emotion recognition engine analyzes the emotional state and provides the results to the server.

[0872] Input: Emotion data sent from the device.

[0873] Output: The analyzed emotional state results.

[0874] Specifically, the emotion recognition engine analyzes the emotion "tired."

[0875] Step 16:

[0876] The server uses this emotional information to generate more suitable suggestions for outing destinations.

[0877] Input: Parsed emotional state results and user profile information.

[0878] Output: Personalized suggestions.

[0879] Specifically, if the system analyzes that the user is "tired," it generates and displays suggestions for relaxing spas and hot springs to the user.

[0880] (Application example 2)

[0881] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0882] In modern society, eating out or visiting recreational spots is common to escape from busy daily lives, but the system often fails to provide optimal suggestions based on the user's emotional state. This makes it difficult for users to find a place or meal that suits their mood, and as a result, the selected place or meal may be unsatisfying. Therefore, there is a need for a system that can suggest appropriate places to go or meals based on the user's emotional state.

[0883] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting and storing user profile information, means for recognizing the user's emotional state, means for optimizing meal suggestions based on the user's emotional information, means for analyzing questions in natural language from the user, means for analyzing photos provided by the user and suggesting similar places, and means for collecting feedback from the user and using it to train the generative AI model. This makes it possible to suggest places and meals that match the user's emotional state.

[0884] "User profile information" refers to data such as a user's basic information, interests, past behavior, and preferred activities.

[0885] "Personalized destinations" refer to destinations and activities that are individually optimized based on a user's profile information and emotional state.

[0886] A "natural language question" is a question typed by a user in a normal, everyday conversational format for the system to recognize and interpret.

[0887] "Image recognition technology" refers to the technology of extracting and analyzing features from image data using algorithms such as machine learning and deep learning.

[0888] A "generative AI model" refers to a model that uses artificial intelligence to generate new information based on data, and is primarily used for learning suggestions and feedback.

[0889] "Emotional state recognition" refers to a technology that analyzes a user's facial expressions, voice, input text, etc. to identify their emotions at that time.

[0890] "Meal suggestion optimization" refers to the process of suggesting the most suitable meals and restaurants based on the user's current emotional state and profile information.

[0891] "Feedback collection" refers to the process of collecting user evaluations and opinions on services and proposals they have experienced and using them to improve the system.

[0892] The present invention provides a system that provides personalized outing and dining recommendations based on a user's profile information. It also recognizes the user's emotional state and optimizes the recommendations accordingly. A specific embodiment of the system is described below.

[0893] User Profile Defaults

[0894] The device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. The device then displays a profile setting screen, where the user selects their interests, favorite meals, and past visit history. The server receives this information and stores it as a user profile.

[0895] Emotion Recognition and Personalized Suggestions

[0896] The user then inputs their emotional state through the device's conversational interface using natural language or voice. The server sends this input to an emotion recognition engine, which analyzes the emotional state. Based on the analysis, the server searches a database for suitable meals, restaurants, or outings and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[0897] Feedback and Learning

[0898] The device provides an interface for users to enter feedback about the places they visited and the meals they ate. Users enter their ratings and impressions, and the device sends them to a server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy, allowing future suggestions to better adapt to the user's needs.

[0899] For example, if a user says, "I'm tired today," the server will use an emotion recognition engine to analyze the user's state of fatigue and suggest relaxing cafes and restaurants that serve healthy snacks. If the user also says, "I'm feeling very energetic today," the server will suggest restaurants and events that involve vigorous activity.

[0900] Prompt Sentence Examples

[0901] User Input: I'm feeling a bit down today

[0902] Response: You'll see suggestions for warm, inviting cafes and soup restaurants to soothe your thirst.

[0903] User Input: Feeling adventurous today

[0904] Response: Suggestions include restaurants serving exotic and new cuisines and information about food festivals.

[0905] In this way, the system will be able to provide personalized suggestions based on the user's emotional state, further improving the user experience.

[0906] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0907] Step 1: Register your user profile information

[0908] The server receives the user's basic information (name, email address, password, etc.) sent from the device and stores it in a database. The device then displays a profile setting screen for the user, where the user enters their interests, favorite meals, and past visit history. The server receives this input information and stores it in a database as a user profile.

[0909] Input: User's basic information, interests, favorite foods, past visit history

[0910] Output: User profile information stored in a database

[0911] Step 2: Recognizing your emotional state

[0912] The user inputs their emotional state through the terminal using natural language or voice. The server sends this input to the emotion recognition engine and analyzes the emotional state. The analyzed emotional state information is returned to the server.

[0913] Input: Natural language or voice input about the user's emotional state

[0914] Output: Parsed emotional state information

[0915] Step 3: Generate personalized suggestions

[0916] The server uses the analyzed emotional state information to search a database for suitable meals, restaurants, or outings, and then uses a generative AI model to compile these suggestions into personalized suggestions, which are then sent back to the device and displayed to the user.

[0917] Input: Analyzed emotional state information, user profile information

[0918] Output: Personalized dining, restaurant, and outing suggestions

[0919] Step 4: Gather feedback

[0920] The device provides an interface for users to input feedback about the places they visited and the meals they ate. Users enter their ratings and impressions, and the device sends them to a server, which stores the feedback in a database.

[0921] Input: User ratings and comments

[0922] Output: Feedback information stored in a database

[0923] Step 5: Retraining the generative AI model

[0924] The server retrains the generative AI model based on the collected feedback information, improving the model's prediction accuracy and making future suggestions more tailored to the user's needs.

[0925] Input: Feedback information stored in the database

[0926] Output: Retrained generative AI model

[0927] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0928] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0929] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0930] [Third embodiment]

[0931] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0932] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0933] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0934] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0935] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0936] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0937] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0938] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0939] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0940] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0941] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0942] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0943] The present invention relates to a system that provides personalized travel destination suggestions based on a user's profile information. This system operates primarily through interactions between a server, a terminal, and a user. Specific embodiments of the system are described below.

[0944] User Profile Defaults

[0945] First, the device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, and preferred activities. The server receives this information and stores it as a user profile.

[0946] For example, when a user selects "hiking" or "visiting museums" from various travel activities, that information is stored on the server.

[0947] Generate personalized suggestions

[0948] The user then inputs a natural language question about the places they want to go or the activities they want to do through the device's conversational interface. The server sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[0949] For example, if a user asks, "What are some natural places I can go to this weekend?" the server will take into account the user's past profile information and current location to suggest suitable parks or mountains.

[0950] Photo-based recommendations

[0951] The device provides an interface for users to upload photos they have taken, and the user uploads the photo they want. The server passes the uploaded photo to an image recognition module to analyze its features. Based on the analysis results, the server searches a database for similar places and uses a generative AI model to create personalized suggestions. These suggestions are returned to the device and displayed to the user.

[0952] For example, if a user uploads a photo of a lake they visited last month, the server analyzes the lake's characteristics and suggests places with similar natural scenery.

[0953] Learning from user feedback

[0954] The device then provides an interface for users to enter feedback about the places they have visited. Users enter their ratings and impressions of the destinations, and the device sends this to the server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy, allowing future suggestions to better suit the user's needs.

[0955] For example, if a user visits a suggested museum and rates the experience as "very satisfying," this feedback information is stored on the server and used to learn how to further personalize future suggestions.

[0956] The system allows users to easily plan their holidays hassle-free and efficiently discover new places and experiences.

[0957] The processing flow will be explained below.

[0958] Program processing flow

[0959] User Profile Defaults

[0960] Step 1:

[0961] The user operates the terminal to access the system and the user registration screen is displayed.

[0962] Step 2:

[0963] The user enters basic information such as name, email address, and password and presses the registration button.

[0964] Step 3:

[0965] The terminal transmits the input information to the server.

[0966] Step 4:

[0967] The server stores the received information in a database and returns a success message to the terminal.

[0968] Step 5:

[0969] The device will then display a profile setup screen where users can set their interests, past travel history, and preferred activities.

[0970] Step 6:

[0971] The user selects their preference from the options and presses the save button.

[0972] Step 7:

[0973] The terminal transmits the selected information to the server.

[0974] Step 8:

[0975] The server receives this information and stores it in a database as a user profile.

[0976] Generate personalized suggestions

[0977] Step 9:

[0978] The terminal displays an interactive interface that allows the user to enter questions in natural language.

[0979] Step 10:

[0980] Users enter questions about places they want to go and activities they want to do.

[0981] Step 11:

[0982] The terminal transmits the entered question to the server.

[0983] Step 12:

[0984] The server passes the received question to a natural language processing module to analyze the user's intent.

[0985] Step 13:

[0986] The server searches the database for suitable location candidates based on the analysis results.

[0987] Step 14:

[0988] The server uses a generative AI model to generate suggestions based on the user's profile and the questions asked.

[0989] Step 15:

[0990] The server sends the generated proposal back to the terminal.

[0991] Step 16:

[0992] The device displays suggested destinations to the user.

[0993] Photo-based recommendations

[0994] Step 17:

[0995] The device provides an interface that allows users to upload photos.

[0996] Step 18:

[0997] The user selects a photo and presses the upload button.

[0998] Step 19:

[0999] The terminal transmits the uploaded photos to the server.

[1000] Step 20:

[1001] The server passes the received photo to an image recognition module to analyze its features.

[1002] Step 21:

[1003] The server searches a database for similar locations based on the analysis results.

[1004] Step 22:

[1005] The server uses a generative AI model to generate personalized suggestions.

[1006] Step 23:

[1007] The server sends the proposal back to the terminal.

[1008] Step 24:

[1009] The device will display suggested similar locations to the user.

[1010] Learning from user feedback

[1011] Step 25:

[1012] The device displays an interface that allows the user to enter feedback about the places they have visited.

[1013] Step 26:

[1014] The user enters a rating and comment and presses the send button.

[1015] Step 27:

[1016] The terminal transmits the feedback information to the server.

[1017] Step 28:

[1018] The server stores the received feedback information in a database.

[1019] Step 29:

[1020] The server uses the stored feedback information to retrain the generative AI model.

[1021] Step 30:

[1022] The server improves the model's predictive accuracy through re-learning and reflects this in future proposals.

[1023] Example 1

[1024] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1025] Conventional tourist information systems have had challenges in making personalized suggestions based on users' interests and past travel history, and in suggesting appropriate locations in response to questions entered in natural language by users. Furthermore, there were no effective systems for making recommendations based on images provided by users or for retraining generative AI models using user feedback. Thus, a comprehensive system was needed to meet the diverse needs of users and improve their experience.

[1026] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1027] In this invention, the server includes means for collecting and storing user profile information, means for generating personalized outing destination suggestions based on the profile information, means for analyzing questions in natural language from users, means for analyzing images provided by users and suggesting similar places, means for collecting user evaluation information and using it for retraining the generation AI model, means for searching and generating suitable suggestions from a database based on questions entered in natural language, and means for accumulating and utilizing information provided by users. This makes it possible to meet the diverse needs of users and provide outing destination suggestions optimized for each individual user.

[1028] "Profile Information" is the basic data used to generate personalized suggestions, such as a user's basic information, interests, past travel history, and preferred activities.

[1029] "Personalized suggestions" are individually optimized suggestions for places to go based on the user's profile information and current situation.

[1030] A "natural language question" is a natural language question or instruction entered by a user through a dialogue interface.

[1031] "Image analysis" is a technology used to extract features from user-provided images and suggest similar locations.

[1032] "Evaluation information" is feedback data such as satisfaction and impressions provided by users regarding places they have visited or experiences they have had.

[1033] A "generative AI model" is an artificial intelligence model used to generate new suggestions based on collected data.

[1034] A "database" is an information storage system for centrally managing user profile information, past travel history, rating information, etc.

[1035] An "interactive interface" is a user interface that allows a user to enter natural language questions and interact with a system.

[1036] The present invention is a comprehensive system that provides personalized travel destination suggestions based on user profile information. This system operates through the interaction of a server, terminals, and users.

[1037] First, the device displays a user registration screen, where the user enters basic information such as name, email address, and password. This information is sent to the server, validated, and stored in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, preferred activities, etc. This information is also sent to the server and stored as a user profile.

[1038] For example, when a user selects an activity such as "hiking" or "visiting a museum," the server stores this information in a database, thereby accumulating data on the user's individual interests and preferences.

[1039] Next, the user enters a natural language question about the places they want to go or the activities they want to do through the device's conversational interface. The server sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[1040] For example, if a user asks, "Where can I go this weekend to enjoy nature?", the server will take into account the user's past profile information and current location to suggest suitable parks and mountains. This process allows for quick and accurate suggestions to be made in response to the user's question.

[1041] The device also provides an interface for users to upload images they have taken. Users upload their desired images, and the server passes the images to an image recognition module to analyze their features. Based on the analysis results, the server searches for similar places and generates personalized suggestions using a generative AI model. These suggestions are returned to the device and displayed to the user.

[1042] For example, if a user uploads an image of a lake they visited last month, the server will analyze the lake's characteristics and suggest places with similar natural scenery, allowing users to discover new places based on their past experiences.

[1043] Finally, the device provides an interface for users to input their ratings and impressions of the places they visited. The user then inputs this information, and the device sends it to the server. The server then stores the rating information in a database and uses it to retrain the generative AI model, thereby improving the accuracy of future suggestions.

[1044] The system allows users to easily plan their holidays hassle-free and efficiently discover new places and experiences.

[1045] An example prompt for using a generative AI model is:

[1046] "Based on the places you've traveled to in the past year and your experiences, what places would you recommend for our next vacation?"

[1047] "Suggest a new travel destination based on photos of places you've recently visited."

[1048] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1049] Step 1:

[1050] The device displays a user registration screen. The user enters basic information such as name, email address, and password. This information is sent by the device to the server. The server receives this basic information and checks the validity of the data. Specifically, it checks whether the email address format is correct and whether the password meets a certain level of strength. After these checks are complete, the server stores the information in a database.

[1051] Step 2:

[1052] The device displays a profile setting screen. The user enters their interests, past travel history, preferred activities, etc. This information is also sent from the device to the server. The server analyzes the received information, creates a user profile, and stores it in a database. For example, if an activity such as "hiking" or "visiting museums" is selected, that information is stored in the database by category.

[1053] Step 3:

[1054] The user inputs a question in natural language about the places they want to go or the activities they want to do through the device's conversational interface. The device then sends the question to the server. The server receives the question and analyzes it using a natural language processing module. Specifically, it extracts keywords from the text and analyzes it to understand its intent. Based on the analysis results, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The generated suggestions are output to the device and displayed to the user.

[1055] For example, if a user asks, "What are some natural places I can go to this weekend?" the server extracts the keyword "natural places" and suggests suitable parks and mountains, taking into account the user's past profile information and current location.

[1056] Step 4:

[1057] The device provides an interface that allows users to upload images they have taken. The user selects and uploads the desired image. This image is then sent from the device to the server. The server passes the received image to an image recognition module, which analyzes its features. Specifically, key objects and landscape features in the image are extracted. The server then uses these features to search a database for similar locations and generates personalized suggestions using a generative AI model. The generated suggestions are output to the device and displayed to the user.

[1058] For example, if a user uploads an image of a lake they visited last month, the server analyzes the lake's characteristics and suggests places with similar natural scenery.

[1059] Step 5:

[1060] The device provides an interface for users to input their ratings and impressions of the places they have visited. The user inputs their ratings and impressions of the places they have visited. This information is sent from the device to the server. The server receives the rating information and stores it in a database. This collected information is also used to retrain the generative AI model. Specifically, the existing model is updated based on the user's rating data, improving the accuracy of future suggestions.

[1061] For example, if a user visits a suggested museum and rates the experience as "very satisfying," this rating information is stored on the server and used as learning data to further personalize future suggestions.

[1062] (Application example 1)

[1063] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1064] Conventional food delivery services have the problem that users must research many menus and restaurants themselves, which makes it time-consuming to make the best choice. Furthermore, simply providing general menu recommendations without fully considering the user's preferences or past ordering history fails to increase user satisfaction. Furthermore, the system lacks a mechanism for utilizing user feedback to make more appropriate suggestions in the future. It also lacks a function to suggest restaurants that serve similar dishes based on a photo of a dish, making it impossible to provide suggestions that meet diverse needs.

[1065] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1066] In this invention, the server includes means for collecting and storing user profile information, means for generating personalized outing suggestions based on the profile information, means for analyzing natural language questions from the user, means for analyzing photos provided by the user and suggesting similar places, means for collecting user feedback and using it to train a generative AI model, means for suggesting restaurants and menus based on the user's food and beverage preferences and order history, and means for receiving questions from the user in real time and providing an interactive interface for suggesting appropriate eating and drinking locations, thereby reducing the burden on the user and enabling personalized and optimal food delivery suggestions.

[1067] "User profile information" refers to individual attributes and behavioral history, including basic information about the user, past order history, preferences, etc.

[1068] "Personalized Suggestions" means suggestions that are individually optimized based on a user's profile information.

[1069] A "natural language question" is a question that the user enters in text format and the system analyzes based on that content.

[1070] "Means for analyzing photos and suggesting similar places" refers to a method for analyzing photos provided by users using image recognition technology and suggesting places with similar characteristics.

[1071] A "generative AI model" is an algorithm that uses machine learning technology to learn from user profile information and feedback, and then generates suggestions based on the results.

[1072] "Feedback" refers to the evaluations and impressions of suggestions provided by users, and is information that is used to improve and learn from the system.

[1073] "Food and beverage preferences and order history" refers to individual preference information for food delivery services, including the dishes a user has ordered in the past, the types of dishes they like, and allergy information.

[1074] A "conversational interface" is a user interface that allows users to enter questions in natural language and receive responses in real time.

[1075] The present invention relates to a system that provides personalized recommendations for places to go and food delivery based on a user's profile information. This system operates primarily through interactions between a server, a terminal, and a user. Specific embodiments of the system are described below.

[1076] User Profile Defaults

[1077] First, the device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, preferred activities, food and beverage preferences, and order history. The server receives this information and stores it as a user profile.

[1078] Generate personalized suggestions

[1079] The user then inputs a natural language question about the places they want to go, the activities they want to do, or the food they want to eat through the device's conversational interface. The server then sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches its database for suitable destinations, restaurants, and menus, and uses a generative AI model to generate personalized suggestions. The suggestions are then sent back to the device and displayed to the user.

[1080] Photo-based recommendations

[1081] The device provides an interface for users to upload photos they have taken, and the user uploads the photo they want. The server passes the uploaded photo to an image recognition module, which analyzes its features. Based on the analysis results, the server searches a database for similar places and dishes and uses a generative AI model to create personalized suggestions. These suggestions are then returned to the device and displayed to the user.

[1082] Learning from user feedback

[1083] The device then provides an interface for users to input feedback about the places they visited and the food they ordered. Users enter their ratings and impressions of the destination and food, and the device sends this to the server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy and allows future suggestions to better adapt to the user's needs.

[1084] Specific examples

[1085] The user enters the following information into the app:

[1086] Name: "Yamada Taro"

[1087] Email address: "yamada.taro@example.com"

[1088] Preferences: Say "I like spicy food"

[1089] After that, when the user asks the app, "What's your recommended lunch today?", the server will suggest the best place to eat and the best food based on the user's preferences. For example, it might suggest a restaurant called "Spicy Corner."

[1090] Prompt Sentence Examples

[1091] "What's your recommended lunch today?"

[1092] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1093] Step 1: Initial User Profile Setup

[1094] The device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The device sends the input data to the server, which receives this information and stores it in a database. The user then uses the device to enter their interests, past orders, and preferences in a profile setting screen. The server also receives this additional information, which is stored in a database.

[1095] Input: User's basic information, interests, past orders, preferences

[1096] Output: User profile information stored in the database

[1097] Step 2: Receiving and parsing the user's question

[1098] The user inputs a question in natural language through the device's dialogue interface. The device then sends the question to the server, which uses a natural language processing module to analyze the question and extract the user's intent. For example, a question might be, "What lunch do you recommend today?"

[1099] Input: User's natural language question

[1100] Output: Parsed question (user intent)

[1101] Step 3: Generate personalized suggestions

[1102] The server searches for suitable candidates from its database based on the analyzed question and user profile information. The server uses a generative AI model to select the most suitable restaurant and menu. For example, if a user's profile states that they "like spicy food," restaurants that serve spicy food will be selected.

[1103] Input: Parsed question content, user profile information

[1104] Output: Personalized suggestions (restaurants and menus)

[1105] Step 4: Sending suggestions to users

[1106] The server sends the generated personalized suggestions to the device, which receives the information and displays the suggestions to the user, for example, a restaurant called "Spicy Corner."

[1107] Input: Personalized suggestions

[1108] Output: Proposal displayed on user's device

[1109] Step 5: Photo-based recommendations

[1110] Users upload photos using the device's interface, which then sends them to a server. The server uses an image recognition module to analyze the photo and extract its features. It then searches a database for places and dishes with similar features and uses a generative AI model to generate suggestions.

[1111] Input: User uploaded photo

[1112] Output: Personalized suggestions for similar places and dishes

[1113] Step 6: Gather user feedback and learn

[1114] Users enter feedback about the places they visited and the food they ordered into the device, which then sends this feedback to the server, which stores it in a database and uses it to retrain the generative AI model, improving the accuracy of future suggestions.

[1115] Input: User feedback

[1116] Output: Feedback stored in a database, trained generative AI model

[1117] Through these steps, the system of the present invention can provide personalized suggestions that match the user's preferences and needs.

[1118] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1119] The present invention combines a system that proposes personalized outing destinations based on a user's profile information with an emotion engine that recognizes the user's emotions. This system operates mainly through interactions between a server, a terminal, and the user. Specific embodiments of the system are described below.

[1120] User Profile Defaults

[1121] First, the device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, and preferred activities. The server receives this information and stores it as a user profile.

[1122] For example, when a user selects "hiking" or "visiting museums" from various travel activities, that information is stored on the server.

[1123] Generate personalized suggestions

[1124] The user then inputs a natural language question about the places they want to go or the activities they want to do through the device's conversational interface. The server sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[1125] For example, if a user asks, "What are some natural places I can go to this weekend?" the server will take into account the user's past profile information and current location to suggest suitable parks or mountains.

[1126] Photo-based recommendations

[1127] The device provides an interface for users to upload photos they have taken, and the user uploads the photo they want. The server passes the uploaded photo to an image recognition module to analyze its features. Based on the analysis results, the server searches a database for similar places and uses a generative AI model to create personalized suggestions. These suggestions are returned to the device and displayed to the user.

[1128] For example, if a user uploads a photo of a lake they visited last month, the server analyzes the lake's characteristics and suggests places with similar natural scenery.

[1129] Learning from user feedback

[1130] The device then provides an interface for users to enter feedback about the places they have visited. Users enter their ratings and impressions of the destinations, and the device sends this to the server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy, allowing future suggestions to better suit the user's needs.

[1131] For example, if a user visits a suggested museum and rates the experience as "very satisfying," this feedback information is stored on the server and used to learn how to further personalize future suggestions.

[1132] Incorporating an emotion engine

[1133] The present invention further incorporates an emotion recognition engine to analyze the user's emotional state and optimize suggestions based on this. The device captures the user's facial expressions, voice, or input text and sends them to the emotion recognition engine. The emotion recognition engine analyzes the emotional state and provides the results to the server. The server then generates more suitable outing suggestions based on this emotional information.

[1134] For example, if the user is perceived as tired, suggestions for relaxing spas and hot springs will be prioritized, while if the user is excited, active activities and events will be suggested.

[1135] In this way, incorporating an emotion engine into this system enables more appropriate suggestions based on the user's current emotional state, further improving the user experience. This configuration allows users to easily plan their holidays without hassle, and efficiently discover new places and experiences.

[1136] The processing flow will be explained below.

[1137] Program processing flow

[1138] User Profile Defaults

[1139] Step 1:

[1140] The user operates the terminal to access the system and the user registration screen is displayed.

[1141] Step 2:

[1142] The user enters basic information such as name, email address, and password and presses the registration button.

[1143] Step 3:

[1144] The terminal transmits the input information to the server.

[1145] Step 4:

[1146] The server stores the received information in a database and returns a success message to the terminal.

[1147] Step 5:

[1148] The device will then display a profile setup screen where users can set their interests, past travel history, and preferred activities.

[1149] Step 6:

[1150] The user selects their preference from the options and presses the save button.

[1151] Step 7:

[1152] The terminal transmits the selected information to the server.

[1153] Step 8:

[1154] The server receives this information and stores it in a database as a user profile.

[1155] Generate personalized suggestions

[1156] Step 9:

[1157] The terminal displays an interactive interface that allows the user to enter questions in natural language.

[1158] Step 10:

[1159] Users enter questions about places they want to go and activities they want to do.

[1160] Step 11:

[1161] The terminal transmits the entered question to the server.

[1162] Step 12:

[1163] The server passes the received question to a natural language processing module to analyze the user's intent.

[1164] Step 13:

[1165] The server searches the database for suitable location candidates based on the analysis results.

[1166] Step 14:

[1167] The server uses a generative AI model to generate suggestions based on the user's profile and the questions asked.

[1168] Step 15:

[1169] The server sends the generated proposal back to the terminal.

[1170] Step 16:

[1171] The device displays suggested destinations to the user.

[1172] Photo-based recommendations

[1173] Step 17:

[1174] The device provides an interface that allows users to upload photos.

[1175] Step 18:

[1176] The user selects a photo and presses the upload button.

[1177] Step 19:

[1178] The terminal transmits the uploaded photos to the server.

[1179] Step 20:

[1180] The server passes the received photo to an image recognition module to analyze its features.

[1181] Step 21:

[1182] The server searches a database for similar locations based on the analysis results.

[1183] Step 22:

[1184] The server uses a generative AI model to generate personalized suggestions.

[1185] Step 23:

[1186] The server sends the proposal back to the terminal.

[1187] Step 24:

[1188] The device will display suggested similar locations to the user.

[1189] Learning from user feedback

[1190] Step 25:

[1191] The device displays an interface that allows the user to enter feedback about the places they have visited.

[1192] Step 26:

[1193] The user enters a rating and comment and presses the send button.

[1194] Step 27:

[1195] The terminal transmits the feedback information to the server.

[1196] Step 28:

[1197] The server stores the received feedback information in a database.

[1198] Step 29:

[1199] The server uses the stored feedback information to retrain the generative AI model.

[1200] Step 30:

[1201] The server improves the model's predictive accuracy through re-learning and reflects this in future proposals.

[1202] Incorporating an emotion engine

[1203] Step 31:

[1204] The device displays an interface that captures the user's facial expressions, voice, or input text.

[1205] Step 32:

[1206] Users input facial expressions and voice through a camera and microphone, and then enter text.

[1207] Step 33:

[1208] The device sends the collected data to an emotion recognition engine.

[1209] Step 34:

[1210] The emotion recognition engine analyzes the user's emotional state and sends the results to the server.

[1211] Step 35:

[1212] The server searches a database for suitable location candidates based on the emotional state.

[1213] Step 36:

[1214] The server uses a generative AI model to generate suggestions based on the user's profile, questions, and emotional state.

[1215] Step 37:

[1216] The server sends the generated proposal back to the terminal.

[1217] Step 38:

[1218] The device displays suggested destinations to the user.

[1219] For example, if the server detects that the user is tired, it will suggest a relaxing spa or hot spring, and if the server detects that the user is excited, it will suggest an active activity or event. This allows the user to receive the most appropriate suggestions based on their emotional state at that time.

[1220] Example 2

[1221] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1222] Conventional personalized recommendation systems make suggestions based solely on the user's profile information and feedback, and therefore are unable to take into account the user's current emotional state. This results in a limited user experience, as the system is unable to provide appropriate suggestions based on the user's current emotions and mood. Furthermore, photo-based recommendation systems are limited to simple similarity analysis, and are therefore unable to provide personalized suggestions that are insufficient to address the user's emotions and interests.

[1223] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1224] In this invention, the server includes means for collecting and storing user profile information, means for generating personalized outing suggestions based on the profile information, means for analyzing natural language questions from the user, means for analyzing photos provided by the user and suggesting similar places, means for collecting user feedback and using it to train the generative AI model, and means for analyzing the user's emotional state and optimizing the suggestions based thereon, thereby enabling more appropriate personalized suggestions according to the user's current emotional state and mood, improving the user experience.

[1225] "User profile information" refers to data such as a user's basic information, interests, past travel history, and preferred activities.

[1226] "Personalized suggestions" are trip suggestions that are individually generated based on the user's profile information and emotional state.

[1227] A "natural language question" is a natural language text question entered by a user through an interactive interface.

[1228] "Image recognition technology" is a technology that uses computer vision to extract and analyze image features.

[1229] A "generative AI model" is a model that uses artificial intelligence to analyze and generate data and provide personalized suggestions to users.

[1230] "User feedback" refers to the ratings and impressions users enter about the places they visit and the experiences they have had.

[1231] "Analyzing emotional state" refers to analyzing and understanding the user's emotions at that time from their facial expressions, voice, or text.

[1232] An "emotion recognition engine" is a software or hardware system for analyzing a user's emotional state.

[1233] The present invention combines a system that proposes personalized outing destinations based on a user's profile information with an emotion engine that recognizes the user's emotions. This system operates mainly through interactions between a server, a terminal, and a user. Specific embodiments of the system are described in detail below.

[1234] User Profile Defaults

[1235] First, the device displays a user registration screen, where the user enters basic information such as name, email address, and password. Next, the server receives the basic information sent from the device and stores it in a database.

[1236] The device then displays a profile setup screen where the user selects their interests, past travel history, and preferred activities. The server receives this information and stores it in a database as a user profile.

[1237] For example, when a user selects "hiking" or "visiting museums" and presses the save button, the server saves this in the database.

[1238] Generate personalized suggestions

[1239] Through the device's conversational interface, users input questions in natural language about places they want to go or activities they want to do. For example, they input a question like, "What are some natural places I can go to this weekend?" The server sends this question to a natural language processing module (e.g., SpaCy or NLTK), which analyzes the question.

[1240] Based on the analysis results, the server searches the database for suitable destinations and generates personalized suggestions using a generative AI model (e.g., GPT-3).The generated suggestions are returned to the device and displayed to the user.

[1241] Photo-based recommendations

[1242] The device provides an interface for users to upload photos they have taken. The user selects the photos they want to upload. Specifically, the user uploads a photo of a lake they visited last month.

[1243] The server passes the uploaded photo to an image recognition module (e.g., TensorFlow or OpenCV) to analyze its features. Based on the analysis results, the server searches a database for similar places and uses a generative AI model to create personalized suggestions. These suggestions are then returned to the device and displayed to the user.

[1244] Learning from user feedback

[1245] The device then provides an interface for users to input their ratings and impressions of the places they visited. The user enters their feedback, and the information is sent from the device to the server.

[1246] The server stores this feedback in a database and uses the collected information to retrain the generative AI model, improving its prediction accuracy and making future suggestions more personalized.

[1247] For example, a user can visit a suggested museum and rate their experience as "very satisfied." This feedback information is stored on the server, and the next suggestion is optimized accordingly.

[1248] Incorporating an emotion engine

[1249] The present invention further incorporates an emotion recognition engine to analyze the user's emotional state and optimize suggestions based on it. Specifically, the device captures the user's facial expressions, voice, or input text and sends them to an emotion recognition engine (e.g., Affectiva or Microsoft Azure Emotion API). The emotion recognition engine analyzes the emotional state and provides the results to the server.

[1250] The server then uses this emotional information to generate more appropriate recommendations for outings. For example, if the user's emotional state is analyzed as "tired," recommendations for relaxing spas and hot springs will be prioritized. If the user's emotional state is analyzed as "excited," active activities and events will be suggested.

[1251] In this way, by incorporating an emotion recognition engine into this system, it becomes possible to make more appropriate suggestions based on the user's current emotional state, further improving the user experience.

[1252] Examples of prompts include, "What are some natural places I can go to this weekend?" or "Can you tell me some places similar to this photo?"

[1253] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1254] Step 1:

[1255] The device displays a user registration screen, where the user enters basic information such as name, email address, and password.

[1256] Input: The user enters their name, email address, and password.

[1257] Output: Basic information about the user that is sent to the server.

[1258] Specifically, the user enters "Yamada Taro," "yamada@example.com," and "password123," and presses the send button.

[1259] Step 2:

[1260] The server receives the basic information sent from the terminal and stores it in a database.

[1261] Input: Basic user information sent from the device.

[1262] Output: Basic information of the user stored in the database.

[1263] Specifically, the server stores "Yamada Taro", "yamada@example.com", and "password123" in the database.

[1264] Step 3:

[1265] The device will then display a profile setup screen where the user can select their interests, past travel history and preferred activities.

[1266] Input: User-selected interests and activities.

[1267] Output: The profile information sent to the server.

[1268] Specifically, the user selects "Hiking" or "Museum Tour" and presses the save button.

[1269] Step 4:

[1270] The server receives this input information and stores it in a database as a user profile.

[1271] Input: Profile information sent from the device.

[1272] Output: User profile stored in the database.

[1273] Specifically, the server stores information about "hiking" and "visiting museums" in a database.

[1274] Step 5:

[1275] Through the device's interactive interface, users input questions in natural language about places they want to go and activities they want to do.

[1276] Input: The user's natural language question.

[1277] Output: The question sent to the server.

[1278] Specifically, the user types, "What are some natural places I can go to this weekend?"

[1279] Step 6:

[1280] The server sends this question to a natural language processing module (e.g., SpaCy or NLTK) to analyze the question.

[1281] Input: The user's natural language question.

[1282] Output: Analysis result of the question.

[1283] Specifically, the server extracts keywords such as "nature" and "this weekend."

[1284] Step 7:

[1285] Based on the analysis results, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model (e.g., GPT-3).

[1286] Input: Question analysis results and user profile information.

[1287] Output: Personalized suggestions.

[1288] Specifically, the server generates a list of parks and mountains based on the user's profile and the criteria "nature" and "this weekend."

[1289] Step 8:

[1290] The generated suggestions are returned to the device and displayed to the user.

[1291] Enter: personalized suggestions.

[1292] Output: The suggestions displayed on the terminal.

[1293] Specifically, the device will display a list of parks and mountains to the user.

[1294] Step 9:

[1295] The device provides an interface for users to upload photos they have taken, and users can select the photos they want to upload.

[1296] Input: A photo uploaded by the user.

[1297] Output: The photo data sent to the server.

[1298] Specifically, the user uploads a photo of a lake they visited last month.

[1299] Step 10:

[1300] The server passes the uploaded photo to an image recognition module (e.g., TensorFlow or OpenCV) to analyze its features.

[1301] Input: The uploaded photo.

[1302] Output: Photo feature analysis results.

[1303] Specifically, the server extracts the features "lake" and "nature" from the photo.

[1304] Step 11:

[1305] Based on the analysis results, the server searches for similar places in its database and uses a generative AI model to create personalized suggestions.

[1306] Input: Photo feature analysis results and user profile information.

[1307] Output: Personalized suggestions.

[1308] Specifically, the server generates and displays similar place suggestions to the user.

[1309] Step 12:

[1310] The device then provides an interface for users to input their ratings and impressions of the places they visited. The user enters their feedback, and the information is sent from the device to the server.

[1311] Input: Feedback entered by the user.

[1312] Output: Feedback data sent to the server.

[1313] Specifically, the user inputs a rating of "very satisfied."

[1314] Step 13:

[1315] The server stores the feedback in a database and uses the collected information to retrain the generative AI model.

[1316] Input: User feedback.

[1317] Output: Feedback information stored in a database and a retrained generative AI model.

[1318] Specifically, the server stores the response "very satisfied" and reflects it in the next proposal.

[1319] Step 14:

[1320] The device captures the user's facial expressions, voice, or input text and sends it to an emotion recognition engine (e.g., Affectiva or Microsoft Azure Emotion API).

[1321] Input: The user's facial expression, voice, or input text.

[1322] Output: Emotion data sent to the emotion recognition engine.

[1323] Specifically, the device takes a photo of the user's facial expression with its camera.

[1324] Step 15:

[1325] The emotion recognition engine analyzes the emotional state and provides the results to the server.

[1326] Input: Emotion data sent from the device.

[1327] Output: The analyzed emotional state results.

[1328] Specifically, the emotion recognition engine analyzes the emotion "tired."

[1329] Step 16:

[1330] The server uses this emotional information to generate more suitable suggestions for outing destinations.

[1331] Input: Parsed emotional state results and user profile information.

[1332] Output: Personalized suggestions.

[1333] Specifically, if the system analyzes that the user is "tired," it generates and displays suggestions for relaxing spas and hot springs to the user.

[1334] (Application example 2)

[1335] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1336] In modern society, eating out or visiting recreational spots is common to escape from busy daily lives, but the system often fails to provide optimal suggestions based on the user's emotional state. This makes it difficult for users to find a place or meal that suits their mood, and as a result, the selected place or meal may be unsatisfying. Therefore, there is a need for a system that can suggest appropriate places to go or meals based on the user's emotional state.

[1337] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting and storing user profile information, means for recognizing the user's emotional state, means for optimizing meal suggestions based on the user's emotional information, means for analyzing questions in natural language from the user, means for analyzing photos provided by the user and suggesting similar places, and means for collecting feedback from the user and using it to train the generative AI model. This makes it possible to suggest places and meals that match the user's emotional state.

[1338] "User profile information" refers to data such as a user's basic information, interests, past behavior, and preferred activities.

[1339] "Personalized destinations" refer to destinations and activities that are individually optimized based on a user's profile information and emotional state.

[1340] A "natural language question" is a question typed by a user in a normal, everyday conversational format for the system to recognize and interpret.

[1341] "Image recognition technology" refers to the technology of extracting and analyzing features from image data using algorithms such as machine learning and deep learning.

[1342] A "generative AI model" refers to a model that uses artificial intelligence to generate new information based on data, and is primarily used for learning suggestions and feedback.

[1343] "Emotional state recognition" refers to a technology that analyzes a user's facial expressions, voice, input text, etc. to identify their emotions at that time.

[1344] "Meal suggestion optimization" refers to the process of suggesting the most suitable meals and restaurants based on the user's current emotional state and profile information.

[1345] "Feedback collection" refers to the process of collecting user evaluations and opinions on services and proposals they have experienced and using them to improve the system.

[1346] The present invention provides a system that provides personalized outing and dining recommendations based on a user's profile information. It also recognizes the user's emotional state and optimizes the recommendations accordingly. A specific embodiment of the system is described below.

[1347] User Profile Defaults

[1348] The device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. The device then displays a profile setting screen, where the user selects their interests, favorite meals, and past visit history. The server receives this information and stores it as a user profile.

[1349] Emotion Recognition and Personalized Suggestions

[1350] The user then inputs their emotional state through the device's conversational interface using natural language or voice. The server sends this input to an emotion recognition engine, which analyzes the emotional state. Based on the analysis, the server searches a database for suitable meals, restaurants, or outings and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[1351] Feedback and Learning

[1352] The device provides an interface for users to enter feedback about the places they visited and the meals they ate. Users enter their ratings and impressions, and the device sends them to a server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy, allowing future suggestions to better adapt to the user's needs.

[1353] For example, if a user says, "I'm tired today," the server will use an emotion recognition engine to analyze the user's state of fatigue and suggest relaxing cafes and restaurants that serve healthy snacks. If the user also says, "I'm feeling very energetic today," the server will suggest restaurants and events that involve vigorous activity.

[1354] Prompt Sentence Examples

[1355] User Input: I'm feeling a bit down today

[1356] Response: You'll see suggestions for warm, inviting cafes and soup restaurants to soothe your thirst.

[1357] User Input: Feeling adventurous today

[1358] Response: Suggestions include restaurants serving exotic and new cuisines and information about food festivals.

[1359] In this way, the system will be able to provide personalized suggestions based on the user's emotional state, further improving the user experience.

[1360] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1361] Step 1: Register your user profile information

[1362] The server receives the user's basic information (name, email address, password, etc.) sent from the device and stores it in a database. The device then displays a profile setting screen for the user, where the user enters their interests, favorite meals, and past visit history. The server receives this input information and stores it in a database as a user profile.

[1363] Input: User's basic information, interests, favorite foods, past visit history

[1364] Output: User profile information stored in a database

[1365] Step 2: Recognizing your emotional state

[1366] The user inputs their emotional state through the terminal using natural language or voice. The server sends this input to the emotion recognition engine and analyzes the emotional state. The analyzed emotional state information is returned to the server.

[1367] Input: Natural language or voice input about the user's emotional state

[1368] Output: Parsed emotional state information

[1369] Step 3: Generate personalized suggestions

[1370] The server uses the analyzed emotional state information to search a database for suitable meals, restaurants, or outings, and then uses a generative AI model to compile these suggestions into personalized suggestions, which are then sent back to the device and displayed to the user.

[1371] Input: Analyzed emotional state information, user profile information

[1372] Output: Personalized dining, restaurant, and outing suggestions

[1373] Step 4: Gather feedback

[1374] The device provides an interface for users to input feedback about the places they visited and the meals they ate. Users enter their ratings and impressions, and the device sends them to a server, which stores the feedback in a database.

[1375] Input: User ratings and comments

[1376] Output: Feedback information stored in a database

[1377] Step 5: Retraining the generative AI model

[1378] The server retrains the generative AI model based on the collected feedback information, improving the model's prediction accuracy and making future suggestions more tailored to the user's needs.

[1379] Input: Feedback information stored in the database

[1380] Output: Retrained generative AI model

[1381] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1382] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1383] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1384] [Fourth embodiment]

[1385] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1386] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1387] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1388] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1389] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1390] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1391] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1392] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1393] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1394] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1395] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1396] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1397] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1398] The present invention relates to a system that provides personalized travel destination suggestions based on a user's profile information. This system operates primarily through interactions between a server, a terminal, and a user. Specific embodiments of the system are described below.

[1399] User Profile Defaults

[1400] First, the device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, and preferred activities. The server receives this information and stores it as a user profile.

[1401] For example, when a user selects "hiking" or "visiting museums" from various travel activities, that information is stored on the server.

[1402] Generate personalized suggestions

[1403] The user then inputs a natural language question about the places they want to go or the activities they want to do through the device's conversational interface. The server sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[1404] For example, if a user asks, "What are some natural places I can go to this weekend?" the server will take into account the user's past profile information and current location to suggest suitable parks or mountains.

[1405] Photo-based recommendations

[1406] The device provides an interface for users to upload photos they have taken, and the user uploads the photo they want. The server passes the uploaded photo to an image recognition module to analyze its features. Based on the analysis results, the server searches a database for similar places and uses a generative AI model to create personalized suggestions. These suggestions are returned to the device and displayed to the user.

[1407] For example, if a user uploads a photo of a lake they visited last month, the server analyzes the lake's characteristics and suggests places with similar natural scenery.

[1408] Learning from user feedback

[1409] The device then provides an interface for users to enter feedback about the places they have visited. Users enter their ratings and impressions of the destinations, and the device sends this to the server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy, allowing future suggestions to better suit the user's needs.

[1410] For example, if a user visits a suggested museum and rates the experience as "very satisfying," this feedback information is stored on the server and used to learn how to further personalize future suggestions.

[1411] The system allows users to easily plan their holidays hassle-free and efficiently discover new places and experiences.

[1412] The processing flow will be explained below.

[1413] Program processing flow

[1414] User Profile Defaults

[1415] Step 1:

[1416] The user operates the terminal to access the system and the user registration screen is displayed.

[1417] Step 2:

[1418] The user enters basic information such as name, email address, and password and presses the registration button.

[1419] Step 3:

[1420] The terminal transmits the input information to the server.

[1421] Step 4:

[1422] The server stores the received information in a database and returns a success message to the terminal.

[1423] Step 5:

[1424] The device will then display a profile setup screen where users can set their interests, past travel history, and preferred activities.

[1425] Step 6:

[1426] The user selects their preference from the options and presses the save button.

[1427] Step 7:

[1428] The terminal transmits the selected information to the server.

[1429] Step 8:

[1430] The server receives this information and stores it in a database as a user profile.

[1431] Generate personalized suggestions

[1432] Step 9:

[1433] The terminal displays an interactive interface that allows the user to enter questions in natural language.

[1434] Step 10:

[1435] Users enter questions about places they want to go and activities they want to do.

[1436] Step 11:

[1437] The terminal transmits the entered question to the server.

[1438] Step 12:

[1439] The server passes the received question to a natural language processing module to analyze the user's intent.

[1440] Step 13:

[1441] The server searches the database for suitable location candidates based on the analysis results.

[1442] Step 14:

[1443] The server uses a generative AI model to generate suggestions based on the user's profile and the questions asked.

[1444] Step 15:

[1445] The server sends the generated proposal back to the terminal.

[1446] Step 16:

[1447] The device displays suggested destinations to the user.

[1448] Photo-based recommendations

[1449] Step 17:

[1450] The device provides an interface that allows users to upload photos.

[1451] Step 18:

[1452] The user selects a photo and presses the upload button.

[1453] Step 19:

[1454] The terminal transmits the uploaded photos to the server.

[1455] Step 20:

[1456] The server passes the received photo to an image recognition module to analyze its features.

[1457] Step 21:

[1458] The server searches a database for similar locations based on the analysis results.

[1459] Step 22:

[1460] The server uses a generative AI model to generate personalized suggestions.

[1461] Step 23:

[1462] The server sends the proposal back to the terminal.

[1463] Step 24:

[1464] The device will display suggested similar locations to the user.

[1465] Learning from user feedback

[1466] Step 25:

[1467] The device displays an interface that allows the user to enter feedback about the places they have visited.

[1468] Step 26:

[1469] The user enters a rating and comment and presses the send button.

[1470] Step 27:

[1471] The terminal transmits the feedback information to the server.

[1472] Step 28:

[1473] The server stores the received feedback information in a database.

[1474] Step 29:

[1475] The server uses the stored feedback information to retrain the generative AI model.

[1476] Step 30:

[1477] The server improves the model's predictive accuracy through re-learning and reflects this in future proposals.

[1478] Example 1

[1479] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1480] Conventional tourist information systems have had challenges in making personalized suggestions based on users' interests and past travel history, and in suggesting appropriate locations in response to questions entered in natural language by users. Furthermore, there were no effective systems for making recommendations based on images provided by users or for retraining generative AI models using user feedback. Thus, a comprehensive system was needed to meet the diverse needs of users and improve their experience.

[1481] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1482] In this invention, the server includes means for collecting and storing user profile information, means for generating personalized outing destination suggestions based on the profile information, means for analyzing questions in natural language from users, means for analyzing images provided by users and suggesting similar places, means for collecting user evaluation information and using it for retraining the generation AI model, means for searching and generating suitable suggestions from a database based on questions entered in natural language, and means for accumulating and utilizing information provided by users. This makes it possible to meet the diverse needs of users and provide outing destination suggestions optimized for each individual user.

[1483] "Profile Information" is the basic data used to generate personalized suggestions, such as a user's basic information, interests, past travel history, and preferred activities.

[1484] "Personalized suggestions" are individually optimized suggestions for places to go based on the user's profile information and current situation.

[1485] A "natural language question" is a natural language question or instruction entered by a user through a dialogue interface.

[1486] "Image analysis" is a technology used to extract features from user-provided images and suggest similar locations.

[1487] "Evaluation information" is feedback data such as satisfaction and impressions provided by users regarding places they have visited or experiences they have had.

[1488] A "generative AI model" is an artificial intelligence model used to generate new suggestions based on collected data.

[1489] A "database" is an information storage system for centrally managing user profile information, past travel history, rating information, etc.

[1490] An "interactive interface" is a user interface that allows a user to enter natural language questions and interact with a system.

[1491] The present invention is a comprehensive system that provides personalized travel destination suggestions based on user profile information. This system operates through the interaction of a server, terminals, and users.

[1492] First, the device displays a user registration screen, where the user enters basic information such as name, email address, and password. This information is sent to the server, validated, and stored in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, preferred activities, etc. This information is also sent to the server and stored as a user profile.

[1493] For example, when a user selects an activity such as "hiking" or "visiting a museum," the server stores this information in a database, thereby accumulating data on the user's individual interests and preferences.

[1494] Next, the user enters a natural language question about the places they want to go or the activities they want to do through the device's conversational interface. The server sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[1495] For example, if a user asks, "Where can I go this weekend to enjoy nature?", the server will take into account the user's past profile information and current location to suggest suitable parks and mountains. This process allows for quick and accurate suggestions to be made in response to the user's question.

[1496] The device also provides an interface for users to upload images they have taken. Users upload their desired images, and the server passes the images to an image recognition module to analyze their features. Based on the analysis results, the server searches for similar places and generates personalized suggestions using a generative AI model. These suggestions are returned to the device and displayed to the user.

[1497] For example, if a user uploads an image of a lake they visited last month, the server will analyze the lake's characteristics and suggest places with similar natural scenery, allowing users to discover new places based on their past experiences.

[1498] Finally, the device provides an interface for users to input their ratings and impressions of the places they visited. The user then inputs this information, and the device sends it to the server. The server then stores the rating information in a database and uses it to retrain the generative AI model, thereby improving the accuracy of future suggestions.

[1499] The system allows users to easily plan their holidays hassle-free and efficiently discover new places and experiences.

[1500] An example prompt for using a generative AI model is:

[1501] "Based on the places you've traveled to in the past year and your experiences, what places would you recommend for our next vacation?"

[1502] "Suggest a new travel destination based on photos of places you've recently visited."

[1503] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1504] Step 1:

[1505] The device displays a user registration screen. The user enters basic information such as name, email address, and password. This information is sent by the device to the server. The server receives this basic information and checks the validity of the data. Specifically, it checks whether the email address format is correct and whether the password meets a certain level of strength. After these checks are complete, the server stores the information in a database.

[1506] Step 2:

[1507] The device displays a profile setting screen. The user enters their interests, past travel history, preferred activities, etc. This information is also sent from the device to the server. The server analyzes the received information, creates a user profile, and stores it in a database. For example, if an activity such as "hiking" or "visiting museums" is selected, that information is stored in the database by category.

[1508] Step 3:

[1509] The user inputs a question in natural language about the places they want to go or the activities they want to do through the device's conversational interface. The device then sends the question to the server. The server receives the question and analyzes it using a natural language processing module. Specifically, it extracts keywords from the text and analyzes it to understand its intent. Based on the analysis results, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The generated suggestions are output to the device and displayed to the user.

[1510] For example, if a user asks, "What are some natural places I can go to this weekend?" the server extracts the keyword "natural places" and suggests suitable parks and mountains, taking into account the user's past profile information and current location.

[1511] Step 4:

[1512] The device provides an interface that allows users to upload images they have taken. The user selects and uploads the desired image. This image is then sent from the device to the server. The server passes the received image to an image recognition module, which analyzes its features. Specifically, key objects and landscape features in the image are extracted. The server then uses these features to search a database for similar locations and generates personalized suggestions using a generative AI model. The generated suggestions are output to the device and displayed to the user.

[1513] For example, if a user uploads an image of a lake they visited last month, the server analyzes the lake's characteristics and suggests places with similar natural scenery.

[1514] Step 5:

[1515] The device provides an interface for users to input their ratings and impressions of the places they have visited. The user inputs their ratings and impressions of the places they have visited. This information is sent from the device to the server. The server receives the rating information and stores it in a database. This collected information is also used to retrain the generative AI model. Specifically, the existing model is updated based on the user's rating data, improving the accuracy of future suggestions.

[1516] For example, if a user visits a suggested museum and rates the experience as "very satisfying," this rating information is stored on the server and used as learning data to further personalize future suggestions.

[1517] (Application example 1)

[1518] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1519] Conventional food delivery services have the problem that users must research many menus and restaurants themselves, which makes it time-consuming to make the best choice. Furthermore, simply providing general menu recommendations without fully considering the user's preferences or past ordering history fails to increase user satisfaction. Furthermore, the system lacks a mechanism for utilizing user feedback to make more appropriate suggestions in the future. It also lacks a function to suggest restaurants that serve similar dishes based on a photo of a dish, making it impossible to provide suggestions that meet diverse needs.

[1520] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1521] In this invention, the server includes means for collecting and storing user profile information, means for generating personalized outing suggestions based on the profile information, means for analyzing natural language questions from the user, means for analyzing photos provided by the user and suggesting similar places, means for collecting user feedback and using it to train a generative AI model, means for suggesting restaurants and menus based on the user's food and beverage preferences and order history, and means for receiving questions from the user in real time and providing an interactive interface for suggesting appropriate eating and drinking locations, thereby reducing the burden on the user and enabling personalized and optimal food delivery suggestions.

[1522] "User profile information" refers to individual attributes and behavioral history, including basic information about the user, past order history, preferences, etc.

[1523] "Personalized Suggestions" means suggestions that are individually optimized based on a user's profile information.

[1524] A "natural language question" is a question that the user enters in text format and the system analyzes based on that content.

[1525] "Means for analyzing photos and suggesting similar places" refers to a method for analyzing photos provided by users using image recognition technology and suggesting places with similar characteristics.

[1526] A "generative AI model" is an algorithm that uses machine learning technology to learn from user profile information and feedback, and then generates suggestions based on the results.

[1527] "Feedback" refers to the evaluations and impressions of suggestions provided by users, and is information that is used to improve and learn from the system.

[1528] "Food and beverage preferences and order history" refers to individual preference information for food delivery services, including the dishes a user has ordered in the past, the types of dishes they like, and allergy information.

[1529] A "conversational interface" is a user interface that allows users to enter questions in natural language and receive responses in real time.

[1530] The present invention relates to a system that provides personalized recommendations for places to go and food delivery based on a user's profile information. This system operates primarily through interactions between a server, a terminal, and a user. Specific embodiments of the system are described below.

[1531] User Profile Defaults

[1532] First, the device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, preferred activities, food and beverage preferences, and order history. The server receives this information and stores it as a user profile.

[1533] Generate personalized suggestions

[1534] The user then inputs a natural language question about the places they want to go, the activities they want to do, or the food they want to eat through the device's conversational interface. The server then sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches its database for suitable destinations, restaurants, and menus, and uses a generative AI model to generate personalized suggestions. The suggestions are then sent back to the device and displayed to the user.

[1535] Photo-based recommendations

[1536] The device provides an interface for users to upload photos they have taken, and the user uploads the photo they want. The server passes the uploaded photo to an image recognition module, which analyzes its features. Based on the analysis results, the server searches a database for similar places and dishes and uses a generative AI model to create personalized suggestions. These suggestions are then returned to the device and displayed to the user.

[1537] Learning from user feedback

[1538] The device then provides an interface for users to input feedback about the places they visited and the food they ordered. Users enter their ratings and impressions of the destination and food, and the device sends this to the server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy and allows future suggestions to better adapt to the user's needs.

[1539] Specific examples

[1540] The user enters the following information into the app:

[1541] Name: "Yamada Taro"

[1542] Email address: "yamada.taro@example.com"

[1543] Preferences: Say "I like spicy food"

[1544] After that, when the user asks the app, "What's your recommended lunch today?", the server will suggest the best place to eat and the best food based on the user's preferences. For example, it might suggest a restaurant called "Spicy Corner."

[1545] Prompt Sentence Examples

[1546] "What's your recommended lunch today?"

[1547] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1548] Step 1: Initial User Profile Setup

[1549] The device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The device sends the input data to the server, which receives this information and stores it in a database. The user then uses the device to enter their interests, past orders, and preferences in a profile setting screen. The server also receives this additional information, which is stored in a database.

[1550] Input: User's basic information, interests, past orders, preferences

[1551] Output: User profile information stored in the database

[1552] Step 2: Receiving and parsing the user's question

[1553] The user inputs a question in natural language through the device's dialogue interface. The device then sends the question to the server, which uses a natural language processing module to analyze the question and extract the user's intent. For example, a question might be, "What lunch do you recommend today?"

[1554] Input: User's natural language question

[1555] Output: Parsed question (user intent)

[1556] Step 3: Generate personalized suggestions

[1557] The server searches for suitable candidates from its database based on the analyzed question and user profile information. The server uses a generative AI model to select the most suitable restaurant and menu. For example, if a user's profile states that they "like spicy food," restaurants that serve spicy food will be selected.

[1558] Input: Parsed question content, user profile information

[1559] Output: Personalized suggestions (restaurants and menus)

[1560] Step 4: Sending suggestions to users

[1561] The server sends the generated personalized suggestions to the device, which receives the information and displays the suggestions to the user, for example, a restaurant called "Spicy Corner."

[1562] Input: Personalized suggestions

[1563] Output: Proposal displayed on user's device

[1564] Step 5: Photo-based recommendations

[1565] Users upload photos using the device's interface, which then sends them to a server. The server uses an image recognition module to analyze the photo and extract its features. It then searches a database for places and dishes with similar features and uses a generative AI model to generate suggestions.

[1566] Input: User uploaded photo

[1567] Output: Personalized suggestions for similar places and dishes

[1568] Step 6: Gather user feedback and learn

[1569] Users enter feedback about the places they visited and the food they ordered into the device, which then sends this feedback to the server, which stores it in a database and uses it to retrain the generative AI model, improving the accuracy of future suggestions.

[1570] Input: User feedback

[1571] Output: Feedback stored in a database, trained generative AI model

[1572] Through these steps, the system of the present invention can provide personalized suggestions that match the user's preferences and needs.

[1573] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1574] The present invention combines a system that proposes personalized outing destinations based on a user's profile information with an emotion engine that recognizes the user's emotions. This system operates mainly through interactions between a server, a terminal, and the user. Specific embodiments of the system are described below.

[1575] User Profile Defaults

[1576] First, the device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. Next, the device displays a profile setting screen, where the user selects their interests, past travel history, and preferred activities. The server receives this information and stores it as a user profile.

[1577] For example, when a user selects "hiking" or "visiting museums" from various travel activities, that information is stored on the server.

[1578] Generate personalized suggestions

[1579] The user then inputs a natural language question about the places they want to go or the activities they want to do through the device's conversational interface. The server sends the question to a natural language processing module, which analyzes it. Based on the analysis, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[1580] For example, if a user asks, "What are some natural places I can go to this weekend?" the server will take into account the user's past profile information and current location to suggest suitable parks or mountains.

[1581] Photo-based recommendations

[1582] The device provides an interface for users to upload photos they have taken, and the user uploads the photo they want. The server passes the uploaded photo to an image recognition module to analyze its features. Based on the analysis results, the server searches a database for similar places and uses a generative AI model to create personalized suggestions. These suggestions are returned to the device and displayed to the user.

[1583] For example, if a user uploads a photo of a lake they visited last month, the server analyzes the lake's characteristics and suggests places with similar natural scenery.

[1584] Learning from user feedback

[1585] The device then provides an interface for users to enter feedback about the places they have visited. Users enter their ratings and impressions of the destinations, and the device sends this to the server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy, allowing future suggestions to better suit the user's needs.

[1586] For example, if a user visits a suggested museum and rates the experience as "very satisfying," this feedback information is stored on the server and used to learn how to further personalize future suggestions.

[1587] Incorporating an emotion engine

[1588] The present invention further incorporates an emotion recognition engine to analyze the user's emotional state and optimize suggestions based on this. The device captures the user's facial expressions, voice, or input text and sends them to the emotion recognition engine. The emotion recognition engine analyzes the emotional state and provides the results to the server. The server then generates more suitable outing suggestions based on this emotional information.

[1589] For example, if the user is perceived as tired, suggestions for relaxing spas and hot springs will be prioritized, while if the user is excited, active activities and events will be suggested.

[1590] In this way, incorporating an emotion engine into this system enables more appropriate suggestions based on the user's current emotional state, further improving the user experience. This configuration allows users to easily plan their holidays without hassle, and efficiently discover new places and experiences.

[1591] The processing flow will be explained below.

[1592] Program processing flow

[1593] User Profile Defaults

[1594] Step 1:

[1595] The user operates the terminal to access the system and the user registration screen is displayed.

[1596] Step 2:

[1597] The user enters basic information such as name, email address, and password and presses the registration button.

[1598] Step 3:

[1599] The terminal transmits the input information to the server.

[1600] Step 4:

[1601] The server stores the received information in a database and returns a success message to the terminal.

[1602] Step 5:

[1603] The device will then display a profile setup screen where users can set their interests, past travel history, and preferred activities.

[1604] Step 6:

[1605] The user selects their preference from the options and presses the save button.

[1606] Step 7:

[1607] The terminal transmits the selected information to the server.

[1608] Step 8:

[1609] The server receives this information and stores it in a database as a user profile.

[1610] Generate personalized suggestions

[1611] Step 9:

[1612] The terminal displays an interactive interface that allows the user to enter questions in natural language.

[1613] Step 10:

[1614] Users enter questions about places they want to go and activities they want to do.

[1615] Step 11:

[1616] The terminal transmits the entered question to the server.

[1617] Step 12:

[1618] The server passes the received question to a natural language processing module to analyze the user's intent.

[1619] Step 13:

[1620] The server searches the database for suitable location candidates based on the analysis results.

[1621] Step 14:

[1622] The server uses a generative AI model to generate suggestions based on the user's profile and the questions asked.

[1623] Step 15:

[1624] The server sends the generated proposal back to the terminal.

[1625] Step 16:

[1626] The device displays suggested destinations to the user.

[1627] Photo-based recommendations

[1628] Step 17:

[1629] The device provides an interface that allows users to upload photos.

[1630] Step 18:

[1631] The user selects a photo and presses the upload button.

[1632] Step 19:

[1633] The terminal transmits the uploaded photos to the server.

[1634] Step 20:

[1635] The server passes the received photo to an image recognition module to analyze its features.

[1636] Step 21:

[1637] The server searches a database for similar locations based on the analysis results.

[1638] Step 22:

[1639] The server uses a generative AI model to generate personalized suggestions.

[1640] Step 23:

[1641] The server sends the proposal back to the terminal.

[1642] Step 24:

[1643] The device will display suggested similar locations to the user.

[1644] Learning from user feedback

[1645] Step 25:

[1646] The device displays an interface that allows the user to enter feedback about the places they have visited.

[1647] Step 26:

[1648] The user enters a rating and comment and presses the send button.

[1649] Step 27:

[1650] The terminal transmits the feedback information to the server.

[1651] Step 28:

[1652] The server stores the received feedback information in a database.

[1653] Step 29:

[1654] The server uses the stored feedback information to retrain the generative AI model.

[1655] Step 30:

[1656] The server improves the model's predictive accuracy through re-learning and reflects this in future proposals.

[1657] Incorporating an emotion engine

[1658] Step 31:

[1659] The device displays an interface that captures the user's facial expressions, voice, or input text.

[1660] Step 32:

[1661] Users input facial expressions and voice through a camera and microphone, and then enter text.

[1662] Step 33:

[1663] The device sends the collected data to an emotion recognition engine.

[1664] Step 34:

[1665] The emotion recognition engine analyzes the user's emotional state and sends the results to the server.

[1666] Step 35:

[1667] The server searches a database for suitable location candidates based on the emotional state.

[1668] Step 36:

[1669] The server uses a generative AI model to generate suggestions based on the user's profile, questions, and emotional state.

[1670] Step 37:

[1671] The server sends the generated proposal back to the terminal.

[1672] Step 38:

[1673] The device displays suggested destinations to the user.

[1674] For example, if the server detects that the user is tired, it will suggest a relaxing spa or hot spring, and if the server detects that the user is excited, it will suggest an active activity or event. This allows the user to receive the most appropriate suggestions based on their emotional state at that time.

[1675] Example 2

[1676] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1677] Conventional personalized recommendation systems make suggestions based solely on the user's profile information and feedback, and therefore are unable to take into account the user's current emotional state. This results in a limited user experience, as the system is unable to provide appropriate suggestions based on the user's current emotions and mood. Furthermore, photo-based recommendation systems are limited to simple similarity analysis, and are therefore unable to provide personalized suggestions that are insufficient to address the user's emotions and interests.

[1678] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1679] In this invention, the server includes means for collecting and storing user profile information, means for generating personalized outing suggestions based on the profile information, means for analyzing natural language questions from the user, means for analyzing photos provided by the user and suggesting similar places, means for collecting user feedback and using it to train the generative AI model, and means for analyzing the user's emotional state and optimizing the suggestions based thereon, thereby enabling more appropriate personalized suggestions according to the user's current emotional state and mood, improving the user experience.

[1680] "User profile information" refers to data such as a user's basic information, interests, past travel history, and preferred activities.

[1681] "Personalized suggestions" are trip suggestions that are individually generated based on the user's profile information and emotional state.

[1682] A "natural language question" is a natural language text question entered by a user through an interactive interface.

[1683] "Image recognition technology" is a technology that uses computer vision to extract and analyze image features.

[1684] A "generative AI model" is a model that uses artificial intelligence to analyze and generate data and provide personalized suggestions to users.

[1685] "User feedback" refers to the ratings and impressions users enter about the places they visit and the experiences they have had.

[1686] "Analyzing emotional state" refers to analyzing and understanding the user's emotions at that time from their facial expressions, voice, or text.

[1687] An "emotion recognition engine" is a software or hardware system for analyzing a user's emotional state.

[1688] The present invention combines a system that proposes personalized outing destinations based on a user's profile information with an emotion engine that recognizes the user's emotions. This system operates mainly through interactions between a server, a terminal, and a user. Specific embodiments of the system are described in detail below.

[1689] User Profile Defaults

[1690] First, the device displays a user registration screen, where the user enters basic information such as name, email address, and password. Next, the server receives the basic information sent from the device and stores it in a database.

[1691] The device then displays a profile setup screen where the user selects their interests, past travel history, and preferred activities. The server receives this information and stores it in a database as a user profile.

[1692] For example, when a user selects "hiking" or "visiting museums" and presses the save button, the server saves this in the database.

[1693] Generate personalized suggestions

[1694] Through the device's conversational interface, users input questions in natural language about places they want to go or activities they want to do. For example, they input a question like, "What are some natural places I can go to this weekend?" The server sends this question to a natural language processing module (e.g., SpaCy or NLTK), which analyzes the question.

[1695] Based on the analysis results, the server searches the database for suitable destinations and generates personalized suggestions using a generative AI model (e.g., GPT-3).The generated suggestions are returned to the device and displayed to the user.

[1696] Photo-based recommendations

[1697] The device provides an interface for users to upload photos they have taken. The user selects the photos they want to upload. Specifically, the user uploads a photo of a lake they visited last month.

[1698] The server passes the uploaded photo to an image recognition module (e.g., TensorFlow or OpenCV) to analyze its features. Based on the analysis results, the server searches a database for similar places and uses a generative AI model to create personalized suggestions. These suggestions are then returned to the device and displayed to the user.

[1699] Learning from user feedback

[1700] The device then provides an interface for users to input their ratings and impressions of the places they visited. The user enters their feedback, and the information is sent from the device to the server.

[1701] The server stores this feedback in a database and uses the collected information to retrain the generative AI model, improving its prediction accuracy and making future suggestions more personalized.

[1702] For example, a user visits a suggested museum and rates the experience as "very satisfying." This feedback information is stored on the server, and the next suggestion is optimized based on it.

[1703] Incorporating an emotion engine

[1704] The present invention further incorporates an emotion recognition engine to analyze the user's emotional state and optimize suggestions based on it. Specifically, the device captures the user's facial expressions, voice, or input text and sends them to an emotion recognition engine (e.g., Affectiva or Microsoft Azure Emotion API). The emotion recognition engine analyzes the emotional state and provides the results to the server.

[1705] The server then uses this emotional information to generate more appropriate recommendations for outings. For example, if the user's emotional state is analyzed as "tired," recommendations for relaxing spas and hot springs will be prioritized. If the user's emotional state is analyzed as "excited," active activities and events will be suggested.

[1706] In this way, by incorporating an emotion recognition engine into this system, it becomes possible to make more appropriate suggestions based on the user's current emotional state, further improving the user experience.

[1707] Examples of prompts include, "What are some natural places I can go to this weekend?" or "Can you tell me some places similar to this photo?"

[1708] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1709] Step 1:

[1710] The device displays a user registration screen, where the user enters basic information such as name, email address, and password.

[1711] Input: The user enters their name, email address, and password.

[1712] Output: Basic information about the user that is sent to the server.

[1713] Specifically, the user enters "Yamada Taro," "yamada@example.com," and "password123," and presses the send button.

[1714] Step 2:

[1715] The server receives the basic information sent from the terminal and stores it in a database.

[1716] Input: Basic user information sent from the device.

[1717] Output: Basic information of the user stored in the database.

[1718] Specifically, the server stores "Yamada Taro", "yamada@example.com", and "password123" in the database.

[1719] Step 3:

[1720] The device will then display a profile setup screen where the user can select their interests, past travel history and preferred activities.

[1721] Input: User-selected interests and activities.

[1722] Output: The profile information sent to the server.

[1723] Specifically, the user selects "Hiking" or "Museum Tour" and presses the save button.

[1724] Step 4:

[1725] The server receives this input information and stores it in a database as a user profile.

[1726] Input: Profile information sent from the device.

[1727] Output: User profile stored in the database.

[1728] Specifically, the server stores information about "hiking" and "visiting museums" in a database.

[1729] Step 5:

[1730] Through the device's interactive interface, users input questions in natural language about places they want to go and activities they want to do.

[1731] Input: The user's natural language question.

[1732] Output: The question sent to the server.

[1733] Specifically, the user types, "What are some natural places I can go to this weekend?"

[1734] Step 6:

[1735] The server sends this question to a natural language processing module (e.g., SpaCy or NLTK) to analyze the question.

[1736] Input: The user's natural language question.

[1737] Output: Analysis result of the question.

[1738] Specifically, the server extracts keywords such as "nature" and "this weekend."

[1739] Step 7:

[1740] Based on the analysis results, the server searches a database for suitable destinations and generates personalized suggestions using a generative AI model (e.g., GPT-3).

[1741] Input: Question analysis results and user profile information.

[1742] Output: Personalized suggestions.

[1743] Specifically, the server generates a list of parks and mountains based on the user's profile and the criteria "nature" and "this weekend."

[1744] Step 8:

[1745] The generated suggestions are returned to the device and displayed to the user.

[1746] Enter: personalized suggestions.

[1747] Output: The suggestions displayed on the terminal.

[1748] Specifically, the device will display a list of parks and mountains to the user.

[1749] Step 9:

[1750] The device provides an interface for users to upload photos they have taken, and users can select the photos they want to upload.

[1751] Input: A photo uploaded by the user.

[1752] Output: The photo data sent to the server.

[1753] Specifically, the user uploads a photo of a lake they visited last month.

[1754] Step 10:

[1755] The server passes the uploaded photo to an image recognition module (e.g., TensorFlow or OpenCV) to analyze its features.

[1756] Input: The uploaded photo.

[1757] Output: Photo feature analysis results.

[1758] Specifically, the server extracts the features "lake" and "nature" from the photo.

[1759] Step 11:

[1760] Based on the analysis results, the server searches for similar places in its database and uses a generative AI model to create personalized suggestions.

[1761] Input: Photo feature analysis results and user profile information.

[1762] Output: Personalized suggestions.

[1763] Specifically, the server generates and displays similar place suggestions to the user.

[1764] Step 12:

[1765] The device then provides an interface for users to input their ratings and impressions of the places they visited. The user enters their feedback, and the information is sent from the device to the server.

[1766] Input: Feedback entered by the user.

[1767] Output: Feedback data sent to the server.

[1768] Specifically, the user inputs a rating of "very satisfied."

[1769] Step 13:

[1770] The server stores the feedback in a database and uses the collected information to retrain the generative AI model.

[1771] Input: User feedback.

[1772] Output: Feedback information stored in a database and a retrained generative AI model.

[1773] Specifically, the server stores the response "very satisfied" and reflects it in the next proposal.

[1774] Step 14:

[1775] The device captures the user's facial expressions, voice, or input text and sends it to an emotion recognition engine (e.g., Affectiva or Microsoft Azure Emotion API).

[1776] Input: The user's facial expression, voice, or input text.

[1777] Output: Emotion data sent to the emotion recognition engine.

[1778] Specifically, the device takes a photo of the user's facial expression with its camera.

[1779] Step 15:

[1780] The emotion recognition engine analyzes the emotional state and provides the results to the server.

[1781] Input: Emotion data sent from the device.

[1782] Output: The analyzed emotional state results.

[1783] Specifically, the emotion recognition engine analyzes the emotion "tired."

[1784] Step 16:

[1785] The server uses this emotional information to generate more suitable suggestions for outing destinations.

[1786] Input: Parsed emotional state results and user profile information.

[1787] Output: Personalized suggestions.

[1788] Specifically, if the system analyzes that the user is "tired," it generates and displays suggestions for relaxing spas and hot springs to the user.

[1789] (Application example 2)

[1790] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1791] In modern society, eating out or visiting recreational spots is common to escape from busy daily lives, but the system often fails to provide optimal suggestions based on the user's emotional state. This makes it difficult for users to find a place or meal that suits their mood, and as a result, the selected place or meal may be unsatisfying. Therefore, there is a need for a system that can suggest appropriate places to go or meals based on the user's emotional state.

[1792] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting and storing user profile information, means for recognizing the user's emotional state, means for optimizing meal suggestions based on the user's emotional information, means for analyzing questions in natural language from the user, means for analyzing photos provided by the user and suggesting similar places, and means for collecting feedback from the user and using it to train the generative AI model. This makes it possible to suggest places and meals that match the user's emotional state.

[1793] "User profile information" refers to data such as a user's basic information, interests, past behavior, and preferred activities.

[1794] "Personalized destinations" refer to destinations and activities that are individually optimized based on a user's profile information and emotional state.

[1795] A "natural language question" is a question typed by a user in a normal, everyday conversational format for the system to recognize and interpret.

[1796] "Image recognition technology" refers to the technology of extracting and analyzing features from image data using algorithms such as machine learning and deep learning.

[1797] A "generative AI model" refers to a model that uses artificial intelligence to generate new information based on data, and is primarily used for learning suggestions and feedback.

[1798] "Emotional state recognition" refers to a technology that analyzes a user's facial expressions, voice, input text, etc. to identify their emotions at that time.

[1799] "Meal suggestion optimization" refers to the process of suggesting the most suitable meals and restaurants based on the user's current emotional state and profile information.

[1800] "Feedback collection" refers to the process of collecting user evaluations and opinions on services and proposals they have experienced and using them to improve the system.

[1801] The present invention provides a system that provides personalized outing and dining recommendations based on a user's profile information. It also recognizes the user's emotional state and optimizes the recommendations accordingly. A specific embodiment of the system is described below.

[1802] User Profile Defaults

[1803] The device displays a user registration screen, where the user enters basic information (such as name, email address, and password). The server receives the entered information and stores it in a database. The device then displays a profile setting screen, where the user selects their interests, favorite meals, and past visit history. The server receives this information and stores it as a user profile.

[1804] Emotion Recognition and Personalized Suggestions

[1805] The user then inputs their emotional state through the device's conversational interface using natural language or voice. The server sends this input to an emotion recognition engine, which analyzes the emotional state. Based on the analysis, the server searches a database for suitable meals, restaurants, or outings and generates personalized suggestions using a generative AI model. The suggestions are then returned to the device and displayed to the user.

[1806] Feedback and Learning

[1807] The device provides an interface for users to enter feedback about the places they visited and the meals they ate. Users enter their ratings and impressions, and the device sends them to a server. The server stores the feedback in a database and uses the collected information to retrain the generative AI model. This improves the model's prediction accuracy, allowing future suggestions to better adapt to the user's needs.

[1808] For example, if a user says, "I'm tired today," the server will use an emotion recognition engine to analyze the user's state of fatigue and suggest relaxing cafes and restaurants that serve healthy snacks. If the user also says, "I'm feeling very energetic today," the server will suggest restaurants and events that involve vigorous activity.

[1809] Prompt Sentence Examples

[1810] User Input: I'm feeling a bit down today

[1811] Response: You'll see suggestions for warm, inviting cafes and soup restaurants to soothe your thirst.

[1812] User Input: Feeling adventurous today

[1813] Response: Suggestions include restaurants serving exotic and new cuisines and information about food festivals.

[1814] In this way, the system will be able to provide personalized suggestions based on the user's emotional state, further improving the user experience.

[1815] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1816] Step 1: Register your user profile information

[1817] The server receives the user's basic information (name, email address, password, etc.) sent from the device and stores it in a database. The device then displays a profile setting screen for the user, where the user enters their interests, favorite meals, and past visit history. The server receives this input information and stores it in a database as a user profile.

[1818] Input: User's basic information, interests, favorite foods, past visit history

[1819] Output: User profile information stored in a database

[1820] Step 2: Recognizing your emotional state

[1821] The user inputs their emotional state through the terminal using natural language or voice. The server sends this input to the emotion recognition engine and analyzes the emotional state. The analyzed emotional state information is returned to the server.

[1822] Input: Natural language or voice input about the user's emotional state

[1823] Output: Parsed emotional state information

[1824] Step 3: Generate personalized suggestions

[1825] The server uses the analyzed emotional state information to search a database for suitable meals, restaurants, or outings, and then uses a generative AI model to compile these suggestions into personalized suggestions, which are then sent back to the device and displayed to the user.

[1826] Input: Analyzed emotional state information, user profile information

[1827] Output: Personalized dining, restaurant, and outing suggestions

[1828] Step 4: Gather feedback

[1829] The device provides an interface for users to input feedback about the places they visited and the meals they ate. Users enter their ratings and impressions, and the device sends them to a server, which stores the feedback in a database.

[1830] Input: User ratings and comments

[1831] Output: Feedback information stored in a database

[1832] Step 5: Retraining the generative AI model

[1833] The server retrains the generative AI model based on the collected feedback information, improving the model's prediction accuracy and making future suggestions more tailored to the user's needs.

[1834] Input: Feedback information stored in the database

[1835] Output: Retrained generative AI model

[1836] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1837] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1838] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1839] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1840] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1841] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1842] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1843] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1844] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1845] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1846] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1847] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1848] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1849] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1850] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1851] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1852] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1853] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1854] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1855] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1856] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1857] The following is further disclosed regarding the above embodiment.

[1858] (Claim 1)

[1859] means for collecting and storing user profile information;

[1860] means for generating personalized destination suggestions based on the profile information;

[1861] A means of analyzing natural language questions from users;

[1862] A means of analyzing user-provided photos and suggesting similar places;

[1863] A means to collect user feedback and use it to train generative AI models

[1864] A system including:

[1865] (Claim 2)

[1866] 10. The system of claim 1, further comprising means for providing an interactive interface for a user to input a natural language question.

[1867] (Claim 3)

[1868] 10. The system of claim 1, further comprising means for extracting features by using image recognition techniques to analyze the photograph.

[1869] "Example 1"

[1870] (Claim 1)

[1871] means for collecting and storing user profile information;

[1872] means for generating personalized destination suggestions based on the profile information;

[1873] A means of analyzing natural language questions from users;

[1874] A means of analyzing user-provided images and suggesting similar locations;

[1875] A means of collecting user evaluation information and using it to retrain the generative AI model;

[1876] A means for searching and generating suitable suggestions from a database based on a question entered in natural language;

[1877] A means of accumulating and utilizing information from users

[1878] A system including:

[1879] (Claim 2)

[1880] 10. The system of claim 1, further comprising means for providing an interactive interface for a user to input a natural language question.

[1881] (Claim 3)

[1882] 10. The system of claim 1, further comprising means for analyzing the image using image recognition techniques to extract features.

[1883] "Application Example 1"

[1884] (Claim 1)

[1885] means for collecting and storing user profile information;

[1886] means for generating personalized destination suggestions based on the profile information;

[1887] A means of analyzing natural language questions from users;

[1888] A means of analyzing user-provided photos and suggesting similar places;

[1889] A means of collecting user feedback and using it to train the generative AI model; and

[1890] a means of suggesting restaurants and menus based on a user's food and beverage preferences and ordering history;

[1891] The system includes a means for providing an interactive interface that receives real-time questions from a user and suggests suitable dining locations.

[1892] (Claim 2)

[1893] means for providing an interactive interface for a user to input a natural language question;

[1894] The system of claim 1, further comprising means for analyzing the user's food photos and suggesting places to eat that serve similar dishes.

[1895] (Claim 3)

[1896] 10. The system of claim 1, further comprising means for extracting features by using image recognition techniques to analyze the photograph.

[1897] (Claim 4)

[1898] The system of claim 1, further comprising means for collecting feedback on the user's eating and drinking habits and using it to retrain the generative AI model.

[1899] "Example 2: Combining Emotion Engines"

[1900] (Claim 1)

[1901] means for collecting and storing user profile information;

[1902] means for generating personalized destination suggestions based on the profile information;

[1903] A means of analyzing natural language questions from users;

[1904] A means of analyzing user-provided photos and suggesting similar places;

[1905] A means of collecting user feedback and using it to train the generative AI model; and

[1906] A means of analyzing the user's emotional state and optimizing suggestions based on this

[1907] A system including:

[1908] (Claim 2)

[1909] 10. The system of claim 1, further comprising means for providing an interactive interface for a user to input a natural language question.

[1910] (Claim 3)

[1911] 10. The system of claim 1, further comprising means for extracting features by using image recognition techniques to analyze the photograph.

[1912] "Application example 2 when combining emotion engines"

[1913] (Claim 1)

[1914] means for collecting and storing user profile information;

[1915] means for generating personalized destination suggestions based on the profile information;

[1916] A means of analyzing natural language questions from users;

[1917] A means of analyzing user-provided photos and suggesting similar places;

[1918] A means of collecting user feedback and using it to train the generative AI model; and

[1919] a means for recognizing the emotional state of a user;

[1920] A means to optimize meal recommendations based on user sentiment

[1921] A system including:

[1922] (Claim 2)

[1923] 10. The system of claim 1, further comprising means for providing an interactive interface for a user to input a natural language question.

[1924] (Claim 3)

[1925] 10. The system of claim 1, further comprising means for extracting features by using image recognition techniques to analyze the photograph. [Explanation of symbols]

[1926] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for collecting and storing user profile information; means for generating personalized destination suggestions based on the profile information; A means of analyzing natural language questions from users; A means of analyzing user-provided photos and suggesting similar places; A means to collect user feedback and use it to train generative AI models A system including:

2. 10. The system of claim 1, further comprising means for providing an interactive interface for a user to input a natural language question.

3. 10. The system of claim 1, further comprising means for analyzing the photograph using image recognition techniques to extract features.

Images