system

Abstract

Provide a system. A method for receiving a request from a user includes: means for obtaining user historical usage and profile data; a means for referencing a database for generating suggestions; means for generating optimal suggestions according to user requirements; means for transmitting the generated suggestions to a user; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, many people are extremely busy in their daily lives, and they need to efficiently complete even small daily tasks. For example, the time it takes to decide what to make for dinner, plan a trip, or choose an outfit, etc. Providing ways to streamline these tasks and save users time is an important challenge for improving quality of life. [Means for solving the problem]

[0005] The present invention provides a system that receives requests from users, acquires their past usage history and profile data, references a database, generates suggestions that best fit the user's needs, and sends the suggestions to the user. Specifically, the system can provide suggestions for dinner menus, travel plans and transportation options, and fashion coordination. This system allows users to streamline small daily tasks and save time.

[0006] The "means for receiving requests from users" is a function for receiving requests sent by users through a chat application or other interface.

[0007] "Profile data" refers to a set of information related to a user, such as the user's past preferences, allergy information, usage history, and ingredient availability.

[0008] The "means for generating optimal proposals" is a function that automatically generates proposals that match the user's requirements based on profile data.

[0009] The "means for referencing a database" is a function for accessing a database that stores user profile data, recipe data, travel information, fashion information, and the like.

[0010] The "dinner menu suggestion means" is a function for suggesting an appropriate dinner menu based on the user's food preferences, allergy information, and food ingredient availability.

[0011] The "travel planning and transportation suggestion means" is a function for suggesting optimal travel plans and transportation means based on the user's travel history, interests, budget, available time, etc.

[0012] The "fashion coordination suggestion means" is a function that suggests appropriate fashion coordination based on the user's past fashion history, preferences, and weather information.

[0013] "User usage history" refers to a record of requests and selections made by a user through an application or system in the past. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The present invention is a system that streamlines users' daily tasks by receiving requests from users and providing optimal suggestions based on the users' past usage history and profile data. This system is mainly composed of a server, a terminal, and a user.

[0036] Menu suggestion function

[0037] Sending and Receiving Requests

[0038] A user sends a request using a chat application, such as "Tell me what's for dinner." The device (smartphone or computer) sends this request to a server. The server analyzes the received request and starts the process of proposing the menu desired by the user.

[0039] Acquisition and analysis of user information

[0040] The server retrieves profile data from the database based on the user ID. The profile data includes the user's past preferences, allergy information, current food stock status, etc. This allows the server to understand the user's preferences and restrictions.

[0041] Menu generation and suggestions

[0042] The server searches and selects recipes that can be suggested to the user from the recipe database, taking into account the current season and the availability of ingredients. Once the optimal menu is determined, the server sends it back to the chat application. The user can then view the suggested menu through the chat application on their device.

[0043] Examples:

[0044] When a user requests, "What would you like for dinner tonight?" the server responds, "Taking into account your past preferences and the current season, I would like to suggest grilled salmon and a salad."

[0045] Travel planning and transportation suggestions

[0046] Sending and Receiving Requests

[0047] A user sends a request to a chat application saying, "Plan a trip for next weekend." The device sends this request to a server, which receives the request and analyzes its contents.

[0048] Obtaining user information and travel conditions

[0049] Based on the user ID, the server retrieves profile data such as past travel history, interests, available budget and time from the database, and also calls external travel information APIs to collect local information and accommodation information.

[0050] Travel plan and transportation generation

[0051] The server generates an optimal travel plan based on this information and suggests the most suitable means of transportation for the user (e.g., train, plane, rental car). The generated plan and means of transportation are sent to the chat application, where the user can confirm them.

[0052] Examples:

[0053] When a user requests, "I'd like to go on a trip somewhere next weekend," the server responds, "Taking into account your past travel history and current interests, I would like to suggest a day trip to Kyoto. The Shinkansen would be a convenient means of transportation."

[0054] Clothing coordination suggestion function

[0055] Sending and Receiving Requests

[0056] The user sends a request to the chat application saying, "Tell me what outfit I'm wearing today." The device sends this request to the server. The server receives the request and analyzes its contents.

[0057] Obtaining user information and weather information

[0058] The server retrieves profile data such as past fashion history and preferences from a database based on the user ID, and then calls an external weather API to retrieve current weather information.

[0059] Coordination generation and suggestions

[0060] The server generates the optimal coordinate based on this information, and the proposed coordinate is sent to the chat application, where the user can view it on their device.

[0061] Examples:

[0062] When a user requests, "Tell me what clothes to wear to suit today's weather," the server responds, "Today is sunny and a little hot, so I'd like to suggest a light summer outfit (e.g., a shirt and shorts)."

[0063] The system of the present invention allows users to efficiently handle small tasks in daily life and save time, which not only improves the quality of life but also enhances the user's experience.

[0064] The processing flow will be explained below.

[0065] Menu suggestion function

[0066] Step 1:

[0067] User submits request:

[0068] A user types into a chat application, "What's for dinner?"

[0069] Step 2:

[0070] The device sends a request:

[0071] The device (smartphone or computer) sends this request to the server.

[0072] Step 3:

[0073] Server receives request:

[0074] The server receives the request from the user and begins parsing it.

[0075] Step 4:

[0076] Server gets user information:

[0077] The server retrieves profile data from the database, including the user's past preferences, allergy information, and food availability.

[0078] Step 5:

[0079] Server references recipe database:

[0080] The server searches the recipe database based on the acquired user information.

[0081] Step 6:

[0082] Server filters recipes:

[0083] The server selects recipes that match the user's criteria (allergy information, past preferences, and ingredient availability), and filters them taking into account seasonality and health factors.

[0084] Step 7:

[0085] Your server will select the perfect menu for you:

[0086] The server determines the best menu from the filtered recipes.

[0087] Step 8:

[0088] The server generates a response:

[0089] The server generates a message containing the menu information.

[0090] Step 9:

[0091] The server sends a response:

[0092] The server sends the generated message back to the chat application.

[0093] Step 10:

[0094] Device receives response:

[0095] The user's terminal receives the response from the server.

[0096] Step 11:

[0097] The terminal displays the message:

[0098] The chat application displays the suggested menu to the user.

[0099] Travel planning and transportation suggestion function

[0100] Step 1:

[0101] User submits request:

[0102] A user types into a chat application, "Plan a trip for next weekend."

[0103] Step 2:

[0104] The device sends a request:

[0105] The device sends a request to the server.

[0106] Step 3:

[0107] Server receives request:

[0108] The server receives the request from the user and begins parsing it.

[0109] Step 4:

[0110] Server gets user information:

[0111] The server retrieves profile data from the database, such as the user's travel history, interests, budget, and available time.

[0112] Step 5:

[0113] Server collects trip information:

[0114] The server calls an external travel information API to obtain local information and accommodation information.

[0115] Step 6:

[0116] Server generates itinerary:

[0117] The server generates an optimal travel plan based on the collected information and the user's requirements.

[0118] Step 7:

[0119] Server searches for transportation:

[0120] The server searches for the best means of transportation (e.g., train, plane, rental car) based on the user's location and destination.

[0121] Step 8:

[0122] Server selects travel mode:

[0123] The server selects the most suitable means of transportation based on the user's conditions (budget, travel time).

[0124] Step 9:

[0125] The server generates a response:

[0126] The server generates a message containing the travel plan and transportation information.

[0127] Step 10:

[0128] The server sends a response:

[0129] The server sends the generated message back to the chat application.

[0130] Step 11:

[0131] Device receives response:

[0132] The user's terminal receives the response from the server.

[0133] Step 12:

[0134] The terminal displays the message:

[0135] A chat application displays travel plans and transportation options to the user.

[0136] Clothing coordination suggestion function

[0137] Step 1:

[0138] User submits request:

[0139] The user types "Tell me what to wear today" into the chat application.

[0140] Step 2:

[0141] The device sends a request:

[0142] The device sends a request to the server.

[0143] Step 3:

[0144] Server receives request:

[0145] The server receives the request from the user and begins parsing it.

[0146] Step 4:

[0147] Server gets user information:

[0148] The server retrieves the user's past fashion history and preference profile data from the database.

[0149] Step 5:

[0150] Server gets weather information:

[0151] The server calls an external weather API to obtain current weather information.

[0152] Step 6:

[0153] The server consults the coordinate database:

[0154] The server searches for the optimal outfit based on the user's fashion history, preferences, and acquired weather information.

[0155] Step 7:

[0156] Server selects coordinates:

[0157] The server selects the best coordinates using filters and a scoring algorithm.

[0158] Step 8:

[0159] The server generates a response:

[0160] The server generates a message containing information about the selected coordinates.

[0161] Step 9:

[0162] The server sends a response:

[0163] The server sends the generated message back to the chat application.

[0164] Step 10:

[0165] Device receives response:

[0166] The user's terminal receives the response from the server.

[0167] Step 11:

[0168] The terminal displays the message:

[0169] The chat application displays suggested outfits to the user.

[0170] Example 1

[0171] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0172] In users' daily lives, tasks such as deciding on meal plans, planning trips, and coordinating fashion are often complicated and time-consuming. It is also difficult to consider the conditions and constraints required for these tasks. It is particularly challenging to obtain optimal suggestions that reflect individual users' preferences and constraints. This leads to inefficient time management and a decline in quality of life. There is a need for technology that can solve these issues and improve the quality of users' lives while making them more efficient.

[0173] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0174] In this invention, the server includes means for receiving requests from users, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, means for generating optimal proposals in response to the user's requests, means for transmitting the generated proposals to the user, means for analyzing the request content and determining which function is being requested, and means for acquiring information from an external API based on the user information. This makes it possible to quickly provide optimal proposals tailored to the preferences and needs of individual users in response to various requests in their daily lives (such as menu suggestions, travel plans, and fashion coordination).

[0175] A "user" is an entity that uses the system to send requests and receive optimal suggestions.

[0176] A "server" is a device or system that receives requests from users, analyzes data and generates suggestions, and transmits the results to the users.

[0177] A "terminal" is a device (e.g., a smartphone or computer) through which a user inputs requests and receives suggestions from a server.

[0178] A "request" is a request that a user sends to a server via a chat application or the like.

[0179] "Profile data" refers to data that includes information necessary for generating suggestions, such as the user's past usage history, preferences, and allergy information.

[0180] A "database" is a storage device that stores information necessary for generating proposals, such as profile data and recipe information.

[0181] A "suggestion" is a specific recommendation that is generated by the server based on the user's request and provided to the user.

[0182] "External API" means an external application programming interface that the server uses to obtain additional data based on user information.

[0183] In order to implement the present invention, it is necessary to build a system that generates optimal proposals based on user requests and provides them to the user. This system is mainly composed of a server, a terminal, and a user.

[0184] Hardware and Software

[0185] Hardware

[0186] 1. Server: Consists of computers with high-performance data processing capabilities, and can be a cloud server or an on-premise server.

[0187] 2. Terminal: A user device such as a smartphone, tablet, or personal computer (PC).

[0188] 3. Network: The network infrastructure that allows the server and devices to communicate through an Internet connection.

[0189] software

[0190] 1. Chat application: An application that users use to send requests to a server. It runs on smartphones and PCs.

[0191] 2. Generative AI model: An artificial intelligence model for generating recommendations (e.g., a natural language processing model or a recommender system).

[0192] 3. Database Management System (DBMS): Software that manages databases that store profile data, recipe information, travel information, fashion items, etc.

[0193] 4. External API: An external application programming interface used to obtain weather information, travel information, etc.

[0194] System Operation

[0195] 1. User submits request

[0196] A user uses a chat application to submit a request, for example, "What's for dinner?"

[0197] 2. The device sends the request to the server

[0198] The device receives the user's request, converts it into JSON format, and sends it to the server.

[0199] 3. Request Analysis by the Server

[0200] The server analyzes the received request and determines which function is being requested. For example, it selects the "menu suggestion function."

[0201] 4. Server obtains user information

[0202] Based on the user ID, the server retrieves profile data (past usage history, allergy information, inventory ingredients, etc.) from the database.

[0203] 5. Data analysis and proposal generation by the server

[0204] The server uses the generative AI model to generate optimal suggestions based on the user information and the request, and calls external APIs to obtain auxiliary information as needed.

[0205] 6. Sending results from the server to the device

[0206] The server converts the generated proposal into JSON format and sends it to the device.

[0207] 7. User confirmation of results

[0208] The user checks the proposed content in the chat application.

[0209] Specific examples

[0210] Specific examples of menu suggestions

[0211] The user types "Please suggest a dinner menu" into a chat application and sends it. The server analyzes this request and selects a "menu suggestion function." The server retrieves the user's past preferences, allergy information, and available ingredients from a database, and then uses a generative AI model to generate an optimal menu. For example, it might suggest "grilled salmon and salad." The generated suggestion is sent to the device, and the user can view the results in the chat application.

[0212] Prompt Sentence Examples

[0213] Please suggest a dinner menu

[0214] I want to go on a trip somewhere next weekend

[0215] What clothes should I wear for today's weather?

[0216] This system can provide prompt and optimal suggestions for various requests in users' daily lives, which will improve the efficiency of users' time management and enhance their quality of life.

[0217] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0218] Step 1:

[0219] The user submits a request.

[0220] Input: A user types "What's for dinner" in a chat application.

[0221] Output: A request sent by the chat application.

[0222] Specific operation: The user opens a chat application on their smartphone or computer, enters a request, and presses the send button.

[0223] Step 2:

[0224] The device sends a request to the server.

[0225] Input: A user request via a chat application.

[0226] Output: The request is sent via an API call to the server.

[0227] Specific operation: The device converts the request content into JSON format and sends a POST request to the server's API endpoint.

[0228] Step 3:

[0229] The server parses the request.

[0230] Input: The request sent to the server in JSON format.

[0231] Output: The appropriate function is selected as a result of analyzing the request content.

[0232] Specific operation: The server analyzes the request message and determines the appropriate processing route based on its content. For example, the "Menu suggestion function" is selected for the request "Tell me what's for dinner."

[0233] Step 4:

[0234] The server retrieves the user information.

[0235] Input: User ID and request details.

[0236] Output: Profile data obtained from the database (past usage history, allergy information, inventory ingredients, etc.).

[0237] What happens: The server queries the database to retrieve data related to the user.

[0238] Step 5:

[0239] The server analyzes the data and generates recommendations.

[0240] Input: Profile data, recipe database, information from external APIs (e.g. weather information).

[0241] Output: The optimal recommendation to provide to the user.

[0242] What it does: The server searches its database to select recipes based on the season and current inventory. It calls external APIs to obtain additional information as needed. It uses generative AI models to generate optimal recommendations.

[0243] Step 6:

[0244] The server sends the generated proposal to the terminal.

[0245] Input: The server-generated proposal.

[0246] Output: Suggestion information sent to the device in JSON format.

[0247] What happens: The server converts the proposal into JSON format and sends a POST request to the device's API endpoint.

[0248] Step 7:

[0249] The user checks the results.

[0250] Input: The proposal information sent back to the device.

[0251] Output: The suggestion that will be displayed in the chat application.

[0252] What happens: A user opens a chat application and sees a new message with suggested information, such as "Suggesting grilled salmon with salad."

[0253] (Application example 1)

[0254] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0255] In users' everyday shopping experiences, selecting the most suitable product from a vast amount of product information is a time-consuming task. In particular, there is a lack of systems that can make individually customized suggestions based on a user's purchasing history and profile data. This can lead to users missing potentially suitable products, so a system that provides an efficient and satisfying shopping experience is needed.

[0256] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0257] In this invention, the server includes means for receiving requests from users, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposals, means for proposing optimal products based on the user's purchase history and interests, and means for transmitting the generated proposals to the user. This allows users to receive optimal product proposals that are individually customized, enabling an efficient and satisfying shopping experience.

[0258] "Users" refers to people who use this system.

[0259] A "request" refers to the action or content of a user requesting some kind of service or information from a system.

[0260] "Usage history" refers to a record of a user's past activities when using the system.

[0261] "Profile Data" refers to a user's personal information, interests, preferences, and past behavioral data.

[0262] "Database" refers to a system for storing and managing user profile data and information for generating proposals.

[0263] "Suggestions" refer to appropriate product recommendations or advice generated based on a user's request.

[0264] "Purchase history" refers to a record of products purchased by a user in the past.

[0265] "Interests" refer to areas or things in which a user is particularly interested.

[0266] "Products" refers to various consumer goods offered to users.

[0267] "Means for suggesting optimal products" refers to the process and function of selecting and recommending the most suitable products for each individual user based on the user's profile data and purchase history.

[0268] "System" refers to an integrated technical configuration and device that includes multiple means and provides services and information to users.

[0269] "Send" refers to transferring data from the server to the user's terminal.

[0270] The system for implementing this invention receives requests from users and proposes optimal products by taking into consideration the user's profile data and purchase history. This system is mainly composed of a server, a terminal, and a user.

[0271] First, a user sends a request using a dedicated application on their device. For example, a request such as "Recommend me a coat." This request is sent from the device to the server. The device used can be a general device such as a smartphone or computer.

[0272] When the server receives the request, it retrieves profile data and purchase history from a database based on the user ID. Specific databases used include MySQL (registered trademark) and MongoDB. Additionally, software such as the Python requests module is used to process HTTP requests to retrieve data on the server side.

[0273] The server analyzes the acquired profile data and purchase history to select appropriate products based on the user's interests and past purchasing habits. This process can be performed using a generative AI model, such as a machine learning algorithm or deep learning model. This generative AI model generates prompts that suggest the best products when the user enters specific keywords or phrases.

[0274] Once the suggestions are generated, the server sends them back to the device. The user can then review the suggestions and select specific products in the application. For example, when a user requests "Recommend me a coat," the server generates a message such as, "Taking into consideration your past purchases and preferences, we suggest the following coats," and provides a list of appropriate products.

[0275] An example of a prompt might be, "Based on user ID 12345, please suggest the best coat for me, taking into account my past purchase history and profile data." This allows users to enjoy a personalized, high-quality shopping experience.

[0276] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0277] Step 1:

[0278] The user inputs a request into a dedicated application on the device and sends it. The input data is the information the user is looking for (e.g., "Recommend a coat"). This request is sent from the device to the server. The specific operation of the device is to receive the user input and send it to the server as an HTTP POST request.

[0279] Step 2:

[0280] The server receives a request from a user. The input data is the user request, and the output data is the user ID and the request content. The specific operation of the server is to receive the HTTP request, analyze the request parameters, and extract the user ID and the request content.

[0281] Step 3:

[0282] The server retrieves the user's profile data and purchase history from the database. The input data is the user ID, and the output data is the profile data and purchase history. The specific operation of the server is to query the database for the user ID and retrieve the corresponding profile data and purchase history.

[0283] Step 4:

[0284] The server analyzes the profile data and purchase history and generates optimal product suggestions using a generative AI model. The input data is the profile data and purchase history, and the output data is the optimal product suggestions. The specific operation of the server is to analyze the acquired data and input a prompt statement to the generative AI model. As a specific example, the prompt statement "Please suggest the best coat based on user ID 12345, taking into consideration past purchase history and profile data" is input to the generative AI model.

[0285] Step 5:

[0286] The server then sends the generated proposal back to the terminal. The input data is the optimal product proposal, and the output data is the proposal to the user terminal. The specific operation of the server is to send the proposal to the terminal as an HTTP response and notify the user.

[0287] Step 6:

[0288] The terminal displays the suggestions to the user. The input data is the suggestions received from the server, and the output data is what is displayed to the user. The specific operation of the terminal is to analyze the received suggestions and display them on the user interface. As a concrete example, a list of products is displayed along with a message such as "Here are some recommended coats."

[0289] Through the above steps, the user can receive optimal product proposals that are individually customized.

[0290] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0291] This invention is a system that streamlines users' daily tasks and improves the accuracy and effectiveness of suggestions by incorporating an emotion engine. This system is mainly composed of a server, a terminal, and a user, and includes an emotion engine for recognizing the user's emotions.

[0292] Menu suggestion function

[0293] Sending and Receiving Requests

[0294] A user sends a request using a chat application, such as "Tell me what's for dinner." The device (smartphone or computer) sends this request to a server. The server analyzes the received request and starts the process of proposing the menu desired by the user.

[0295] Acquisition and analysis of user information

[0296] The server retrieves profile data from the database based on the user ID. The profile data includes the user's past preferences, allergy information, current food availability, etc. The server then uses an emotion engine to analyze the user's current emotional state.

[0297] Menu generation and suggestions

[0298] The server generates suggestions from a recipe database, taking into account not only the current season and ingredient availability, but also the user's emotional state. For example, if the user is tired, it will suggest easy-to-make dishes, and if they are energetic, it will suggest new and challenging recipes. The generated menu is sent to a chat application, where the user can check the suggested menu.

[0299] Examples:

[0300] When a user requests, "What would you like for dinner tonight?" the server responds, "Taking into account your past preferences and the current season, and because we recognize your current emotional state as tired, we'd like to suggest a quick and easy chicken and vegetable stir-fry."

[0301] Travel planning and transportation suggestions

[0302] Sending and Receiving Requests

[0303] A user sends a request to a chat application saying, "Plan a trip for next weekend." The device sends this request to a server, which receives the request and analyzes its contents.

[0304] Obtaining user information and travel conditions

[0305] Based on the user ID, the server retrieves profile data from the database, such as past travel history, interests, available budget and time, and analyzes the user's current emotional state using an emotion engine.

[0306] Travel plan and transportation generation

[0307] The server uses this information to generate an optimal travel plan. For example, if the user is feeling stressed, it might suggest a relaxing resort, or if the user is in an adventure-seeking emotional state, it might suggest a plan including active activities. Once the optimal travel plan and means of transportation are determined, they are sent to the chat application, where the user can confirm the plan.

[0308] Examples:

[0309] If a user requests, "I'd like to go on a trip somewhere next weekend," the server responds, "Based on your past travel history, current interests, and current emotional state, I recognize that you are seeking relaxation, so I will suggest a trip to a hot spring resort. The train would be a convenient means of transportation."

[0310] Clothing coordination suggestion function

[0311] Sending and Receiving Requests

[0312] The user sends a request to the chat application saying, "Tell me what outfit I'm wearing today." The device sends this request to the server. The server receives the request and analyzes its contents.

[0313] Obtaining user information and weather information

[0314] The server retrieves profile data such as past fashion history and preferences from a database based on the user ID, analyzes the user's current emotional state using an emotion engine, and calls an external weather API to retrieve current weather information.

[0315] Coordination generation and suggestions

[0316] The server uses this information to generate the optimal outfit for the user. For example, if the user is in a positive emotional state, it will suggest brightly colored clothes, and if they are feeling down, it will suggest more subdued clothing. The generated outfit is then sent to a chat application where the user can view it.

[0317] Examples:

[0318] When a user requests, "Tell me what clothes to wear to suit today's weather," the server responds, "Today is sunny and a little hot. Furthermore, taking into account that your current emotional state is positive, I would like to suggest a light summer outfit (e.g., a light-colored shirt and shorts)."

[0319] In this way, the system of the present invention can take into account the user's emotional state and provide optimal suggestions to the user, thereby enabling the user to efficiently handle daily tasks and improve their quality of life.

[0320] The processing flow will be explained below.

[0321] Menu suggestion function

[0322] Step 1:

[0323] User submits request:

[0324] A user types into a chat application, "What's for dinner?"

[0325] Step 2:

[0326] The device sends a request:

[0327] The device (smartphone or computer) sends this request to the server.

[0328] Step 3:

[0329] Server receives request:

[0330] The server receives the request from the user and begins parsing it.

[0331] Step 4:

[0332] Server gets user information:

[0333] The server retrieves the user's profile data (past preferences, allergy information, and food availability) from the database.

[0334] Step 5:

[0335] Server calls emotion engine:

[0336] The server invokes the emotion engine to analyze the user's emotional state.

[0337] Step 6:

[0338] Server references recipe database:

[0339] The server searches a recipe database based on the user's profile data and emotional state.

[0340] Step 7:

[0341] Server filters recipes:

[0342] The server selects recipes that match the user's criteria (allergy information, past preferences, ingredient availability) and emotional state, and filters them taking into account seasonality and health factors.

[0343] Step 8:

[0344] Your server will select the perfect menu for you:

[0345] The server determines the best menu from the filtered recipes.

[0346] Step 9:

[0347] The server generates a response:

[0348] The server generates a message containing the menu information.

[0349] Step 10:

[0350] The server sends a response:

[0351] The server sends the generated message back to the chat application.

[0352] Step 11:

[0353] Device receives response:

[0354] The user's terminal receives the response from the server.

[0355] Step 12:

[0356] The terminal displays the message:

[0357] The chat application displays the suggested menu to the user.

[0358] Travel planning and transportation suggestion function

[0359] Step 1:

[0360] User submits request:

[0361] A user types into a chat application, "Plan a trip for next weekend."

[0362] Step 2:

[0363] The device sends a request:

[0364] The device sends a request to the server.

[0365] Step 3:

[0366] Server receives request:

[0367] The server receives the request from the user and begins parsing it.

[0368] Step 4:

[0369] Server gets user information:

[0370] The server retrieves profile data from the database, such as the user's travel history, interests, budget, and available time.

[0371] Step 5:

[0372] Server calls emotion engine:

[0373] The server invokes the emotion engine to analyze the user's emotional state.

[0374] Step 6:

[0375] Server collects trip information:

[0376] The server calls an external travel information API to obtain local information and accommodation information.

[0377] Step 7:

[0378] Server generates itinerary:

[0379] The server generates an optimal travel plan based on the collected information and the user's conditions and emotional state.

[0380] Step 8:

[0381] Server searches for transportation:

[0382] The server searches for the best means of transportation (e.g., train, plane, rental car) based on the user's location and destination.

[0383] Step 9:

[0384] Server selects travel mode:

[0385] The server selects the most suitable mode of transportation based on the user's conditions (budget, travel time) and emotional state.

[0386] Step 10:

[0387] The server generates a response:

[0388] The server generates a message containing the travel plan and transportation information.

[0389] Step 11:

[0390] The server sends a response:

[0391] The server sends the generated message back to the chat application.

[0392] Step 12:

[0393] Device receives response:

[0394] The user's terminal receives the response from the server.

[0395] Step 13:

[0396] The terminal displays the message:

[0397] A chat application displays travel plans and transportation options to the user.

[0398] Clothing coordination suggestion function

[0399] Step 1:

[0400] User submits request:

[0401] The user types "Tell me what to wear today" into the chat application.

[0402] Step 2:

[0403] The device sends a request:

[0404] The device sends a request to the server.

[0405] Step 3:

[0406] Server receives request:

[0407] The server receives the request from the user and begins parsing it.

[0408] Step 4:

[0409] Server gets user information:

[0410] The server retrieves the user's past fashion history and preference profile data from the database.

[0411] Step 5:

[0412] Server calls emotion engine:

[0413] The server invokes the emotion engine to analyze the user's emotional state.

[0414] Step 6:

[0415] Server gets weather information:

[0416] The server calls an external weather API to obtain current weather information.

[0417] Step 7:

[0418] The server consults the coordinate database:

[0419] The server searches for the optimal outfit based on the user's fashion history and preferences, acquired weather information, and emotional state.

[0420] Step 8:

[0421] Server selects coordinates:

[0422] The server selects the best coordinates using filters and a scoring algorithm.

[0423] Step 9:

[0424] The server generates a response:

[0425] The server generates a message containing information about the selected coordinates.

[0426] Step 10:

[0427] The server sends a response:

[0428] The server sends the generated message back to the chat application.

[0429] Step 11:

[0430] Device receives response:

[0431] The user's terminal receives the response from the server.

[0432] Step 12:

[0433] The terminal displays the message:

[0434] The chat application displays suggested outfits to the user.

[0435] In this way, by incorporating an emotion engine, the system of the present invention is designed to provide personalized suggestions based on the user's emotional state, allowing them to handle everyday tasks more efficiently.

[0436] Example 2

[0437] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0438] To streamline users' daily tasks, a system that quickly provides optimal suggestions in response to individual requests is required. Conventional systems were able to provide suggestions based on a user's past preferences and profile data, but they were unable to adequately consider the user's current emotional state. This made it difficult to provide suggestions that matched the user's needs.

[0439] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving a request from a user, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, means including an emotion engine for analyzing the user's emotional state, means for generating optimal proposals in response to the user's request, and means for transmitting the generated proposals to the user. This enables proposals that take the user's emotional state into consideration.

[0440] "User" means an individual or corporation that uses the system.

[0441] A "request" is a specific request or inquiry sent from a user to a system.

[0442] A "terminal" is a device that a user uses to access and operate the system, and includes smartphones, computers, tablets, etc.

[0443] A "server" is a computer system that receives and processes requests from users.

[0444] "Profile data" refers to data that includes personal information such as a user's past usage history, preferences, allergy information, and food ingredient availability.

[0445] A "database" is a collection of stored data that is referenced to generate recommendations.

[0446] An "emotion engine" is a technology for analyzing a user's current emotional state, analyzing the user's emotions and generating data based on the analysis.

[0447] A "suggestion" is the optimal answer or advice that the system generates in response to a user's request.

[0448] The "menu suggestion means" is a function that suggests the most suitable meal menu based on the user's meal requests.

[0449] The "travel planning and transportation suggestion means" is a function that suggests optimal travel plans and transportation means based on the user's travel requests.

[0450] The "fashion coordination suggestion means" is a function that suggests the most suitable fashion coordination based on the user's clothing requests.

[0451] A "natural language processing (NLP) engine" is a technology for analyzing and understanding user requests.

[0452] A "generative AI model" is an artificial intelligence technique that generates specific outputs based on user requests and other data.

[0453] This invention is a system that streamlines users' daily tasks and improves the accuracy and effectiveness of suggestions by incorporating an emotion engine. This system consists of three main components: a server, a terminal, and a user.

[0454] System configuration and hardware / software integration

[0455] server

[0456] The server plays a central role in receiving requests from users, analyzing them, and generating optimal suggestions. The following specific software is used:

[0457] NLP engine: Uses spaCy and NLTK to analyze the content of requests using natural language processing.

[0458] Emotion Engine: Analyzes the user's emotional state using Affectiva and Microsoft® Emotion API.

[0459] Database: Relational databases such as MySQL and PostgreSQL, and NoSQL databases such as MongoDB and ElasticSearch (registered trademark) are used to manage data such as user profile data, recipe data, and travel plans.

[0460] Terminal

[0461] A terminal is a device that allows a user to access and operate a system. Typically, this is a smartphone or computer. A terminal has the following functions:

[0462] Chat application: Sends user requests to a server, receives responses from the server and displays them to the user.

[0463] User

[0464] A user is an individual or legal entity that uses a terminal to send requests to the system and receive offers from the server.

[0465] System Functions and Prompt Sentence Examples

[0466] Menu suggestion function

[0467] Sending and Receiving Requests

[0468] A user sends a request using a chat application saying, "Tell me what's for dinner." The device then sends this request to the server.

[0469] Example: A user types, "What's for dinner tonight?"

[0470] Acquisition and analysis of user information

[0471] The server retrieves profile data (past preferences, allergy information, food stock status, etc.) from a database based on the user ID and analyzes the emotional state using an emotion engine.

[0472] Example: The server responds, "Considering your past preferences and the current season, and your emotional state of fatigue, I would like to suggest a quick chicken and vegetable stir fry."

[0473] Travel planning and transportation suggestions

[0474] Sending and Receiving Requests

[0475] A user sends a request to a chat application saying, "Plan a trip for next weekend." The device sends this request to a server.

[0476] Example: A user types, "I want to go on a trip somewhere next weekend."

[0477] Obtaining user information and travel conditions

[0478] Based on the user ID, the server obtains data such as past travel history, interests, budget, and time, and analyzes the user's emotional state using an emotion engine.

[0479] Example: The server responds, "Since your past travel history, current interests, and emotional state indicate a desire for relaxation, we suggest a trip to a hot spring resort. Trains would be a convenient means of transportation."

[0480] Clothing coordination suggestion function

[0481] Sending and Receiving Requests

[0482] The user sends a request to the chat application saying, "Tell me what outfit I'm wearing today." The device then sends this request to the server.

[0483] Example: A user types, "What should I wear for today's weather?"

[0484] Obtaining user information and weather information

[0485] The server obtains fashion history and preference data based on the user ID, analyzes the user's emotional state using an emotion engine, and obtains current weather information using a weather API.

[0486] Example: The server responds, "Today is sunny and slightly hot, and your emotional state is positive, so I suggest a light summer outfit (e.g., a light-colored shirt and shorts)."

[0487] Prompt Sentence Examples

[0488] Meal suggestion feature: "Generate prompts to suggest dinner menus for tired users."

[0489] Trip Planning & Transportation Suggestion: "Generate prompts to suggest trip plans for users looking to relax."

[0490] Clothing Coordination Suggestion: "Generate prompts that suggest fashion coordination for users who are in a positive emotional state."

[0491] In this way, the system of the present invention can take into account the user's emotional state to provide more personalized suggestions and efficiently handle everyday tasks.

[0492] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0493] Menu suggestion function

[0494] Processing Steps

[0495] Step 1:

[0496] Terminal: A user types a request into a chat application: "What's for dinner?"

[0497] Input: User request

[0498] Output: Sends an API request to the server

[0499] What it does: Use your smartphone or computer to access a chat app, type in your request, and press send.

[0500] Step 2:

[0501] Server: Receives requests sent from the device and analyzes them using the NLP engine.

[0502] Input: API request

[0503] Output: Parsed request data

[0504] What it does: The server uses a natural language processing engine (e.g., spaCy or NLTK) to analyze the user's request and extract meaning.

[0505] Step 3:

[0506] Server: Retrieves profile data from the database based on the user ID.

[0507] Input: User ID

[0508] Output: User profile data (past preferences, allergy information, current ingredient availability)

[0509] Specific operation: The server issues a query to a database (e.g., MySQL or PostgreSQL) to retrieve profile data.

[0510] Step 4:

[0511] Server: Analyzes the user's current emotional state using the emotion engine.

[0512] Input: User profile data

[0513] Output: Current emotional state data

[0514] What it does: The server inputs profile data into an emotion engine (e.g., Affectiva or Microsoft Emotion API) to analyze the emotional state.

[0515] Step 5:

[0516] Server: Searches the recipe database for candidate recipes that match the criteria and generates the optimal menu.

[0517] Input: Emotional state data, profile data, database query

[0518] Output: Optimal meal plan

[0519] Specific operation: The server issues a query to a recipe database (e.g., MongoDB or Elasticsearch), searches for recipes that meet the criteria, and generates the optimal menu.

[0520] Step 6:

[0521] Server: Formats the generated menu information into a response message and sends it to the device.

[0522] Input: Optimal meal plan

[0523] Output: The formatted response message

[0524] Specific operation: The server formats the generated menu data in rich text or JSON format and sends it to the terminal.

[0525] Step 7:

[0526] Terminal: Messages received from the server are displayed within the chat application for the user to review.

[0527] Input: Response message from the server

[0528] Output: Menu display in a chat application

[0529] Specific behavior: The device receives the response message and displays it in the chat application UI.

[0530] Travel planning and transportation suggestions

[0531] Processing Steps

[0532] Step 1:

[0533] Terminal: A user types a request into a chat application: "Plan a trip for next weekend."

[0534] Input: User request

[0535] Output: Sends an API request to the server

[0536] What it does: Access a chat app on your smartphone or computer, type in your request, and press send.

[0537] Step 2:

[0538] Server: Receives requests sent from the device and analyzes them using the NLP engine.

[0539] Input: API request

[0540] Output: Parsed request data

[0541] What happens: The server uses a natural language processing engine to analyze the request and extract meaning.

[0542] Step 3:

[0543] Server: Retrieves profile data from the database based on the user ID.

[0544] Input: User ID

[0545] Output: Past travel history, interests, budget, time data

[0546] What happens next: The server queries the database to retrieve profile data.

[0547] Step 4:

[0548] Server: Analyzes the user's current emotional state using the emotion engine.

[0549] Input: Profile data

[0550] Output: Current emotional state data

[0551] Specific operation: The server inputs profile data into the emotion engine and analyzes the emotional state.

[0552] Step 5:

[0553] Server: Generates the optimal travel plan based on the acquired information.

[0554] Input: Emotional state data, profile data, database query

[0555] Output: Optimal travel plan

[0556] Specific operation: The server executes the travel plan generation algorithm and generates a plan that meets the conditions.

[0557] Step 6:

[0558] Server: Formats the itinerary and transportation methods into a response message and sends it to the device.

[0559] Input: Best Travel Plan

[0560] Output: The formatted response message

[0561] Specific operation: The server formats the generated plan data and sends it to the device.

[0562] Step 7:

[0563] On the device: The proposed itinerary is displayed in the chat app for the user to review.

[0564] Input: Response message from the server

[0565] Output: Plan display in a chat application

[0566] Specific behavior: The device receives the response message and displays it in the chat app's UI.

[0567] Clothing coordination suggestion function

[0568] Processing Steps

[0569] Step 1:

[0570] Terminal: A user types a request into a chat application, such as "Tell me what outfit to wear today."

[0571] Input: User request

[0572] Output: Sends an API request to the server

[0573] What it does: Access a chat app on your smartphone or computer, type in your request, and press send.

[0574] Step 2:

[0575] Server: Receives requests sent from the device and analyzes them using the NLP engine.

[0576] Input: API request

[0577] Output: Parsed request data

[0578] What happens: The server uses a natural language processing engine to analyze the content of the request.

[0579] Step 3:

[0580] Server: Retrieves profile data from the database based on the user ID.

[0581] Input: User ID

[0582] Output: Past fashion history, preference data

[0583] What happens next: The server queries the database to retrieve profile data.

[0584] Step 4:

[0585] Server: Analyzes the user's current emotional state using the emotion engine.

[0586] Input: Profile data

[0587] Output: Current emotional state data

[0588] Specific operation: The server inputs profile data into the emotion engine and analyzes the emotional state.

[0589] Step 5:

[0590] Server: Calls the weather API to get current weather information.

[0591] Input: Weather API request

[0592] Output: Current weather information

[0593] What happens: The server sends a request to an external weather API to get current weather information.

[0594] Step 6:

[0595] Server: Generates optimal coordination based on the acquired information.

[0596] Input: Emotional state data, fashion history data, weather information

[0597] Output: Optimal outfit ideas

[0598] Specific operation: The server executes the coordination generation algorithm and generates fashion suggestions that meet the conditions.

[0599] Step 7:

[0600] Server: Formats the proposed coordinates into a response message and sends it to the device.

[0601] Input: Best outfit ideas

[0602] Output: The formatted response message

[0603] Specific operation: The server formats the generated coordinate data and sends it to the terminal.

[0604] Step 8:

[0605] Device: The suggested outfits are displayed in the chat app so that the user can check them.

[0606] Input: Response message from the server

[0607] Output: Coordinate display in a chat application

[0608] Specific behavior: The device receives the response message and displays it in the chat app's UI.

[0609] Example prompts for generative AI models

[0610] Meal suggestion feature: "Generate prompts to suggest dinner menus for tired users."

[0611] Trip Planning & Transportation Suggestion: "Generate prompts to suggest trip plans for users looking to relax."

[0612] Clothing Coordination Suggestion: "Generate prompts that suggest fashion coordination for users who are in a positive emotional state."

[0613] (Application example 2)

[0614] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0615] Conventional recommendation systems make suggestions based on user requests and past data, but because they do not take the user's emotional state into account, they are unable to make suggestions that are optimal for the user's current mood or situation, limiting the improvement of user satisfaction.In addition, it has been difficult to make personalized suggestions based on real-time emotion recognition in physical stores, etc.

[0616] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0617] In this invention, the server includes means for receiving requests from a user, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, means including an emotion engine for analyzing the user's emotional state, means for generating optimal proposals in accordance with the user's requests and emotional state, and means for transmitting the generated proposals to the user, thereby enabling optimal proposals that take into account the user's current emotional state.

[0618] The "means for receiving a request from a user" refers to a device or software that allows a user to input a request for information or a service they desire into the system and receives the request electronically.

[0619] "Means for obtaining a user's past usage history and profile data" refers to a device or software that extracts records of a user's past use of the system, personal information, and preference data.

[0620] The "means for referencing a database for generating proposal content" refers to a device or software that accesses and refers to a database that stores information required for generating various proposal content.

[0621] The "means including an emotion engine for analyzing the user's emotional state" refers to hardware and software for recognizing and analyzing emotions from the user's facial expressions, tone of voice, etc.

[0622] The "means for generating optimal suggestions according to the user's requests and emotional state" refers to a device or software for generating the most appropriate suggestions based on the user's request content and analyzed emotional state.

[0623] The "means for transmitting the generated proposal to the user" refers to a device or software that electronically transmits the content of the generated proposal to the user's terminal.

[0624] The system of the present invention makes suggestions to improve the efficiency of a user's daily tasks and includes an emotion engine that analyzes the user's emotional state. This system is primarily composed of a server, a terminal, and a user. Specific embodiments for realizing this system are described below.

[0625] The server includes means for receiving requests from users, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, an emotion engine for analyzing the user's emotional state, means for generating optimal proposals according to the user's requests and emotional state, and means for transmitting the generated proposals to the user.

[0626] Specifically, when a user accesses product information using a smartphone or smart glasses, the system analyzes the user's emotional state at that time and proposes the most suitable products and sets. For example, if the user is tired, the system will propose relaxing items (bath additives, aromatherapy, relaxation goods, etc.), and if the user is energetic, it will propose products suitable for new challenges or activities (sports goods, games, activity kits, etc.).

[0627] This system mainly uses the following hardware and software:

[0628] The camera on a smartphone or smart glasses is used to capture the user's facial expressions.

[0629] Using OpenCV (image processing library), the user's facial expression is processed from the captured image.

[0630] Use emotion_recognition (emotion recognition library) to analyze the user's current emotional state from their facial expressions.

[0631] Use requests (an HTTP request library) to communicate with the user database and retrieve user profile data.

[0632] Use the recommendation_engine to generate optimal recommendations based on emotional state and profile data.

[0633] As a concrete example, a user enters a physical store through smart glasses. It seems that the user has been very busy that weekend and is feeling stressed. The camera analyzes the user's facial expressions to determine their emotions and determines that they are "tired." As a result, the smart glasses' display displays a message saying, "We'll suggest products that will help you relax today!" The user can then choose a product based on the suggestions.

[0634] Example prompt sentence:

[0635] "Relax, we've found the perfect product for you."

[0636] "Here are some recommended activities for you who are feeling energetic!"

[0637] In this way, by providing individually optimized suggestions that take into account the user's emotional state, user satisfaction can be improved.

[0638] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0639] Step 1:

[0640] A user inputs a request using a smartphone or smart glasses. The user sends a question to the system, such as "What is today's recommended product?" The input is the user's request, and the output is the request data.

[0641] Step 2:

[0642] The device sends the user's request to the server. After receiving the user's request, the smartphone or smart glasses transmit the request to the server via the Internet. The input is the user's request data, and the output is a confirmation of the request transmission to the server.

[0643] Step 3:

[0644] The server receives and analyzes the request. The received request data is analyzed by a program on the server to identify the type of information or service the user is seeking. The input is the received request data, and the output is the analysis result.

[0645] Step 4:

[0646] The server retrieves the user's profile data. After analyzing the request, the server retrieves past usage history and profile data from the database based on the user ID. The input is the user ID and the output is the profile data.

[0647] Step 5:

[0648] The server uses an emotion engine to analyze the user's emotional state. The emotion engine analyzes the user's emotional state based on the user's facial expressions captured in real time by a camera installed in a smartphone or smart glasses. The input is the user's facial expression data, and the output is the analyzed emotional state.

[0649] Step 6:

[0650] The server generates optimal suggestions based on the acquired profile data and analyzed emotional state. The server refers to a recipe database and recommends products and services that match the user's profile data and emotional state. The input is the profile data and emotional state, and the output is the optimal suggestions.

[0651] Step 7:

[0652] The server sends the generated proposal to the terminal. Once the optimal proposal is generated, the server sends it to the user's terminal. The input is the optimal proposal, and the output is a confirmation of the proposal transmission to the terminal.

[0653] Step 8:

[0654] The device displays the proposed content to the user. The smartphone or smart glasses displays the received proposed content on the user's display. The input is the proposed content sent from the server, and the output is the proposed content displayed on the user's display.

[0655] Specifically, the user speaks to the smart glasses, asking, "What is today's recommended product?" The server analyzes the response and displays, "Considering that you are tired, we recommend an aroma candle to help you relax today."

[0656] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0657] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0658] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0659] [Second embodiment]

[0660] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0661] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0662] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0663] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0664] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0665] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0666] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0667] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0668] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0669] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0670] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0671] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0672] The present invention is a system that streamlines users' daily tasks by receiving requests from users and providing optimal suggestions based on the users' past usage history and profile data. This system is mainly composed of a server, a terminal, and a user.

[0673] Menu suggestion function

[0674] Sending and Receiving Requests

[0675] A user sends a request using a chat application, such as "Tell me what's for dinner." The device (smartphone or computer) sends this request to a server. The server analyzes the received request and starts the process of proposing the menu desired by the user.

[0676] Acquisition and analysis of user information

[0677] The server retrieves profile data from the database based on the user ID. The profile data includes the user's past preferences, allergy information, current food stock status, etc. This allows the server to understand the user's preferences and restrictions.

[0678] Menu generation and suggestions

[0679] The server searches and selects recipes that can be suggested to the user from the recipe database, taking into account the current season and the availability of ingredients. Once the optimal menu is determined, the server sends it back to the chat application. The user can then view the suggested menu through the chat application on their device.

[0680] Examples:

[0681] When a user requests, "What would you like for dinner tonight?" the server responds, "Taking into account your past preferences and the current season, I would like to suggest grilled salmon and a salad."

[0682] Travel planning and transportation suggestions

[0683] Sending and Receiving Requests

[0684] A user sends a request to a chat application saying, "Plan a trip for next weekend." The device sends this request to a server, which receives the request and analyzes its contents.

[0685] Obtaining user information and travel conditions

[0686] Based on the user ID, the server retrieves profile data such as past travel history, interests, available budget and time from the database, and also calls external travel information APIs to collect local information and accommodation information.

[0687] Travel plan and transportation generation

[0688] The server generates an optimal travel plan based on this information and suggests the most suitable means of transportation for the user (e.g., train, plane, rental car). The generated plan and means of transportation are sent to the chat application, where the user can confirm them.

[0689] Examples:

[0690] When a user requests, "I'd like to go on a trip somewhere next weekend," the server responds, "Taking into account your past travel history and current interests, I would like to suggest a day trip to Kyoto. The Shinkansen would be a convenient means of transportation."

[0691] Clothing coordination suggestion function

[0692] Sending and Receiving Requests

[0693] The user sends a request to the chat application saying, "Tell me what outfit I'm wearing today." The device sends this request to the server. The server receives the request and analyzes its contents.

[0694] Obtaining user information and weather information

[0695] The server retrieves profile data such as past fashion history and preferences from a database based on the user ID, and then calls an external weather API to retrieve current weather information.

[0696] Coordination generation and suggestions

[0697] The server generates the optimal coordinate based on this information, and the proposed coordinate is sent to the chat application, where the user can view it on their device.

[0698] Examples:

[0699] When a user requests, "Tell me what clothes to wear to suit today's weather," the server responds, "Today is sunny and a little hot, so I'd like to suggest a light summer outfit (e.g., a shirt and shorts)."

[0700] The system of the present invention allows users to efficiently handle small tasks in daily life and save time, which not only improves the quality of life but also enhances the user's experience.

[0701] The processing flow will be explained below.

[0702] Menu suggestion function

[0703] Step 1:

[0704] User submits request:

[0705] A user types into a chat application, "What's for dinner?"

[0706] Step 2:

[0707] The device sends a request:

[0708] The device (smartphone or computer) sends this request to the server.

[0709] Step 3:

[0710] Server receives request:

[0711] The server receives the request from the user and begins parsing it.

[0712] Step 4:

[0713] Server gets user information:

[0714] The server retrieves profile data from the database, including the user's past preferences, allergy information, and food availability.

[0715] Step 5:

[0716] Server references recipe database:

[0717] The server searches the recipe database based on the acquired user information.

[0718] Step 6:

[0719] Server filters recipes:

[0720] The server selects recipes that match the user's criteria (allergy information, past preferences, and ingredient availability), and filters them taking into account seasonality and health factors.

[0721] Step 7:

[0722] Your server will select the perfect menu for you:

[0723] The server determines the best menu from the filtered recipes.

[0724] Step 8:

[0725] The server generates a response:

[0726] The server generates a message containing the menu information.

[0727] Step 9:

[0728] The server sends a response:

[0729] The server sends the generated message back to the chat application.

[0730] Step 10:

[0731] Device receives response:

[0732] The user's terminal receives the response from the server.

[0733] Step 11:

[0734] The terminal displays the message:

[0735] The chat application displays the suggested menu to the user.

[0736] Travel planning and transportation suggestion function

[0737] Step 1:

[0738] User submits request:

[0739] A user types into a chat application, "Plan a trip for next weekend."

[0740] Step 2:

[0741] The device sends a request:

[0742] The device sends a request to the server.

[0743] Step 3:

[0744] Server receives request:

[0745] The server receives the request from the user and begins parsing it.

[0746] Step 4:

[0747] Server gets user information:

[0748] The server retrieves profile data from the database, such as the user's travel history, interests, budget, and available time.

[0749] Step 5:

[0750] Server collects trip information:

[0751] The server calls an external travel information API to obtain local information and accommodation information.

[0752] Step 6:

[0753] Server generates itinerary:

[0754] The server generates an optimal travel plan based on the collected information and the user's requirements.

[0755] Step 7:

[0756] Server searches for transportation:

[0757] The server searches for the best means of transportation (e.g., train, plane, rental car) based on the user's location and destination.

[0758] Step 8:

[0759] Server selects travel mode:

[0760] The server selects the most suitable means of transportation based on the user's conditions (budget, travel time).

[0761] Step 9:

[0762] The server generates a response:

[0763] The server generates a message containing the travel plan and transportation information.

[0764] Step 10:

[0765] The server sends a response:

[0766] The server sends the generated message back to the chat application.

[0767] Step 11:

[0768] Device receives response:

[0769] The user's terminal receives the response from the server.

[0770] Step 12:

[0771] The terminal displays the message:

[0772] A chat application displays travel plans and transportation options to the user.

[0773] Clothing coordination suggestion function

[0774] Step 1:

[0775] User submits request:

[0776] The user types "Tell me what to wear today" into the chat application.

[0777] Step 2:

[0778] The device sends a request:

[0779] The device sends a request to the server.

[0780] Step 3:

[0781] Server receives request:

[0782] The server receives the request from the user and begins parsing it.

[0783] Step 4:

[0784] Server gets user information:

[0785] The server retrieves the user's past fashion history and preference profile data from the database.

[0786] Step 5:

[0787] Server gets weather information:

[0788] The server calls an external weather API to obtain current weather information.

[0789] Step 6:

[0790] The server consults the coordinate database:

[0791] The server searches for the optimal outfit based on the user's fashion history, preferences, and acquired weather information.

[0792] Step 7:

[0793] Server selects coordinates:

[0794] The server selects the best coordinates using filters and a scoring algorithm.

[0795] Step 8:

[0796] The server generates a response:

[0797] The server generates a message containing information about the selected coordinates.

[0798] Step 9:

[0799] The server sends a response:

[0800] The server sends the generated message back to the chat application.

[0801] Step 10:

[0802] Device receives response:

[0803] The user's terminal receives the response from the server.

[0804] Step 11:

[0805] The terminal displays the message:

[0806] The chat application displays suggested outfits to the user.

[0807] Example 1

[0808] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0809] In users' daily lives, tasks such as deciding on meal plans, planning trips, and coordinating fashion are often complicated and time-consuming. It is also difficult to consider the conditions and constraints required for these tasks. It is particularly challenging to obtain optimal suggestions that reflect individual users' preferences and constraints. This leads to inefficient time management and a decline in quality of life. There is a need for technology that can solve these issues and improve the quality of users' lives while making them more efficient.

[0810] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0811] In this invention, the server includes means for receiving requests from users, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, means for generating optimal proposals in response to the user's requests, means for transmitting the generated proposals to the user, means for analyzing the request content and determining which function is being requested, and means for acquiring information from an external API based on the user information. This makes it possible to quickly provide optimal proposals tailored to the preferences and needs of individual users in response to various requests in their daily lives (such as menu suggestions, travel plans, and fashion coordination).

[0812] A "user" is an entity that uses the system to send requests and receive optimal suggestions.

[0813] A "server" is a device or system that receives requests from users, analyzes data and generates suggestions, and transmits the results to the users.

[0814] A "terminal" is a device (e.g., a smartphone or computer) through which a user inputs requests and receives suggestions from a server.

[0815] A "request" is a request that a user sends to a server via a chat application or the like.

[0816] "Profile data" refers to data that includes information necessary for generating suggestions, such as the user's past usage history, preferences, and allergy information.

[0817] A "database" is a storage device that stores information necessary for generating proposals, such as profile data and recipe information.

[0818] A "suggestion" is a specific recommendation that is generated by the server based on the user's request and provided to the user.

[0819] "External API" means an external application programming interface that the server uses to obtain additional data based on user information.

[0820] In order to implement the present invention, it is necessary to build a system that generates optimal proposals based on user requests and provides them to the user. This system is mainly composed of a server, a terminal, and a user.

[0821] Hardware and Software

[0822] Hardware

[0823] 1. Server: Consists of computers with high-performance data processing capabilities, and can be a cloud server or an on-premise server.

[0824] 2. Terminal: A user device such as a smartphone, tablet, or personal computer (PC).

[0825] 3. Network: The network infrastructure that allows the server and devices to communicate through an Internet connection.

[0826] software

[0827] 1. Chat application: An application that users use to send requests to a server. It runs on smartphones and PCs.

[0828] 2. Generative AI model: An artificial intelligence model for generating recommendations (e.g., a natural language processing model or a recommender system).

[0829] 3. Database Management System (DBMS): Software that manages databases that store profile data, recipe information, travel information, fashion items, etc.

[0830] 4. External API: An external application programming interface used to obtain weather information, travel information, etc.

[0831] System Operation

[0832] 1. User submits request

[0833] A user uses a chat application to submit a request, for example, "What's for dinner?"

[0834] 2. The device sends the request to the server

[0835] The device receives the user's request, converts it into JSON format, and sends it to the server.

[0836] 3. Request Analysis by the Server

[0837] The server analyzes the received request and determines which function is being requested. For example, it selects the "menu suggestion function."

[0838] 4. Server obtains user information

[0839] Based on the user ID, the server retrieves profile data (past usage history, allergy information, inventory ingredients, etc.) from the database.

[0840] 5. Data analysis and proposal generation by the server

[0841] The server uses the generative AI model to generate optimal suggestions based on the user information and the request, and calls external APIs to obtain auxiliary information as needed.

[0842] 6. Sending results from the server to the device

[0843] The server converts the generated proposal into JSON format and sends it to the device.

[0844] 7. User confirmation of results

[0845] The user checks the proposed content in the chat application.

[0846] Specific examples

[0847] Specific examples of menu suggestions

[0848] The user types "Please suggest a dinner menu" into a chat application and sends it. The server analyzes this request and selects a "menu suggestion function." The server retrieves the user's past preferences, allergy information, and available ingredients from a database, and then uses a generative AI model to generate an optimal menu. For example, it might suggest "grilled salmon and salad." The generated suggestion is sent to the device, and the user can view the results in the chat application.

[0849] Prompt Sentence Examples

[0850] Please suggest a dinner menu

[0851] I want to go on a trip somewhere next weekend

[0852] What clothes should I wear for today's weather?

[0853] This system can provide prompt and optimal suggestions for various requests in users' daily lives, which will improve the efficiency of users' time management and enhance their quality of life.

[0854] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0855] Step 1:

[0856] The user submits a request.

[0857] Input: A user types "What's for dinner" in a chat application.

[0858] Output: A request sent by the chat application.

[0859] Specific operation: The user opens a chat application on their smartphone or computer, enters a request, and presses the send button.

[0860] Step 2:

[0861] The device sends a request to the server.

[0862] Input: A user request via a chat application.

[0863] Output: The request is sent via an API call to the server.

[0864] Specific operation: The device converts the request content into JSON format and sends a POST request to the server's API endpoint.

[0865] Step 3:

[0866] The server parses the request.

[0867] Input: The request sent to the server in JSON format.

[0868] Output: The appropriate function is selected as a result of analyzing the request content.

[0869] Specific operation: The server analyzes the request message and determines the appropriate processing route based on its content. For example, the "Menu suggestion function" is selected for the request "Tell me what's for dinner."

[0870] Step 4:

[0871] The server retrieves the user information.

[0872] Input: User ID and request details.

[0873] Output: Profile data obtained from the database (past usage history, allergy information, inventory ingredients, etc.).

[0874] What happens: The server queries the database to retrieve data related to the user.

[0875] Step 5:

[0876] The server analyzes the data and generates recommendations.

[0877] Input: Profile data, recipe database, information from external APIs (e.g. weather information).

[0878] Output: The optimal recommendation to provide to the user.

[0879] What it does: The server searches its database to select recipes based on the season and current inventory. It calls external APIs to obtain additional information as needed. It uses generative AI models to generate optimal recommendations.

[0880] Step 6:

[0881] The server sends the generated proposal to the terminal.

[0882] Input: The server-generated proposal.

[0883] Output: Suggestion information sent to the device in JSON format.

[0884] What happens: The server converts the proposal into JSON format and sends a POST request to the device's API endpoint.

[0885] Step 7:

[0886] The user checks the results.

[0887] Input: The proposal information sent back to the device.

[0888] Output: The suggestion that will be displayed in the chat application.

[0889] What happens: A user opens a chat application and sees a new message with suggested information, such as "Suggesting grilled salmon with salad."

[0890] (Application example 1)

[0891] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0892] In users' everyday shopping experiences, selecting the most suitable product from a vast amount of product information is a time-consuming task. In particular, there is a lack of systems that can make individually customized suggestions based on a user's purchasing history and profile data. This can lead to users missing potentially suitable products, so a system that provides an efficient and satisfying shopping experience is needed.

[0893] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0894] In this invention, the server includes means for receiving requests from users, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposals, means for proposing optimal products based on the user's purchase history and interests, and means for transmitting the generated proposals to the user. This allows users to receive optimal product proposals that are individually customized, enabling an efficient and satisfying shopping experience.

[0895] "Users" refers to people who use this system.

[0896] A "request" refers to the action or content of a user requesting some kind of service or information from a system.

[0897] "Usage history" refers to a record of a user's past activities when using the system.

[0898] "Profile Data" refers to a user's personal information, interests, preferences, and past behavioral data.

[0899] "Database" refers to a system for storing and managing user profile data and information for generating proposals.

[0900] "Suggestions" refer to appropriate product recommendations or advice generated based on a user's request.

[0901] "Purchase history" refers to a record of products purchased by a user in the past.

[0902] "Interests" refer to areas or things in which a user is particularly interested.

[0903] "Products" refers to various consumer goods offered to users.

[0904] "Means for suggesting optimal products" refers to the process and function of selecting and recommending the most suitable products for each individual user based on the user's profile data and purchase history.

[0905] "System" refers to an integrated technical configuration and device that includes multiple means and provides services and information to users.

[0906] "Send" refers to transferring data from the server to the user's terminal.

[0907] The system for implementing this invention receives requests from users and proposes optimal products by taking into consideration the user's profile data and purchase history. This system is mainly composed of a server, a terminal, and a user.

[0908] First, a user sends a request using a dedicated application on their device. For example, a request such as "Recommend me a coat." This request is sent from the device to the server. The device used can be a general device such as a smartphone or computer.

[0909] When the server receives the request, it retrieves profile data and purchase history from a database based on the user ID. Specific databases used include MySQL and MongoDB. Additionally, software such as the Python requests module is used to process HTTP requests to retrieve data on the server side.

[0910] The server analyzes the acquired profile data and purchase history to select appropriate products based on the user's interests and past purchasing habits. This process can be performed using a generative AI model, such as a machine learning algorithm or deep learning model. This generative AI model generates prompts that suggest the best products when the user enters specific keywords or phrases.

[0911] Once the suggestions are generated, the server sends them back to the device. The user can then review the suggestions and select specific products in the application. For example, when a user requests "Recommend me a coat," the server generates a message such as, "Taking into consideration your past purchases and preferences, we suggest the following coats," and provides a list of appropriate products.

[0912] An example of a prompt might be, "Based on user ID 12345, please suggest the best coat for me, taking into account my past purchase history and profile data." This allows users to enjoy a personalized, high-quality shopping experience.

[0913] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0914] Step 1:

[0915] The user inputs a request into a dedicated application on the device and sends it. The input data is the information the user is looking for (e.g., "Recommend a coat"). This request is sent from the device to the server. The specific operation of the device is to receive the user input and send it to the server as an HTTP POST request.

[0916] Step 2:

[0917] The server receives a request from a user. The input data is the user request, and the output data is the user ID and the request content. The specific operation of the server is to receive the HTTP request, analyze the request parameters, and extract the user ID and the request content.

[0918] Step 3:

[0919] The server retrieves the user's profile data and purchase history from the database. The input data is the user ID, and the output data is the profile data and purchase history. The specific operation of the server is to query the database for the user ID and retrieve the corresponding profile data and purchase history.

[0920] Step 4:

[0921] The server analyzes the profile data and purchase history and generates optimal product suggestions using a generative AI model. The input data is the profile data and purchase history, and the output data is the optimal product suggestions. The specific operation of the server is to analyze the acquired data and input a prompt statement to the generative AI model. As a specific example, the prompt statement "Please suggest the best coat based on user ID 12345, taking into consideration past purchase history and profile data" is input to the generative AI model.

[0922] Step 5:

[0923] The server then sends the generated proposal back to the terminal. The input data is the optimal product proposal, and the output data is the proposal to the user terminal. The specific operation of the server is to send the proposal to the terminal as an HTTP response and notify the user.

[0924] Step 6:

[0925] The terminal displays the suggestions to the user. The input data is the suggestions received from the server, and the output data is what is displayed to the user. The specific operation of the terminal is to analyze the received suggestions and display them on the user interface. As a concrete example, a list of products is displayed along with a message such as "Here are some recommended coats."

[0926] Through the above steps, the user can receive optimal product proposals that are individually customized.

[0927] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0928] This invention is a system that streamlines users' daily tasks and improves the accuracy and effectiveness of suggestions by incorporating an emotion engine. This system is mainly composed of a server, a terminal, and a user, and includes an emotion engine for recognizing the user's emotions.

[0929] Menu suggestion function

[0930] Sending and Receiving Requests

[0931] A user sends a request using a chat application, such as "Tell me what's for dinner." The device (smartphone or computer) sends this request to a server. The server analyzes the received request and starts the process of proposing the menu desired by the user.

[0932] Acquisition and analysis of user information

[0933] The server retrieves profile data from the database based on the user ID. The profile data includes the user's past preferences, allergy information, current food availability, etc. The server then uses an emotion engine to analyze the user's current emotional state.

[0934] Menu generation and suggestions

[0935] The server generates suggestions from a recipe database, taking into account not only the current season and ingredient availability, but also the user's emotional state. For example, if the user is tired, it will suggest easy-to-make dishes, and if they are energetic, it will suggest new and challenging recipes. The generated menu is sent to a chat application, where the user can check the suggested menu.

[0936] Examples:

[0937] When a user requests, "What would you like for dinner tonight?" the server responds, "Taking into account your past preferences and the current season, and because we recognize your current emotional state as tired, we'd like to suggest a quick and easy chicken and vegetable stir-fry."

[0938] Travel planning and transportation suggestions

[0939] Sending and Receiving Requests

[0940] A user sends a request to a chat application saying, "Plan a trip for next weekend." The device sends this request to a server, which receives the request and analyzes its contents.

[0941] Obtaining user information and travel conditions

[0942] Based on the user ID, the server retrieves profile data from the database, such as past travel history, interests, available budget and time, and analyzes the user's current emotional state using an emotion engine.

[0943] Travel plan and transportation generation

[0944] The server uses this information to generate an optimal travel plan. For example, if the user is feeling stressed, it might suggest a relaxing resort, or if the user is in an adventure-seeking emotional state, it might suggest a plan including active activities. Once the optimal travel plan and means of transportation are determined, they are sent to the chat application, where the user can confirm the plan.

[0945] Examples:

[0946] If a user requests, "I'd like to go on a trip somewhere next weekend," the server responds, "Based on your past travel history, current interests, and current emotional state, I recognize that you are seeking relaxation, so I will suggest a trip to a hot spring resort. The train would be a convenient means of transportation."

[0947] Clothing coordination suggestion function

[0948] Sending and Receiving Requests

[0949] The user sends a request to the chat application saying, "Tell me what outfit I'm wearing today." The device sends this request to the server. The server receives the request and analyzes its contents.

[0950] Obtaining user information and weather information

[0951] The server retrieves profile data such as past fashion history and preferences from a database based on the user ID, analyzes the user's current emotional state using an emotion engine, and calls an external weather API to retrieve current weather information.

[0952] Coordination generation and suggestions

[0953] The server uses this information to generate the optimal outfit for the user. For example, if the user is in a positive emotional state, it will suggest brightly colored clothes, and if they are feeling down, it will suggest more subdued clothing. The generated outfit is then sent to a chat application where the user can view it.

[0954] Examples:

[0955] When a user requests, "Tell me what clothes to wear to suit today's weather," the server responds, "Today is sunny and a little hot. Furthermore, taking into account that your current emotional state is positive, I would like to suggest a light summer outfit (e.g., a light-colored shirt and shorts)."

[0956] In this way, the system of the present invention can take into account the user's emotional state and provide optimal suggestions to the user, thereby enabling the user to efficiently handle daily tasks and improve their quality of life.

[0957] The processing flow will be explained below.

[0958] Menu suggestion function

[0959] Step 1:

[0960] User submits request:

[0961] A user types into a chat application, "What's for dinner?"

[0962] Step 2:

[0963] The device sends a request:

[0964] The device (smartphone or computer) sends this request to the server.

[0965] Step 3:

[0966] Server receives request:

[0967] The server receives the request from the user and begins parsing it.

[0968] Step 4:

[0969] Server gets user information:

[0970] The server retrieves the user's profile data (past preferences, allergy information, and food availability) from the database.

[0971] Step 5:

[0972] Server calls emotion engine:

[0973] The server invokes the emotion engine to analyze the user's emotional state.

[0974] Step 6:

[0975] Server references recipe database:

[0976] The server searches a recipe database based on the user's profile data and emotional state.

[0977] Step 7:

[0978] Server filters recipes:

[0979] The server selects recipes that match the user's criteria (allergy information, past preferences, ingredient availability) and emotional state, and filters them taking into account seasonality and health factors.

[0980] Step 8:

[0981] Your server will select the perfect menu for you:

[0982] The server determines the best menu from the filtered recipes.

[0983] Step 9:

[0984] The server generates a response:

[0985] The server generates a message containing the menu information.

[0986] Step 10:

[0987] The server sends a response:

[0988] The server sends the generated message back to the chat application.

[0989] Step 11:

[0990] Device receives response:

[0991] The user's terminal receives the response from the server.

[0992] Step 12:

[0993] The terminal displays the message:

[0994] The chat application displays the suggested menu to the user.

[0995] Travel planning and transportation suggestion function

[0996] Step 1:

[0997] User submits request:

[0998] A user types into a chat application, "Plan a trip for next weekend."

[0999] Step 2:

[1000] The device sends a request:

[1001] The device sends a request to the server.

[1002] Step 3:

[1003] Server receives request:

[1004] The server receives the request from the user and begins parsing it.

[1005] Step 4:

[1006] Server gets user information:

[1007] The server retrieves profile data from the database, such as the user's travel history, interests, budget, and available time.

[1008] Step 5:

[1009] Server calls emotion engine:

[1010] The server invokes the emotion engine to analyze the user's emotional state.

[1011] Step 6:

[1012] Server collects trip information:

[1013] The server calls an external travel information API to obtain local information and accommodation information.

[1014] Step 7:

[1015] Server generates itinerary:

[1016] The server generates an optimal travel plan based on the collected information and the user's conditions and emotional state.

[1017] Step 8:

[1018] Server searches for transportation:

[1019] The server searches for the best means of transportation (e.g., train, plane, rental car) based on the user's location and destination.

[1020] Step 9:

[1021] Server selects travel mode:

[1022] The server selects the most suitable mode of transportation based on the user's conditions (budget, travel time) and emotional state.

[1023] Step 10:

[1024] The server generates a response:

[1025] The server generates a message containing the travel plan and transportation information.

[1026] Step 11:

[1027] The server sends a response:

[1028] The server sends the generated message back to the chat application.

[1029] Step 12:

[1030] Device receives response:

[1031] The user's terminal receives the response from the server.

[1032] Step 13:

[1033] The terminal displays the message:

[1034] A chat application displays travel plans and transportation options to the user.

[1035] Clothing coordination suggestion function

[1036] Step 1:

[1037] User submits request:

[1038] The user types "Tell me what to wear today" into the chat application.

[1039] Step 2:

[1040] The device sends a request:

[1041] The device sends a request to the server.

[1042] Step 3:

[1043] Server receives request:

[1044] The server receives the request from the user and begins parsing it.

[1045] Step 4:

[1046] Server gets user information:

[1047] The server retrieves the user's past fashion history and preference profile data from the database.

[1048] Step 5:

[1049] Server calls emotion engine:

[1050] The server invokes the emotion engine to analyze the user's emotional state.

[1051] Step 6:

[1052] Server gets weather information:

[1053] The server calls an external weather API to obtain current weather information.

[1054] Step 7:

[1055] The server consults the coordinate database:

[1056] The server searches for the optimal outfit based on the user's fashion history and preferences, acquired weather information, and emotional state.

[1057] Step 8:

[1058] Server selects coordinates:

[1059] The server selects the best coordinates using filters and a scoring algorithm.

[1060] Step 9:

[1061] The server generates a response:

[1062] The server generates a message containing information about the selected coordinates.

[1063] Step 10:

[1064] The server sends a response:

[1065] The server sends the generated message back to the chat application.

[1066] Step 11:

[1067] Device receives response:

[1068] The user's terminal receives the response from the server.

[1069] Step 12:

[1070] The terminal displays the message:

[1071] The chat application displays suggested outfits to the user.

[1072] In this way, by incorporating an emotion engine, the system of the present invention is designed to provide personalized suggestions based on the user's emotional state, allowing them to handle everyday tasks more efficiently.

[1073] Example 2

[1074] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1075] To streamline users' daily tasks, a system that quickly provides optimal suggestions in response to individual requests is required. Conventional systems were able to provide suggestions based on a user's past preferences and profile data, but they were unable to adequately consider the user's current emotional state. This made it difficult to provide suggestions that matched the user's needs.

[1076] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving a request from a user, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, means including an emotion engine for analyzing the user's emotional state, means for generating optimal proposals in response to the user's request, and means for transmitting the generated proposals to the user. This enables proposals that take the user's emotional state into consideration.

[1077] "User" means an individual or corporation that uses the system.

[1078] A "request" is a specific request or inquiry sent from a user to a system.

[1079] A "terminal" is a device that a user uses to access and operate the system, and includes smartphones, computers, tablets, etc.

[1080] A "server" is a computer system that receives and processes requests from users.

[1081] "Profile data" refers to data that includes personal information such as a user's past usage history, preferences, allergy information, and food ingredient availability.

[1082] A "database" is a collection of stored data that is referenced to generate recommendations.

[1083] An "emotion engine" is a technology for analyzing a user's current emotional state, analyzing the user's emotions and generating data based on the analysis.

[1084] A "suggestion" is the optimal answer or advice that the system generates in response to a user's request.

[1085] The "menu suggestion means" is a function that suggests the most suitable meal menu based on the user's meal requests.

[1086] The "travel planning and transportation suggestion means" is a function that suggests optimal travel plans and transportation means based on the user's travel requests.

[1087] The "fashion coordination suggestion means" is a function that suggests the most suitable fashion coordination based on the user's clothing requests.

[1088] A "natural language processing (NLP) engine" is a technology for analyzing and understanding user requests.

[1089] A "generative AI model" is an artificial intelligence technique that generates specific outputs based on user requests and other data.

[1090] This invention is a system that streamlines users' daily tasks and improves the accuracy and effectiveness of suggestions by incorporating an emotion engine. This system consists of three main components: a server, a terminal, and a user.

[1091] System configuration and hardware / software integration

[1092] server

[1093] The server plays a central role in receiving requests from users, analyzing them, and generating optimal suggestions. The following specific software is used:

[1094] NLP engine: Uses spaCy and NLTK to analyze the content of requests using natural language processing.

[1095] Emotion Engine: Analyzes the user's emotional state using Affectiva and Microsoft Emotion API.

[1096] Databases: Relational databases such as MySQL and PostgreSQL, and NoSQL databases such as MongoDB and Elasticsearch are used to manage data such as user profile data, recipe data, and travel plans.

[1097] Terminal

[1098] A terminal is a device that allows a user to access and operate a system. Typically, this is a smartphone or computer. A terminal has the following functions:

[1099] Chat application: Sends user requests to a server, receives responses from the server and displays them to the user.

[1100] User

[1101] A user is an individual or legal entity that uses a terminal to send requests to the system and receive offers from the server.

[1102] System Functions and Prompt Sentence Examples

[1103] Menu suggestion function

[1104] Sending and Receiving Requests

[1105] A user sends a request using a chat application saying, "Tell me what's for dinner." The device then sends this request to the server.

[1106] Example: A user types, "What's for dinner tonight?"

[1107] Acquisition and analysis of user information

[1108] The server retrieves profile data (past preferences, allergy information, food stock status, etc.) from a database based on the user ID and analyzes the emotional state using an emotion engine.

[1109] Example: The server responds, "Considering your past preferences and the current season, and your emotional state of fatigue, I would like to suggest a quick chicken and vegetable stir fry."

[1110] Travel planning and transportation suggestions

[1111] Sending and Receiving Requests

[1112] A user sends a request to a chat application saying, "Plan a trip for next weekend." The device sends this request to a server.

[1113] Example: A user types, "I want to go on a trip somewhere next weekend."

[1114] Obtaining user information and travel conditions

[1115] Based on the user ID, the server obtains data such as past travel history, interests, budget, and time, and analyzes the user's emotional state using an emotion engine.

[1116] Example: The server responds, "Since your past travel history, current interests, and emotional state indicate a desire for relaxation, we suggest a trip to a hot spring resort. Trains would be a convenient means of transportation."

[1117] Clothing coordination suggestion function

[1118] Sending and Receiving Requests

[1119] The user sends a request to the chat application saying, "Tell me what outfit I'm wearing today." The device then sends this request to the server.

[1120] Example: A user types, "What should I wear for today's weather?"

[1121] Obtaining user information and weather information

[1122] The server obtains fashion history and preference data based on the user ID, analyzes the user's emotional state using an emotion engine, and obtains current weather information using a weather API.

[1123] Example: The server responds, "Today is sunny and slightly hot, and your emotional state is positive, so I suggest a light summer outfit (e.g., a light-colored shirt and shorts)."

[1124] Prompt Sentence Examples

[1125] Meal suggestion feature: "Generate prompts to suggest dinner menus for tired users."

[1126] Trip Planning & Transportation Suggestion: "Generate prompts to suggest trip plans for users looking to relax."

[1127] Clothing Coordination Suggestion: "Generate prompts that suggest fashion coordination for users who are in a positive emotional state."

[1128] In this way, the system of the present invention can take into account the user's emotional state to provide more personalized suggestions and efficiently handle everyday tasks.

[1129] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1130] Menu suggestion function

[1131] Processing Steps

[1132] Step 1:

[1133] Terminal: A user types a request into a chat application: "What's for dinner?"

[1134] Input: User request

[1135] Output: Sends an API request to the server

[1136] What it does: Use your smartphone or computer to access a chat app, type in your request, and press send.

[1137] Step 2:

[1138] Server: Receives requests sent from the device and analyzes them using the NLP engine.

[1139] Input: API request

[1140] Output: Parsed request data

[1141] What it does: The server uses a natural language processing engine (e.g., spaCy or NLTK) to analyze the user's request and extract meaning.

[1142] Step 3:

[1143] Server: Retrieves profile data from the database based on the user ID.

[1144] Input: User ID

[1145] Output: User profile data (past preferences, allergy information, current ingredient availability)

[1146] Specific operation: The server issues a query to a database (e.g., MySQL or PostgreSQL) to retrieve profile data.

[1147] Step 4:

[1148] Server: Analyzes the user's current emotional state using the emotion engine.

[1149] Input: User profile data

[1150] Output: Current emotional state data

[1151] What it does: The server inputs profile data into an emotion engine (e.g., Affectiva or Microsoft Emotion API) to analyze the emotional state.

[1152] Step 5:

[1153] Server: Searches the recipe database for candidate recipes that match the criteria and generates the optimal menu.

[1154] Input: Emotional state data, profile data, database query

[1155] Output: Optimal meal plan

[1156] Specific operation: The server issues a query to a recipe database (e.g., MongoDB or Elasticsearch), searches for recipes that meet the criteria, and generates the optimal menu.

[1157] Step 6:

[1158] Server: Formats the generated menu information into a response message and sends it to the device.

[1159] Input: Optimal meal plan

[1160] Output: The formatted response message

[1161] Specific operation: The server formats the generated menu data in rich text or JSON format and sends it to the terminal.

[1162] Step 7:

[1163] Terminal: Messages received from the server are displayed within the chat application for the user to review.

[1164] Input: Response message from the server

[1165] Output: Menu display in a chat application

[1166] Specific behavior: The device receives the response message and displays it in the chat application UI.

[1167] Travel planning and transportation suggestions

[1168] Processing Steps

[1169] Step 1:

[1170] Terminal: A user types a request into a chat application: "Plan a trip for next weekend."

[1171] Input: User request

[1172] Output: Sends an API request to the server

[1173] What it does: Access a chat app on your smartphone or computer, type in your request, and press send.

[1174] Step 2:

[1175] Server: Receives requests sent from the device and analyzes them using the NLP engine.

[1176] Input: API request

[1177] Output: Parsed request data

[1178] What happens: The server uses a natural language processing engine to analyze the request and extract meaning.

[1179] Step 3:

[1180] Server: Retrieves profile data from the database based on the user ID.

[1181] Input: User ID

[1182] Output: Past travel history, interests, budget, time data

[1183] What happens next: The server queries the database to retrieve profile data.

[1184] Step 4:

[1185] Server: Analyzes the user's current emotional state using the emotion engine.

[1186] Input: Profile data

[1187] Output: Current emotional state data

[1188] Specific operation: The server inputs profile data into the emotion engine and analyzes the emotional state.

[1189] Step 5:

[1190] Server: Generates the optimal travel plan based on the acquired information.

[1191] Input: Emotional state data, profile data, database query

[1192] Output: Optimal travel plan

[1193] Specific operation: The server executes the travel plan generation algorithm and generates a plan that meets the conditions.

[1194] Step 6:

[1195] Server: Formats the itinerary and transportation methods into a response message and sends it to the device.

[1196] Input: Best Travel Plan

[1197] Output: The formatted response message

[1198] Specific operation: The server formats the generated plan data and sends it to the device.

[1199] Step 7:

[1200] On the device: The proposed itinerary is displayed in the chat app for the user to review.

[1201] Input: Response message from the server

[1202] Output: Plan display in a chat application

[1203] Specific behavior: The device receives the response message and displays it in the chat app's UI.

[1204] Clothing coordination suggestion function

[1205] Processing Steps

[1206] Step 1:

[1207] Terminal: A user types a request into a chat application, such as "Tell me what outfit to wear today."

[1208] Input: User request

[1209] Output: Sends an API request to the server

[1210] What it does: Access a chat app on your smartphone or computer, type in your request, and press send.

[1211] Step 2:

[1212] Server: Receives requests sent from the device and analyzes them using the NLP engine.

[1213] Input: API request

[1214] Output: Parsed request data

[1215] What happens: The server uses a natural language processing engine to analyze the content of the request.

[1216] Step 3:

[1217] Server: Retrieves profile data from the database based on the user ID.

[1218] Input: User ID

[1219] Output: Past fashion history, preference data

[1220] What happens next: The server queries the database to retrieve profile data.

[1221] Step 4:

[1222] Server: Analyzes the user's current emotional state using the emotion engine.

[1223] Input: Profile data

[1224] Output: Current emotional state data

[1225] Specific operation: The server inputs profile data into the emotion engine and analyzes the emotional state.

[1226] Step 5:

[1227] Server: Calls the weather API to get current weather information.

[1228] Input: Weather API request

[1229] Output: Current weather information

[1230] What happens: The server sends a request to an external weather API to get current weather information.

[1231] Step 6:

[1232] Server: Generates optimal coordination based on the acquired information.

[1233] Input: Emotional state data, fashion history data, weather information

[1234] Output: Optimal outfit ideas

[1235] Specific operation: The server executes the coordination generation algorithm and generates fashion suggestions that meet the conditions.

[1236] Step 7:

[1237] Server: Formats the proposed coordinates into a response message and sends it to the device.

[1238] Input: Best outfit ideas

[1239] Output: The formatted response message

[1240] Specific operation: The server formats the generated coordinate data and sends it to the terminal.

[1241] Step 8:

[1242] Device: The suggested outfits are displayed in the chat app so that the user can check them.

[1243] Input: Response message from the server

[1244] Output: Coordinate display in a chat application

[1245] Specific behavior: The device receives the response message and displays it in the chat app's UI.

[1246] Example prompts for generative AI models

[1247] Meal suggestion feature: "Generate prompts to suggest dinner menus for tired users."

[1248] Trip Planning & Transportation Suggestion: "Generate prompts to suggest trip plans for users looking to relax."

[1249] Clothing Coordination Suggestion: "Generate prompts that suggest fashion coordination for users who are in a positive emotional state."

[1250] (Application example 2)

[1251] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1252] Conventional recommendation systems make suggestions based on user requests and past data, but because they do not take the user's emotional state into account, they are unable to make suggestions that are optimal for the user's current mood or situation, limiting the improvement of user satisfaction.In addition, it has been difficult to make personalized suggestions based on real-time emotion recognition in physical stores, etc.

[1253] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1254] In this invention, the server includes means for receiving requests from a user, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, means including an emotion engine for analyzing the user's emotional state, means for generating optimal proposals in accordance with the user's requests and emotional state, and means for transmitting the generated proposals to the user, thereby enabling optimal proposals that take into account the user's current emotional state.

[1255] The "means for receiving a request from a user" refers to a device or software that allows a user to input a request for information or a service they desire into the system and receives the request electronically.

[1256] "Means for obtaining a user's past usage history and profile data" refers to a device or software that extracts records of a user's past use of the system, personal information, and preference data.

[1257] The "means for referencing a database for generating proposal content" refers to a device or software that accesses and refers to a database that stores information required for generating various proposal content.

[1258] The "means including an emotion engine for analyzing the user's emotional state" refers to hardware and software for recognizing and analyzing emotions from the user's facial expressions, tone of voice, etc.

[1259] The "means for generating optimal suggestions according to the user's requests and emotional state" refers to a device or software for generating the most appropriate suggestions based on the user's request content and analyzed emotional state.

[1260] The "means for transmitting the generated proposal to the user" refers to a device or software that electronically transmits the content of the generated proposal to the user's terminal.

[1261] The system of the present invention makes suggestions to improve the efficiency of a user's daily tasks and includes an emotion engine that analyzes the user's emotional state. This system is primarily composed of a server, a terminal, and a user. Specific embodiments for realizing this system are described below.

[1262] The server includes means for receiving requests from users, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, an emotion engine for analyzing the user's emotional state, means for generating optimal proposals according to the user's requests and emotional state, and means for transmitting the generated proposals to the user.

[1263] Specifically, when a user accesses product information using a smartphone or smart glasses, the system analyzes the user's emotional state at that time and proposes the most suitable products and sets. For example, if the user is tired, the system will propose relaxing items (bath additives, aromatherapy, relaxation goods, etc.), and if the user is energetic, it will propose products suitable for new challenges or activities (sports goods, games, activity kits, etc.).

[1264] This system mainly uses the following hardware and software:

[1265] The camera on a smartphone or smart glasses is used to capture the user's facial expressions.

[1266] Using OpenCV (image processing library), the user's facial expression is processed from the captured image.

[1267] Use emotion_recognition (emotion recognition library) to analyze the user's current emotional state from their facial expressions.

[1268] Use requests (an HTTP request library) to communicate with the user database and retrieve user profile data.

[1269] Use the recommendation_engine to generate optimal recommendations based on emotional state and profile data.

[1270] As a concrete example, a user enters a physical store through smart glasses. It seems that the user has been very busy that weekend and is feeling stressed. The camera analyzes the user's facial expressions to determine their emotions and determines that they are "tired." As a result, the smart glasses' display displays a message saying, "We'll suggest products that will help you relax today!" The user can then choose a product based on the suggestions.

[1271] Example prompt sentence:

[1272] "Relax, we've found the perfect product for you."

[1273] "Here are some recommended activities for you who are feeling energetic!"

[1274] In this way, by providing individually optimized suggestions that take into account the user's emotional state, user satisfaction can be improved.

[1275] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1276] Step 1:

[1277] A user inputs a request using a smartphone or smart glasses. The user sends a question to the system, such as "What is today's recommended product?" The input is the user's request, and the output is the request data.

[1278] Step 2:

[1279] The device sends the user's request to the server. After receiving the user's request, the smartphone or smart glasses transmit the request to the server via the Internet. The input is the user's request data, and the output is a confirmation of the request transmission to the server.

[1280] Step 3:

[1281] The server receives and analyzes the request. The received request data is analyzed by a program on the server to identify the type of information or service the user is seeking. The input is the received request data, and the output is the analysis result.

[1282] Step 4:

[1283] The server retrieves the user's profile data. After analyzing the request, the server retrieves past usage history and profile data from the database based on the user ID. The input is the user ID and the output is the profile data.

[1284] Step 5:

[1285] The server uses an emotion engine to analyze the user's emotional state. The emotion engine analyzes the user's emotional state based on the user's facial expressions captured in real time by a camera installed in a smartphone or smart glasses. The input is the user's facial expression data, and the output is the analyzed emotional state.

[1286] Step 6:

[1287] The server generates optimal suggestions based on the acquired profile data and analyzed emotional state. The server refers to a recipe database and recommends products and services that match the user's profile data and emotional state. The input is the profile data and emotional state, and the output is the optimal suggestions.

[1288] Step 7:

[1289] The server sends the generated proposal to the terminal. Once the optimal proposal is generated, the server sends it to the user's terminal. The input is the optimal proposal, and the output is a confirmation of the proposal transmission to the terminal.

[1290] Step 8:

[1291] The device displays the proposed content to the user. The smartphone or smart glasses displays the received proposed content on the user's display. The input is the proposed content sent from the server, and the output is the proposed content displayed on the user's display.

[1292] Specifically, the user speaks to the smart glasses, asking, "What is today's recommended product?" The server analyzes the response and displays, "Considering that you are tired, we recommend an aroma candle to help you relax today."

[1293] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1294] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1295] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1296] [Third embodiment]

[1297] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1298] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1299] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1300] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1301] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1302] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1303] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1304] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1305] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1306] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1307] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1308] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1309] The present invention is a system that streamlines users' daily tasks by receiving requests from users and providing optimal suggestions based on the users' past usage history and profile data. This system is mainly composed of a server, a terminal, and a user.

[1310] Menu suggestion function

[1311] Sending and Receiving Requests

[1312] A user sends a request using a chat application, such as "Tell me what's for dinner." The device (smartphone or computer) sends this request to a server. The server analyzes the received request and starts the process of proposing the menu desired by the user.

[1313] Acquisition and analysis of user information

[1314] The server retrieves profile data from the database based on the user ID. The profile data includes the user's past preferences, allergy information, current food stock status, etc. This allows the server to understand the user's preferences and restrictions.

[1315] Menu generation and suggestions

[1316] The server searches and selects recipes that can be suggested to the user from the recipe database, taking into account the current season and the availability of ingredients. Once the optimal menu is determined, the server sends it back to the chat application. The user can then view the suggested menu through the chat application on their device.

[1317] Examples:

[1318] When a user requests, "What would you like for dinner tonight?" the server responds, "Taking into account your past preferences and the current season, I would like to suggest grilled salmon and a salad."

[1319] Travel planning and transportation suggestions

[1320] Sending and Receiving Requests

[1321] A user sends a request to a chat application saying, "Plan a trip for next weekend." The device sends this request to a server, which receives the request and analyzes its contents.

[1322] Obtaining user information and travel conditions

[1323] Based on the user ID, the server retrieves profile data such as past travel history, interests, available budget and time from the database, and also calls external travel information APIs to collect local information and accommodation information.

[1324] Travel plan and transportation generation

[1325] The server generates an optimal travel plan based on this information and suggests the most suitable means of transportation for the user (e.g., train, plane, rental car). The generated plan and means of transportation are sent to the chat application, where the user can confirm them.

[1326] Examples:

[1327] When a user requests, "I'd like to go on a trip somewhere next weekend," the server responds, "Taking into account your past travel history and current interests, I would like to suggest a day trip to Kyoto. The Shinkansen would be a convenient means of transportation."

[1328] Clothing coordination suggestion function

[1329] Sending and Receiving Requests

[1330] The user sends a request to the chat application saying, "Tell me what outfit I'm wearing today." The device sends this request to the server. The server receives the request and analyzes its contents.

[1331] Obtaining user information and weather information

[1332] The server retrieves profile data such as past fashion history and preferences from a database based on the user ID, and then calls an external weather API to retrieve current weather information.

[1333] Coordination generation and suggestions

[1334] The server generates the optimal coordinate based on this information, and the proposed coordinate is sent to the chat application, where the user can view it on their device.

[1335] Examples:

[1336] When a user requests, "Tell me what clothes to wear to suit today's weather," the server responds, "Today is sunny and a little hot, so I'd like to suggest a light summer outfit (e.g., a shirt and shorts)."

[1337] The system of the present invention allows users to efficiently handle small tasks in daily life and save time, which not only improves the quality of life but also enhances the user's experience.

[1338] The processing flow will be explained below.

[1339] Menu suggestion function

[1340] Step 1:

[1341] User submits request:

[1342] A user types into a chat application, "What's for dinner?"

[1343] Step 2:

[1344] The device sends a request:

[1345] The device (smartphone or computer) sends this request to the server.

[1346] Step 3:

[1347] Server receives request:

[1348] The server receives the request from the user and begins parsing it.

[1349] Step 4:

[1350] Server gets user information:

[1351] The server retrieves profile data from the database, including the user's past preferences, allergy information, and food availability.

[1352] Step 5:

[1353] Server references recipe database:

[1354] The server searches the recipe database based on the acquired user information.

[1355] Step 6:

[1356] Server filters recipes:

[1357] The server selects recipes that match the user's criteria (allergy information, past preferences, and ingredient availability), and filters them taking into account seasonality and health factors.

[1358] Step 7:

[1359] Your server will select the perfect menu for you:

[1360] The server determines the best menu from the filtered recipes.

[1361] Step 8:

[1362] The server generates a response:

[1363] The server generates a message containing the menu information.

[1364] Step 9:

[1365] The server sends a response:

[1366] The server sends the generated message back to the chat application.

[1367] Step 10:

[1368] Device receives response:

[1369] The user's terminal receives the response from the server.

[1370] Step 11:

[1371] The terminal displays the message:

[1372] The chat application displays the suggested menu to the user.

[1373] Travel planning and transportation suggestion function

[1374] Step 1:

[1375] User submits request:

[1376] A user types into a chat application, "Plan a trip for next weekend."

[1377] Step 2:

[1378] The device sends a request:

[1379] The device sends a request to the server.

[1380] Step 3:

[1381] Server receives request:

[1382] The server receives the request from the user and begins parsing it.

[1383] Step 4:

[1384] Server gets user information:

[1385] The server retrieves profile data from the database, such as the user's travel history, interests, budget, and available time.

[1386] Step 5:

[1387] Server collects trip information:

[1388] The server calls an external travel information API to obtain local information and accommodation information.

[1389] Step 6:

[1390] Server generates itinerary:

[1391] The server generates an optimal travel plan based on the collected information and the user's requirements.

[1392] Step 7:

[1393] Server searches for transportation:

[1394] The server searches for the best means of transportation (e.g., train, plane, rental car) based on the user's location and destination.

[1395] Step 8:

[1396] Server selects travel mode:

[1397] The server selects the most suitable means of transportation based on the user's conditions (budget, travel time).

[1398] Step 9:

[1399] The server generates a response:

[1400] The server generates a message containing the travel plan and transportation information.

[1401] Step 10:

[1402] The server sends a response:

[1403] The server sends the generated message back to the chat application.

[1404] Step 11:

[1405] Device receives response:

[1406] The user's terminal receives the response from the server.

[1407] Step 12:

[1408] The terminal displays the message:

[1409] A chat application displays travel plans and transportation options to the user.

[1410] Clothing coordination suggestion function

[1411] Step 1:

[1412] User submits request:

[1413] The user types "Tell me what to wear today" into the chat application.

[1414] Step 2:

[1415] The device sends a request:

[1416] The device sends a request to the server.

[1417] Step 3:

[1418] Server receives request:

[1419] The server receives the request from the user and begins parsing it.

[1420] Step 4:

[1421] Server gets user information:

[1422] The server retrieves the user's past fashion history and preference profile data from the database.

[1423] Step 5:

[1424] Server gets weather information:

[1425] The server calls an external weather API to obtain current weather information.

[1426] Step 6:

[1427] The server consults the coordinate database:

[1428] The server searches for the optimal outfit based on the user's fashion history, preferences, and acquired weather information.

[1429] Step 7:

[1430] Server selects coordinates:

[1431] The server selects the best coordinates using filters and a scoring algorithm.

[1432] Step 8:

[1433] The server generates a response:

[1434] The server generates a message containing information about the selected coordinates.

[1435] Step 9:

[1436] The server sends a response:

[1437] The server sends the generated message back to the chat application.

[1438] Step 10:

[1439] Device receives response:

[1440] The user's terminal receives the response from the server.

[1441] Step 11:

[1442] The terminal displays the message:

[1443] The chat application displays suggested outfits to the user.

[1444] Example 1

[1445] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1446] In users' daily lives, tasks such as deciding on meal plans, planning trips, and coordinating fashion are often complicated and time-consuming. It is also difficult to consider the conditions and constraints required for these tasks. It is particularly challenging to obtain optimal suggestions that reflect individual users' preferences and constraints. This leads to inefficient time management and a decline in quality of life. There is a need for technology that can solve these issues and improve the quality of users' lives while making them more efficient.

[1447] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1448] In this invention, the server includes means for receiving requests from users, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, means for generating optimal proposals in response to the user's requests, means for transmitting the generated proposals to the user, means for analyzing the request content and determining which function is being requested, and means for acquiring information from an external API based on the user information. This makes it possible to quickly provide optimal proposals tailored to the preferences and needs of individual users in response to various requests in their daily lives (such as menu suggestions, travel plans, and fashion coordination).

[1449] A "user" is an entity that uses the system to send requests and receive optimal suggestions.

[1450] A "server" is a device or system that receives requests from users, analyzes data and generates suggestions, and transmits the results to the users.

[1451] A "terminal" is a device (e.g., a smartphone or computer) through which a user inputs requests and receives suggestions from a server.

[1452] A "request" is a request that a user sends to a server via a chat application or the like.

[1453] "Profile data" refers to data that includes information necessary for generating suggestions, such as the user's past usage history, preferences, and allergy information.

[1454] A "database" is a storage device that stores information necessary for generating proposals, such as profile data and recipe information.

[1455] A "suggestion" is a specific recommendation that is generated by the server based on the user's request and provided to the user.

[1456] "External API" means an external application programming interface that the server uses to obtain additional data based on user information.

[1457] In order to implement the present invention, it is necessary to build a system that generates optimal proposals based on user requests and provides them to the user. This system is mainly composed of a server, a terminal, and a user.

[1458] Hardware and Software

[1459] Hardware

[1460] 1. Server: Consists of computers with high-performance data processing capabilities, and can be a cloud server or an on-premise server.

[1461] 2. Terminal: A user device such as a smartphone, tablet, or personal computer (PC).

[1462] 3. Network: The network infrastructure that allows the server and devices to communicate through an Internet connection.

[1463] software

[1464] 1. Chat application: An application that users use to send requests to a server. It runs on smartphones and PCs.

[1465] 2. Generative AI model: An artificial intelligence model for generating recommendations (e.g., a natural language processing model or a recommender system).

[1466] 3. Database Management System (DBMS): Software that manages databases that store profile data, recipe information, travel information, fashion items, etc.

[1467] 4. External API: An external application programming interface used to obtain weather information, travel information, etc.

[1468] System Operation

[1469] 1. User submits request

[1470] A user uses a chat application to submit a request, for example, "What's for dinner?"

[1471] 2. The device sends the request to the server

[1472] The device receives the user's request, converts it into JSON format, and sends it to the server.

[1473] 3. Request Analysis by the Server

[1474] The server analyzes the received request and determines which function is being requested. For example, it selects the "menu suggestion function."

[1475] 4. Server obtains user information

[1476] Based on the user ID, the server retrieves profile data (past usage history, allergy information, inventory ingredients, etc.) from the database.

[1477] 5. Data analysis and proposal generation by the server

[1478] The server uses the generative AI model to generate optimal suggestions based on the user information and the request, and calls external APIs to obtain auxiliary information as needed.

[1479] 6. Sending results from the server to the device

[1480] The server converts the generated proposal into JSON format and sends it to the device.

[1481] 7. User confirmation of results

[1482] The user checks the proposed content in the chat application.

[1483] Specific examples

[1484] Specific examples of menu suggestions

[1485] The user types "Please suggest a dinner menu" into a chat application and sends it. The server analyzes this request and selects a "menu suggestion function." The server retrieves the user's past preferences, allergy information, and available ingredients from a database, and then uses a generative AI model to generate an optimal menu. For example, it might suggest "grilled salmon and salad." The generated suggestion is sent to the device, and the user can view the results in the chat application.

[1486] Prompt Sentence Examples

[1487] Please suggest a dinner menu

[1488] I want to go on a trip somewhere next weekend

[1489] What clothes should I wear for today's weather?

[1490] This system can provide prompt and optimal suggestions for various requests in users' daily lives, which will improve the efficiency of users' time management and enhance their quality of life.

[1491] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1492] Step 1:

[1493] The user submits a request.

[1494] Input: A user types "What's for dinner" in a chat application.

[1495] Output: A request sent by the chat application.

[1496] Specific operation: The user opens a chat application on their smartphone or computer, enters a request, and presses the send button.

[1497] Step 2:

[1498] The device sends a request to the server.

[1499] Input: A user request via a chat application.

[1500] Output: The request is sent via an API call to the server.

[1501] Specific operation: The device converts the request content into JSON format and sends a POST request to the server's API endpoint.

[1502] Step 3:

[1503] The server parses the request.

[1504] Input: The request sent to the server in JSON format.

[1505] Output: The appropriate function is selected as a result of analyzing the request content.

[1506] Specific operation: The server analyzes the request message and determines the appropriate processing route based on its content. For example, the "Menu suggestion function" is selected for the request "Tell me what's for dinner."

[1507] Step 4:

[1508] The server retrieves the user information.

[1509] Input: User ID and request details.

[1510] Output: Profile data obtained from the database (past usage history, allergy information, inventory ingredients, etc.).

[1511] What happens: The server queries the database to retrieve data related to the user.

[1512] Step 5:

[1513] The server analyzes the data and generates recommendations.

[1514] Input: Profile data, recipe database, information from external APIs (e.g. weather information).

[1515] Output: The optimal recommendation to provide to the user.

[1516] What it does: The server searches its database to select recipes based on the season and current inventory. It calls external APIs to obtain additional information as needed. It uses generative AI models to generate optimal recommendations.

[1517] Step 6:

[1518] The server sends the generated proposal to the terminal.

[1519] Input: The server-generated proposal.

[1520] Output: Suggestion information sent to the device in JSON format.

[1521] What happens: The server converts the proposal into JSON format and sends a POST request to the device's API endpoint.

[1522] Step 7:

[1523] The user checks the results.

[1524] Input: The proposal information sent back to the device.

[1525] Output: The suggestion that will be displayed in the chat application.

[1526] What happens: A user opens a chat application and sees a new message with suggested information, such as "Suggesting grilled salmon with salad."

[1527] (Application example 1)

[1528] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1529] In users' everyday shopping experiences, selecting the most suitable product from a vast amount of product information is a time-consuming task. In particular, there is a lack of systems that can make individually customized suggestions based on a user's purchasing history and profile data. This can lead to users missing potentially suitable products, so a system that provides an efficient and satisfying shopping experience is needed.

[1530] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1531] In this invention, the server includes means for receiving requests from users, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposals, means for proposing optimal products based on the user's purchase history and interests, and means for transmitting the generated proposals to the user. This allows users to receive optimal product proposals that are individually customized, enabling an efficient and satisfying shopping experience.

[1532] "Users" refers to people who use this system.

[1533] A "request" refers to the action or content of a user requesting some kind of service or information from a system.

[1534] "Usage history" refers to a record of a user's past activities when using the system.

[1535] "Profile Data" refers to a user's personal information, interests, preferences, and past behavioral data.

[1536] "Database" refers to a system for storing and managing user profile data and information for generating proposals.

[1537] "Suggestions" refer to appropriate product recommendations or advice generated based on a user's request.

[1538] "Purchase history" refers to a record of products purchased by a user in the past.

[1539] "Interests" refer to areas or things in which a user is particularly interested.

[1540] "Products" refers to various consumer goods offered to users.

[1541] "Means for suggesting optimal products" refers to the process and function of selecting and recommending the most suitable products for each individual user based on the user's profile data and purchase history.

[1542] "System" refers to an integrated technical configuration and device that includes multiple means and provides services and information to users.

[1543] "Send" refers to transferring data from the server to the user's terminal.

[1544] The system for implementing this invention receives requests from users and proposes optimal products by taking into consideration the user's profile data and purchase history. This system is mainly composed of a server, a terminal, and a user.

[1545] First, a user sends a request using a dedicated application on their device. For example, a request such as "Recommend me a coat." This request is sent from the device to the server. The device used can be a general device such as a smartphone or computer.

[1546] When the server receives the request, it retrieves profile data and purchase history from a database based on the user ID. Specific databases used include MySQL and MongoDB. Additionally, software such as the Python requests module is used to process HTTP requests to retrieve data on the server side.

[1547] The server analyzes the acquired profile data and purchase history to select appropriate products based on the user's interests and past purchasing habits. This process can be performed using a generative AI model, such as a machine learning algorithm or deep learning model. This generative AI model generates prompts that suggest the best products when the user enters specific keywords or phrases.

[1548] Once the suggestions are generated, the server sends them back to the device. The user can then review the suggestions and select specific products in the application. For example, when a user requests "Recommend me a coat," the server generates a message such as, "Taking into consideration your past purchases and preferences, we suggest the following coats," and provides a list of appropriate products.

[1549] An example of a prompt might be, "Based on user ID 12345, please suggest the best coat for me, taking into account my past purchase history and profile data." This allows users to enjoy a personalized, high-quality shopping experience.

[1550] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1551] Step 1:

[1552] The user inputs a request into a dedicated application on the device and sends it. The input data is the information the user is looking for (e.g., "Recommend a coat"). This request is sent from the device to the server. The specific operation of the device is to receive the user input and send it to the server as an HTTP POST request.

[1553] Step 2:

[1554] The server receives a request from a user. The input data is the user request, and the output data is the user ID and the request content. The specific operation of the server is to receive the HTTP request, analyze the request parameters, and extract the user ID and the request content.

[1555] Step 3:

[1556] The server retrieves the user's profile data and purchase history from the database. The input data is the user ID, and the output data is the profile data and purchase history. The specific operation of the server is to query the database for the user ID and retrieve the corresponding profile data and purchase history.

[1557] Step 4:

[1558] The server analyzes the profile data and purchase history and generates optimal product suggestions using a generative AI model. The input data is the profile data and purchase history, and the output data is the optimal product suggestions. The specific operation of the server is to analyze the acquired data and input a prompt statement to the generative AI model. As a specific example, the prompt statement "Please suggest the best coat based on user ID 12345, taking into consideration past purchase history and profile data" is input to the generative AI model.

[1559] Step 5:

[1560] The server then sends the generated proposal back to the terminal. The input data is the optimal product proposal, and the output data is the proposal to the user terminal. The specific operation of the server is to send the proposal to the terminal as an HTTP response and notify the user.

[1561] Step 6:

[1562] The terminal displays the suggestions to the user. The input data is the suggestions received from the server, and the output data is what is displayed to the user. The specific operation of the terminal is to analyze the received suggestions and display them on the user interface. As a concrete example, a list of products is displayed along with a message such as "Here are some recommended coats."

[1563] Through the above steps, the user can receive optimal product proposals that are individually customized.

[1564] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1565] This invention is a system that streamlines users' daily tasks and improves the accuracy and effectiveness of suggestions by incorporating an emotion engine. This system is mainly composed of a server, a terminal, and a user, and includes an emotion engine for recognizing the user's emotions.

[1566] Menu suggestion function

[1567] Sending and Receiving Requests

[1568] A user sends a request using a chat application, such as "Tell me what's for dinner." The device (smartphone or computer) sends this request to a server. The server analyzes the received request and starts the process of proposing the menu desired by the user.

[1569] Acquisition and analysis of user information

[1570] The server retrieves profile data from the database based on the user ID. The profile data includes the user's past preferences, allergy information, current food availability, etc. The server then uses an emotion engine to analyze the user's current emotional state.

[1571] Menu generation and suggestions

[1572] The server generates suggestions from a recipe database, taking into account not only the current season and ingredient availability, but also the user's emotional state. For example, if the user is tired, it will suggest easy-to-make dishes, and if they are energetic, it will suggest new and challenging recipes. The generated menu is sent to a chat application, where the user can check the suggested menu.

[1573] Examples:

[1574] When a user requests, "What would you like for dinner tonight?" the server responds, "Taking into account your past preferences and the current season, and because we recognize your current emotional state as tired, we'd like to suggest a quick and easy chicken and vegetable stir-fry."

[1575] Travel planning and transportation suggestions

[1576] Sending and Receiving Requests

[1577] A user sends a request to a chat application saying, "Plan a trip for next weekend." The device sends this request to a server, which receives the request and analyzes its contents.

[1578] Obtaining user information and travel conditions

[1579] Based on the user ID, the server retrieves profile data from the database, such as past travel history, interests, available budget and time, and analyzes the user's current emotional state using an emotion engine.

[1580] Travel plan and transportation generation

[1581] The server uses this information to generate an optimal travel plan. For example, if the user is feeling stressed, it might suggest a relaxing resort, or if the user is in an adventure-seeking emotional state, it might suggest a plan including active activities. Once the optimal travel plan and means of transportation are determined, they are sent to the chat application, where the user can confirm the plan.

[1582] Examples:

[1583] If a user requests, "I'd like to go on a trip somewhere next weekend," the server responds, "Based on your past travel history, current interests, and current emotional state, I recognize that you are seeking relaxation, so I will suggest a trip to a hot spring resort. The train would be a convenient means of transportation."

[1584] Clothing coordination suggestion function

[1585] Sending and Receiving Requests

[1586] The user sends a request to the chat application saying, "Tell me what outfit I'm wearing today." The device sends this request to the server. The server receives the request and analyzes its contents.

[1587] Obtaining user information and weather information

[1588] The server retrieves profile data such as past fashion history and preferences from a database based on the user ID, analyzes the user's current emotional state using an emotion engine, and calls an external weather API to retrieve current weather information.

[1589] Coordination generation and suggestions

[1590] The server uses this information to generate the optimal outfit for the user. For example, if the user is in a positive emotional state, it will suggest brightly colored clothes, and if they are feeling down, it will suggest more subdued clothing. The generated outfit is then sent to a chat application where the user can view it.

[1591] Examples:

[1592] When a user requests, "Tell me what clothes to wear to suit today's weather," the server responds, "Today is sunny and a little hot. Furthermore, taking into account that your current emotional state is positive, I would like to suggest a light summer outfit (e.g., a light-colored shirt and shorts)."

[1593] In this way, the system of the present invention can take into account the user's emotional state and provide optimal suggestions to the user, thereby enabling the user to efficiently handle daily tasks and improve their quality of life.

[1594] The processing flow will be explained below.

[1595] Menu suggestion function

[1596] Step 1:

[1597] User submits request:

[1598] A user types into a chat application, "What's for dinner?"

[1599] Step 2:

[1600] The device sends a request:

[1601] The device (smartphone or computer) sends this request to the server.

[1602] Step 3:

[1603] Server receives request:

[1604] The server receives the request from the user and begins parsing it.

[1605] Step 4:

[1606] Server gets user information:

[1607] The server retrieves the user's profile data (past preferences, allergy information, and food availability) from the database.

[1608] Step 5:

[1609] Server calls emotion engine:

[1610] The server invokes the emotion engine to analyze the user's emotional state.

[1611] Step 6:

[1612] Server references recipe database:

[1613] The server searches a recipe database based on the user's profile data and emotional state.

[1614] Step 7:

[1615] Server filters recipes:

[1616] The server selects recipes that match the user's criteria (allergy information, past preferences, ingredient availability) and emotional state, and filters them taking into account seasonality and health factors.

[1617] Step 8:

[1618] Your server will select the perfect menu for you:

[1619] The server determines the best menu from the filtered recipes.

[1620] Step 9:

[1621] The server generates a response:

[1622] The server generates a message containing the menu information.

[1623] Step 10:

[1624] The server sends a response:

[1625] The server sends the generated message back to the chat application.

[1626] Step 11:

[1627] Device receives response:

[1628] The user's terminal receives the response from the server.

[1629] Step 12:

[1630] The terminal displays the message:

[1631] The chat application displays the suggested menu to the user.

[1632] Travel planning and transportation suggestion function

[1633] Step 1:

[1634] User submits request:

[1635] A user types into a chat application, "Plan a trip for next weekend."

[1636] Step 2:

[1637] The device sends a request:

[1638] The device sends a request to the server.

[1639] Step 3:

[1640] Server receives request:

[1641] The server receives the request from the user and begins parsing it.

[1642] Step 4:

[1643] Server gets user information:

[1644] The server retrieves profile data from the database, such as the user's travel history, interests, budget, and available time.

[1645] Step 5:

[1646] Server calls emotion engine:

[1647] The server invokes the emotion engine to analyze the user's emotional state.

[1648] Step 6:

[1649] Server collects trip information:

[1650] The server calls an external travel information API to obtain local information and accommodation information.

[1651] Step 7:

[1652] Server generates itinerary:

[1653] The server generates an optimal travel plan based on the collected information and the user's conditions and emotional state.

[1654] Step 8:

[1655] Server searches for transportation:

[1656] The server searches for the best means of transportation (e.g., train, plane, rental car) based on the user's location and destination.

[1657] Step 9:

[1658] Server selects travel mode:

[1659] The server selects the most suitable mode of transportation based on the user's conditions (budget, travel time) and emotional state.

[1660] Step 10:

[1661] The server generates a response:

[1662] The server generates a message containing the travel plan and transportation information.

[1663] Step 11:

[1664] The server sends a response:

[1665] The server sends the generated message back to the chat application.

[1666] Step 12:

[1667] Device receives response:

[1668] The user's terminal receives the response from the server.

[1669] Step 13:

[1670] The terminal displays the message:

[1671] A chat application displays travel plans and transportation options to the user.

[1672] Clothing coordination suggestion function

[1673] Step 1:

[1674] User submits request:

[1675] The user types "Tell me what to wear today" into the chat application.

[1676] Step 2:

[1677] The device sends a request:

[1678] The device sends a request to the server.

[1679] Step 3:

[1680] Server receives request:

[1681] The server receives the request from the user and begins parsing it.

[1682] Step 4:

[1683] Server gets user information:

[1684] The server retrieves the user's past fashion history and preference profile data from the database.

[1685] Step 5:

[1686] Server calls emotion engine:

[1687] The server invokes the emotion engine to analyze the user's emotional state.

[1688] Step 6:

[1689] Server gets weather information:

[1690] The server calls an external weather API to obtain current weather information.

[1691] Step 7:

[1692] The server consults the coordinate database:

[1693] The server searches for the optimal outfit based on the user's fashion history and preferences, acquired weather information, and emotional state.

[1694] Step 8:

[1695] Server selects coordinates:

[1696] The server selects the best coordinates using filters and a scoring algorithm.

[1697] Step 9:

[1698] The server generates a response:

[1699] The server generates a message containing information about the selected coordinates.

[1700] Step 10:

[1701] The server sends a response:

[1702] The server sends the generated message back to the chat application.

[1703] Step 11:

[1704] Device receives response:

[1705] The user's terminal receives the response from the server.

[1706] Step 12:

[1707] The terminal displays the message:

[1708] The chat application displays suggested outfits to the user.

[1709] In this way, by incorporating an emotion engine, the system of the present invention is designed to provide personalized suggestions based on the user's emotional state, allowing them to handle everyday tasks more efficiently.

[1710] Example 2

[1711] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1712] To streamline users' daily tasks, a system that quickly provides optimal suggestions in response to individual requests is required. Conventional systems were able to provide suggestions based on a user's past preferences and profile data, but they were unable to adequately consider the user's current emotional state. This made it difficult to provide suggestions that matched the user's needs.

[1713] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving a request from a user, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, means including an emotion engine for analyzing the user's emotional state, means for generating optimal proposals in response to the user's request, and means for transmitting the generated proposals to the user. This enables proposals that take the user's emotional state into consideration.

[1714] "User" means an individual or corporation that uses the system.

[1715] A "request" is a specific request or inquiry sent from a user to a system.

[1716] A "terminal" is a device that a user uses to access and operate the system, and includes smartphones, computers, tablets, etc.

[1717] A "server" is a computer system that receives and processes requests from users.

[1718] "Profile data" refers to data that includes personal information such as a user's past usage history, preferences, allergy information, and food ingredient availability.

[1719] A "database" is a collection of stored data that is referenced to generate recommendations.

[1720] An "emotion engine" is a technology for analyzing a user's current emotional state, analyzing the user's emotions and generating data based on the analysis.

[1721] A "suggestion" is the optimal answer or advice that the system generates in response to a user's request.

[1722] The "menu suggestion means" is a function that suggests the most suitable meal menu based on the user's meal requests.

[1723] The "travel planning and transportation suggestion means" is a function that suggests optimal travel plans and transportation means based on the user's travel requests.

[1724] The "fashion coordination suggestion means" is a function that suggests the most suitable fashion coordination based on the user's clothing requests.

[1725] A "natural language processing (NLP) engine" is a technology for analyzing and understanding user requests.

[1726] A "generative AI model" is an artificial intelligence technique that generates specific outputs based on user requests and other data.

[1727] This invention is a system that streamlines users' daily tasks and improves the accuracy and effectiveness of suggestions by incorporating an emotion engine. This system consists of three main components: a server, a terminal, and a user.

[1728] System configuration and hardware / software integration

[1729] server

[1730] The server plays a central role in receiving requests from users, analyzing them, and generating optimal suggestions. The following specific software is used:

[1731] NLP engine: Uses spaCy and NLTK to analyze the content of requests using natural language processing.

[1732] Emotion Engine: Analyzes the user's emotional state using Affectiva and Microsoft Emotion API.

[1733] Databases: Relational databases such as MySQL and PostgreSQL, and NoSQL databases such as MongoDB and Elasticsearch are used to manage data such as user profile data, recipe data, and travel plans.

[1734] Terminal

[1735] A terminal is a device that allows a user to access and operate a system. Typically, this is a smartphone or computer. A terminal has the following functions:

[1736] Chat application: Sends user requests to a server, receives responses from the server and displays them to the user.

[1737] User

[1738] A user is an individual or legal entity that uses a terminal to send requests to the system and receive offers from the server.

[1739] System Functions and Prompt Sentence Examples

[1740] Menu suggestion function

[1741] Sending and Receiving Requests

[1742] A user sends a request using a chat application saying, "Tell me what's for dinner." The device then sends this request to the server.

[1743] Example: A user types, "What's for dinner tonight?"

[1744] Acquisition and analysis of user information

[1745] The server retrieves profile data (past preferences, allergy information, food stock status, etc.) from a database based on the user ID and analyzes the emotional state using an emotion engine.

[1746] Example: The server responds, "Considering your past preferences and the current season, and your emotional state of fatigue, I would like to suggest a quick chicken and vegetable stir fry."

[1747] Travel planning and transportation suggestions

[1748] Sending and Receiving Requests

[1749] A user sends a request to a chat application saying, "Plan a trip for next weekend." The device sends this request to a server.

[1750] Example: A user types, "I want to go on a trip somewhere next weekend."

[1751] Obtaining user information and travel conditions

[1752] Based on the user ID, the server obtains data such as past travel history, interests, budget, and time, and analyzes the user's emotional state using an emotion engine.

[1753] Example: The server responds, "Since your past travel history, current interests, and emotional state indicate a desire for relaxation, we suggest a trip to a hot spring resort. Trains would be a convenient means of transportation."

[1754] Clothing coordination suggestion function

[1755] Sending and Receiving Requests

[1756] The user sends a request to the chat application saying, "Tell me what outfit I'm wearing today." The device then sends this request to the server.

[1757] Example: A user types, "What should I wear for today's weather?"

[1758] Obtaining user information and weather information

[1759] The server obtains fashion history and preference data based on the user ID, analyzes the user's emotional state using an emotion engine, and obtains current weather information using a weather API.

[1760] Example: The server responds, "Today is sunny and slightly hot, and your emotional state is positive, so I suggest a light summer outfit (e.g., a light-colored shirt and shorts)."

[1761] Prompt Sentence Examples

[1762] Meal suggestion feature: "Generate prompts to suggest dinner menus for tired users."

[1763] Trip Planning & Transportation Suggestion: "Generate prompts to suggest trip plans for users looking to relax."

[1764] Clothing Coordination Suggestion: "Generate prompts that suggest fashion coordination for users who are in a positive emotional state."

[1765] In this way, the system of the present invention can take into account the user's emotional state to provide more personalized suggestions and efficiently handle everyday tasks.

[1766] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1767] Menu suggestion function

[1768] Processing Steps

[1769] Step 1:

[1770] Terminal: A user types a request into a chat application: "What's for dinner?"

[1771] Input: User request

[1772] Output: Sends an API request to the server

[1773] What it does: Use your smartphone or computer to access a chat app, type in your request, and press send.

[1774] Step 2:

[1775] Server: Receives requests sent from the device and analyzes them using the NLP engine.

[1776] Input: API request

[1777] Output: Parsed request data

[1778] What it does: The server uses a natural language processing engine (e.g., spaCy or NLTK) to analyze the user's request and extract meaning.

[1779] Step 3:

[1780] Server: Retrieves profile data from the database based on the user ID.

[1781] Input: User ID

[1782] Output: User profile data (past preferences, allergy information, current ingredient availability)

[1783] Specific operation: The server issues a query to a database (e.g., MySQL or PostgreSQL) to retrieve profile data.

[1784] Step 4:

[1785] Server: Analyzes the user's current emotional state using the emotion engine.

[1786] Input: User profile data

[1787] Output: Current emotional state data

[1788] What it does: The server inputs profile data into an emotion engine (e.g., Affectiva or Microsoft Emotion API) to analyze the emotional state.

[1789] Step 5:

[1790] Server: Searches the recipe database for candidate recipes that match the criteria and generates the optimal menu.

[1791] Input: Emotional state data, profile data, database query

[1792] Output: Optimal meal plan

[1793] Specific operation: The server issues a query to a recipe database (e.g., MongoDB or Elasticsearch), searches for recipes that meet the criteria, and generates the optimal menu.

[1794] Step 6:

[1795] Server: Formats the generated menu information into a response message and sends it to the device.

[1796] Input: Optimal meal plan

[1797] Output: The formatted response message

[1798] Specific operation: The server formats the generated menu data in rich text or JSON format and sends it to the terminal.

[1799] Step 7:

[1800] Terminal: Messages received from the server are displayed within the chat application for the user to review.

[1801] Input: Response message from the server

[1802] Output: Menu display in a chat application

[1803] Specific behavior: The device receives the response message and displays it in the chat application UI.

[1804] Travel planning and transportation suggestions

[1805] Processing Steps

[1806] Step 1:

[1807] Terminal: A user types a request into a chat application: "Plan a trip for next weekend."

[1808] Input: User request

[1809] Output: Sends an API request to the server

[1810] What it does: Access a chat app on your smartphone or computer, type in your request, and press send.

[1811] Step 2:

[1812] Server: Receives requests sent from the device and analyzes them using the NLP engine.

[1813] Input: API request

[1814] Output: Parsed request data

[1815] What happens: The server uses a natural language processing engine to analyze the request and extract meaning.

[1816] Step 3:

[1817] Server: Retrieves profile data from the database based on the user ID.

[1818] Input: User ID

[1819] Output: Past travel history, interests, budget, time data

[1820] What happens next: The server queries the database to retrieve profile data.

[1821] Step 4:

[1822] Server: Analyzes the user's current emotional state using the emotion engine.

[1823] Input: Profile data

[1824] Output: Current emotional state data

[1825] Specific operation: The server inputs profile data into the emotion engine and analyzes the emotional state.

[1826] Step 5:

[1827] Server: Generates the optimal travel plan based on the acquired information.

[1828] Input: Emotional state data, profile data, database query

[1829] Output: Optimal travel plan

[1830] Specific operation: The server executes the travel plan generation algorithm and generates a plan that meets the conditions.

[1831] Step 6:

[1832] Server: Formats the itinerary and transportation methods into a response message and sends it to the device.

[1833] Input: Best Travel Plan

[1834] Output: The formatted response message

[1835] Specific operation: The server formats the generated plan data and sends it to the device.

[1836] Step 7:

[1837] On the device: The proposed itinerary is displayed in the chat app for the user to review.

[1838] Input: Response message from the server

[1839] Output: Plan display in a chat application

[1840] Specific behavior: The device receives the response message and displays it in the chat app's UI.

[1841] Clothing coordination suggestion function

[1842] Processing Steps

[1843] Step 1:

[1844] Terminal: A user types a request into a chat application, such as "Tell me what outfit to wear today."

[1845] Input: User request

[1846] Output: Sends an API request to the server

[1847] What it does: Access a chat app on your smartphone or computer, type in your request, and press send.

[1848] Step 2:

[1849] Server: Receives requests sent from the device and analyzes them using the NLP engine.

[1850] Input: API request

[1851] Output: Parsed request data

[1852] What happens: The server uses a natural language processing engine to analyze the content of the request.

[1853] Step 3:

[1854] Server: Retrieves profile data from the database based on the user ID.

[1855] Input: User ID

[1856] Output: Past fashion history, preference data

[1857] What happens next: The server queries the database to retrieve profile data.

[1858] Step 4:

[1859] Server: Analyzes the user's current emotional state using the emotion engine.

[1860] Input: Profile data

[1861] Output: Current emotional state data

[1862] Specific operation: The server inputs profile data into the emotion engine and analyzes the emotional state.

[1863] Step 5:

[1864] Server: Calls the weather API to get current weather information.

[1865] Input: Weather API request

[1866] Output: Current weather information

[1867] What happens: The server sends a request to an external weather API to get current weather information.

[1868] Step 6:

[1869] Server: Generates optimal coordination based on the acquired information.

[1870] Input: Emotional state data, fashion history data, weather information

[1871] Output: Optimal outfit ideas

[1872] Specific operation: The server executes the coordination generation algorithm and generates fashion suggestions that meet the conditions.

[1873] Step 7:

[1874] Server: Formats the proposed coordinates into a response message and sends it to the device.

[1875] Input: Best outfit ideas

[1876] Output: The formatted response message

[1877] Specific operation: The server formats the generated coordinate data and sends it to the terminal.

[1878] Step 8:

[1879] Device: The suggested outfits are displayed in the chat app so that the user can check them.

[1880] Input: Response message from the server

[1881] Output: Coordinate display in a chat application

[1882] Specific behavior: The device receives the response message and displays it in the chat app's UI.

[1883] Example prompts for generative AI models

[1884] Meal suggestion feature: "Generate prompts to suggest dinner menus for tired users."

[1885] Trip Planning & Transportation Suggestion: "Generate prompts to suggest trip plans for users looking to relax."

[1886] Clothing Coordination Suggestion: "Generate prompts that suggest fashion coordination for users who are in a positive emotional state."

[1887] (Application example 2)

[1888] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1889] Conventional recommendation systems make suggestions based on user requests and past data, but because they do not take the user's emotional state into account, they are unable to make suggestions that are optimal for the user's current mood or situation, limiting the improvement of user satisfaction.In addition, it has been difficult to make personalized suggestions based on real-time emotion recognition in physical stores, etc.

[1890] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1891] In this invention, the server includes means for receiving requests from a user, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, means including an emotion engine for analyzing the user's emotional state, means for generating optimal proposals in accordance with the user's requests and emotional state, and means for transmitting the generated proposals to the user, thereby enabling optimal proposals that take into account the user's current emotional state.

[1892] The "means for receiving a request from a user" refers to a device or software that allows a user to input a request for information or a service they desire into the system and receives the request electronically.

[1893] "Means for obtaining a user's past usage history and profile data" refers to a device or software that extracts records of a user's past use of the system, personal information, and preference data.

[1894] The "means for referencing a database for generating proposal content" refers to a device or software that accesses and refers to a database that stores information required for generating various proposal content.

[1895] The "means including an emotion engine for analyzing the user's emotional state" refers to hardware and software for recognizing and analyzing emotions from the user's facial expressions, tone of voice, etc.

[1896] The "means for generating optimal suggestions according to the user's requests and emotional state" refers to a device or software for generating the most appropriate suggestions based on the user's request content and analyzed emotional state.

[1897] The "means for transmitting the generated proposal to the user" refers to a device or software that electronically transmits the content of the generated proposal to the user's terminal.

[1898] The system of the present invention makes suggestions to improve the efficiency of a user's daily tasks and includes an emotion engine that analyzes the user's emotional state. This system is primarily composed of a server, a terminal, and a user. Specific embodiments for realizing this system are described below.

[1899] The server includes means for receiving requests from users, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, an emotion engine for analyzing the user's emotional state, means for generating optimal proposals according to the user's requests and emotional state, and means for transmitting the generated proposals to the user.

[1900] Specifically, when a user accesses product information using a smartphone or smart glasses, the system analyzes the user's emotional state at that time and proposes the most suitable products and sets. For example, if the user is tired, the system will propose relaxing items (bath additives, aromatherapy, relaxation goods, etc.), and if the user is energetic, it will propose products suitable for new challenges or activities (sports goods, games, activity kits, etc.).

[1901] This system mainly uses the following hardware and software:

[1902] The camera on a smartphone or smart glasses is used to capture the user's facial expressions.

[1903] Using OpenCV (image processing library), the user's facial expression is processed from the captured image.

[1904] Use emotion_recognition (emotion recognition library) to analyze the user's current emotional state from their facial expressions.

[1905] Use requests (an HTTP request library) to communicate with the user database and retrieve user profile data.

[1906] Use the recommendation_engine to generate optimal recommendations based on emotional state and profile data.

[1907] As a concrete example, a user enters a physical store through smart glasses. It seems that the user has been very busy that weekend and is feeling stressed. The camera analyzes the user's facial expressions to determine their emotions and determines that they are "tired." As a result, the smart glasses' display displays a message saying, "We'll suggest products that will help you relax today!" The user can then choose a product based on the suggestions.

[1908] Example prompt sentence:

[1909] "Relax, we've found the perfect product for you."

[1910] "Here are some recommended activities for you who are feeling energetic!"

[1911] In this way, by providing individually optimized suggestions that take into account the user's emotional state, user satisfaction can be improved.

[1912] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1913] Step 1:

[1914] A user inputs a request using a smartphone or smart glasses. The user sends a question to the system, such as "What is today's recommended product?" The input is the user's request, and the output is the request data.

[1915] Step 2:

[1916] The device sends the user's request to the server. After receiving the user's request, the smartphone or smart glasses transmit the request to the server via the Internet. The input is the user's request data, and the output is a confirmation of the request transmission to the server.

[1917] Step 3:

[1918] The server receives and analyzes the request. The received request data is analyzed by a program on the server to identify the type of information or service the user is seeking. The input is the received request data, and the output is the analysis result.

[1919] Step 4:

[1920] The server retrieves the user's profile data. After analyzing the request, the server retrieves past usage history and profile data from the database based on the user ID. The input is the user ID and the output is the profile data.

[1921] Step 5:

[1922] The server uses an emotion engine to analyze the user's emotional state. The emotion engine analyzes the user's emotional state based on the user's facial expressions captured in real time by a camera installed in a smartphone or smart glasses. The input is the user's facial expression data, and the output is the analyzed emotional state.

[1923] Step 6:

[1924] The server generates optimal suggestions based on the acquired profile data and analyzed emotional state. The server refers to a recipe database and recommends products and services that match the user's profile data and emotional state. The input is the profile data and emotional state, and the output is the optimal suggestions.

[1925] Step 7:

[1926] The server sends the generated proposal to the terminal. Once the optimal proposal is generated, the server sends it to the user's terminal. The input is the optimal proposal, and the output is a confirmation of the proposal transmission to the terminal.

[1927] Step 8:

[1928] The device displays the proposed content to the user. The smartphone or smart glasses displays the received proposed content on the user's display. The input is the proposed content sent from the server, and the output is the proposed content displayed on the user's display.

[1929] Specifically, the user speaks to the smart glasses, asking, "What is today's recommended product?" The server analyzes the response and displays, "Considering that you are tired, we recommend an aroma candle to help you relax today."

[1930] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1931] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1932] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1933] [Fourth embodiment]

[1934] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1935] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1936] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1937] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1938] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1939] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1940] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1941] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1942] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1943] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1944] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1945] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1946] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1947] The present invention is a system that streamlines users' daily tasks by receiving requests from users and providing optimal suggestions based on the users' past usage history and profile data. This system is mainly composed of a server, a terminal, and a user.

[1948] Menu suggestion function

[1949] Sending and Receiving Requests

[1950] A user sends a request using a chat application, such as "Tell me what's for dinner." The device (smartphone or computer) sends this request to a server. The server analyzes the received request and starts the process of proposing the menu desired by the user.

[1951] Acquisition and analysis of user information

[1952] The server retrieves profile data from the database based on the user ID. The profile data includes the user's past preferences, allergy information, current food stock status, etc. This allows the server to understand the user's preferences and restrictions.

[1953] Menu generation and suggestions

[1954] The server searches and selects recipes that can be suggested to the user from the recipe database, taking into account the current season and the availability of ingredients. Once the optimal menu is determined, the server sends it back to the chat application. The user can then view the suggested menu through the chat application on their device.

[1955] Examples:

[1956] When a user requests, "What would you like for dinner tonight?" the server responds, "Taking into account your past preferences and the current season, I would like to suggest grilled salmon and a salad."

[1957] Travel planning and transportation suggestions

[1958] Sending and Receiving Requests

[1959] A user sends a request to a chat application saying, "Plan a trip for next weekend." The device sends this request to a server, which receives the request and analyzes its contents.

[1960] Obtaining user information and travel conditions

[1961] Based on the user ID, the server retrieves profile data such as past travel history, interests, available budget and time from the database, and also calls external travel information APIs to collect local information and accommodation information.

[1962] Travel plan and transportation generation

[1963] The server generates an optimal travel plan based on this information and suggests the most suitable means of transportation for the user (e.g., train, plane, rental car). The generated plan and means of transportation are sent to the chat application, where the user can confirm them.

[1964] Examples:

[1965] When a user requests, "I'd like to go on a trip somewhere next weekend," the server responds, "Taking into account your past travel history and current interests, I would like to suggest a day trip to Kyoto. The Shinkansen would be a convenient means of transportation."

[1966] Clothing coordination suggestion function

[1967] Sending and Receiving Requests

[1968] The user sends a request to the chat application saying, "Tell me what outfit I'm wearing today." The device sends this request to the server. The server receives the request and analyzes its contents.

[1969] Obtaining user information and weather information

[1970] The server retrieves profile data such as past fashion history and preferences from a database based on the user ID, and then calls an external weather API to retrieve current weather information.

[1971] Coordination generation and suggestions

[1972] The server generates the optimal coordinate based on this information, and the proposed coordinate is sent to the chat application, where the user can view it on their device.

[1973] Examples:

[1974] When a user requests, "Tell me what clothes to wear to suit today's weather," the server responds, "Today is sunny and a little hot, so I'd like to suggest a light summer outfit (e.g., a shirt and shorts)."

[1975] The system of the present invention allows users to efficiently handle small tasks in daily life and save time, which not only improves the quality of life but also enhances the user's experience.

[1976] The processing flow will be explained below.

[1977] Menu suggestion function

[1978] Step 1:

[1979] User submits request:

[1980] A user types into a chat application, "What's for dinner?"

[1981] Step 2:

[1982] The device sends a request:

[1983] The device (smartphone or computer) sends this request to the server.

[1984] Step 3:

[1985] Server receives request:

[1986] The server receives the request from the user and begins parsing it.

[1987] Step 4:

[1988] Server gets user information:

[1989] The server retrieves profile data from the database, including the user's past preferences, allergy information, and food availability.

[1990] Step 5:

[1991] Server references recipe database:

[1992] The server searches the recipe database based on the acquired user information.

[1993] Step 6:

[1994] Server filters recipes:

[1995] The server selects recipes that match the user's criteria (allergy information, past preferences, and ingredient availability), and filters them taking into account seasonality and health factors.

[1996] Step 7:

[1997] Your server will select the perfect menu for you:

[1998] The server determines the best menu from the filtered recipes.

[1999] Step 8:

[2000] The server generates a response:

[2001] The server generates a message containing the menu information.

[2002] Step 9:

[2003] The server sends a response:

[2004] The server sends the generated message back to the chat application.

[2005] Step 10:

[2006] Device receives response:

[2007] The user's terminal receives the response from the server.

[2008] Step 11:

[2009] The terminal displays the message:

[2010] The chat application displays the suggested menu to the user.

[2011] Travel planning and transportation suggestion function

[2012] Step 1:

[2013] User submits request:

[2014] A user types into a chat application, "Plan a trip for next weekend."

[2015] Step 2:

[2016] The device sends a request:

[2017] The device sends a request to the server.

[2018] Step 3:

[2019] Server receives request:

[2020] The server receives the request from the user and begins parsing it.

[2021] Step 4:

[2022] Server gets user information:

[2023] The server retrieves profile data from the database, such as the user's travel history, interests, budget, and available time.

[2024] Step 5:

[2025] Server collects trip information:

[2026] The server calls an external travel information API to obtain local information and accommodation information.

[2027] Step 6:

[2028] Server generates itinerary:

[2029] The server generates an optimal travel plan based on the collected information and the user's requirements.

[2030] Step 7:

[2031] Server searches for transportation:

[2032] The server searches for the best means of transportation (e.g., train, plane, rental car) based on the user's location and destination.

[2033] Step 8:

[2034] Server selects travel mode:

[2035] The server selects the most suitable means of transportation based on the user's conditions (budget, travel time).

[2036] Step 9:

[2037] The server generates a response:

[2038] The server generates a message containing the travel plan and transportation information.

[2039] Step 10:

[2040] The server sends a response:

[2041] The server sends the generated message back to the chat application.

[2042] Step 11:

[2043] Device receives response:

[2044] The user's terminal receives the response from the server.

[2045] Step 12:

[2046] The terminal displays the message:

[2047] A chat application displays travel plans and transportation options to the user.

[2048] Clothing coordination suggestion function

[2049] Step 1:

[2050] User submits request:

[2051] The user types "Tell me what to wear today" into the chat application.

[2052] Step 2:

[2053] The device sends a request:

[2054] The device sends a request to the server.

[2055] Step 3:

[2056] Server receives request:

[2057] The server receives the request from the user and begins parsing it.

[2058] Step 4:

[2059] Server gets user information:

[2060] The server retrieves the user's past fashion history and preference profile data from the database.

[2061] Step 5:

[2062] Server gets weather information:

[2063] The server calls an external weather API to obtain current weather information.

[2064] Step 6:

[2065] The server consults the coordinate database:

[2066] The server searches for the optimal outfit based on the user's fashion history, preferences, and acquired weather information.

[2067] Step 7:

[2068] Server selects coordinates:

[2069] The server selects the best coordinates using filters and a scoring algorithm.

[2070] Step 8:

[2071] The server generates a response:

[2072] The server generates a message containing information about the selected coordinates.

[2073] Step 9:

[2074] The server sends a response:

[2075] The server sends the generated message back to the chat application.

[2076] Step 10:

[2077] Device receives response:

[2078] The user's terminal receives the response from the server.

[2079] Step 11:

[2080] The terminal displays the message:

[2081] The chat application displays suggested outfits to the user.

[2082] Example 1

[2083] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2084] In users' daily lives, tasks such as deciding on meal plans, planning trips, and coordinating fashion are often complicated and time-consuming. It is also difficult to consider the conditions and constraints required for these tasks. It is particularly challenging to obtain optimal suggestions that reflect individual users' preferences and constraints. This leads to inefficient time management and a decline in quality of life. There is a need for technology that can solve these issues and improve the quality of users' lives while making them more efficient.

[2085] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2086] In this invention, the server includes means for receiving requests from users, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, means for generating optimal proposals in response to the user's requests, means for transmitting the generated proposals to the user, means for analyzing the request content and determining which function is being requested, and means for acquiring information from an external API based on the user information. This makes it possible to quickly provide optimal proposals tailored to the preferences and needs of individual users in response to various requests in their daily lives (such as menu suggestions, travel plans, and fashion coordination).

[2087] A "user" is an entity that uses the system to send requests and receive optimal suggestions.

[2088] A "server" is a device or system that receives requests from users, analyzes data and generates suggestions, and transmits the results to the users.

[2089] A "terminal" is a device (e.g., a smartphone or computer) through which a user inputs requests and receives suggestions from a server.

[2090] A "request" is a request that a user sends to a server via a chat application or the like.

[2091] "Profile data" refers to data that includes information necessary for generating suggestions, such as the user's past usage history, preferences, and allergy information.

[2092] A "database" is a storage device that stores information necessary for generating proposals, such as profile data and recipe information.

[2093] A "suggestion" is a specific recommendation that is generated by the server based on the user's request and provided to the user.

[2094] "External API" means an external application programming interface that the server uses to obtain additional data based on user information.

[2095] In order to implement the present invention, it is necessary to build a system that generates optimal proposals based on user requests and provides them to the user. This system is mainly composed of a server, a terminal, and a user.

[2096] Hardware and Software

[2097] Hardware

[2098] 1. Server: Consists of computers with high-performance data processing capabilities, and can be a cloud server or an on-premise server.

[2099] 2. Terminal: A user device such as a smartphone, tablet, or personal computer (PC).

[2100] 3. Network: The network infrastructure that allows the server and devices to communicate through an Internet connection.

[2101] software

[2102] 1. Chat application: An application that users use to send requests to a server. It runs on smartphones and PCs.

[2103] 2. Generative AI model: An artificial intelligence model for generating recommendations (e.g., a natural language processing model or a recommender system).

[2104] 3. Database Management System (DBMS): Software that manages databases that store profile data, recipe information, travel information, fashion items, etc.

[2105] 4. External API: An external application programming interface used to obtain weather information, travel information, etc.

[2106] System Operation

[2107] 1. User submits request

[2108] A user uses a chat application to submit a request, for example, "What's for dinner?"

[2109] 2. The device sends the request to the server

[2110] The device receives the user's request, converts it into JSON format, and sends it to the server.

[2111] 3. Request Analysis by the Server

[2112] The server analyzes the received request and determines which function is being requested. For example, it selects the "menu suggestion function."

[2113] 4. Server obtains user information

[2114] Based on the user ID, the server retrieves profile data (past usage history, allergy information, inventory ingredients, etc.) from the database.

[2115] 5. Data analysis and proposal generation by the server

[2116] The server uses the generative AI model to generate optimal suggestions based on the user information and the request, and calls external APIs to obtain auxiliary information as needed.

[2117] 6. Sending results from the server to the device

[2118] The server converts the generated proposal into JSON format and sends it to the device.

[2119] 7. User confirmation of results

[2120] The user checks the proposed content in the chat application.

[2121] Specific examples

[2122] Specific examples of menu suggestions

[2123] The user types "Please suggest a dinner menu" into a chat application and sends it. The server analyzes this request and selects a "menu suggestion function." The server retrieves the user's past preferences, allergy information, and available ingredients from a database, and then uses a generative AI model to generate an optimal menu. For example, it might suggest "grilled salmon and salad." The generated suggestion is sent to the device, and the user can view the results in the chat application.

[2124] Prompt Sentence Examples

[2125] Please suggest a dinner menu

[2126] I want to go on a trip somewhere next weekend

[2127] What clothes should I wear for today's weather?

[2128] This system can provide prompt and optimal suggestions for various requests in users' daily lives, which will improve the efficiency of users' time management and enhance their quality of life.

[2129] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2130] Step 1:

[2131] The user submits a request.

[2132] Input: A user types "What's for dinner" in a chat application.

[2133] Output: A request sent by the chat application.

[2134] Specific operation: The user opens a chat application on their smartphone or computer, enters a request, and presses the send button.

[2135] Step 2:

[2136] The device sends a request to the server.

[2137] Input: A user request via a chat application.

[2138] Output: The request is sent via an API call to the server.

[2139] Specific operation: The device converts the request content into JSON format and sends a POST request to the server's API endpoint.

[2140] Step 3:

[2141] The server parses the request.

[2142] Input: The request sent to the server in JSON format.

[2143] Output: The appropriate function is selected as a result of analyzing the request content.

[2144] Specific operation: The server analyzes the request message and determines the appropriate processing route based on its content. For example, the "Menu suggestion function" is selected for the request "Tell me what's for dinner."

[2145] Step 4:

[2146] The server retrieves the user information.

[2147] Input: User ID and request details.

[2148] Output: Profile data obtained from the database (past usage history, allergy information, inventory ingredients, etc.).

[2149] What happens: The server queries the database to retrieve data related to the user.

[2150] Step 5:

[2151] The server analyzes the data and generates recommendations.

[2152] Input: Profile data, recipe database, information from external APIs (e.g. weather information).

[2153] Output: The optimal recommendation to provide to the user.

[2154] What it does: The server searches its database to select recipes based on the season and current inventory. It calls external APIs to obtain additional information as needed. It uses generative AI models to generate optimal recommendations.

[2155] Step 6:

[2156] The server sends the generated proposal to the terminal.

[2157] Input: The server-generated proposal.

[2158] Output: Suggestion information sent to the device in JSON format.

[2159] What happens: The server converts the proposal into JSON format and sends a POST request to the device's API endpoint.

[2160] Step 7:

[2161] The user checks the results.

[2162] Input: The proposal information sent back to the device.

[2163] Output: The suggestion that will be displayed in the chat application.

[2164] What happens: A user opens a chat application and sees a new message with suggested information, such as "Suggesting grilled salmon with salad."

[2165] (Application example 1)

[2166] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2167] In users' everyday shopping experiences, selecting the most suitable product from a vast amount of product information is a time-consuming task. In particular, there is a lack of systems that can make individually customized suggestions based on a user's purchasing history and profile data. This can lead to users missing potentially suitable products, so a system that provides an efficient and satisfying shopping experience is needed.

[2168] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2169] In this invention, the server includes means for receiving requests from users, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposals, means for proposing optimal products based on the user's purchase history and interests, and means for transmitting the generated proposals to the user. This allows users to receive optimal product proposals that are individually customized, enabling an efficient and satisfying shopping experience.

[2170] "Users" refers to people who use this system.

[2171] A "request" refers to the action or content of a user requesting some kind of service or information from a system.

[2172] "Usage history" refers to a record of a user's past activities when using the system.

[2173] "Profile Data" refers to a user's personal information, interests, preferences, and past behavioral data.

[2174] "Database" refers to a system for storing and managing user profile data and information for generating proposals.

[2175] "Suggestions" refer to appropriate product recommendations or advice generated based on a user's request.

[2176] "Purchase history" refers to a record of products purchased by a user in the past.

[2177] "Interests" refer to areas or things in which a user is particularly interested.

[2178] "Products" refers to various consumer goods offered to users.

[2179] "Means for suggesting optimal products" refers to the process and function of selecting and recommending the most suitable products for each individual user based on the user's profile data and purchase history.

[2180] "System" refers to an integrated technical configuration and device that includes multiple means and provides services and information to users.

[2181] "Send" refers to transferring data from the server to the user's terminal.

[2182] The system for implementing this invention receives requests from users and proposes optimal products by taking into consideration the user's profile data and purchase history. This system is mainly composed of a server, a terminal, and a user.

[2183] First, a user sends a request using a dedicated application on their device. For example, a request such as "Recommend me a coat." This request is sent from the device to the server. The device used can be a general device such as a smartphone or computer.

[2184] When the server receives the request, it retrieves profile data and purchase history from a database based on the user ID. Specific databases used include MySQL and MongoDB. Additionally, software such as the Python requests module is used to process HTTP requests to retrieve data on the server side.

[2185] The server analyzes the acquired profile data and purchase history to select appropriate products based on the user's interests and past purchasing habits. This process can be performed using a generative AI model, such as a machine learning algorithm or deep learning model. This generative AI model generates prompts that suggest the best products when the user enters specific keywords or phrases.

[2186] Once the suggestions are generated, the server sends them back to the device. The user can then review the suggestions and select specific products in the application. For example, when a user requests "Recommend me a coat," the server generates a message such as, "Taking into consideration your past purchases and preferences, we suggest the following coats," and provides a list of appropriate products.

[2187] An example of a prompt might be, "Based on user ID 12345, please suggest the best coat for me, taking into account my past purchase history and profile data." This allows users to enjoy a personalized, high-quality shopping experience.

[2188] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2189] Step 1:

[2190] The user inputs a request into a dedicated application on the device and sends it. The input data is the information the user is looking for (e.g., "Recommend a coat"). This request is sent from the device to the server. The specific operation of the device is to receive the user input and send it to the server as an HTTP POST request.

[2191] Step 2:

[2192] The server receives a request from a user. The input data is the user request, and the output data is the user ID and the request content. The specific operation of the server is to receive the HTTP request, analyze the request parameters, and extract the user ID and the request content.

[2193] Step 3:

[2194] The server retrieves the user's profile data and purchase history from the database. The input data is the user ID, and the output data is the profile data and purchase history. The specific operation of the server is to query the database for the user ID and retrieve the corresponding profile data and purchase history.

[2195] Step 4:

[2196] The server analyzes the profile data and purchase history and generates optimal product suggestions using a generative AI model. The input data is the profile data and purchase history, and the output data is the optimal product suggestions. The specific operation of the server is to analyze the acquired data and input a prompt statement to the generative AI model. As a specific example, the prompt statement "Please suggest the best coat based on user ID 12345, taking into consideration past purchase history and profile data" is input to the generative AI model.

[2197] Step 5:

[2198] The server then sends the generated proposal back to the terminal. The input data is the optimal product proposal, and the output data is the proposal to the user terminal. The specific operation of the server is to send the proposal to the terminal as an HTTP response and notify the user.

[2199] Step 6:

[2200] The terminal displays the suggestions to the user. The input data is the suggestions received from the server, and the output data is what is displayed to the user. The specific operation of the terminal is to analyze the received suggestions and display them on the user interface. As a concrete example, a list of products is displayed along with a message such as "Here are some recommended coats."

[2201] Through the above steps, the user can receive optimal product proposals that are individually customized.

[2202] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2203] This invention is a system that streamlines users' daily tasks and improves the accuracy and effectiveness of suggestions by incorporating an emotion engine. This system is mainly composed of a server, a terminal, and a user, and includes an emotion engine for recognizing the user's emotions.

[2204] Menu suggestion function

[2205] Sending and Receiving Requests

[2206] A user sends a request using a chat application, such as "Tell me what's for dinner." The device (smartphone or computer) sends this request to a server. The server analyzes the received request and starts the process of proposing the menu desired by the user.

[2207] Acquisition and analysis of user information

[2208] The server retrieves profile data from the database based on the user ID. The profile data includes the user's past preferences, allergy information, current food availability, etc. The server then uses an emotion engine to analyze the user's current emotional state.

[2209] Menu generation and suggestions

[2210] The server generates suggestions from a recipe database, taking into account not only the current season and ingredient availability, but also the user's emotional state. For example, if the user is tired, it will suggest easy-to-make dishes, and if they are energetic, it will suggest new and challenging recipes. The generated menu is sent to a chat application, where the user can check the suggested menu.

[2211] Examples:

[2212] When a user requests, "What would you like for dinner tonight?" the server responds, "Taking into account your past preferences and the current season, and because we recognize your current emotional state as tired, we'd like to suggest a quick and easy chicken and vegetable stir-fry."

[2213] Travel planning and transportation suggestions

[2214] Sending and Receiving Requests

[2215] A user sends a request to a chat application saying, "Plan a trip for next weekend." The device sends this request to a server, which receives the request and analyzes its contents.

[2216] Obtaining user information and travel conditions

[2217] Based on the user ID, the server retrieves profile data from the database, such as past travel history, interests, available budget and time, and analyzes the user's current emotional state using an emotion engine.

[2218] Travel plan and transportation generation

[2219] The server uses this information to generate an optimal travel plan. For example, if the user is feeling stressed, it might suggest a relaxing resort, or if the user is in an adventure-seeking emotional state, it might suggest a plan including active activities. Once the optimal travel plan and means of transportation are determined, they are sent to the chat application, where the user can confirm the plan.

[2220] Examples:

[2221] If a user requests, "I'd like to go on a trip somewhere next weekend," the server responds, "Based on your past travel history, current interests, and current emotional state, I recognize that you are seeking relaxation, so I will suggest a trip to a hot spring resort. The train would be a convenient means of transportation."

[2222] Clothing coordination suggestion function

[2223] Sending and Receiving Requests

[2224] The user sends a request to the chat application saying, "Tell me what outfit I'm wearing today." The device sends this request to the server. The server receives the request and analyzes its contents.

[2225] Obtaining user information and weather information

[2226] The server retrieves profile data such as past fashion history and preferences from a database based on the user ID, analyzes the user's current emotional state using an emotion engine, and calls an external weather API to retrieve current weather information.

[2227] Coordination generation and suggestions

[2228] The server uses this information to generate the optimal outfit for the user. For example, if the user is in a positive emotional state, it will suggest brightly colored clothes, and if they are feeling down, it will suggest more subdued clothing. The generated outfit is then sent to a chat application where the user can view it.

[2229] Examples:

[2230] When a user requests, "Tell me what clothes to wear to suit today's weather," the server responds, "Today is sunny and a little hot. Furthermore, taking into account that your current emotional state is positive, I would like to suggest a light summer outfit (e.g., a light-colored shirt and shorts)."

[2231] In this way, the system of the present invention can take into account the user's emotional state and provide optimal suggestions to the user, thereby enabling the user to efficiently handle daily tasks and improve their quality of life.

[2232] The processing flow will be explained below.

[2233] Menu suggestion function

[2234] Step 1:

[2235] User submits request:

[2236] A user types into a chat application, "What's for dinner?"

[2237] Step 2:

[2238] The device sends a request:

[2239] The device (smartphone or computer) sends this request to the server.

[2240] Step 3:

[2241] Server receives request:

[2242] The server receives the request from the user and begins parsing it.

[2243] Step 4:

[2244] Server gets user information:

[2245] The server retrieves the user's profile data (past preferences, allergy information, and food availability) from the database.

[2246] Step 5:

[2247] Server calls emotion engine:

[2248] The server invokes the emotion engine to analyze the user's emotional state.

[2249] Step 6:

[2250] Server references recipe database:

[2251] The server searches a recipe database based on the user's profile data and emotional state.

[2252] Step 7:

[2253] Server filters recipes:

[2254] The server selects recipes that match the user's criteria (allergy information, past preferences, ingredient availability) and emotional state, and filters them taking into account seasonality and health factors.

[2255] Step 8:

[2256] Your server will select the perfect menu for you:

[2257] The server determines the best menu from the filtered recipes.

[2258] Step 9:

[2259] The server generates a response:

[2260] The server generates a message containing the menu information.

[2261] Step 10:

[2262] The server sends a response:

[2263] The server sends the generated message back to the chat application.

[2264] Step 11:

[2265] Device receives response:

[2266] The user's terminal receives the response from the server.

[2267] Step 12:

[2268] The terminal displays the message:

[2269] The chat application displays the suggested menu to the user.

[2270] Travel planning and transportation suggestion function

[2271] Step 1:

[2272] User submits request:

[2273] A user types into a chat application, "Plan a trip for next weekend."

[2274] Step 2:

[2275] The device sends a request:

[2276] The device sends a request to the server.

[2277] Step 3:

[2278] Server receives request:

[2279] The server receives the request from the user and begins parsing it.

[2280] Step 4:

[2281] Server gets user information:

[2282] The server retrieves profile data from the database, such as the user's travel history, interests, budget, and available time.

[2283] Step 5:

[2284] Server calls emotion engine:

[2285] The server invokes the emotion engine to analyze the user's emotional state.

[2286] Step 6:

[2287] Server collects trip information:

[2288] The server calls an external travel information API to obtain local information and accommodation information.

[2289] Step 7:

[2290] Server generates itinerary:

[2291] The server generates an optimal travel plan based on the collected information and the user's conditions and emotional state.

[2292] Step 8:

[2293] Server searches for transportation:

[2294] The server searches for the best means of transportation (e.g., train, plane, rental car) based on the user's location and destination.

[2295] Step 9:

[2296] Server selects travel mode:

[2297] The server selects the most suitable mode of transportation based on the user's conditions (budget, travel time) and emotional state.

[2298] Step 10:

[2299] The server generates a response:

[2300] The server generates a message containing the travel plan and transportation information.

[2301] Step 11:

[2302] The server sends a response:

[2303] The server sends the generated message back to the chat application.

[2304] Step 12:

[2305] Device receives response:

[2306] The user's terminal receives the response from the server.

[2307] Step 13:

[2308] The terminal displays the message:

[2309] A chat application displays travel plans and transportation options to the user.

[2310] Clothing coordination suggestion function

[2311] Step 1:

[2312] User submits request:

[2313] The user types "Tell me what to wear today" into the chat application.

[2314] Step 2:

[2315] The device sends a request:

[2316] The device sends a request to the server.

[2317] Step 3:

[2318] Server receives request:

[2319] The server receives the request from the user and begins parsing it.

[2320] Step 4:

[2321] Server gets user information:

[2322] The server retrieves the user's past fashion history and preference profile data from the database.

[2323] Step 5:

[2324] Server calls emotion engine:

[2325] The server invokes the emotion engine to analyze the user's emotional state.

[2326] Step 6:

[2327] Server gets weather information:

[2328] The server calls an external weather API to obtain current weather information.

[2329] Step 7:

[2330] The server consults the coordinate database:

[2331] The server searches for the optimal outfit based on the user's fashion history and preferences, acquired weather information, and emotional state.

[2332] Step 8:

[2333] Server selects coordinates:

[2334] The server selects the best coordinates using filters and a scoring algorithm.

[2335] Step 9:

[2336] The server generates a response:

[2337] The server generates a message containing information about the selected coordinates.

[2338] Step 10:

[2339] The server sends a response:

[2340] The server sends the generated message back to the chat application.

[2341] Step 11:

[2342] Device receives response:

[2343] The user's terminal receives the response from the server.

[2344] Step 12:

[2345] The terminal displays the message:

[2346] The chat application displays suggested outfits to the user.

[2347] In this way, by incorporating an emotion engine, the system of the present invention is designed to provide personalized suggestions based on the user's emotional state, allowing them to handle everyday tasks more efficiently.

[2348] Example 2

[2349] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2350] To streamline users' daily tasks, a system that quickly provides optimal suggestions in response to individual requests is required. Conventional systems were able to provide suggestions based on a user's past preferences and profile data, but they were unable to adequately consider the user's current emotional state. This made it difficult to provide suggestions that matched the user's needs.

[2351] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving a request from a user, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, means including an emotion engine for analyzing the user's emotional state, means for generating optimal proposals in response to the user's request, and means for transmitting the generated proposals to the user. This enables proposals that take the user's emotional state into consideration.

[2352] "User" means an individual or corporation that uses the system.

[2353] A "request" is a specific request or inquiry sent from a user to a system.

[2354] A "terminal" is a device that a user uses to access and operate the system, and includes smartphones, computers, tablets, etc.

[2355] A "server" is a computer system that receives and processes requests from users.

[2356] "Profile data" refers to data that includes personal information such as a user's past usage history, preferences, allergy information, and food ingredient availability.

[2357] A "database" is a collection of stored data that is referenced to generate recommendations.

[2358] An "emotion engine" is a technology for analyzing a user's current emotional state, analyzing the user's emotions and generating data based on the analysis.

[2359] A "suggestion" is the optimal answer or advice that the system generates in response to a user's request.

[2360] The "menu suggestion means" is a function that suggests the most suitable meal menu based on the user's meal requests.

[2361] The "travel planning and transportation suggestion means" is a function that suggests optimal travel plans and transportation means based on the user's travel requests.

[2362] The "fashion coordination suggestion means" is a function that suggests the most suitable fashion coordination based on the user's clothing requests.

[2363] A "natural language processing (NLP) engine" is a technology for analyzing and understanding user requests.

[2364] A "generative AI model" is an artificial intelligence technique that generates specific outputs based on user requests and other data.

[2365] This invention is a system that streamlines users' daily tasks and improves the accuracy and effectiveness of suggestions by incorporating an emotion engine. This system consists of three main components: a server, a terminal, and a user.

[2366] System configuration and hardware / software integration

[2367] server

[2368] The server plays a central role in receiving requests from users, analyzing them, and generating optimal suggestions. The following specific software is used:

[2369] NLP engine: Uses spaCy and NLTK to analyze the content of requests using natural language processing.

[2370] Emotion Engine: Analyzes the user's emotional state using Affectiva and Microsoft Emotion API.

[2371] Databases: Relational databases such as MySQL and PostgreSQL, and NoSQL databases such as MongoDB and Elasticsearch are used to manage data such as user profile data, recipe data, and travel plans.

[2372] Terminal

[2373] A terminal is a device that allows a user to access and operate a system. Typically, this is a smartphone or computer. A terminal has the following functions:

[2374] Chat application: Sends user requests to a server, receives responses from the server and displays them to the user.

[2375] User

[2376] A user is an individual or legal entity that uses a terminal to send requests to the system and receive offers from the server.

[2377] System Functions and Prompt Sentence Examples

[2378] Menu suggestion function

[2379] Sending and Receiving Requests

[2380] A user sends a request using a chat application saying, "Tell me what's for dinner." The device then sends this request to the server.

[2381] Example: A user types, "What's for dinner tonight?"

[2382] Acquisition and analysis of user information

[2383] The server retrieves profile data (past preferences, allergy information, food stock status, etc.) from a database based on the user ID and analyzes the emotional state using an emotion engine.

[2384] Example: The server responds, "Considering your past preferences and the current season, and your emotional state of fatigue, I would like to suggest a quick chicken and vegetable stir fry."

[2385] Travel planning and transportation suggestions

[2386] Sending and Receiving Requests

[2387] A user sends a request to a chat application saying, "Plan a trip for next weekend." The device sends this request to a server.

[2388] Example: A user types, "I want to go on a trip somewhere next weekend."

[2389] Obtaining user information and travel conditions

[2390] Based on the user ID, the server obtains data such as past travel history, interests, budget, and time, and analyzes the user's emotional state using an emotion engine.

[2391] Example: The server responds, "Since your past travel history, current interests, and emotional state indicate a desire for relaxation, we suggest a trip to a hot spring resort. Trains would be a convenient means of transportation."

[2392] Clothing coordination suggestion function

[2393] Sending and Receiving Requests

[2394] The user sends a request to the chat application saying, "Tell me what outfit I'm wearing today." The device then sends this request to the server.

[2395] Example: A user types, "What should I wear for today's weather?"

[2396] Obtaining user information and weather information

[2397] The server obtains fashion history and preference data based on the user ID, analyzes the user's emotional state using an emotion engine, and obtains current weather information using a weather API.

[2398] Example: The server responds, "Today is sunny and slightly hot, and your emotional state is positive, so I suggest a light summer outfit (e.g., a light-colored shirt and shorts)."

[2399] Prompt Sentence Examples

[2400] Meal suggestion feature: "Generate prompts to suggest dinner menus for tired users."

[2401] Trip Planning & Transportation Suggestion: "Generate prompts to suggest trip plans for users looking to relax."

[2402] Clothing Coordination Suggestion: "Generate prompts that suggest fashion coordination for users who are in a positive emotional state."

[2403] In this way, the system of the present invention can take into account the user's emotional state to provide more personalized suggestions and efficiently handle everyday tasks.

[2404] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2405] Menu suggestion function

[2406] Processing Steps

[2407] Step 1:

[2408] Terminal: A user types a request into a chat application: "What's for dinner?"

[2409] Input: User request

[2410] Output: Sends an API request to the server

[2411] What it does: Use your smartphone or computer to access a chat app, type in your request, and press send.

[2412] Step 2:

[2413] Server: Receives requests sent from the device and analyzes them using the NLP engine.

[2414] Input: API request

[2415] Output: Parsed request data

[2416] What it does: The server uses a natural language processing engine (e.g., spaCy or NLTK) to analyze the user's request and extract meaning.

[2417] Step 3:

[2418] Server: Retrieves profile data from the database based on the user ID.

[2419] Input: User ID

[2420] Output: User profile data (past preferences, allergy information, current ingredient availability)

[2421] Specific operation: The server issues a query to a database (e.g., MySQL or PostgreSQL) to retrieve profile data.

[2422] Step 4:

[2423] Server: Analyzes the user's current emotional state using the emotion engine.

[2424] Input: User profile data

[2425] Output: Current emotional state data

[2426] What it does: The server inputs profile data into an emotion engine (e.g., Affectiva or Microsoft Emotion API) to analyze the emotional state.

[2427] Step 5:

[2428] Server: Searches the recipe database for candidate recipes that match the criteria and generates the optimal menu.

[2429] Input: Emotional state data, profile data, database query

[2430] Output: Optimal meal plan

[2431] Specific operation: The server issues a query to a recipe database (e.g., MongoDB or Elasticsearch), searches for recipes that meet the criteria, and generates the optimal menu.

[2432] Step 6:

[2433] Server: Formats the generated menu information into a response message and sends it to the device.

[2434] Input: Optimal meal plan

[2435] Output: The formatted response message

[2436] Specific operation: The server formats the generated menu data in rich text or JSON format and sends it to the terminal.

[2437] Step 7:

[2438] Terminal: Messages received from the server are displayed within the chat application for the user to review.

[2439] Input: Response message from the server

[2440] Output: Menu display in a chat application

[2441] Specific behavior: The device receives the response message and displays it in the chat application UI.

[2442] Travel planning and transportation suggestions

[2443] Processing Steps

[2444] Step 1:

[2445] Terminal: A user types a request into a chat application: "Plan a trip for next weekend."

[2446] Input: User request

[2447] Output: Sends an API request to the server

[2448] What it does: Access a chat app on your smartphone or computer, type in your request, and press send.

[2449] Step 2:

[2450] Server: Receives requests sent from the device and analyzes them using the NLP engine.

[2451] Input: API request

[2452] Output: Parsed request data

[2453] What happens: The server uses a natural language processing engine to analyze the request and extract meaning.

[2454] Step 3:

[2455] Server: Retrieves profile data from the database based on the user ID.

[2456] Input: User ID

[2457] Output: Past travel history, interests, budget, time data

[2458] What happens next: The server queries the database to retrieve profile data.

[2459] Step 4:

[2460] Server: Analyzes the user's current emotional state using the emotion engine.

[2461] Input: Profile data

[2462] Output: Current emotional state data

[2463] Specific operation: The server inputs profile data into the emotion engine and analyzes the emotional state.

[2464] Step 5:

[2465] Server: Generates the optimal travel plan based on the acquired information.

[2466] Input: Emotional state data, profile data, database query

[2467] Output: Optimal travel plan

[2468] Specific operation: The server executes the travel plan generation algorithm and generates a plan that meets the conditions.

[2469] Step 6:

[2470] Server: Formats the itinerary and transportation methods into a response message and sends it to the device.

[2471] Input: Best Travel Plan

[2472] Output: The formatted response message

[2473] Specific operation: The server formats the generated plan data and sends it to the device.

[2474] Step 7:

[2475] On the device: The proposed itinerary is displayed in the chat app for the user to review.

[2476] Input: Response message from the server

[2477] Output: Plan display in a chat application

[2478] Specific behavior: The device receives the response message and displays it in the chat app's UI.

[2479] Clothing coordination suggestion function

[2480] Processing Steps

[2481] Step 1:

[2482] Terminal: A user types a request into a chat application, such as "Tell me what outfit to wear today."

[2483] Input: User request

[2484] Output: Sends an API request to the server

[2485] What it does: Access a chat app on your smartphone or computer, type in your request, and press send.

[2486] Step 2:

[2487] Server: Receives requests sent from the device and analyzes them using the NLP engine.

[2488] Input: API request

[2489] Output: Parsed request data

[2490] What happens: The server uses a natural language processing engine to analyze the content of the request.

[2491] Step 3:

[2492] Server: Retrieves profile data from the database based on the user ID.

[2493] Input: User ID

[2494] Output: Past fashion history, preference data

[2495] What happens next: The server queries the database to retrieve profile data.

[2496] Step 4:

[2497] Server: Analyzes the user's current emotional state using the emotion engine.

[2498] Input: Profile data

[2499] Output: Current emotional state data

[2500] Specific operation: The server inputs profile data into the emotion engine and analyzes the emotional state.

[2501] Step 5:

[2502] Server: Calls the weather API to get current weather information.

[2503] Input: Weather API request

[2504] Output: Current weather information

[2505] What happens: The server sends a request to an external weather API to get current weather information.

[2506] Step 6:

[2507] Server: Generates optimal coordination based on the acquired information.

[2508] Input: Emotional state data, fashion history data, weather information

[2509] Output: Optimal outfit ideas

[2510] Specific operation: The server executes the coordination generation algorithm and generates fashion suggestions that meet the conditions.

[2511] Step 7:

[2512] Server: Formats the proposed coordinates into a response message and sends it to the device.

[2513] Input: Best outfit ideas

[2514] Output: The formatted response message

[2515] Specific operation: The server formats the generated coordinate data and sends it to the terminal.

[2516] Step 8:

[2517] Device: The suggested outfits are displayed in the chat app so that the user can check them.

[2518] Input: Response message from the server

[2519] Output: Coordinate display in a chat application

[2520] Specific behavior: The device receives the response message and displays it in the chat app's UI.

[2521] Example prompts for generative AI models

[2522] Meal suggestion feature: "Generate prompts to suggest dinner menus for tired users."

[2523] Trip Planning & Transportation Suggestion: "Generate prompts to suggest trip plans for users looking to relax."

[2524] Clothing Coordination Suggestion: "Generate prompts that suggest fashion coordination for users who are in a positive emotional state."

[2525] (Application example 2)

[2526] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2527] Conventional recommendation systems make suggestions based on user requests and past data, but because they do not take the user's emotional state into account, they are unable to make suggestions that are optimal for the user's current mood or situation, limiting the improvement of user satisfaction.In addition, it has been difficult to make personalized suggestions based on real-time emotion recognition in physical stores, etc.

[2528] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2529] In this invention, the server includes means for receiving requests from a user, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, means including an emotion engine for analyzing the user's emotional state, means for generating optimal proposals in accordance with the user's requests and emotional state, and means for transmitting the generated proposals to the user, thereby enabling optimal proposals that take into account the user's current emotional state.

[2530] The "means for receiving a request from a user" refers to a device or software that allows a user to input a request for information or a service they desire into the system and receives the request electronically.

[2531] "Means for obtaining a user's past usage history and profile data" refers to a device or software that extracts records of a user's past use of the system, personal information, and preference data.

[2532] The "means for referencing a database for generating proposal content" refers to a device or software that accesses and refers to a database that stores information required for generating various proposal content.

[2533] The "means including an emotion engine for analyzing the user's emotional state" refers to hardware and software for recognizing and analyzing emotions from the user's facial expressions, tone of voice, etc.

[2534] The "means for generating optimal suggestions according to the user's requests and emotional state" refers to a device or software for generating the most appropriate suggestions based on the user's request content and analyzed emotional state.

[2535] The "means for transmitting the generated proposal to the user" refers to a device or software that electronically transmits the content of the generated proposal to the user's terminal.

[2536] The system of the present invention makes suggestions to improve the efficiency of a user's daily tasks and includes an emotion engine that analyzes the user's emotional state. This system is primarily composed of a server, a terminal, and a user. Specific embodiments for realizing this system are described below.

[2537] The server includes means for receiving requests from users, means for acquiring the user's past usage history and profile data, means for referencing a database for generating proposal content, an emotion engine for analyzing the user's emotional state, means for generating optimal proposals according to the user's requests and emotional state, and means for transmitting the generated proposals to the user.

[2538] Specifically, when a user accesses product information using a smartphone or smart glasses, the system analyzes the user's emotional state at that time and proposes the most suitable products and sets. For example, if the user is tired, the system will propose relaxing items (bath additives, aromatherapy, relaxation goods, etc.), and if the user is energetic, it will propose products suitable for new challenges or activities (sports goods, games, activity kits, etc.).

[2539] This system mainly uses the following hardware and software:

[2540] The camera on a smartphone or smart glasses is used to capture the user's facial expressions.

[2541] Using OpenCV (image processing library), the user's facial expression is processed from the captured image.

[2542] Use emotion_recognition (emotion recognition library) to analyze the user's current emotional state from their facial expressions.

[2543] Use requests (an HTTP request library) to communicate with the user database and retrieve user profile data.

[2544] Use the recommendation_engine to generate optimal recommendations based on emotional state and profile data.

[2545] As a concrete example, a user enters a physical store through smart glasses. It seems that the user has been very busy that weekend and is feeling stressed. The camera analyzes the user's facial expressions to determine their emotions and determines that they are "tired." As a result, the smart glasses' display displays a message saying, "We'll suggest products that will help you relax today!" The user can then choose a product based on the suggestions.

[2546] Example prompt sentence:

[2547] "Relax, we've found the perfect product for you."

[2548] "Here are some recommended activities for you who are feeling energetic!"

[2549] In this way, by providing individually optimized suggestions that take into account the user's emotional state, user satisfaction can be improved.

[2550] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2551] Step 1:

[2552] A user inputs a request using a smartphone or smart glasses. The user sends a question to the system, such as "What is today's recommended product?" The input is the user's request, and the output is the request data.

[2553] Step 2:

[2554] The device sends the user's request to the server. After receiving the user's request, the smartphone or smart glasses transmit the request to the server via the Internet. The input is the user's request data, and the output is a confirmation of the request transmission to the server.

[2555] Step 3:

[2556] The server receives and analyzes the request. The received request data is analyzed by a program on the server to identify the type of information or service the user is seeking. The input is the received request data, and the output is the analysis result.

[2557] Step 4:

[2558] The server retrieves the user's profile data. After analyzing the request, the server retrieves past usage history and profile data from the database based on the user ID. The input is the user ID and the output is the profile data.

[2559] Step 5:

[2560] The server uses an emotion engine to analyze the user's emotional state. The emotion engine analyzes the user's emotional state based on the user's facial expressions captured in real time by a camera installed in a smartphone or smart glasses. The input is the user's facial expression data, and the output is the analyzed emotional state.

[2561] Step 6:

[2562] The server generates optimal suggestions based on the acquired profile data and analyzed emotional state. The server refers to a recipe database and recommends products and services that match the user's profile data and emotional state. The input is the profile data and emotional state, and the output is the optimal suggestions.

[2563] Step 7:

[2564] The server sends the generated proposal to the terminal. Once the optimal proposal is generated, the server sends it to the user's terminal. The input is the optimal proposal, and the output is a confirmation of the proposal transmission to the terminal.

[2565] Step 8:

[2566] The device displays the proposed content to the user. The smartphone or smart glasses displays the received proposed content on the user's display. The input is the proposed content sent from the server, and the output is the proposed content displayed on the user's display.

[2567] Specifically, the user speaks to the smart glasses, asking, "What is today's recommended product?" The server analyzes the response and displays, "Considering that you are tired, we recommend an aroma candle to help you relax today."

[2568] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2569] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2570] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2571] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2572] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2573] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2574] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2575] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2576] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2577] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2578] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2579] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2580] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2581] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2582] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2583] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2584] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2585] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2586] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2587] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2588] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2589] The following is further disclosed regarding the above embodiment.

[2590] (Claim 1)

[2591] means for receiving a request from a user;

[2592] means for obtaining user historical usage and profile data;

[2593] a means for referencing a database for generating suggestions;

[2594] means for generating optimal suggestions according to user requirements;

[2595] means for transmitting the generated suggestions to a user;

[2596] A system including:

[2597] (Claim 2)

[2598] The system according to claim 1, further comprising a dinner menu suggestion means, a travel plan and transportation suggestion means, and a fashion coordination suggestion means for receiving various requests related to the user's daily tasks and providing optimal suggestions in response thereto.

[2599] (Claim 3)

[2600] 2. The system according to claim 1, further comprising means for generating an optimal menu based on the user's profile data, taking into consideration dietary preferences, allergy information, and availability of ingredients, and suggesting the menu to the user.

[2601] "Example 1"

[2602] (Claim 1)

[2603] means for receiving a request from a user;

[2604] means for obtaining user historical usage and profile data;

[2605] a means for referencing a database for generating suggestions;

[2606] means for generating optimal suggestions according to user requirements;

[2607] means for transmitting the generated suggestions to a user;

[2608] A means of analyzing the request and determining which functionality is being requested;

[2609] A means of obtaining information from external APIs based on user information;

[2610] A system including:

[2611] (Claim 2)

[2612] The system of claim 1, further comprising a meal menu suggestion means, a travel plan and transportation suggestion means, and a fashion coordination suggestion means for receiving various requests related to the user's daily tasks and providing optimal suggestions in response thereto.

[2613] (Claim 3)

[2614] The system of claim 1 further comprises means for generating an optimal menu based on the user's profile data, taking into consideration dietary preferences, allergy information, ingredient availability, and information from an external API, and suggesting the menu to the user.

[2615] "Application Example 1"

[2616] (Claim 1)

[2617] means for receiving a request from a user;

[2618] means for obtaining user historical usage and profile data;

[2619] a means for referencing a database for generating suggestions;

[2620] means for generating optimal suggestions according to user requirements;

[2621] means for transmitting the generated suggestions to a user;

[2622] A means for suggesting optimal products based on a user's purchasing history and interests;

[2623] A system including:

[2624] (Claim 2)

[2625] The system of claim 1, further comprising a dinner menu suggestion means, a travel plan and transportation suggestion means, a fashion coordination suggestion means, and a means for suggesting optimal products, for receiving various requests related to the user's daily tasks and providing optimal suggestions in response thereto.

[2626] (Claim 3)

[2627] The system of claim 1 further comprises a means for generating an optimal menu based on the user's profile data, taking into consideration dietary preferences, allergy information, and ingredient availability, and suggesting this to the user, as well as a means for suggesting optimal products based on the user's purchasing history and interests.

[2628] "Example 2: Combining Emotion Engines"

[2629] (Claim 1)

[2630] means for receiving a request from a user;

[2631] means for obtaining user historical usage and profile data;

[2632] a means for referencing a database for generating suggestions;

[2633] means including an emotion engine for analyzing an emotional state of a user; ...

Claims

1. means for receiving a request from a user; means for obtaining user historical usage and profile data; a means for referencing a database for generating suggestions; means for generating optimal suggestions according to user requirements; means for transmitting the generated suggestions to a user; A system including:

2. The system according to claim 1, further comprising a dinner menu suggestion means, a travel plan and transportation suggestion means, and a fashion coordination suggestion means for receiving various requests related to the user's daily tasks and providing optimal suggestions in response thereto.

3. 2. The system according to claim 1, further comprising means for generating an optimal menu based on the user's profile data, taking into consideration dietary preferences, allergy information, and availability of ingredients, and for suggesting the menu to the user.

Images