System

Abstract

A system is provided.SOLUTION: A system comprising: means for authenticating a user; means for collecting profile information of each user; means for selecting a generated AI model suitable for each user; means for combining the selected generated AI models to construct an integrated model; means for analyzing needs and preferences of an entire group using the integrated model; means for generating an optimal proposal for the entire group; means for transmitting the generated proposal to terminals of the respective users; means for collecting feedback of the users; and means for analyzing the collected feedback to improve the generated AI model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] While conventional generative AI models can provide personalized suggestions to individual users, they face the challenge of integrating the different needs and preferences of each user to provide optimal suggestions when multiple users make decisions as a group. This limitation is particularly pronounced in scenarios that require group collaboration, such as choosing a restaurant or creating a travel plan. This invention aims to solve this problem by combining individual generative AI models to provide optimal suggestions that reflect the needs and preferences of the entire group. [Means for solving the problem]

[0005] The present invention provides a system including means for authenticating users, means for collecting profile information of each user, means for selecting a generative AI model suitable for each user, means for combining the selected generative AI models to construct an integrated model, means for analyzing the needs and preferences of an entire group using the integrated model, means for generating optimal proposals for the entire group, means for transmitting the generated proposals to each user's device, means for collecting user feedback, and means for analyzing the collected feedback to improve the generative AI model, thereby enabling the integration of different preferences and needs of multiple users and making optimal proposals for the entire group in real time.

[0006] "User" refers to an individual or individuals who use the system.

[0007] "Authentication" refers to the process by which a system verifies a user's identity and grants access rights.

[0008] "Profile information" refers to data that the system stores and manages about users, such as their preferences, past behavioral history, and settings.

[0009] "Generative AI model" refers to an artificial intelligence algorithm that generates natural language and other data based on user requests.

[0010] "Combining" refers to the process of integrating multiple generative AI models and treating them as a single integrated model.

[0011] An "integrated model" refers to a model created by combining multiple generative AI models to analyze and reflect the preferences and needs of an entire population.

[0012] "Suggestion" refers to a specific action or option that is generated for the user based on the integrated model.

[0013] "Terminal" refers to the device (e.g., smartphone, PC, tablet, etc.) used by a user to access the system.

[0014] "Feedback" refers to reactions such as ratings, comments, and opinions provided by users in response to suggestions.

[0015] "Analysis" refers to the process of processing and evaluating collected data to extract meaningful information.

[0016] "Improvement" refers to the process of improving the performance and capabilities of a generative AI model based on collected data and feedback. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention is a system that integrates the different preferences and needs of multiple users and makes optimal suggestions to the entire group. An embodiment of this system will be described in detail below.

[0039] System configuration and program processing

[0040] The system is mainly implemented by three main components: "server", "terminal", and "user".

[0041] 1. User authentication and information collection

[0042] A terminal is a device that a user uses to access the system, such as a smartphone or a PC. When a user logs in to the system using a terminal, the server authenticates the user. If authentication is successful, the server retrieves each user's profile information from the database. The profile information includes the user's preferences, past selection history, and customization settings.

[0043] 2. Selecting and combining generative AI models

[0044] The server selects the generative AI model that is most suitable for each user based on the acquired profile information. For example, "Italian Restaurant GPT" is selected for Person A, "Japanese Restaurant GPT" for Person B, and "Healthy Restaurant GPT" for Person C. The selected generative AI models are combined by the server to build an integrated model. This integrated model reflects each user's preferences and can perform analysis according to overall needs.

[0045] 3. Generate and send optimal proposals

[0046] The server uses the integrated model to analyze the needs and preferences of the entire group and generate optimal suggestions. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine" and describes in detail the benefits and features of each suggestion. The server then sends the generated suggestions to each user's device, and the device displays the received suggestions to the user. Users can provide feedback on the suggestions, which is then sent back to the server.

[0047] 4. Analyze feedback and improve the model

[0048] The server analyzes the collected feedback and uses it to improve the generative AI model. This process improves the accuracy of the system's suggestions, ensuring that future suggestions are more in line with the user's preferences.

[0049] Specific examples

[0050] For example, suppose three users (A, B, and C) want to choose a restaurant to have dinner together.

[0051] 1. Device: Persons A, B, and C each log in to the app on their own smartphones.

[0052] 2. Server: Retrieves each user's preference information (A prefers "Italian food," B prefers "Japanese food," and C prefers "healthy food") from the database.

[0053] 3. Server: Selects the optimal generative AI model for each user and combines the "Italian Restaurant GPT," "Japanese Restaurant GPT," and "Healthy Restaurant GPT."

[0054] 4. Server: Uses the combined model to search for restaurants that match everyone's preferences and generate multiple candidates.

[0055] 5. Server: Sends the generated proposals to each user's device, which displays them to the user.

[0056] 6. User: Each user enters feedback on the proposal, and the device sends it to the server.

[0057] 7. Server: Analyzes feedback and improves the generative AI model.

[0058] In this way, the system can effectively integrate the different preferences and needs of multiple users and provide optimal recommendations for the entire population.

[0059] The processing flow will be explained below.

[0060] Step 1:

[0061] The terminal displays an authentication screen for the user to access the system. The user enters their own authentication information (user ID and password).

[0062] Step 2:

[0063] The terminal transmits the input authentication information to the server.

[0064] Step 3:

[0065] The server compares the received authentication information with the database to authenticate the user. If authentication is successful, the session begins.

[0066] Step 4:

[0067] The server retrieves the authenticated user's profile information from the database, which includes the user's preferences, past behavior history, and customization settings.

[0068] Step 5:

[0069] The server selects the optimal generative AI model for each user based on the acquired profile information. For example, it might select "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[0070] Step 6:

[0071] The server combines the selected generative AI models to build an integrated model, which can integrate information from each combined generative AI model and analyze the needs and preferences of the entire population.

[0072] Step 7:

[0073] The server uses the integrated model to generate optimal recommendations that reflect the needs and preferences of the entire group, such as generating multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine."

[0074] Step 8:

[0075] The server transmits the generated proposals to each user's terminal.

[0076] Step 9:

[0077] The device displays the received suggestions to the user, including the restaurant name, location, menu details, and ratings.

[0078] Step 10:

[0079] Users select their preferred options from the suggestions and enter feedback, including their satisfaction level, additions, and corrections.

[0080] Step 11:

[0081] The terminal transmits the user's feedback to the server.

[0082] Step 12:

[0083] The server analyzes the received feedback and improves the generative AI model, so that future suggestions will better match the user's preferences.

[0084] In this way, the system effectively integrates the different preferences and needs of multiple users to provide optimal recommendations for the entire group.

[0085] Example 1

[0086] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0087] There is a need for a system that can integrate the different preferences and needs of multiple users to generate optimal proposals for the entire group. However, conventional methods have difficulty analyzing each user's preferences individually, making it difficult to efficiently generate proposals that meet the needs of the entire group. In addition, there is a lack of technology to effectively incorporate user feedback and improve the model. To solve this problem, a collective optimization method using an advanced generative AI model is required.

[0088] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0089] In this invention, the server includes means for authenticating users, means for collecting profile information for each user, means for selecting a generative AI model suitable for each user, means for combining the selected generative AI models to build an integrated model, means for analyzing the needs and preferences of the entire group using the integrated model, means for generating optimal proposals for the entire group, means for transmitting the generated proposals to each user's device, means for collecting user feedback, means for analyzing the collected feedback to improve the generative AI model, means for describing the optimal proposals in detail using prompt sentences, and means for notifying and displaying the proposals. This makes it possible to effectively integrate the preferences and needs of multiple users, generate optimal proposals for the entire group, and further improve the accuracy of the proposals by utilizing user feedback.

[0090] "Means for authenticating users" refers to a mechanism for verifying the user's login information and determining whether the user is legitimate.

[0091] The "means for collecting profile information for each user" is a function for obtaining information such as the user's preferences, past selection history, and customization settings from a database.

[0092] The "means for selecting a generative AI model that suits each user" is a mechanism for identifying the generative AI model that best suits the user's preferences and needs based on collected profile information.

[0093] "Means for combining selected generative AI models to construct an integrated model" refers to a method for integrating generative AI models selected for each user to create a common model that reflects the preferences and needs of multiple users.

[0094] "Means for analyzing needs and preferences across a population using an integrated model" means a process for utilizing an integrated generative AI model to comprehensively assess the needs and preferences of a group of multiple users.

[0095] The "means for generating optimal proposals for the entire group" is a function that creates optimal proposals that satisfy the preferences and needs of the entire group based on the analysis results of the entire group.

[0096] The "means for transmitting the generated proposal to each user's terminal" is a mechanism for distributing the generated proposal to each user's device via a network.

[0097] The "means for collecting user feedback" is a mechanism for collecting opinions and ratings provided by users on suggestions.

[0098] "Means for analyzing collected feedback to improve generative AI models" refers to methods for analyzing feedback collected from users and using the results to improve the accuracy and applicability of generative AI models.

[0099] The "means for describing the optimal proposal in detail using a prompt sentence" is a function for expressing the generated proposal in detail using a prompt sentence including specific sentences and instructions.

[0100] A "means for notifying and displaying suggestions" is a method for informing the user of and visually displaying the generated suggestions.

[0101] This invention is a system that integrates the different preferences and needs of multiple users and makes optimal suggestions to the entire group. How this system is implemented will be described in detail below.

[0102] System configuration

[0103] This system is mainly composed of three main components: "server," "terminal," and "user."

[0104] User authentication and information collection

[0105] A terminal is a device that a user uses to access the system, such as a smartphone or a PC. When a user logs in to the system using a terminal, the login information (user name and password) is sent from the terminal to the server. At this time, the communication is securely protected using the HTTPS protocol.

[0106] The server passes the received login information to the authentication processing module and performs user authentication. If authentication is successful, the server retrieves the user's profile information from the database. This profile information includes the user's preferences, past selection history, and customization settings. The database uses an RDBMS such as MySQL or PostgreSQL.

[0107] Selecting and combining generative AI models

[0108] The server selects the most suitable generative AI model for each user based on the acquired profile information. For example, "Italian Restaurant GPT" is selected for Person A, "Japanese Restaurant GPT" for Person B, and "Healthy Restaurant GPT" for Person C. These generative AI models are trained using training data specialized for each user's specific needs.

[0109] The server combines the selected generative AI models to construct an integrated model that comprehensively reflects the user's preferences. For this purpose, technologies such as reinforcement learning (RL) and ensemble learning may be used.

[0110] Generate and send optimal proposals

[0111] The server uses the integrated model to analyze the needs and preferences of the entire group and generate specific suggestions. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine," and includes detailed information about each suggestion (location, menu, recommended points, etc.). This process uses prompts from the generative AI model.

[0112] Here are some examples of prompts:

[0113] "Person A likes Italian restaurants, Person B likes Japanese food, and Person C likes healthy meals. Please provide restaurant suggestions that would be best for these three people."

[0114] The server sends the generated suggestions to each user's device, which then displays them to the user. The suggestions are displayed visually in rich text and image formats, and push notifications are also used if necessary.

[0115] Collecting and analyzing feedback

[0116] The user provides feedback on the displayed suggestions. The feedback is sent from the device to the server through a dedicated input form. The feedback includes the degree of satisfaction with the suggestions and specific comments.

[0117] The server analyzes this feedback and uses it to improve the generative AI model. The feedback data is fed into the machine learning algorithm, and the generative AI model is retrained to improve the accuracy of future suggestions.

[0118] Specific examples

[0119] For example, suppose three users (A, B, and C) want to choose a restaurant to have dinner together.

[0120] 1. Device: Persons A, B, and C each log in to the app on their own smartphones.

[0121] 2. Terminal: Once the login information is entered, the terminal sends it to the server.

[0122] 3. Server: The server receives the login information and performs authentication. If authentication is successful, it retrieves the profile information from the database.

[0123] 4. Server: Analyzes the profile information and selects "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[0124] 5. Server: Combines the selected generative AI models to create an integrated model.

[0125] 6. Server: Uses the integrated model to generate multiple restaurant options that match everyone's preferences, and generates detailed information using prompts.

[0126] 7. Server: Sends the generated proposals to each user's device.

[0127] 8. Device: The device displays the received suggestions to the user and optionally sends push notifications.

[0128] 9. User: Each user provides feedback on the proposal, which is sent to the server via their device.

[0129] 10. Server: Analyzes feedback and helps improve the generative AI model.

[0130] In this way, the system can effectively integrate the different preferences and needs of multiple users and provide optimal recommendations for the entire population.

[0131] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0132] Program processing flow

[0133] Step 1: User Login

[0134] Step 2: User authentication and profile information retrieval

[0135] Step 3: Selecting a generative AI model

[0136] Step 4: Combining generative AI models

[0137] Step 5: Generate optimal proposals

[0138] Step 6: Submit and view your proposal

[0139] Step 7: Gather feedback

[0140] Step 8: Analyze feedback and improve the generative AI model

[0141] Detailed explanation of each processing step

[0142] Step 1: User Login

[0143] A terminal is a device that users use to access the system. Users access the login screen using a smartphone or computer and enter their username and password. The terminal receives this login information and sends it to the server using the HTTPS protocol.

[0144] Input: Username, Password

[0145] Output: Login information sent to the server

[0146] Step 2: User authentication and profile information retrieval

[0147] The server passes the received login information to the authentication processing module and performs user authentication. If authentication is successful, the server retrieves the user's profile information (preferences, past selection history, customization settings, etc.) from the database. The database uses an RDBMS such as MySQL or PostgreSQL.

[0148] Input: Login information

[0149] Output: Authentication result (success / failure), profile information

[0150] Step 3: Selecting a generative AI model

[0151] The server analyzes the acquired profile information and selects the most suitable generative AI model for each user. For example, "Italian Restaurant GPT" is selected for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C. The selection criteria are based on the user's preferences and past selection history.

[0152] Input: Profile Information

[0153] Output: The selected generative AI model

[0154] Step 4: Combining generative AI models

[0155] The server combines the selected generative AI models to build an integrated model. This integrated model reflects each user's preferences and analyzes the overall needs. Specific methods include ensemble learning and reinforcement learning.

[0156] Input: A selected generative AI model

[0157] Output: Integrated model

[0158] Step 5: Generate optimal proposals

[0159] The server uses the integrated model to analyze the needs and preferences of the entire group and generate specific suggestions. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine," including detailed information (location, menu, recommended features, etc.). This process uses prompts from the generative AI model.

[0160] Input: Integration model

[0161] Output: Proposal

[0162] Step 6: Submit and view your proposal

[0163] The server sends the generated suggestions to each user's device, which displays the received suggestions to the user and, if necessary, sends push notifications. Suggestions are visually displayed in rich text and image formats.

[0164] Input: Proposal

[0165] Output: Proposal data sent to the user's device

[0166] Step 7: Gather feedback

[0167] The user provides feedback on the displayed suggestions. The feedback is sent from the device to the server via a dedicated input form. The feedback includes the degree of satisfaction with the suggestions and specific comments.

[0168] Input: Feedback information

[0169] Output: Feedback data sent to the server

[0170] Step 8: Analyze feedback and improve the generative AI model

[0171] The server analyzes the collected feedback and feeds the feedback data to a machine learning algorithm, which uses it as training data for the generative AI model. This improves the generative AI model so that future suggestions will better match the user's preferences.

[0172] Input: Feedback data

[0173] Output: An improved generative AI model

[0174] In this way, the system can effectively integrate the different preferences and needs of multiple users and provide optimal recommendations for the entire population.

[0175] (Application example 1)

[0176] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0177] Conventional recommendation systems face the challenge of effectively integrating the different preferences and needs of individual users and making optimal recommendations for the entire group. In particular, when multiple users have different food preferences, it is necessary to quickly and accurately recommend restaurants and dishes that will satisfy everyone. Furthermore, there is a lack of a way to continuously improve the system using user feedback, making it difficult to improve the accuracy of the recommendations.

[0178] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0179] In this invention, the server includes means for authenticating users, means for collecting profile information for each user, means for selecting a generative AI model suitable for each user, means for combining the selected generative AI models to build an integrated model, means for analyzing the needs and preferences of the entire group using the integrated model, means for generating optimal proposals for the entire group, means for sending the generated proposals to each user's device, means for collecting user feedback, and means for analyzing the collected feedback to improve the generative AI model. This makes it possible to effectively integrate the different preferences and needs of multiple users, quickly suggest restaurants and dishes that will satisfy everyone, and continuously improve the accuracy of the suggestions by reflecting user feedback.

[0180] "User authentication" is the process of verifying the identity and access rights of a user attempting to access a system and confirming that the user is a legitimate user.

[0181] "Profile information" is data that includes personal information such as each user's preferences, past selection history, and customization settings.

[0182] A "generative AI model" is an AI model that generates appropriate suggestions based on a user's preferences and needs, and is specialized for a specific genre or theme.

[0183] An "integrated model" is a model created by combining individual generative AI models, and is capable of integrating the preferences of multiple users to analyze and make suggestions based on overall needs.

[0184] "The needs and preferences of the entire group" refers to a comprehensive understanding of the different interests, concerns, and desired conditions of multiple users.

[0185] An "optimal suggestion" is one that provides the most satisfying options to the entire group and is generated based on the different preferences and needs of users.

[0186] A "terminal" is a device that a user uses to access the system, such as a smartphone, tablet, or PC.

[0187] "Feedback" refers to data such as ratings, opinions, and comments provided by users regarding proposed options, and is used to improve the system.

[0188] "Improvement" is the process of improving a generative AI model or the entire system based on feedback.

[0189] This invention is a system for integrating the different preferences and needs of multiple users and making optimal suggestions to the entire group. The system is implemented mainly by a "server," "terminals," and "users."

[0190] 1. User authentication and information collection

[0191] A terminal is a device that a user uses to access the system. Examples include smartphones, tablets, and PCs. When a user logs into the system using a terminal, the server authenticates the user. If authentication is successful, the server retrieves each user's profile information from the database. The profile information includes the user's preferences, past selection history, and customization settings.

[0192] 2. Selecting and combining generative AI models

[0193] The server selects the generative AI model that is most suitable for each user based on the acquired profile information. For example, "Italian GPT" is selected for User A, "Japanese Food GPT" for User B, and "Healthy GPT" for User C. The selected generative AI models are combined by the server to build an integrated model. This integrated model reflects each user's preferences and can perform analysis according to overall needs.

[0194] 3. Generate and send optimal proposals

[0195] The server uses the integrated model to analyze the needs and preferences of the entire population and generate optimal suggestions. For example, it generates multiple healthy restaurant options that combine Italian and Japanese cuisine, detailing the benefits and features of each suggestion. The server then sends the suggestions to each user's device, which displays them to the user. Users can provide feedback on the suggestions, which is then sent back to the server.

[0196] 4. Analyze feedback and improve the model

[0197] The server analyzes the collected feedback and uses it to improve the generative AI model. This process improves the accuracy of the system's suggestions, ensuring that future suggestions are more in line with the user's preferences.

[0198] Hardware and Software Used

[0199] Hardware: Smartphones, tablets, computers

[0200] Server: AWS Compute Services, Database (e.g., Amazon RDS)

[0201] Generative AI models: large-scale language models such as OpenAI GPT-3

[0202] Data analysis tools: Python, Pandas, Numpy

[0203] Examples:

[0204] For example, consider a scenario where three users want to have dinner together. If User A prefers Italian food, User B prefers Japanese food, and User C prefers healthy food, the server selects a generative AI model suited to each user. By integrating the selected models, restaurant and food recommendations that satisfy everyone's preferences are generated.

[0205] Example prompt sentence:

[0206] "User A likes Italian food, User B likes Japanese food, and User C prefers healthy food. Please combine these preferences and suggest restaurant candidates that will satisfy all of them."

[0207] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0208] Step 1:

[0209] The server authenticates the user. When a user logs in to the system using a terminal, the terminal sends the authentication information to the server. The server authenticates the user based on the authentication information, and if authentication is successful, returns the user ID.

[0210] Input: User credentials

[0211] Output: authenticated user ID

[0212] Step 2:

[0213] The server retrieves each user's profile information from the database based on the authenticated user ID, including the user's preferences, past selection history, and customization settings.

[0214] Input: authenticated user ID

[0215] Output: User profile information

[0216] Step 3:

[0217] The server selects a generative AI model that suits each user based on the acquired profile information. For example, "Italian GPT" is selected for User A, "Japanese Food GPT" for User B, and "Healthy GPT" for User C.

[0218] Input: User profile information

[0219] Output: A generative AI model suited to each user

[0220] Step 4:

[0221] The server combines the selected generative AI models to build an integrated model, which combines the capabilities of the individual models and is used to analyze the needs and preferences of the entire population.

[0222] Input: A generative AI model tailored to each user

[0223] Output: Integrated model

[0224] Step 5:

[0225] The server uses the integrated model to analyze the needs and preferences of the entire population and generate optimal recommendations, such as a list of healthy restaurants that combine Italian and Japanese cuisine, detailing the benefits and features of each recommendation.

[0226] Input: Integration model

[0227] Output: A list of the best suggestions

[0228] Step 6:

[0229] The server sends the generated proposals to each user's terminal, which then displays the received proposals to the user.

[0230] Input: List of best suggestions

[0231] Output: Proposal displayed on the user's device

[0232] Step 7:

[0233] The user provides feedback on the suggestion, and the terminal transmits the user-entered feedback to the server.

[0234] Input: User feedback

[0235] Output: Feedback sent to the server

[0236] Step 8:

[0237] The server analyzes the collected feedback, which is then used to improve the generative AI model, adjusting it so that future suggestions better match the user's preferences.

[0238] Input: User feedback

[0239] Output: An improved generative AI model

[0240] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0241] This invention is a system that authenticates users, collects profile information, selects and combines appropriate generative AI models, generates and shares suggestions, collects user feedback, and improves the generative AI models based on the feedback. Furthermore, by combining it with an emotion engine that recognizes user emotions, it provides more accurate and personalized suggestions.

[0242] System configuration and program processing

[0243] This system consists of the following main components: an "emotion engine" that recognizes the user's emotions and reflects them in feedback, a "server" that manages the user's profile information, and a "terminal."

[0244] 1. User authentication and information collection

[0245] The terminal displays an authentication screen for the user to access the system. The user enters authentication information (user ID and password) and sends the authentication information to the server. The server verifies the authentication information and, if authentication is successful, retrieves the user's profile information from the database.

[0246] 2. Selecting and combining generative AI models

[0247] The server selects the optimal generative AI model for each user based on their profile information, and also takes into account the user's emotional state using an emotion engine. For example, it selects "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C. The user's emotional state is reflected in the profile information, making it possible to select a more accurate generative AI model. The selected generative AI models are then combined to build an integrated model.

[0248] 3. Generate and send optimal proposals

[0249] The server uses the integrated model to generate optimal suggestions that reflect the needs, preferences, and emotional state of the entire group. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine." The generated suggestions are sent to each user's device, which then displays the received suggestions to the user.

[0250] 4. User Emotion Recognition Feedback and Model Improvement

[0251] The user selects their preferred option from the suggested options and enters their feedback. The feedback includes not only their satisfaction level, desired additions, and corrections, but also reflects the user's emotional state as recognized by the emotion engine. The device sends the user's feedback to the server, which then analyzes the received feedback.

[0252] 5. Analyzing feedback and improving generative AI models

[0253] The server analyzes the feedback and continuously improves the generative AI model and emotion engine, thereby improving the accuracy of the system's suggestions so that future suggestions are more in line with the user's preferences and emotions.

[0254] Specific examples

[0255] For example, consider the case where three users (A, B, and C) are choosing a restaurant for dinner.

[0256] 1. Device: Persons A, B, and C log in to the app on their respective devices.

[0257] 2. Server: Each user's preference information (A-san prefers "Italian food," B-san prefers "Japanese food," and C-san prefers "healthy food") and their emotional state as recognized by the emotion engine at that time are obtained from the database.

[0258] 3. Server: Selects the optimal generative AI model for each user and combines the "Italian Restaurant GPT," "Japanese Restaurant GPT," and "Healthy Restaurant GPT" with the emotion engine.

[0259] 4. Server: Using the integrated model, it searches for restaurants that match the preferences and emotions of all users and generates multiple suggestions.

[0260] 5. Server: Sends the generated proposals to each user's device.

[0261] 6. Terminal: Displays the received suggestions to the user.

[0262] 7. User: Enters feedback on the suggestions and the emotional state is also reflected by the emotion engine.

[0263] 8. Device: Sends feedback to the server.

[0264] 9. Server: Analyzes feedback and improves generative AI models and emotion engines.

[0265] This system can effectively integrate the different preferences and emotions of multiple users and provide optimal suggestions to the entire group.

[0266] The processing flow will be explained below.

[0267] Step 1:

[0268] The terminal displays an authentication screen for the user to access the system. The user enters authentication information (user ID and password).

[0269] Step 2:

[0270] The terminal transmits the input authentication information to the server.

[0271] Step 3:

[0272] The server compares the received authentication information with the database to authenticate the user. If authentication is successful, the session begins.

[0273] Step 4:

[0274] The server retrieves profile information of the authenticated user from the database, including the user's preferences, past behavior history, customization settings, and emotional state data from the emotion engine.

[0275] Step 5:

[0276] The server selects the optimal generative AI model for each user based on the acquired profile information. Taking into account the user's emotional state, for example, it might select "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[0277] Step 6:

[0278] The server combines the selected generative AI models to build an integrated model, which integrates information from the combined generative AI models with emotional data from the emotion engine to analyze the needs, preferences, and emotions of the entire population.

[0279] Step 7:

[0280] The server uses the integrated model to generate optimal recommendations that reflect the needs, preferences, and emotional state of the entire population, such as generating multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine."

[0281] Step 8:

[0282] The server transmits the generated proposals to each user's terminal.

[0283] Step 9:

[0284] The device displays the received suggestions to the user, including the restaurant name, location, menu details, and ratings.

[0285] Step 10:

[0286] Users select their preferred option from the options presented and enter their feedback, which can include their level of satisfaction, what they would like to add, or what needs to be corrected. The emotion engine also analyzes the user's emotional state in real time and reflects this in the feedback.

[0287] Step 11:

[0288] The terminal transmits the user's feedback and emotion data to the server.

[0289] Step 12:

[0290] The server analyzes the received feedback and emotion data to improve the generative AI model and emotion engine, so that future suggestions will better match the user's preferences and emotions.

[0291] Specific examples

[0292] Consider three users (A, B, and C) having dinner together.

[0293] Step 1:

[0294] Device: Persons A, B, and C each log in to the app on their own smartphones.

[0295] Step 2:

[0296] Terminal: Sends the entered authentication information (user ID and password) to the server.

[0297] Step 3:

[0298] Server: Compares the received authentication information with the database and performs authentication. If authentication is successful, a session begins.

[0299] Step 4:

[0300] Server: Retrieves profile information for each authenticated user from a database, including preferences, past behavioral history, customization settings, and current emotional state.

[0301] Step 5:

[0302] Server: Based on the profile information and emotional state, select "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[0303] Step 6:

[0304] Server: Combines selected generative AI models to build an integrated model that integrates everyone's preferences and emotions.

[0305] Step 7:

[0306] Server: Using the integrated model, it generates multiple restaurant candidates that take into account everyone's preferences and emotions. For example, it generates candidates for "healthy restaurants that combine Italian and Japanese cuisine."

[0307] Step 8:

[0308] Server: Sends the generated proposals to the devices of A, B, and C.

[0309] Step 9:

[0310] Terminal: Displays the received suggestions to the user, including detailed restaurant information.

[0311] Step 10:

[0312] User: Selects the preferred option from the suggested options and enters feedback. The emotion engine analyzes the user's emotional state in real time and reflects that in the feedback.

[0313] Step 11:

[0314] Terminal: Sends feedback and emotion data to the server.

[0315] Step 12:

[0316] Server: Analyzes feedback and sentiment data to improve generative AI models and sentiment engines.

[0317] In this way, the system can effectively integrate the different preferences and emotions of multiple users and provide optimal suggestions for the entire population.

[0318] Example 2

[0319] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0320] Conventional suggestion systems make suggestions based solely on the user's preferences, resulting in a lack of personalized suggestions that reflect the user's real-time emotional state. Furthermore, the selection and integration of generative AI models was not centralized, making it difficult to improve the quality of the generated suggestions. This resulted in a decline in user satisfaction and an inability to fully utilize the system's effectiveness.

[0321] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0322] In this invention, the server includes means for authenticating users, means for collecting profile information for each user, means for selecting a generative AI model that suits each user, means for recognizing the user's emotional state, means for combining the selected generative AI model with an emotion engine to build an integrated model, means for analyzing the needs, preferences, and emotional state of the entire group using the integrated model, means for generating optimal proposals for the entire group, means for transmitting the generated proposals to each user's device, means for collecting user feedback and emotion recognition results, and means for analyzing the collected feedback and emotion recognition results to improve the generative AI model and emotion engine, thereby enabling more accurate and personalized proposals that reflect the user's real-time emotional state.

[0323] "Means of user authentication" refers to the method of having a user enter the necessary information (such as a user ID and password) to verify that they are a legitimate user when they access a system.

[0324] "Profile information" refers to detailed information about each user, such as personal data, preference information, and history information about the user.

[0325] The "means for selecting a generative AI model" is a method for selecting the optimal generative AI model based on the user's profile information and emotional state.

[0326] "Means for recognizing emotional state" refers to technologies (such as voice analysis and facial expression recognition) used to analyze a user's emotions in real time and identify their state.

[0327] "Means for constructing an integrated model by combining a generative AI model and an emotion engine" refers to a method for linking a selected generative AI model with an emotion engine to construct an integrated model.

[0328] The "means for analyzing the needs, preferences, and emotional states of an entire population" is a method for analyzing the needs, preferences, and emotional states of multiple users using an integrated model.

[0329] The "means for generating optimal proposals" is a method for creating the most suitable proposals for the user based on the analysis results.

[0330] The "means for transmitting the generated proposal to the terminal of each user" is a method for distributing the generated proposal to the terminal used by the user.

[0331] "Means for collecting feedback and emotion recognition results" refers to a method for collecting user evaluations and additional opinions on suggestions, as well as emotion analysis results from the emotion engine.

[0332] "Means for analyzing feedback and emotion recognition results to improve generative AI models and emotion engines" refers to methods for adjusting and improving generative AI models and emotion engines based on collected feedback and emotion data.

[0333] This invention is a system that authenticates users, collects profile information, selects and combines generative AI models, generates and shares suggestions, collects user feedback, and improves the generative AI models based on the feedback. Furthermore, by combining it with an emotion engine that recognizes user emotions, it provides more accurate and personalized suggestions.

[0334] The main components of the system are:

[0335] "Emotion Engine" that recognizes user emotions and reflects them in feedback

[0336] "Server" and "Terminal" that manage user profile information

[0337] 1. User authentication and information collection

[0338] The terminal displays an authentication screen for the user to access the system. The terminal collects the user ID and password entered by the user and sends this information to the server. The server verifies the received authentication information and retrieves the corresponding user profile information from a database. At this time, encrypted communication such as SSL is used for authentication to ensure the security of the information.

[0339] 2. Selecting and combining generative AI models

[0340] The server analyzes the acquired profile information. Data analysis tools such as Python are used for the analysis to select the optimal generative AI model based on the user's preferences and past behavioral history. At this time, the emotion engine analyzes the user's emotional state in real time, and this information is also added to the profile. The selected generative AI model and the emotion engine are then combined to create an integrated model. This integrated model is realized using machine learning frameworks such as TensorFlow and PyTorch.

[0341] 3. Generate and send optimal proposals

[0342] The server uses the integrated generative AI model to generate optimal suggestions based on the user's preferences and emotional state. For example, if a user prefers Italian restaurants with a relaxed atmosphere, the server generates multiple suggestions that reflect that information. The generated suggestions are sent to each user's device via a REST API. The device then displays the received suggestions on its screen, allowing the user to easily access them.

[0343] 4. User Emotion Recognition Feedback and Model Improvement

[0344] The user selects the preferred suggestion from the displayed suggestions and enters feedback regarding satisfaction, additions, and corrections. Specifically, the user uses a feedback form or rating slider. Furthermore, the emotion engine analyzes the user's emotional state and includes this information in the feedback. The device then sends this feedback information to the server.

[0345] 5. Analyzing feedback and improving generative AI models

[0346] The server performs a detailed analysis of the received feedback, using natural language processing and data mining techniques to extract patterns from the collected data. Based on the results, the generative AI model and emotion engine are refined. This continuous improvement ensures that future suggestions are more in line with the user's preferences and emotions.

[0347] Specific examples

[0348] For example, consider the case where three users (user A, user B, and user C) are choosing a restaurant for dinner.

[0349] 1. Device: User A, User B, and User C log in to the app on their respective devices.

[0350] 2. Server: Obtains each user's preference information (User A is "Italian food," User B is "Japanese food," and User C is "healthy food") and the emotional state recognized by the emotion engine at that time from the database.

[0351] 3. Server: Selects the optimal generative AI model for each user, for example, "Italian Restaurant GPT," "Japanese Restaurant GPT," and "Healthy Restaurant GPT," and combines them with an emotion engine.

[0352] 4. Server: Using the integrated model, it searches for restaurants that match the preferences and emotions of all users and generates multiple suggestions.

[0353] 5. Server: Sends the generated proposals to each user's device.

[0354] 6. Terminal: Displays the received suggestions to the user.

[0355] 7. User: Enters feedback on the suggestions and the emotional state is also reflected by the emotion engine.

[0356] 8. Device: Sends feedback to the server.

[0357] 9. Server: Analyzes feedback and improves generative AI models and emotion engines.

[0358] Prompt Sentence Examples

[0359] "User A currently likes Italian food, so please display several Italian restaurant suggestions for tonight's dinner. Please also consider the user's emotional state and prioritize restaurants with a relaxing atmosphere."

[0360] This system can effectively integrate the different preferences and emotions of multiple users and provide optimal suggestions to the entire group.

[0361] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0362] Step 1: User authentication and information gathering

[0363] The terminal displays an authentication screen for the user to access the system. The user enters their user ID and password and clicks the "Login button." Based on this input, the terminal sends the user ID and password to the server. The server searches for user information in a database, and if the authentication information is correct, retrieves profile information. At this time, encrypted communication such as SSL is used to ensure the security of the information. If authentication is successful, the user's profile information is sent from the server to the terminal.

[0364] Step 2: Selecting and combining generative AI models

[0365] The server analyzes the acquired profile information. This analysis uses data analysis tools such as Python to select the optimal generative AI model based on the user's preferences and past behavioral history. The emotion engine also becomes active, analyzing the user's real-time emotional state and adding this information to the profile. For example, if the user is looking for a "relaxed atmosphere," a generative AI model that reflects this is selected. The selected generative AI model is integrated with the emotion engine to build an integrated model. The input data is the profile information and emotional state, and the output is the generative AI model and the integrated model.

[0366] Step 3: Generate and submit optimal proposals

[0367] The server uses the integrated generative AI model to generate optimal suggestions based on the user's preferences and emotional state. For example, it generates multiple candidates for "relaxing Italian restaurants." The generative AI model uses data processing and machine learning algorithms to generate suggestions that match the user's preferences and emotions. The generated suggestions are sent to each user's device via a REST API. The input data is the integrated model, and the output is the generated suggestions. The device displays the received suggestions to the user.

[0368] Step 4: User emotion recognition feedback and model improvement

[0369] The user selects the suggestion they like best from the ones displayed and enters feedback regarding their satisfaction, additions they would like to make, and any corrections they would like to make. Specifically, they use a feedback form or a rating slider. Furthermore, the emotion engine analyzes the user's emotional state and includes this information in the feedback. For example, it may recognize from facial expression analysis that the user is "very satisfied" with the suggestion. The device then sends this feedback information to the server. The input data is the user's feedback and emotional state, and the output is the sent feedback information.

[0370] Step 5: Analyze feedback and improve the generative AI model

[0371] The server performs a detailed analysis of the received feedback. This analysis uses natural language processing and data mining techniques to extract patterns from the collected data. Based on the results, the generative AI model and emotion engine are improved. For example, the reason why a particular suggestion was not liked can be analyzed and the algorithm adjusted. This continuous improvement ensures that future suggestions will better match the user's preferences and emotions. The input data is feedback information and emotional state, and the output is an improved generative AI model and emotion engine.

[0372] (Application example 2)

[0373] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0374] Conventional recommendation systems have a problem in that they make suggestions based solely on the user's preference information. In particular, if the user's emotional state is not taken into account, the accuracy of the suggestions and user satisfaction may decrease. Furthermore, it is not easy to address the preferences and needs of an entire group, making it difficult to make appropriate suggestions to individual users. Therefore, there is a need for a system that takes into account the user's preference information and emotional state in an integrated manner and provides more personalized and appropriate suggestions.

[0375] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0376] In this invention, the server includes means for utilizing an emotion engine to reflect the user's emotional state, means for selecting a generative AI model suited to each user, and means for combining the selected generative AI models to construct an integrated model. This allows for the generation of suggestions that take into consideration the user's preference information and emotional state in an integrated manner, thereby improving user satisfaction.

[0377] "Means for authenticating users" refers to a function that allows a user to input authentication information required to access the system and verify that information.

[0378] The "means for collecting profile information of each user" is a function for collecting personal information, preference information, and history data provided by the user.

[0379] "Means for selecting a generative AI model that suits each user" is a function that selects the optimal generative AI model based on the user's profile information and emotional state.

[0380] "Means for combining selected generative AI models to construct an integrated model" refers to a function for combining multiple generative AI models to construct a single integrated model.

[0381] "Means for analyzing the needs and preferences of an entire group using an integrated model" is a function that utilizes an integrated model to analyze the needs and preferences of multiple users.

[0382] "Means for utilizing the emotion engine" refers to a function that recognizes the user's emotional state from facial expressions, voice, etc., and captures this as data.

[0383] The "means for generating optimal proposals" is a function that generates the most suitable proposals for the user based on all collected data.

[0384] The "means for transmitting the generated proposal to each user's terminal" is a function for transmitting the generated proposal from the server to the user's terminal.

[0385] The "means for collecting user feedback" is a function for obtaining reactions and evaluations made by users to suggestions.

[0386] "Means for analyzing collected feedback to improve the generative AI model and emotion engine" refers to a function for analyzing feedback obtained from users and continuously improving the generative AI model and emotion engine based on that feedback.

[0387] MODE FOR CARRYING OUT THE INVENTION

[0388] System Overview

[0389] This invention is a system that makes optimal suggestions based on a user's preferences and emotional state. The system consists of the following main components: an "emotion engine" that recognizes the user's emotions and reflects them in feedback, a "server" that manages the user's profile information, and a "terminal." Specifically, the system performs user authentication, profile collection, selection and integration of appropriate generative AI models, generation and sharing of suggestions, feedback collection, and model improvement based on the feedback.

[0390] Program processing overview

[0391] The program for this system is implemented as an application installed on a smartphone or smart glasses. Below, we will explain the processing overview of each component.

[0392] User authentication and information collection

[0393] A user logs in to the system using a smartphone or smart glasses. The device sends user authentication information to the server, which then verifies the authentication information. If authentication is successful, the server retrieves the user's profile information from the database.

[0394] Selecting and combining generative AI models

[0395] The server selects the optimal generative AI model based on the user's profile information and the emotion data recognized by the emotion engine. For example, it builds an integrated model by combining multiple generative AI models, such as "Chinese cuisine GPT," "Hamburger GPT," and "Vegan cuisine GPT."

[0396] Generate and send optimal proposals

[0397] The server uses the integrated model to generate optimal food delivery suggestions that reflect the user's preferences and emotional state. The suggestions are then sent to the user's device, which then displays the suggestions to the user.

[0398] User emotion recognition feedback and model improvement

[0399] The user selects from the presented suggestions and enters feedback. The emotion engine also recognizes the user's emotional state at the time of the feedback, and the device sends this feedback to the server. The server analyzes the received feedback and improves the generative AI model and emotion engine.

[0400] Hardware and software used

[0401] Hardware: Smartphones, smart glasses, servers

[0402] Software: Emotion recognition engine (EmotionEngine), AI model selection and combination library (ai_model_selector), database

[0403] Specific examples

[0404] For example, if a user is considering ordering dinner delivery, the system might:

[0405] 1. Authentication and profile information acquisition: The user logs in to the app on their smartphone, their authentication information is sent to the server, and their profile information is acquired.

[0406] 2. Recognizing emotional state: The emotion engine recognizes the user's emotional state, and the server receives the emotion data.

[0407] 3. Selection of generative AI model: The server selects the optimal generative AI model based on the profile information and emotional state, and constructs an integrated model.

[0408] 4. Proposal generation and transmission: The integrated model is used to generate optimal food delivery proposals and transmit them to the user's device.

[0409] 5. Feedback collection and model improvement: The user enters feedback and their emotional state is also recorded, which is sent to the server and used to improve the model.

[0410] Prompt Sentence Examples

[0411] "User Preferences: Chinese food"

[0412] "User's emotional state: Stressed"

[0413] "Proposal requirements: Please propose a quick and delicious Chinese food delivery menu."

[0414] This allows the system to make personalized suggestions that take into account the user's preferences and emotional state, increasing user satisfaction.

[0415] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0416] Step 1:

[0417] The terminal prompts the user to enter authentication information (user ID and password). The entered authentication information is sent to the server. The server compares the received authentication information with the database and determines whether the authentication was successful or not. If the authentication is successful, the user's profile information is retrieved from the database.

[0418] Input: User ID, Password

[0419] Data processing: Verification of authentication information

[0420] Output: User profile information, authentication result (success / failure)

[0421] Step 2:

[0422] The server uses an emotion engine to recognize the user's emotional state based on the acquired profile information. Sensors such as the device's camera and microphone are used to collect emotional data in real time, which is then analyzed by the emotion engine.

[0423] Input: User profile information, real-time emotional data (images, voice)

[0424] Data processing: Analysis of emotional data, recognition of emotional states

[0425] Output: User's emotional state

[0426] Step 3:

[0427] The server selects an appropriate generative AI model based on the user's profile information and emotional state. The server analyzes the profile information and emotional data and selects an appropriate generative AI model, such as "Chinese Food GPT" or "Hamburger GPT."

[0428] Input: User profile information, emotional state

[0429] Data processing: Selection of generative AI model

[0430] Output: A list of selected generative AI models

[0431] Step 4:

[0432] The server combines the selected generative AI models to build an integrated model. For example, it combines the selected "Chinese food GPT" and "Hamburger GPT" to create an integrated model.

[0433] Input: A list of selected generative AI models

[0434] Data processing: Combining generative AI models

[0435] Output: Integrated model

[0436] Step 5:

[0437] The server uses the integrated model to generate food delivery suggestions that reflect the user's preferences and emotional state. The server inputs the prompt sentence into the integrated model to generate suggestions.

[0438] Input: Integrated model, prompt statement

[0439] Data processing: Proposal generation

[0440] Output: Food delivery suggestions

[0441] Step 6:

[0442] The server sends the generated proposals to each user's terminal, which then displays the received proposals on the user's screen.

[0443] Input: Food delivery suggestions

[0444] Data Processing: Submit Proposal

[0445] Output: The proposal displayed on the user's device

[0446] Step 7:

[0447] The user selects from the presented suggestions and inputs feedback, and the terminal transmits the user's feedback and current emotional state to the server.

[0448] Input: User feedback, emotional state

[0449] Data processing: Feedback collection

[0450] Output: Feedback sent to the server

[0451] Step 8:

[0452] The server analyzes the collected feedback and improves the generative AI model and emotion engine. The server analyzes the feedback data and improves the accuracy of future suggestions.

[0453] Input: User feedback, emotional state

[0454] Data processing: analyzing feedback, improving generative AI models and emotion engines

[0455] Output: Improved generative AI models and emotion engines

[0456] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0457] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0458] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0459] [Second embodiment]

[0460] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0461] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0462] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0463] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0464] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0465] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0466] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0467] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0468] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0469] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0470] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0471] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0472] The present invention is a system that integrates the different preferences and needs of multiple users and makes optimal suggestions to the entire group. An embodiment of this system will be described in detail below.

[0473] System configuration and program processing

[0474] The system is mainly implemented by three main components: "server", "terminal", and "user".

[0475] 1. User authentication and information collection

[0476] A terminal is a device that a user uses to access the system, such as a smartphone or a PC. When a user logs in to the system using a terminal, the server authenticates the user. If authentication is successful, the server retrieves each user's profile information from the database. The profile information includes the user's preferences, past selection history, and customization settings.

[0477] 2. Selecting and combining generative AI models

[0478] The server selects the generative AI model that is most suitable for each user based on the acquired profile information. For example, "Italian Restaurant GPT" is selected for Person A, "Japanese Restaurant GPT" for Person B, and "Healthy Restaurant GPT" for Person C. The selected generative AI models are combined by the server to build an integrated model. This integrated model reflects each user's preferences and can perform analysis according to overall needs.

[0479] 3. Generate and send optimal proposals

[0480] The server uses the integrated model to analyze the needs and preferences of the entire group and generate optimal suggestions. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine" and describes in detail the benefits and features of each suggestion. The server then sends the generated suggestions to each user's device, and the device displays the received suggestions to the user. Users can provide feedback on the suggestions, which is then sent back to the server.

[0481] 4. Analyze feedback and improve the model

[0482] The server analyzes the collected feedback and uses it to improve the generative AI model. This process improves the accuracy of the system's suggestions, ensuring that future suggestions are more in line with the user's preferences.

[0483] Specific examples

[0484] For example, suppose three users (A, B, and C) want to choose a restaurant to have dinner together.

[0485] 1. Device: Persons A, B, and C each log in to the app on their own smartphones.

[0486] 2. Server: Retrieves each user's preference information (A prefers "Italian food," B prefers "Japanese food," and C prefers "healthy food") from the database.

[0487] 3. Server: Selects the optimal generative AI model for each user and combines the "Italian Restaurant GPT," "Japanese Restaurant GPT," and "Healthy Restaurant GPT."

[0488] 4. Server: Uses the combined model to search for restaurants that match everyone's preferences and generate multiple candidates.

[0489] 5. Server: Sends the generated proposals to each user's device, which displays them to the user.

[0490] 6. User: Each user enters feedback on the proposal, and the device sends it to the server.

[0491] 7. Server: Analyzes feedback and improves the generative AI model.

[0492] In this way, the system can effectively integrate the different preferences and needs of multiple users and provide optimal recommendations for the entire population.

[0493] The processing flow will be explained below.

[0494] Step 1:

[0495] The terminal displays an authentication screen for the user to access the system. The user enters their own authentication information (user ID and password).

[0496] Step 2:

[0497] The terminal transmits the input authentication information to the server.

[0498] Step 3:

[0499] The server compares the received authentication information with the database to authenticate the user. If authentication is successful, the session begins.

[0500] Step 4:

[0501] The server retrieves the authenticated user's profile information from the database, which includes the user's preferences, past behavior history, and customization settings.

[0502] Step 5:

[0503] The server selects the optimal generative AI model for each user based on the acquired profile information. For example, it might select "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[0504] Step 6:

[0505] The server combines the selected generative AI models to build an integrated model, which can integrate information from each combined generative AI model and analyze the needs and preferences of the entire population.

[0506] Step 7:

[0507] The server uses the integrated model to generate optimal recommendations that reflect the needs and preferences of the entire group, such as generating multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine."

[0508] Step 8:

[0509] The server transmits the generated proposals to each user's terminal.

[0510] Step 9:

[0511] The device displays the received suggestions to the user, including the restaurant name, location, menu details, and ratings.

[0512] Step 10:

[0513] Users select their preferred options from the suggestions and enter feedback, including their satisfaction level, additions, and corrections.

[0514] Step 11:

[0515] The terminal transmits the user's feedback to the server.

[0516] Step 12:

[0517] The server analyzes the received feedback and improves the generative AI model, so that future suggestions will better match the user's preferences.

[0518] In this way, the system effectively integrates the different preferences and needs of multiple users to provide optimal recommendations for the entire group.

[0519] Example 1

[0520] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0521] There is a need for a system that can integrate the different preferences and needs of multiple users to generate optimal proposals for the entire group. However, conventional methods have difficulty analyzing each user's preferences individually, making it difficult to efficiently generate proposals that meet the needs of the entire group. In addition, there is a lack of technology to effectively incorporate user feedback and improve the model. To solve this problem, a collective optimization method using an advanced generative AI model is required.

[0522] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0523] In this invention, the server includes means for authenticating users, means for collecting profile information for each user, means for selecting a generative AI model suitable for each user, means for combining the selected generative AI models to build an integrated model, means for analyzing the needs and preferences of the entire group using the integrated model, means for generating optimal proposals for the entire group, means for transmitting the generated proposals to each user's device, means for collecting user feedback, means for analyzing the collected feedback to improve the generative AI model, means for describing the optimal proposals in detail using prompt sentences, and means for notifying and displaying the proposals. This makes it possible to effectively integrate the preferences and needs of multiple users, generate optimal proposals for the entire group, and further improve the accuracy of the proposals by utilizing user feedback.

[0524] "Means for authenticating users" refers to a mechanism for verifying the user's login information and determining whether the user is legitimate.

[0525] The "means for collecting profile information for each user" is a function for obtaining information such as the user's preferences, past selection history, and customization settings from a database.

[0526] The "means for selecting a generative AI model that suits each user" is a mechanism for identifying the generative AI model that best suits the user's preferences and needs based on collected profile information.

[0527] "Means for combining selected generative AI models to construct an integrated model" refers to a method for integrating generative AI models selected for each user to create a common model that reflects the preferences and needs of multiple users.

[0528] "Means for analyzing needs and preferences across a population using an integrated model" means a process for utilizing an integrated generative AI model to comprehensively assess the needs and preferences of a group of multiple users.

[0529] The "means for generating optimal proposals for the entire group" is a function that creates optimal proposals that satisfy the preferences and needs of the entire group based on the analysis results of the entire group.

[0530] The "means for transmitting the generated proposal to each user's terminal" is a mechanism for distributing the generated proposal to each user's device via a network.

[0531] The "means for collecting user feedback" is a mechanism for collecting opinions and ratings provided by users on suggestions.

[0532] "Means for analyzing collected feedback to improve generative AI models" refers to methods for analyzing feedback collected from users and using the results to improve the accuracy and applicability of generative AI models.

[0533] The "means for describing the optimal proposal in detail using a prompt sentence" is a function for expressing the generated proposal in detail using a prompt sentence including specific sentences and instructions.

[0534] A "means for notifying and displaying suggestions" is a method for informing the user of and visually displaying the generated suggestions.

[0535] This invention is a system that integrates the different preferences and needs of multiple users and makes optimal suggestions to the entire group. How this system is implemented will be described in detail below.

[0536] System configuration

[0537] This system is mainly composed of three main components: "server," "terminal," and "user."

[0538] User authentication and information collection

[0539] A terminal is a device that a user uses to access the system, such as a smartphone or a PC. When a user logs in to the system using a terminal, the login information (user name and password) is sent from the terminal to the server. At this time, the communication is securely protected using the HTTPS protocol.

[0540] The server passes the received login information to the authentication processing module and performs user authentication. If authentication is successful, the server retrieves the user's profile information from the database. This profile information includes the user's preferences, past selection history, and customization settings. The database uses an RDBMS such as MySQL or PostgreSQL.

[0541] Selecting and combining generative AI models

[0542] The server selects the most suitable generative AI model for each user based on the acquired profile information. For example, "Italian Restaurant GPT" is selected for Person A, "Japanese Restaurant GPT" for Person B, and "Healthy Restaurant GPT" for Person C. These generative AI models are trained using training data specialized for each user's specific needs.

[0543] The server combines the selected generative AI models to construct an integrated model that comprehensively reflects the user's preferences. For this purpose, technologies such as reinforcement learning (RL) and ensemble learning may be used.

[0544] Generate and send optimal proposals

[0545] The server uses the integrated model to analyze the needs and preferences of the entire group and generate specific suggestions. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine," and includes detailed information about each suggestion (location, menu, recommended points, etc.). This process uses prompts from the generative AI model.

[0546] Here are some examples of prompts:

[0547] "Person A likes Italian restaurants, Person B likes Japanese food, and Person C likes healthy meals. Please provide restaurant suggestions that would be best for these three people."

[0548] The server sends the generated suggestions to each user's device, which then displays them to the user. The suggestions are displayed visually in rich text and image formats, and push notifications are also used if necessary.

[0549] Collecting and analyzing feedback

[0550] The user provides feedback on the displayed suggestions. The feedback is sent from the device to the server through a dedicated input form. The feedback includes the degree of satisfaction with the suggestions and specific comments.

[0551] The server analyzes this feedback and uses it to improve the generative AI model. The feedback data is fed into the machine learning algorithm, and the generative AI model is retrained to improve the accuracy of future suggestions.

[0552] Specific examples

[0553] For example, suppose three users (A, B, and C) want to choose a restaurant to have dinner together.

[0554] 1. Device: Persons A, B, and C each log in to the app on their own smartphones.

[0555] 2. Terminal: Once the login information is entered, the terminal sends it to the server.

[0556] 3. Server: The server receives the login information and performs authentication. If authentication is successful, it retrieves the profile information from the database.

[0557] 4. Server: Analyzes the profile information and selects "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[0558] 5. Server: Combines the selected generative AI models to create an integrated model.

[0559] 6. Server: Uses the integrated model to generate multiple restaurant options that match everyone's preferences, and generates detailed information using prompts.

[0560] 7. Server: Sends the generated proposals to each user's device.

[0561] 8. Device: The device displays the received suggestions to the user and optionally sends push notifications.

[0562] 9. User: Each user provides feedback on the proposal, which is sent to the server via their device.

[0563] 10. Server: Analyzes feedback and helps improve the generative AI model.

[0564] In this way, the system can effectively integrate the different preferences and needs of multiple users and provide optimal recommendations for the entire population.

[0565] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0566] Program processing flow

[0567] Step 1: User Login

[0568] Step 2: User authentication and profile information retrieval

[0569] Step 3: Selecting a generative AI model

[0570] Step 4: Combining generative AI models

[0571] Step 5: Generate optimal proposals

[0572] Step 6: Submit and view your proposal

[0573] Step 7: Gather feedback

[0574] Step 8: Analyze feedback and improve the generative AI model

[0575] Detailed explanation of each processing step

[0576] Step 1: User Login

[0577] A terminal is a device that users use to access the system. Users access the login screen using a smartphone or computer and enter their username and password. The terminal receives this login information and sends it to the server using the HTTPS protocol.

[0578] Input: Username, Password

[0579] Output: Login information sent to the server

[0580] Step 2: User authentication and profile information retrieval

[0581] The server passes the received login information to the authentication processing module and performs user authentication. If authentication is successful, the server retrieves the user's profile information (preferences, past selection history, customization settings, etc.) from the database. The database uses an RDBMS such as MySQL or PostgreSQL.

[0582] Input: Login information

[0583] Output: Authentication result (success / failure), profile information

[0584] Step 3: Selecting a generative AI model

[0585] The server analyzes the acquired profile information and selects the most suitable generative AI model for each user. For example, "Italian Restaurant GPT" is selected for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C. The selection criteria are based on the user's preferences and past selection history.

[0586] Input: Profile Information

[0587] Output: The selected generative AI model

[0588] Step 4: Combining generative AI models

[0589] The server combines the selected generative AI models to build an integrated model. This integrated model reflects each user's preferences and analyzes the overall needs. Specific methods include ensemble learning and reinforcement learning.

[0590] Input: A selected generative AI model

[0591] Output: Integrated model

[0592] Step 5: Generate optimal proposals

[0593] The server uses the integrated model to analyze the needs and preferences of the entire group and generate specific suggestions. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine," including detailed information (location, menu, recommended features, etc.). This process uses prompts from the generative AI model.

[0594] Input: Integration model

[0595] Output: Proposal

[0596] Step 6: Submit and view your proposal

[0597] The server sends the generated suggestions to each user's device, which displays the received suggestions to the user and, if necessary, sends push notifications. Suggestions are visually displayed in rich text and image formats.

[0598] Input: Proposal

[0599] Output: Proposal data sent to the user's device

[0600] Step 7: Gather feedback

[0601] The user provides feedback on the displayed suggestions. The feedback is sent from the device to the server via a dedicated input form. The feedback includes the degree of satisfaction with the suggestions and specific comments.

[0602] Input: Feedback information

[0603] Output: Feedback data sent to the server

[0604] Step 8: Analyze feedback and improve the generative AI model

[0605] The server analyzes the collected feedback and feeds the feedback data to a machine learning algorithm, which uses it as training data for the generative AI model. This improves the generative AI model so that future suggestions will better match the user's preferences.

[0606] Input: Feedback data

[0607] Output: An improved generative AI model

[0608] In this way, the system can effectively integrate the different preferences and needs of multiple users and provide optimal recommendations for the entire population.

[0609] (Application example 1)

[0610] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0611] Conventional recommendation systems face the challenge of effectively integrating the different preferences and needs of individual users and making optimal recommendations for the entire group. In particular, when multiple users have different food preferences, it is necessary to quickly and accurately recommend restaurants and dishes that will satisfy everyone. Furthermore, there is a lack of a way to continuously improve the system using user feedback, making it difficult to improve the accuracy of the recommendations.

[0612] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0613] In this invention, the server includes means for authenticating users, means for collecting profile information for each user, means for selecting a generative AI model suitable for each user, means for combining the selected generative AI models to build an integrated model, means for analyzing the needs and preferences of the entire group using the integrated model, means for generating optimal proposals for the entire group, means for sending the generated proposals to each user's device, means for collecting user feedback, and means for analyzing the collected feedback to improve the generative AI model. This makes it possible to effectively integrate the different preferences and needs of multiple users, quickly suggest restaurants and dishes that will satisfy everyone, and continuously improve the accuracy of the suggestions by reflecting user feedback.

[0614] "User authentication" is the process of verifying the identity and access rights of a user attempting to access a system and confirming that the user is a legitimate user.

[0615] "Profile information" is data that includes personal information such as each user's preferences, past selection history, and customization settings.

[0616] A "generative AI model" is an AI model that generates appropriate suggestions based on a user's preferences and needs, and is specialized for a specific genre or theme.

[0617] An "integrated model" is a model created by combining individual generative AI models, and is capable of integrating the preferences of multiple users to analyze and make suggestions based on overall needs.

[0618] "The needs and preferences of the entire group" refers to a comprehensive understanding of the different interests, concerns, and desired conditions of multiple users.

[0619] An "optimal suggestion" is one that provides the most satisfying options to the entire group and is generated based on the different preferences and needs of users.

[0620] A "terminal" is a device that a user uses to access the system, such as a smartphone, tablet, or PC.

[0621] "Feedback" refers to data such as ratings, opinions, and comments provided by users regarding proposed options, and is used to improve the system.

[0622] "Improvement" is the process of improving a generative AI model or the entire system based on feedback.

[0623] This invention is a system for integrating the different preferences and needs of multiple users and making optimal suggestions to the entire group. The system is implemented mainly by a "server," "terminals," and "users."

[0624] 1. User authentication and information collection

[0625] A terminal is a device that a user uses to access the system. Examples include smartphones, tablets, and PCs. When a user logs into the system using a terminal, the server authenticates the user. If authentication is successful, the server retrieves each user's profile information from the database. The profile information includes the user's preferences, past selection history, and customization settings.

[0626] 2. Selecting and combining generative AI models

[0627] The server selects the generative AI model that is most suitable for each user based on the acquired profile information. For example, "Italian GPT" is selected for User A, "Japanese Food GPT" for User B, and "Healthy GPT" for User C. The selected generative AI models are combined by the server to build an integrated model. This integrated model reflects each user's preferences and can perform analysis according to overall needs.

[0628] 3. Generate and send optimal proposals

[0629] The server uses the integrated model to analyze the needs and preferences of the entire population and generate optimal suggestions. For example, it generates multiple healthy restaurant options that combine Italian and Japanese cuisine, detailing the benefits and features of each suggestion. The server then sends the suggestions to each user's device, which displays them to the user. Users can provide feedback on the suggestions, which is then sent back to the server.

[0630] 4. Analyze feedback and improve the model

[0631] The server analyzes the collected feedback and uses it to improve the generative AI model. This process improves the accuracy of the system's suggestions, ensuring that future suggestions are more in line with the user's preferences.

[0632] Hardware and Software Used

[0633] Hardware: Smartphones, tablets, computers

[0634] Server: AWS Compute Services, Database (e.g., Amazon RDS)

[0635] Generative AI models: large-scale language models such as OpenAI GPT-3

[0636] Data analysis tools: Python, Pandas, Numpy

[0637] Examples:

[0638] For example, consider a scenario where three users want to have dinner together. If User A prefers Italian food, User B prefers Japanese food, and User C prefers healthy food, the server selects a generative AI model suited to each user. By integrating the selected models, restaurant and food recommendations that satisfy everyone's preferences are generated.

[0639] Example prompt sentence:

[0640] "User A likes Italian food, User B likes Japanese food, and User C prefers healthy food. Please combine these preferences and suggest restaurant candidates that will satisfy all of them."

[0641] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0642] Step 1:

[0643] The server authenticates the user. When a user logs in to the system using a terminal, the terminal sends the authentication information to the server. The server authenticates the user based on the authentication information, and if authentication is successful, returns the user ID.

[0644] Input: User credentials

[0645] Output: authenticated user ID

[0646] Step 2:

[0647] The server retrieves each user's profile information from the database based on the authenticated user ID, including the user's preferences, past selection history, and customization settings.

[0648] Input: authenticated user ID

[0649] Output: User profile information

[0650] Step 3:

[0651] The server selects a generative AI model that suits each user based on the acquired profile information. For example, "Italian GPT" is selected for User A, "Japanese Food GPT" for User B, and "Healthy GPT" for User C.

[0652] Input: User profile information

[0653] Output: A generative AI model suited to each user

[0654] Step 4:

[0655] The server combines the selected generative AI models to build an integrated model, which combines the capabilities of the individual models and is used to analyze the needs and preferences of the entire population.

[0656] Input: A generative AI model tailored to each user

[0657] Output: Integrated model

[0658] Step 5:

[0659] The server uses the integrated model to analyze the needs and preferences of the entire population and generate optimal recommendations, such as a list of healthy restaurants that combine Italian and Japanese cuisine, detailing the benefits and features of each recommendation.

[0660] Input: Integration model

[0661] Output: A list of the best suggestions

[0662] Step 6:

[0663] The server sends the generated proposals to each user's terminal, which then displays the received proposals to the user.

[0664] Input: List of best suggestions

[0665] Output: Proposal displayed on the user's device

[0666] Step 7:

[0667] The user provides feedback on the suggestion, and the terminal transmits the user-entered feedback to the server.

[0668] Input: User feedback

[0669] Output: Feedback sent to the server

[0670] Step 8:

[0671] The server analyzes the collected feedback, which is then used to improve the generative AI model, adjusting it so that future suggestions better match the user's preferences.

[0672] Input: User feedback

[0673] Output: An improved generative AI model

[0674] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0675] This invention is a system that authenticates users, collects profile information, selects and combines appropriate generative AI models, generates and shares suggestions, collects user feedback, and improves the generative AI models based on the feedback. Furthermore, by combining it with an emotion engine that recognizes user emotions, it provides more accurate and personalized suggestions.

[0676] System configuration and program processing

[0677] This system consists of the following main components: an "emotion engine" that recognizes the user's emotions and reflects them in feedback, a "server" that manages the user's profile information, and a "terminal."

[0678] 1. User authentication and information collection

[0679] The terminal displays an authentication screen for the user to access the system. The user enters authentication information (user ID and password) and sends the authentication information to the server. The server verifies the authentication information and, if authentication is successful, retrieves the user's profile information from the database.

[0680] 2. Selecting and combining generative AI models

[0681] The server selects the optimal generative AI model for each user based on their profile information, and also takes into account the user's emotional state using an emotion engine. For example, it selects "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C. The user's emotional state is reflected in the profile information, making it possible to select a more accurate generative AI model. The selected generative AI models are then combined to build an integrated model.

[0682] 3. Generate and send optimal proposals

[0683] The server uses the integrated model to generate optimal suggestions that reflect the needs, preferences, and emotional state of the entire group. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine." The generated suggestions are sent to each user's device, which then displays the received suggestions to the user.

[0684] 4. User Emotion Recognition Feedback and Model Improvement

[0685] The user selects their preferred option from the suggested options and enters their feedback. The feedback includes not only their satisfaction level, desired additions, and corrections, but also reflects the user's emotional state as recognized by the emotion engine. The device sends the user's feedback to the server, which then analyzes the received feedback.

[0686] 5. Analyzing feedback and improving generative AI models

[0687] The server analyzes the feedback and continuously improves the generative AI model and emotion engine, thereby improving the accuracy of the system's suggestions so that future suggestions are more in line with the user's preferences and emotions.

[0688] Specific examples

[0689] For example, consider the case where three users (A, B, and C) are choosing a restaurant for dinner.

[0690] 1. Device: Persons A, B, and C log in to the app on their respective devices.

[0691] 2. Server: Each user's preference information (A-san prefers "Italian food," B-san prefers "Japanese food," and C-san prefers "healthy food") and their emotional state as recognized by the emotion engine at that time are obtained from the database.

[0692] 3. Server: Selects the optimal generative AI model for each user and combines the "Italian Restaurant GPT," "Japanese Restaurant GPT," and "Healthy Restaurant GPT" with the emotion engine.

[0693] 4. Server: Using the integrated model, it searches for restaurants that match the preferences and emotions of all users and generates multiple suggestions.

[0694] 5. Server: Sends the generated proposals to each user's device.

[0695] 6. Terminal: Displays the received suggestions to the user.

[0696] 7. User: Enters feedback on the suggestions and the emotional state is also reflected by the emotion engine.

[0697] 8. Device: Sends feedback to the server.

[0698] 9. Server: Analyzes feedback and improves generative AI models and emotion engines.

[0699] This system can effectively integrate the different preferences and emotions of multiple users and provide optimal suggestions to the entire group.

[0700] The processing flow will be explained below.

[0701] Step 1:

[0702] The terminal displays an authentication screen for the user to access the system. The user enters authentication information (user ID and password).

[0703] Step 2:

[0704] The terminal transmits the input authentication information to the server.

[0705] Step 3:

[0706] The server compares the received authentication information with the database to authenticate the user. If authentication is successful, the session begins.

[0707] Step 4:

[0708] The server retrieves profile information of the authenticated user from the database, including the user's preferences, past behavior history, customization settings, and emotional state data from the emotion engine.

[0709] Step 5:

[0710] The server selects the optimal generative AI model for each user based on the acquired profile information. Taking into account the user's emotional state, for example, it might select "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[0711] Step 6:

[0712] The server combines the selected generative AI models to build an integrated model, which integrates information from the combined generative AI models with emotional data from the emotion engine to analyze the needs, preferences, and emotions of the entire population.

[0713] Step 7:

[0714] The server uses the integrated model to generate optimal recommendations that reflect the needs, preferences, and emotional state of the entire population, such as generating multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine."

[0715] Step 8:

[0716] The server transmits the generated proposals to each user's terminal.

[0717] Step 9:

[0718] The device displays the received suggestions to the user, including the restaurant name, location, menu details, and ratings.

[0719] Step 10:

[0720] Users select their preferred option from the options presented and enter their feedback, which can include their level of satisfaction, what they would like to add, or what needs to be corrected. The emotion engine also analyzes the user's emotional state in real time and reflects this in the feedback.

[0721] Step 11:

[0722] The terminal transmits the user's feedback and emotion data to the server.

[0723] Step 12:

[0724] The server analyzes the received feedback and emotion data to improve the generative AI model and emotion engine, so that future suggestions will better match the user's preferences and emotions.

[0725] Specific examples

[0726] Consider three users (A, B, and C) having dinner together.

[0727] Step 1:

[0728] Device: Persons A, B, and C each log in to the app on their own smartphones.

[0729] Step 2:

[0730] Terminal: Sends the entered authentication information (user ID and password) to the server.

[0731] Step 3:

[0732] Server: Compares the received authentication information with the database and performs authentication. If authentication is successful, a session begins.

[0733] Step 4:

[0734] Server: Retrieves profile information for each authenticated user from a database, including preferences, past behavioral history, customization settings, and current emotional state.

[0735] Step 5:

[0736] Server: Based on the profile information and emotional state, select "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[0737] Step 6:

[0738] Server: Combines selected generative AI models to build an integrated model that integrates everyone's preferences and emotions.

[0739] Step 7:

[0740] Server: Using the integrated model, it generates multiple restaurant candidates that take into account everyone's preferences and emotions. For example, it generates candidates for "healthy restaurants that combine Italian and Japanese cuisine."

[0741] Step 8:

[0742] Server: Sends the generated proposals to the devices of A, B, and C.

[0743] Step 9:

[0744] Terminal: Displays the received suggestions to the user, including detailed restaurant information.

[0745] Step 10:

[0746] User: Selects the preferred option from the suggested options and enters feedback. The emotion engine analyzes the user's emotional state in real time and reflects that in the feedback.

[0747] Step 11:

[0748] Terminal: Sends feedback and emotion data to the server.

[0749] Step 12:

[0750] Server: Analyzes feedback and sentiment data to improve generative AI models and sentiment engines.

[0751] In this way, the system can effectively integrate the different preferences and emotions of multiple users and provide optimal suggestions for the entire population.

[0752] Example 2

[0753] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0754] Conventional suggestion systems make suggestions based solely on the user's preferences, resulting in a lack of personalized suggestions that reflect the user's real-time emotional state. Furthermore, the selection and integration of generative AI models was not centralized, making it difficult to improve the quality of the generated suggestions. This resulted in a decline in user satisfaction and an inability to fully utilize the system's effectiveness.

[0755] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0756] In this invention, the server includes means for authenticating users, means for collecting profile information for each user, means for selecting a generative AI model that suits each user, means for recognizing the user's emotional state, means for combining the selected generative AI model with an emotion engine to build an integrated model, means for analyzing the needs, preferences, and emotional state of the entire group using the integrated model, means for generating optimal proposals for the entire group, means for transmitting the generated proposals to each user's device, means for collecting user feedback and emotion recognition results, and means for analyzing the collected feedback and emotion recognition results to improve the generative AI model and emotion engine, thereby enabling more accurate and personalized proposals that reflect the user's real-time emotional state.

[0757] "Means of user authentication" refers to the method of having a user enter the necessary information (such as a user ID and password) to verify that they are a legitimate user when they access a system.

[0758] "Profile information" refers to detailed information about each user, such as personal data, preference information, and history information about the user.

[0759] The "means for selecting a generative AI model" is a method for selecting the optimal generative AI model based on the user's profile information and emotional state.

[0760] "Means for recognizing emotional state" refers to technologies (such as voice analysis and facial expression recognition) used to analyze a user's emotions in real time and identify their state.

[0761] "Means for constructing an integrated model by combining a generative AI model and an emotion engine" refers to a method for linking a selected generative AI model with an emotion engine to construct an integrated model.

[0762] The "means for analyzing the needs, preferences, and emotional states of an entire population" is a method for analyzing the needs, preferences, and emotional states of multiple users using an integrated model.

[0763] The "means for generating optimal proposals" is a method for creating the most suitable proposals for the user based on the analysis results.

[0764] The "means for transmitting the generated proposal to the terminal of each user" is a method for distributing the generated proposal to the terminal used by the user.

[0765] "Means for collecting feedback and emotion recognition results" refers to a method for collecting user evaluations and additional opinions on suggestions, as well as emotion analysis results from the emotion engine.

[0766] "Means for analyzing feedback and emotion recognition results to improve generative AI models and emotion engines" refers to methods for adjusting and improving generative AI models and emotion engines based on collected feedback and emotion data.

[0767] This invention is a system that authenticates users, collects profile information, selects and combines generative AI models, generates and shares suggestions, collects user feedback, and improves the generative AI models based on the feedback. Furthermore, by combining it with an emotion engine that recognizes user emotions, it provides more accurate and personalized suggestions.

[0768] The main components of the system are:

[0769] "Emotion Engine" that recognizes user emotions and reflects them in feedback

[0770] "Server" and "Terminal" that manage user profile information

[0771] 1. User authentication and information collection

[0772] The terminal displays an authentication screen for the user to access the system. The terminal collects the user ID and password entered by the user and sends this information to the server. The server verifies the received authentication information and retrieves the corresponding user profile information from a database. At this time, encrypted communication such as SSL is used for authentication to ensure the security of the information.

[0773] 2. Selecting and combining generative AI models

[0774] The server analyzes the acquired profile information. Data analysis tools such as Python are used for the analysis to select the optimal generative AI model based on the user's preferences and past behavioral history. At this time, the emotion engine analyzes the user's emotional state in real time, and this information is also added to the profile. The selected generative AI model and the emotion engine are then combined to create an integrated model. This integrated model is realized using machine learning frameworks such as TensorFlow and PyTorch.

[0775] 3. Generate and send optimal proposals

[0776] The server uses the integrated generative AI model to generate optimal suggestions based on the user's preferences and emotional state. For example, if a user prefers Italian restaurants with a relaxed atmosphere, the server generates multiple suggestions that reflect that information. The generated suggestions are sent to each user's device via a REST API. The device then displays the received suggestions on its screen, allowing the user to easily access them.

[0777] 4. User Emotion Recognition Feedback and Model Improvement

[0778] The user selects the preferred suggestion from the displayed suggestions and enters feedback regarding satisfaction, additions, and corrections. Specifically, the user uses a feedback form or rating slider. Furthermore, the emotion engine analyzes the user's emotional state and includes this information in the feedback. The device then sends this feedback information to the server.

[0779] 5. Analyzing feedback and improving generative AI models

[0780] The server performs a detailed analysis of the received feedback, using natural language processing and data mining techniques to extract patterns from the collected data. Based on the results, the generative AI model and emotion engine are refined. This continuous improvement ensures that future suggestions are more in line with the user's preferences and emotions.

[0781] Specific examples

[0782] For example, consider the case where three users (user A, user B, and user C) are choosing a restaurant for dinner.

[0783] 1. Device: User A, User B, and User C log in to the app on their respective devices.

[0784] 2. Server: Obtains each user's preference information (User A is "Italian food," User B is "Japanese food," and User C is "healthy food") and the emotional state recognized by the emotion engine at that time from the database.

[0785] 3. Server: Selects the optimal generative AI model for each user, for example, "Italian Restaurant GPT," "Japanese Restaurant GPT," and "Healthy Restaurant GPT," and combines them with an emotion engine.

[0786] 4. Server: Using the integrated model, it searches for restaurants that match the preferences and emotions of all users and generates multiple suggestions.

[0787] 5. Server: Sends the generated proposals to each user's device.

[0788] 6. Terminal: Displays the received suggestions to the user.

[0789] 7. User: Enters feedback on the suggestions and the emotional state is also reflected by the emotion engine.

[0790] 8. Device: Sends feedback to the server.

[0791] 9. Server: Analyzes feedback and improves generative AI models and emotion engines.

[0792] Prompt Sentence Examples

[0793] "User A currently likes Italian food, so please display several Italian restaurant suggestions for tonight's dinner. Please also consider the user's emotional state and prioritize restaurants with a relaxing atmosphere."

[0794] This system can effectively integrate the different preferences and emotions of multiple users and provide optimal suggestions to the entire group.

[0795] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0796] Step 1: User authentication and information gathering

[0797] The terminal displays an authentication screen for the user to access the system. The user enters their user ID and password and clicks the "Login button." Based on this input, the terminal sends the user ID and password to the server. The server searches for user information in a database, and if the authentication information is correct, retrieves profile information. At this time, encrypted communication such as SSL is used to ensure the security of the information. If authentication is successful, the user's profile information is sent from the server to the terminal.

[0798] Step 2: Selecting and combining generative AI models

[0799] The server analyzes the acquired profile information. This analysis uses data analysis tools such as Python to select the optimal generative AI model based on the user's preferences and past behavioral history. The emotion engine also becomes active, analyzing the user's real-time emotional state and adding this information to the profile. For example, if the user is looking for a "relaxed atmosphere," a generative AI model that reflects this is selected. The selected generative AI model is integrated with the emotion engine to build an integrated model. The input data is the profile information and emotional state, and the output is the generative AI model and the integrated model.

[0800] Step 3: Generate and submit optimal proposals

[0801] The server uses the integrated generative AI model to generate optimal suggestions based on the user's preferences and emotional state. For example, it generates multiple candidates for "relaxing Italian restaurants." The generative AI model uses data processing and machine learning algorithms to generate suggestions that match the user's preferences and emotions. The generated suggestions are sent to each user's device via a REST API. The input data is the integrated model, and the output is the generated suggestions. The device displays the received suggestions to the user.

[0802] Step 4: User emotion recognition feedback and model improvement

[0803] The user selects the suggestion they like best from the ones displayed and enters feedback regarding their satisfaction, additions they would like to make, and any corrections they would like to make. Specifically, they use a feedback form or a rating slider. Furthermore, the emotion engine analyzes the user's emotional state and includes this information in the feedback. For example, it may recognize from facial expression analysis that the user is "very satisfied" with the suggestion. The device then sends this feedback information to the server. The input data is the user's feedback and emotional state, and the output is the sent feedback information.

[0804] Step 5: Analyze feedback and improve the generative AI model

[0805] The server performs a detailed analysis of the received feedback. This analysis uses natural language processing and data mining techniques to extract patterns from the collected data. Based on the results, the generative AI model and emotion engine are improved. For example, the reason why a particular suggestion was not liked can be analyzed and the algorithm adjusted. This continuous improvement ensures that future suggestions will better match the user's preferences and emotions. The input data is feedback information and emotional state, and the output is an improved generative AI model and emotion engine.

[0806] (Application example 2)

[0807] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0808] Conventional recommendation systems have a problem in that they make suggestions based solely on the user's preference information. In particular, if the user's emotional state is not taken into account, the accuracy of the suggestions and user satisfaction may decrease. Furthermore, it is not easy to address the preferences and needs of an entire group, making it difficult to make appropriate suggestions to individual users. Therefore, there is a need for a system that takes into account the user's preference information and emotional state in an integrated manner and provides more personalized and appropriate suggestions.

[0809] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0810] In this invention, the server includes means for utilizing an emotion engine to reflect the user's emotional state, means for selecting a generative AI model suited to each user, and means for combining the selected generative AI models to construct an integrated model. This allows for the generation of suggestions that take into consideration the user's preference information and emotional state in an integrated manner, thereby improving user satisfaction.

[0811] "Means for authenticating users" refers to a function that allows a user to input authentication information required to access the system and verify that information.

[0812] The "means for collecting profile information of each user" is a function for collecting personal information, preference information, and history data provided by the user.

[0813] "Means for selecting a generative AI model that suits each user" is a function that selects the optimal generative AI model based on the user's profile information and emotional state.

[0814] "Means for combining selected generative AI models to construct an integrated model" refers to a function for combining multiple generative AI models to construct a single integrated model.

[0815] "Means for analyzing the needs and preferences of an entire group using an integrated model" is a function that utilizes an integrated model to analyze the needs and preferences of multiple users.

[0816] "Means for utilizing the emotion engine" refers to a function that recognizes the user's emotional state from facial expressions, voice, etc., and captures this as data.

[0817] The "means for generating optimal proposals" is a function that generates the most suitable proposals for the user based on all collected data.

[0818] The "means for transmitting the generated proposal to each user's terminal" is a function for transmitting the generated proposal from the server to the user's terminal.

[0819] The "means for collecting user feedback" is a function for obtaining reactions and evaluations made by users to suggestions.

[0820] "Means for analyzing collected feedback to improve the generative AI model and emotion engine" refers to a function for analyzing feedback obtained from users and continuously improving the generative AI model and emotion engine based on that feedback.

[0821] MODE FOR CARRYING OUT THE INVENTION

[0822] System Overview

[0823] This invention is a system that makes optimal suggestions based on a user's preferences and emotional state. The system consists of the following main components: an "emotion engine" that recognizes the user's emotions and reflects them in feedback, a "server" that manages the user's profile information, and a "terminal." Specifically, the system performs user authentication, profile collection, selection and integration of appropriate generative AI models, generation and sharing of suggestions, feedback collection, and model improvement based on the feedback.

[0824] Program processing overview

[0825] The program for this system is implemented as an application installed on a smartphone or smart glasses. Below, we will explain the processing overview of each component.

[0826] User authentication and information collection

[0827] A user logs in to the system using a smartphone or smart glasses. The device sends user authentication information to the server, which then verifies the authentication information. If authentication is successful, the server retrieves the user's profile information from the database.

[0828] Selecting and combining generative AI models

[0829] The server selects the optimal generative AI model based on the user's profile information and the emotion data recognized by the emotion engine. For example, it builds an integrated model by combining multiple generative AI models, such as "Chinese cuisine GPT," "Hamburger GPT," and "Vegan cuisine GPT."

[0830] Generate and send optimal proposals

[0831] The server uses the integrated model to generate optimal food delivery suggestions that reflect the user's preferences and emotional state. The suggestions are then sent to the user's device, which then displays the suggestions to the user.

[0832] User emotion recognition feedback and model improvement

[0833] The user selects from the presented suggestions and enters feedback. The emotion engine also recognizes the user's emotional state at the time of the feedback, and the device sends this feedback to the server. The server analyzes the received feedback and improves the generative AI model and emotion engine.

[0834] Hardware and software used

[0835] Hardware: Smartphones, smart glasses, servers

[0836] Software: Emotion recognition engine (EmotionEngine), AI model selection and combination library (ai_model_selector), database

[0837] Specific examples

[0838] For example, if a user is considering ordering dinner delivery, the system might:

[0839] 1. Authentication and profile information acquisition: The user logs in to the app on their smartphone, their authentication information is sent to the server, and their profile information is acquired.

[0840] 2. Recognizing emotional state: The emotion engine recognizes the user's emotional state, and the server receives the emotion data.

[0841] 3. Selection of generative AI model: The server selects the optimal generative AI model based on the profile information and emotional state, and constructs an integrated model.

[0842] 4. Proposal generation and transmission: The integrated model is used to generate optimal food delivery proposals and transmit them to the user's device.

[0843] 5. Feedback collection and model improvement: The user enters feedback and their emotional state is also recorded, which is sent to the server and used to improve the model.

[0844] Prompt Sentence Examples

[0845] "User Preferences: Chinese food"

[0846] "User's emotional state: Stressed"

[0847] "Proposal requirements: Please propose a quick and delicious Chinese food delivery menu."

[0848] This allows the system to make personalized suggestions that take into account the user's preferences and emotional state, increasing user satisfaction.

[0849] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0850] Step 1:

[0851] The terminal prompts the user to enter authentication information (user ID and password). The entered authentication information is sent to the server. The server compares the received authentication information with the database and determines whether the authentication was successful or not. If the authentication is successful, the user's profile information is retrieved from the database.

[0852] Input: User ID, Password

[0853] Data processing: Verification of authentication information

[0854] Output: User profile information, authentication result (success / failure)

[0855] Step 2:

[0856] The server uses an emotion engine to recognize the user's emotional state based on the acquired profile information. Sensors such as the device's camera and microphone are used to collect emotional data in real time, which is then analyzed by the emotion engine.

[0857] Input: User profile information, real-time emotional data (images, voice)

[0858] Data processing: Analysis of emotional data, recognition of emotional states

[0859] Output: User's emotional state

[0860] Step 3:

[0861] The server selects an appropriate generative AI model based on the user's profile information and emotional state. The server analyzes the profile information and emotional data and selects an appropriate generative AI model, such as "Chinese Food GPT" or "Hamburger GPT."

[0862] Input: User profile information, emotional state

[0863] Data processing: Selection of generative AI model

[0864] Output: A list of selected generative AI models

[0865] Step 4:

[0866] The server combines the selected generative AI models to build an integrated model. For example, it combines the selected "Chinese food GPT" and "Hamburger GPT" to create an integrated model.

[0867] Input: A list of selected generative AI models

[0868] Data processing: Combining generative AI models

[0869] Output: Integrated model

[0870] Step 5:

[0871] The server uses the integrated model to generate food delivery suggestions that reflect the user's preferences and emotional state. The server inputs the prompt sentence into the integrated model to generate suggestions.

[0872] Input: Integrated model, prompt statement

[0873] Data processing: Proposal generation

[0874] Output: Food delivery suggestions

[0875] Step 6:

[0876] The server sends the generated proposals to each user's terminal, which then displays the received proposals on the user's screen.

[0877] Input: Food delivery suggestions

[0878] Data Processing: Submit Proposal

[0879] Output: The proposal displayed on the user's device

[0880] Step 7:

[0881] The user selects from the presented suggestions and inputs feedback, and the terminal transmits the user's feedback and current emotional state to the server.

[0882] Input: User feedback, emotional state

[0883] Data processing: Feedback collection

[0884] Output: Feedback sent to the server

[0885] Step 8:

[0886] The server analyzes the collected feedback and improves the generative AI model and emotion engine. The server analyzes the feedback data and improves the accuracy of future suggestions.

[0887] Input: User feedback, emotional state

[0888] Data processing: analyzing feedback, improving generative AI models and emotion engines

[0889] Output: Improved generative AI models and emotion engines

[0890] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0891] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0892] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0893] [Third embodiment]

[0894] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0895] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0896] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0897] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0898] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0899] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0900] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0901] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0902] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0903] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0904] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0905] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0906] The present invention is a system that integrates the different preferences and needs of multiple users and makes optimal suggestions to the entire group. An embodiment of this system will be described in detail below.

[0907] System configuration and program processing

[0908] The system is mainly implemented by three main components: "server", "terminal", and "user".

[0909] 1. User authentication and information collection

[0910] A terminal is a device that a user uses to access the system, such as a smartphone or a PC. When a user logs in to the system using a terminal, the server authenticates the user. If authentication is successful, the server retrieves each user's profile information from the database. The profile information includes the user's preferences, past selection history, and customization settings.

[0911] 2. Selecting and combining generative AI models

[0912] The server selects the generative AI model that is most suitable for each user based on the acquired profile information. For example, "Italian Restaurant GPT" is selected for Person A, "Japanese Restaurant GPT" for Person B, and "Healthy Restaurant GPT" for Person C. The selected generative AI models are combined by the server to build an integrated model. This integrated model reflects each user's preferences and can perform analysis according to overall needs.

[0913] 3. Generate and send optimal proposals

[0914] The server uses the integrated model to analyze the needs and preferences of the entire group and generate optimal suggestions. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine" and describes in detail the benefits and features of each suggestion. The server then sends the generated suggestions to each user's device, and the device displays the received suggestions to the user. Users can provide feedback on the suggestions, which is then sent back to the server.

[0915] 4. Analyze feedback and improve the model

[0916] The server analyzes the collected feedback and uses it to improve the generative AI model. This process improves the accuracy of the system's suggestions, ensuring that future suggestions are more in line with the user's preferences.

[0917] Specific examples

[0918] For example, suppose three users (A, B, and C) want to choose a restaurant to have dinner together.

[0919] 1. Device: Persons A, B, and C each log in to the app on their own smartphones.

[0920] 2. Server: Retrieves each user's preference information (A prefers "Italian food," B prefers "Japanese food," and C prefers "healthy food") from the database.

[0921] 3. Server: Selects the optimal generative AI model for each user and combines the "Italian Restaurant GPT," "Japanese Restaurant GPT," and "Healthy Restaurant GPT."

[0922] 4. Server: Uses the combined model to search for restaurants that match everyone's preferences and generate multiple candidates.

[0923] 5. Server: Sends the generated proposals to each user's device, which displays them to the user.

[0924] 6. User: Each user enters feedback on the proposal, and the device sends it to the server.

[0925] 7. Server: Analyzes feedback and improves the generative AI model.

[0926] In this way, the system can effectively integrate the different preferences and needs of multiple users and provide optimal recommendations for the entire population.

[0927] The processing flow will be explained below.

[0928] Step 1:

[0929] The terminal displays an authentication screen for the user to access the system. The user enters their own authentication information (user ID and password).

[0930] Step 2:

[0931] The terminal transmits the input authentication information to the server.

[0932] Step 3:

[0933] The server compares the received authentication information with the database to authenticate the user. If authentication is successful, the session begins.

[0934] Step 4:

[0935] The server retrieves the authenticated user's profile information from the database, which includes the user's preferences, past behavior history, and customization settings.

[0936] Step 5:

[0937] The server selects the optimal generative AI model for each user based on the acquired profile information. For example, it might select "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[0938] Step 6:

[0939] The server combines the selected generative AI models to build an integrated model, which can integrate information from each combined generative AI model and analyze the needs and preferences of the entire population.

[0940] Step 7:

[0941] The server uses the integrated model to generate optimal recommendations that reflect the needs and preferences of the entire group, such as generating multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine."

[0942] Step 8:

[0943] The server transmits the generated proposals to each user's terminal.

[0944] Step 9:

[0945] The device displays the received suggestions to the user, including the restaurant name, location, menu details, and ratings.

[0946] Step 10:

[0947] Users select their preferred options from the suggestions and enter feedback, including their satisfaction level, additions, and corrections.

[0948] Step 11:

[0949] The terminal transmits the user's feedback to the server.

[0950] Step 12:

[0951] The server analyzes the received feedback and improves the generative AI model, so that future suggestions will better match the user's preferences.

[0952] In this way, the system effectively integrates the different preferences and needs of multiple users to provide optimal recommendations for the entire group.

[0953] Example 1

[0954] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0955] There is a need for a system that can integrate the different preferences and needs of multiple users to generate optimal proposals for the entire group. However, conventional methods have difficulty analyzing each user's preferences individually, making it difficult to efficiently generate proposals that meet the needs of the entire group. In addition, there is a lack of technology to effectively incorporate user feedback and improve the model. To solve this problem, a collective optimization method using an advanced generative AI model is required.

[0956] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0957] In this invention, the server includes means for authenticating users, means for collecting profile information for each user, means for selecting a generative AI model suitable for each user, means for combining the selected generative AI models to build an integrated model, means for analyzing the needs and preferences of the entire group using the integrated model, means for generating optimal proposals for the entire group, means for transmitting the generated proposals to each user's device, means for collecting user feedback, means for analyzing the collected feedback to improve the generative AI model, means for describing the optimal proposals in detail using prompt sentences, and means for notifying and displaying the proposals. This makes it possible to effectively integrate the preferences and needs of multiple users, generate optimal proposals for the entire group, and further improve the accuracy of the proposals by utilizing user feedback.

[0958] "Means for authenticating users" refers to a mechanism for verifying the user's login information and determining whether the user is legitimate.

[0959] The "means for collecting profile information for each user" is a function for obtaining information such as the user's preferences, past selection history, and customization settings from a database.

[0960] The "means for selecting a generative AI model that suits each user" is a mechanism for identifying the generative AI model that best suits the user's preferences and needs based on collected profile information.

[0961] "Means for combining selected generative AI models to construct an integrated model" refers to a method for integrating generative AI models selected for each user to create a common model that reflects the preferences and needs of multiple users.

[0962] "Means for analyzing needs and preferences across a population using an integrated model" means a process for utilizing an integrated generative AI model to comprehensively assess the needs and preferences of a group of multiple users.

[0963] The "means for generating optimal proposals for the entire group" is a function that creates optimal proposals that satisfy the preferences and needs of the entire group based on the analysis results of the entire group.

[0964] The "means for transmitting the generated proposal to each user's terminal" is a mechanism for distributing the generated proposal to each user's device via a network.

[0965] The "means for collecting user feedback" is a mechanism for collecting opinions and ratings provided by users on suggestions.

[0966] "Means for analyzing collected feedback to improve generative AI models" refers to methods for analyzing feedback collected from users and using the results to improve the accuracy and applicability of generative AI models.

[0967] The "means for describing the optimal proposal in detail using a prompt sentence" is a function for expressing the generated proposal in detail using a prompt sentence including specific sentences and instructions.

[0968] A "means for notifying and displaying suggestions" is a method for informing the user of and visually displaying the generated suggestions.

[0969] This invention is a system that integrates the different preferences and needs of multiple users and makes optimal suggestions to the entire group. How this system is implemented will be described in detail below.

[0970] System configuration

[0971] This system is mainly composed of three main components: "server," "terminal," and "user."

[0972] User authentication and information collection

[0973] A terminal is a device that a user uses to access the system, such as a smartphone or a PC. When a user logs in to the system using a terminal, the login information (user name and password) is sent from the terminal to the server. At this time, the communication is securely protected using the HTTPS protocol.

[0974] The server passes the received login information to the authentication processing module and performs user authentication. If authentication is successful, the server retrieves the user's profile information from the database. This profile information includes the user's preferences, past selection history, and customization settings. The database uses an RDBMS such as MySQL or PostgreSQL.

[0975] Selecting and combining generative AI models

[0976] The server selects the most suitable generative AI model for each user based on the acquired profile information. For example, "Italian Restaurant GPT" is selected for Person A, "Japanese Restaurant GPT" for Person B, and "Healthy Restaurant GPT" for Person C. These generative AI models are trained using training data specialized for each user's specific needs.

[0977] The server combines the selected generative AI models to construct an integrated model that comprehensively reflects the user's preferences. For this purpose, technologies such as reinforcement learning (RL) and ensemble learning may be used.

[0978] Generate and send optimal proposals

[0979] The server uses the integrated model to analyze the needs and preferences of the entire group and generate specific suggestions. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine," and includes detailed information about each suggestion (location, menu, recommended points, etc.). This process uses prompts from the generative AI model.

[0980] Here are some examples of prompts:

[0981] "Person A likes Italian restaurants, Person B likes Japanese food, and Person C likes healthy meals. Please provide restaurant suggestions that would be best for these three people."

[0982] The server sends the generated suggestions to each user's device, which then displays them to the user. The suggestions are displayed visually in rich text and image formats, and push notifications are also used if necessary.

[0983] Collecting and analyzing feedback

[0984] The user provides feedback on the displayed suggestions. The feedback is sent from the device to the server through a dedicated input form. The feedback includes the degree of satisfaction with the suggestions and specific comments.

[0985] The server analyzes this feedback and uses it to improve the generative AI model. The feedback data is fed into the machine learning algorithm, and the generative AI model is retrained to improve the accuracy of future suggestions.

[0986] Specific examples

[0987] For example, suppose three users (A, B, and C) want to choose a restaurant to have dinner together.

[0988] 1. Device: Persons A, B, and C each log in to the app on their own smartphones.

[0989] 2. Terminal: Once the login information is entered, the terminal sends it to the server.

[0990] 3. Server: The server receives the login information and performs authentication. If authentication is successful, it retrieves the profile information from the database.

[0991] 4. Server: Analyzes the profile information and selects "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[0992] 5. Server: Combines the selected generative AI models to create an integrated model.

[0993] 6. Server: Uses the integrated model to generate multiple restaurant options that match everyone's preferences, and generates detailed information using prompts.

[0994] 7. Server: Sends the generated proposals to each user's device.

[0995] 8. Device: The device displays the received suggestions to the user and optionally sends push notifications.

[0996] 9. User: Each user provides feedback on the proposal, which is sent to the server via their device.

[0997] 10. Server: Analyzes feedback and helps improve the generative AI model.

[0998] In this way, the system can effectively integrate the different preferences and needs of multiple users and provide optimal recommendations for the entire population.

[0999] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1000] Program processing flow

[1001] Step 1: User Login

[1002] Step 2: User authentication and profile information retrieval

[1003] Step 3: Selecting a generative AI model

[1004] Step 4: Combining generative AI models

[1005] Step 5: Generate optimal proposals

[1006] Step 6: Submit and view your proposal

[1007] Step 7: Gather feedback

[1008] Step 8: Analyze feedback and improve the generative AI model

[1009] Detailed explanation of each processing step

[1010] Step 1: User Login

[1011] A terminal is a device that users use to access the system. Users access the login screen using a smartphone or computer and enter their username and password. The terminal receives this login information and sends it to the server using the HTTPS protocol.

[1012] Input: Username, Password

[1013] Output: Login information sent to the server

[1014] Step 2: User authentication and profile information retrieval

[1015] The server passes the received login information to the authentication processing module and performs user authentication. If authentication is successful, the server retrieves the user's profile information (preferences, past selection history, customization settings, etc.) from the database. The database uses an RDBMS such as MySQL or PostgreSQL.

[1016] Input: Login information

[1017] Output: Authentication result (success / failure), profile information

[1018] Step 3: Selecting a generative AI model

[1019] The server analyzes the acquired profile information and selects the most suitable generative AI model for each user. For example, "Italian Restaurant GPT" is selected for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C. The selection criteria are based on the user's preferences and past selection history.

[1020] Input: Profile Information

[1021] Output: The selected generative AI model

[1022] Step 4: Combining generative AI models

[1023] The server combines the selected generative AI models to build an integrated model. This integrated model reflects each user's preferences and analyzes the overall needs. Specific methods include ensemble learning and reinforcement learning.

[1024] Input: A selected generative AI model

[1025] Output: Integrated model

[1026] Step 5: Generate optimal proposals

[1027] The server uses the integrated model to analyze the needs and preferences of the entire group and generate specific suggestions. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine," including detailed information (location, menu, recommended features, etc.). This process uses prompts from the generative AI model.

[1028] Input: Integration model

[1029] Output: Proposal

[1030] Step 6: Submit and view your proposal

[1031] The server sends the generated suggestions to each user's device, which displays the received suggestions to the user and, if necessary, sends push notifications. Suggestions are visually displayed in rich text and image formats.

[1032] Input: Proposal

[1033] Output: Proposal data sent to the user's device

[1034] Step 7: Gather feedback

[1035] The user provides feedback on the displayed suggestions. The feedback is sent from the device to the server via a dedicated input form. The feedback includes the degree of satisfaction with the suggestions and specific comments.

[1036] Input: Feedback information

[1037] Output: Feedback data sent to the server

[1038] Step 8: Analyze feedback and improve the generative AI model

[1039] The server analyzes the collected feedback and feeds the feedback data to a machine learning algorithm, which uses it as training data for the generative AI model. This improves the generative AI model so that future suggestions will better match the user's preferences.

[1040] Input: Feedback data

[1041] Output: An improved generative AI model

[1042] In this way, the system can effectively integrate the different preferences and needs of multiple users and provide optimal recommendations for the entire population.

[1043] (Application example 1)

[1044] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1045] Conventional recommendation systems face the challenge of effectively integrating the different preferences and needs of individual users and making optimal recommendations for the entire group. In particular, when multiple users have different food preferences, it is necessary to quickly and accurately recommend restaurants and dishes that will satisfy everyone. Furthermore, there is a lack of a way to continuously improve the system using user feedback, making it difficult to improve the accuracy of the recommendations.

[1046] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1047] In this invention, the server includes means for authenticating users, means for collecting profile information for each user, means for selecting a generative AI model suitable for each user, means for combining the selected generative AI models to build an integrated model, means for analyzing the needs and preferences of the entire group using the integrated model, means for generating optimal proposals for the entire group, means for sending the generated proposals to each user's device, means for collecting user feedback, and means for analyzing the collected feedback to improve the generative AI model. This makes it possible to effectively integrate the different preferences and needs of multiple users, quickly suggest restaurants and dishes that will satisfy everyone, and continuously improve the accuracy of the suggestions by reflecting user feedback.

[1048] "User authentication" is the process of verifying the identity and access rights of a user attempting to access a system and confirming that the user is a legitimate user.

[1049] "Profile information" is data that includes personal information such as each user's preferences, past selection history, and customization settings.

[1050] A "generative AI model" is an AI model that generates appropriate suggestions based on a user's preferences and needs, and is specialized for a specific genre or theme.

[1051] An "integrated model" is a model created by combining individual generative AI models, and is capable of integrating the preferences of multiple users to analyze and make suggestions based on overall needs.

[1052] "The needs and preferences of the entire group" refers to a comprehensive understanding of the different interests, concerns, and desired conditions of multiple users.

[1053] An "optimal suggestion" is one that provides the most satisfying options to the entire group and is generated based on the different preferences and needs of users.

[1054] A "terminal" is a device that a user uses to access the system, such as a smartphone, tablet, or PC.

[1055] "Feedback" refers to data such as ratings, opinions, and comments provided by users regarding proposed options, and is used to improve the system.

[1056] "Improvement" is the process of improving a generative AI model or the entire system based on feedback.

[1057] This invention is a system for integrating the different preferences and needs of multiple users and making optimal suggestions to the entire group. The system is implemented mainly by a "server," "terminals," and "users."

[1058] 1. User authentication and information collection

[1059] A terminal is a device that a user uses to access the system. Examples include smartphones, tablets, and PCs. When a user logs into the system using a terminal, the server authenticates the user. If authentication is successful, the server retrieves each user's profile information from the database. The profile information includes the user's preferences, past selection history, and customization settings.

[1060] 2. Selecting and combining generative AI models

[1061] The server selects the generative AI model that is most suitable for each user based on the acquired profile information. For example, "Italian GPT" is selected for User A, "Japanese Food GPT" for User B, and "Healthy GPT" for User C. The selected generative AI models are combined by the server to build an integrated model. This integrated model reflects each user's preferences and can perform analysis according to overall needs.

[1062] 3. Generate and send optimal proposals

[1063] The server uses the integrated model to analyze the needs and preferences of the entire population and generate optimal suggestions. For example, it generates multiple healthy restaurant options that combine Italian and Japanese cuisine, detailing the benefits and features of each suggestion. The server then sends the suggestions to each user's device, which displays them to the user. Users can provide feedback on the suggestions, which is then sent back to the server.

[1064] 4. Analyze feedback and improve the model

[1065] The server analyzes the collected feedback and uses it to improve the generative AI model. This process improves the accuracy of the system's suggestions, ensuring that future suggestions are more in line with the user's preferences.

[1066] Hardware and Software Used

[1067] Hardware: Smartphones, tablets, computers

[1068] Server: AWS Compute Services, Database (e.g., Amazon RDS)

[1069] Generative AI models: large-scale language models such as OpenAI GPT-3

[1070] Data analysis tools: Python, Pandas, Numpy

[1071] Examples:

[1072] For example, consider a scenario where three users want to have dinner together. If User A prefers Italian food, User B prefers Japanese food, and User C prefers healthy food, the server selects a generative AI model suited to each user. By integrating the selected models, restaurant and food recommendations that satisfy everyone's preferences are generated.

[1073] Example prompt sentence:

[1074] "User A likes Italian food, User B likes Japanese food, and User C prefers healthy food. Please combine these preferences and suggest restaurant candidates that will satisfy all of them."

[1075] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1076] Step 1:

[1077] The server authenticates the user. When a user logs in to the system using a terminal, the terminal sends the authentication information to the server. The server authenticates the user based on the authentication information, and if authentication is successful, returns the user ID.

[1078] Input: User credentials

[1079] Output: authenticated user ID

[1080] Step 2:

[1081] The server retrieves each user's profile information from the database based on the authenticated user ID, including the user's preferences, past selection history, and customization settings.

[1082] Input: authenticated user ID

[1083] Output: User profile information

[1084] Step 3:

[1085] The server selects a generative AI model that suits each user based on the acquired profile information. For example, "Italian GPT" is selected for User A, "Japanese Food GPT" for User B, and "Healthy GPT" for User C.

[1086] Input: User profile information

[1087] Output: A generative AI model suited to each user

[1088] Step 4:

[1089] The server combines the selected generative AI models to build an integrated model, which combines the capabilities of the individual models and is used to analyze the needs and preferences of the entire population.

[1090] Input: A generative AI model tailored to each user

[1091] Output: Integrated model

[1092] Step 5:

[1093] The server uses the integrated model to analyze the needs and preferences of the entire population and generate optimal recommendations, such as a list of healthy restaurants that combine Italian and Japanese cuisine, detailing the benefits and features of each recommendation.

[1094] Input: Integration model

[1095] Output: A list of the best suggestions

[1096] Step 6:

[1097] The server sends the generated proposals to each user's terminal, which then displays the received proposals to the user.

[1098] Input: List of best suggestions

[1099] Output: Proposal displayed on the user's device

[1100] Step 7:

[1101] The user provides feedback on the suggestion, and the terminal transmits the user-entered feedback to the server.

[1102] Input: User feedback

[1103] Output: Feedback sent to the server

[1104] Step 8:

[1105] The server analyzes the collected feedback, which is then used to improve the generative AI model, adjusting it so that future suggestions better match the user's preferences.

[1106] Input: User feedback

[1107] Output: An improved generative AI model

[1108] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1109] This invention is a system that authenticates users, collects profile information, selects and combines appropriate generative AI models, generates and shares suggestions, collects user feedback, and improves the generative AI models based on the feedback. Furthermore, by combining it with an emotion engine that recognizes user emotions, it provides more accurate and personalized suggestions.

[1110] System configuration and program processing

[1111] This system consists of the following main components: an "emotion engine" that recognizes the user's emotions and reflects them in feedback, a "server" that manages the user's profile information, and a "terminal."

[1112] 1. User authentication and information collection

[1113] The terminal displays an authentication screen for the user to access the system. The user enters authentication information (user ID and password) and sends the authentication information to the server. The server verifies the authentication information and, if authentication is successful, retrieves the user's profile information from the database.

[1114] 2. Selecting and combining generative AI models

[1115] The server selects the optimal generative AI model for each user based on their profile information, and also takes into account the user's emotional state using an emotion engine. For example, it selects "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C. The user's emotional state is reflected in the profile information, making it possible to select a more accurate generative AI model. The selected generative AI models are then combined to build an integrated model.

[1116] 3. Generate and send optimal proposals

[1117] The server uses the integrated model to generate optimal suggestions that reflect the needs, preferences, and emotional state of the entire group. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine." The generated suggestions are sent to each user's device, which then displays the received suggestions to the user.

[1118] 4. User Emotion Recognition Feedback and Model Improvement

[1119] The user selects their preferred option from the suggested options and enters their feedback. The feedback includes not only their satisfaction level, desired additions, and corrections, but also reflects the user's emotional state as recognized by the emotion engine. The device sends the user's feedback to the server, which then analyzes the received feedback.

[1120] 5. Analyzing feedback and improving generative AI models

[1121] The server analyzes the feedback and continuously improves the generative AI model and emotion engine, thereby improving the accuracy of the system's suggestions so that future suggestions are more in line with the user's preferences and emotions.

[1122] Specific examples

[1123] For example, consider the case where three users (A, B, and C) are choosing a restaurant for dinner.

[1124] 1. Device: Persons A, B, and C log in to the app on their respective devices.

[1125] 2. Server: Each user's preference information (A-san prefers "Italian food," B-san prefers "Japanese food," and C-san prefers "healthy food") and their emotional state as recognized by the emotion engine at that time are obtained from the database.

[1126] 3. Server: Selects the optimal generative AI model for each user and combines the "Italian Restaurant GPT," "Japanese Restaurant GPT," and "Healthy Restaurant GPT" with the emotion engine.

[1127] 4. Server: Using the integrated model, it searches for restaurants that match the preferences and emotions of all users and generates multiple suggestions.

[1128] 5. Server: Sends the generated proposals to each user's device.

[1129] 6. Terminal: Displays the received suggestions to the user.

[1130] 7. User: Enters feedback on the suggestions and the emotional state is also reflected by the emotion engine.

[1131] 8. Device: Sends feedback to the server.

[1132] 9. Server: Analyzes feedback and improves generative AI models and emotion engines.

[1133] This system can effectively integrate the different preferences and emotions of multiple users and provide optimal suggestions to the entire group.

[1134] The processing flow will be explained below.

[1135] Step 1:

[1136] The terminal displays an authentication screen for the user to access the system. The user enters authentication information (user ID and password).

[1137] Step 2:

[1138] The terminal transmits the input authentication information to the server.

[1139] Step 3:

[1140] The server compares the received authentication information with the database to authenticate the user. If authentication is successful, the session begins.

[1141] Step 4:

[1142] The server retrieves profile information of the authenticated user from the database, including the user's preferences, past behavior history, customization settings, and emotional state data from the emotion engine.

[1143] Step 5:

[1144] The server selects the optimal generative AI model for each user based on the acquired profile information. Taking into account the user's emotional state, for example, it might select "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[1145] Step 6:

[1146] The server combines the selected generative AI models to build an integrated model, which integrates information from the combined generative AI models with emotional data from the emotion engine to analyze the needs, preferences, and emotions of the entire population.

[1147] Step 7:

[1148] The server uses the integrated model to generate optimal recommendations that reflect the needs, preferences, and emotional state of the entire population, such as generating multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine."

[1149] Step 8:

[1150] The server transmits the generated proposals to each user's terminal.

[1151] Step 9:

[1152] The device displays the received suggestions to the user, including the restaurant name, location, menu details, and ratings.

[1153] Step 10:

[1154] Users select their preferred option from the options presented and enter their feedback, which can include their level of satisfaction, what they would like to add, or what needs to be corrected. The emotion engine also analyzes the user's emotional state in real time and reflects this in the feedback.

[1155] Step 11:

[1156] The terminal transmits the user's feedback and emotion data to the server.

[1157] Step 12:

[1158] The server analyzes the received feedback and emotion data to improve the generative AI model and emotion engine, so that future suggestions will better match the user's preferences and emotions.

[1159] Specific examples

[1160] Consider three users (A, B, and C) having dinner together.

[1161] Step 1:

[1162] Device: Persons A, B, and C each log in to the app on their own smartphones.

[1163] Step 2:

[1164] Terminal: Sends the entered authentication information (user ID and password) to the server.

[1165] Step 3:

[1166] Server: Compares the received authentication information with the database and performs authentication. If authentication is successful, a session begins.

[1167] Step 4:

[1168] Server: Retrieves profile information for each authenticated user from a database, including preferences, past behavioral history, customization settings, and current emotional state.

[1169] Step 5:

[1170] Server: Based on the profile information and emotional state, select "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[1171] Step 6:

[1172] Server: Combines selected generative AI models to build an integrated model that integrates everyone's preferences and emotions.

[1173] Step 7:

[1174] Server: Using the integrated model, it generates multiple restaurant candidates that take into account everyone's preferences and emotions. For example, it generates candidates for "healthy restaurants that combine Italian and Japanese cuisine."

[1175] Step 8:

[1176] Server: Sends the generated proposals to the devices of A, B, and C.

[1177] Step 9:

[1178] Terminal: Displays the received suggestions to the user, including detailed restaurant information.

[1179] Step 10:

[1180] User: Selects the preferred option from the suggested options and enters feedback. The emotion engine analyzes the user's emotional state in real time and reflects that in the feedback.

[1181] Step 11:

[1182] Terminal: Sends feedback and emotion data to the server.

[1183] Step 12:

[1184] Server: Analyzes feedback and sentiment data to improve generative AI models and sentiment engines.

[1185] In this way, the system can effectively integrate the different preferences and emotions of multiple users and provide optimal suggestions for the entire population.

[1186] Example 2

[1187] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1188] Conventional suggestion systems make suggestions based solely on the user's preferences, resulting in a lack of personalized suggestions that reflect the user's real-time emotional state. Furthermore, the selection and integration of generative AI models was not centralized, making it difficult to improve the quality of the generated suggestions. This resulted in a decline in user satisfaction and an inability to fully utilize the system's effectiveness.

[1189] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1190] In this invention, the server includes means for authenticating users, means for collecting profile information for each user, means for selecting a generative AI model that suits each user, means for recognizing the user's emotional state, means for combining the selected generative AI model with an emotion engine to build an integrated model, means for analyzing the needs, preferences, and emotional state of the entire group using the integrated model, means for generating optimal proposals for the entire group, means for transmitting the generated proposals to each user's device, means for collecting user feedback and emotion recognition results, and means for analyzing the collected feedback and emotion recognition results to improve the generative AI model and emotion engine, thereby enabling more accurate and personalized proposals that reflect the user's real-time emotional state.

[1191] "Means of user authentication" refers to the method of having a user enter the necessary information (such as a user ID and password) to verify that they are a legitimate user when they access a system.

[1192] "Profile information" refers to detailed information about each user, such as personal data, preference information, and history information about the user.

[1193] The "means for selecting a generative AI model" is a method for selecting the optimal generative AI model based on the user's profile information and emotional state.

[1194] "Means for recognizing emotional state" refers to technologies (such as voice analysis and facial expression recognition) used to analyze a user's emotions in real time and identify their state.

[1195] "Means for constructing an integrated model by combining a generative AI model and an emotion engine" refers to a method for linking a selected generative AI model with an emotion engine to construct an integrated model.

[1196] The "means for analyzing the needs, preferences, and emotional states of an entire population" is a method for analyzing the needs, preferences, and emotional states of multiple users using an integrated model.

[1197] The "means for generating optimal proposals" is a method for creating the most suitable proposals for the user based on the analysis results.

[1198] The "means for transmitting the generated proposal to the terminal of each user" is a method for distributing the generated proposal to the terminal used by the user.

[1199] "Means for collecting feedback and emotion recognition results" refers to a method for collecting user evaluations and additional opinions on suggestions, as well as emotion analysis results from the emotion engine.

[1200] "Means for analyzing feedback and emotion recognition results to improve generative AI models and emotion engines" refers to methods for adjusting and improving generative AI models and emotion engines based on collected feedback and emotion data.

[1201] This invention is a system that authenticates users, collects profile information, selects and combines generative AI models, generates and shares suggestions, collects user feedback, and improves the generative AI models based on the feedback. Furthermore, by combining it with an emotion engine that recognizes user emotions, it provides more accurate and personalized suggestions.

[1202] The main components of the system are:

[1203] "Emotion Engine" that recognizes user emotions and reflects them in feedback

[1204] "Server" and "Terminal" that manage user profile information

[1205] 1. User authentication and information collection

[1206] The terminal displays an authentication screen for the user to access the system. The terminal collects the user ID and password entered by the user and sends this information to the server. The server verifies the received authentication information and retrieves the corresponding user profile information from a database. At this time, encrypted communication such as SSL is used for authentication to ensure the security of the information.

[1207] 2. Selecting and combining generative AI models

[1208] The server analyzes the acquired profile information. Data analysis tools such as Python are used for the analysis to select the optimal generative AI model based on the user's preferences and past behavioral history. At this time, the emotion engine analyzes the user's emotional state in real time, and this information is also added to the profile. The selected generative AI model and the emotion engine are then combined to create an integrated model. This integrated model is realized using machine learning frameworks such as TensorFlow and PyTorch.

[1209] 3. Generate and send optimal proposals

[1210] The server uses the integrated generative AI model to generate optimal suggestions based on the user's preferences and emotional state. For example, if a user prefers Italian restaurants with a relaxed atmosphere, the server generates multiple suggestions that reflect that information. The generated suggestions are sent to each user's device via a REST API. The device then displays the received suggestions on its screen, allowing the user to easily access them.

[1211] 4. User Emotion Recognition Feedback and Model Improvement

[1212] The user selects the preferred suggestion from the displayed suggestions and enters feedback regarding satisfaction, additions, and corrections. Specifically, the user uses a feedback form or rating slider. Furthermore, the emotion engine analyzes the user's emotional state and includes this information in the feedback. The device then sends this feedback information to the server.

[1213] 5. Analyzing feedback and improving generative AI models

[1214] The server performs a detailed analysis of the received feedback, using natural language processing and data mining techniques to extract patterns from the collected data. Based on the results, the generative AI model and emotion engine are refined. This continuous improvement ensures that future suggestions are more in line with the user's preferences and emotions.

[1215] Specific examples

[1216] For example, consider the case where three users (user A, user B, and user C) are choosing a restaurant for dinner.

[1217] 1. Device: User A, User B, and User C log in to the app on their respective devices.

[1218] 2. Server: Obtains each user's preference information (User A is "Italian food," User B is "Japanese food," and User C is "healthy food") and the emotional state recognized by the emotion engine at that time from the database.

[1219] 3. Server: Selects the optimal generative AI model for each user, for example, "Italian Restaurant GPT," "Japanese Restaurant GPT," and "Healthy Restaurant GPT," and combines them with an emotion engine.

[1220] 4. Server: Using the integrated model, it searches for restaurants that match the preferences and emotions of all users and generates multiple suggestions.

[1221] 5. Server: Sends the generated proposals to each user's device.

[1222] 6. Terminal: Displays the received suggestions to the user.

[1223] 7. User: Enters feedback on the suggestions and the emotional state is also reflected by the emotion engine.

[1224] 8. Device: Sends feedback to the server.

[1225] 9. Server: Analyzes feedback and improves generative AI models and emotion engines.

[1226] Prompt Sentence Examples

[1227] "User A currently likes Italian food, so please display several Italian restaurant suggestions for tonight's dinner. Please also consider the user's emotional state and prioritize restaurants with a relaxing atmosphere."

[1228] This system can effectively integrate the different preferences and emotions of multiple users and provide optimal suggestions to the entire group.

[1229] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1230] Step 1: User authentication and information gathering

[1231] The terminal displays an authentication screen for the user to access the system. The user enters their user ID and password and clicks the "Login button." Based on this input, the terminal sends the user ID and password to the server. The server searches for user information in a database, and if the authentication information is correct, retrieves profile information. At this time, encrypted communication such as SSL is used to ensure the security of the information. If authentication is successful, the user's profile information is sent from the server to the terminal.

[1232] Step 2: Selecting and combining generative AI models

[1233] The server analyzes the acquired profile information. This analysis uses data analysis tools such as Python to select the optimal generative AI model based on the user's preferences and past behavioral history. The emotion engine also becomes active, analyzing the user's real-time emotional state and adding this information to the profile. For example, if the user is looking for a "relaxed atmosphere," a generative AI model that reflects this is selected. The selected generative AI model is integrated with the emotion engine to build an integrated model. The input data is the profile information and emotional state, and the output is the generative AI model and the integrated model.

[1234] Step 3: Generate and submit optimal proposals

[1235] The server uses the integrated generative AI model to generate optimal suggestions based on the user's preferences and emotional state. For example, it generates multiple candidates for "relaxing Italian restaurants." The generative AI model uses data processing and machine learning algorithms to generate suggestions that match the user's preferences and emotions. The generated suggestions are sent to each user's device via a REST API. The input data is the integrated model, and the output is the generated suggestions. The device displays the received suggestions to the user.

[1236] Step 4: User emotion recognition feedback and model improvement

[1237] The user selects the suggestion they like best from the ones displayed and enters feedback regarding their satisfaction, additions they would like to make, and any corrections they would like to make. Specifically, they use a feedback form or a rating slider. Furthermore, the emotion engine analyzes the user's emotional state and includes this information in the feedback. For example, it may recognize from facial expression analysis that the user is "very satisfied" with the suggestion. The device then sends this feedback information to the server. The input data is the user's feedback and emotional state, and the output is the sent feedback information.

[1238] Step 5: Analyze feedback and improve the generative AI model

[1239] The server performs a detailed analysis of the received feedback. This analysis uses natural language processing and data mining techniques to extract patterns from the collected data. Based on the results, the generative AI model and emotion engine are improved. For example, the reason why a particular suggestion was not liked can be analyzed and the algorithm adjusted. This continuous improvement ensures that future suggestions will better match the user's preferences and emotions. The input data is feedback information and emotional state, and the output is an improved generative AI model and emotion engine.

[1240] (Application example 2)

[1241] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1242] Conventional recommendation systems have a problem in that they make suggestions based solely on the user's preference information. In particular, if the user's emotional state is not taken into account, the accuracy of the suggestions and user satisfaction may decrease. Furthermore, it is not easy to address the preferences and needs of an entire group, making it difficult to make appropriate suggestions to individual users. Therefore, there is a need for a system that takes into account the user's preference information and emotional state in an integrated manner and provides more personalized and appropriate suggestions.

[1243] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1244] In this invention, the server includes means for utilizing an emotion engine to reflect the user's emotional state, means for selecting a generative AI model suited to each user, and means for combining the selected generative AI models to construct an integrated model. This allows for the generation of suggestions that take into consideration the user's preference information and emotional state in an integrated manner, thereby improving user satisfaction.

[1245] "Means for authenticating users" refers to a function that allows a user to input authentication information required to access the system and verify that information.

[1246] The "means for collecting profile information of each user" is a function for collecting personal information, preference information, and history data provided by the user.

[1247] "Means for selecting a generative AI model that suits each user" is a function that selects the optimal generative AI model based on the user's profile information and emotional state.

[1248] "Means for combining selected generative AI models to construct an integrated model" refers to a function for combining multiple generative AI models to construct a single integrated model.

[1249] "Means for analyzing the needs and preferences of an entire group using an integrated model" is a function that utilizes an integrated model to analyze the needs and preferences of multiple users.

[1250] "Means for utilizing the emotion engine" refers to a function that recognizes the user's emotional state from facial expressions, voice, etc., and captures this as data.

[1251] The "means for generating optimal proposals" is a function that generates the most suitable proposals for the user based on all collected data.

[1252] The "means for transmitting the generated proposal to each user's terminal" is a function for transmitting the generated proposal from the server to the user's terminal.

[1253] The "means for collecting user feedback" is a function for obtaining reactions and evaluations made by users to suggestions.

[1254] "Means for analyzing collected feedback to improve the generative AI model and emotion engine" refers to a function for analyzing feedback obtained from users and continuously improving the generative AI model and emotion engine based on that feedback.

[1255] MODE FOR CARRYING OUT THE INVENTION

[1256] System Overview

[1257] This invention is a system that makes optimal suggestions based on a user's preferences and emotional state. The system consists of the following main components: an "emotion engine" that recognizes the user's emotions and reflects them in feedback, a "server" that manages the user's profile information, and a "terminal." Specifically, the system performs user authentication, profile collection, selection and integration of appropriate generative AI models, generation and sharing of suggestions, feedback collection, and model improvement based on the feedback.

[1258] Program processing overview

[1259] The program for this system is implemented as an application installed on a smartphone or smart glasses. Below, we will explain the processing overview of each component.

[1260] User authentication and information collection

[1261] A user logs in to the system using a smartphone or smart glasses. The device sends user authentication information to the server, which then verifies the authentication information. If authentication is successful, the server retrieves the user's profile information from the database.

[1262] Selecting and combining generative AI models

[1263] The server selects the optimal generative AI model based on the user's profile information and the emotion data recognized by the emotion engine. For example, it builds an integrated model by combining multiple generative AI models, such as "Chinese cuisine GPT," "Hamburger GPT," and "Vegan cuisine GPT."

[1264] Generate and send optimal proposals

[1265] The server uses the integrated model to generate optimal food delivery suggestions that reflect the user's preferences and emotional state. The suggestions are then sent to the user's device, which then displays the suggestions to the user.

[1266] User emotion recognition feedback and model improvement

[1267] The user selects from the presented suggestions and enters feedback. The emotion engine also recognizes the user's emotional state at the time of the feedback, and the device sends this feedback to the server. The server analyzes the received feedback and improves the generative AI model and emotion engine.

[1268] Hardware and software used

[1269] Hardware: Smartphones, smart glasses, servers

[1270] Software: Emotion recognition engine (EmotionEngine), AI model selection and combination library (ai_model_selector), database

[1271] Specific examples

[1272] For example, if a user is considering ordering dinner delivery, the system might:

[1273] 1. Authentication and profile information acquisition: The user logs in to the app on their smartphone, their authentication information is sent to the server, and their profile information is acquired.

[1274] 2. Recognizing emotional state: The emotion engine recognizes the user's emotional state, and the server receives the emotion data.

[1275] 3. Selection of generative AI model: The server selects the optimal generative AI model based on the profile information and emotional state, and constructs an integrated model.

[1276] 4. Proposal generation and transmission: The integrated model is used to generate optimal food delivery proposals and transmit them to the user's device.

[1277] 5. Feedback collection and model improvement: The user enters feedback and their emotional state is also recorded, which is sent to the server and used to improve the model.

[1278] Prompt Sentence Examples

[1279] "User Preferences: Chinese food"

[1280] "User's emotional state: Stressed"

[1281] "Proposal requirements: Please propose a quick and delicious Chinese food delivery menu."

[1282] This allows the system to make personalized suggestions that take into account the user's preferences and emotional state, increasing user satisfaction.

[1283] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1284] Step 1:

[1285] The terminal prompts the user to enter authentication information (user ID and password). The entered authentication information is sent to the server. The server compares the received authentication information with the database and determines whether the authentication was successful or not. If the authentication is successful, the user's profile information is retrieved from the database.

[1286] Input: User ID, Password

[1287] Data processing: Verification of authentication information

[1288] Output: User profile information, authentication result (success / failure)

[1289] Step 2:

[1290] The server uses an emotion engine to recognize the user's emotional state based on the acquired profile information. Sensors such as the device's camera and microphone are used to collect emotional data in real time, which is then analyzed by the emotion engine.

[1291] Input: User profile information, real-time emotional data (images, voice)

[1292] Data processing: Analysis of emotional data, recognition of emotional states

[1293] Output: User's emotional state

[1294] Step 3:

[1295] The server selects an appropriate generative AI model based on the user's profile information and emotional state. The server analyzes the profile information and emotional data and selects an appropriate generative AI model, such as "Chinese Food GPT" or "Hamburger GPT."

[1296] Input: User profile information, emotional state

[1297] Data processing: Selection of generative AI model

[1298] Output: A list of selected generative AI models

[1299] Step 4:

[1300] The server combines the selected generative AI models to build an integrated model. For example, it combines the selected "Chinese food GPT" and "Hamburger GPT" to create an integrated model.

[1301] Input: A list of selected generative AI models

[1302] Data processing: Combining generative AI models

[1303] Output: Integrated model

[1304] Step 5:

[1305] The server uses the integrated model to generate food delivery suggestions that reflect the user's preferences and emotional state. The server inputs the prompt sentence into the integrated model to generate suggestions.

[1306] Input: Integrated model, prompt statement

[1307] Data processing: Proposal generation

[1308] Output: Food delivery suggestions

[1309] Step 6:

[1310] The server sends the generated proposals to each user's terminal, which then displays the received proposals on the user's screen.

[1311] Input: Food delivery suggestions

[1312] Data Processing: Submit Proposal

[1313] Output: The proposal displayed on the user's device

[1314] Step 7:

[1315] The user selects from the presented suggestions and inputs feedback, and the terminal transmits the user's feedback and current emotional state to the server.

[1316] Input: User feedback, emotional state

[1317] Data processing: Feedback collection

[1318] Output: Feedback sent to the server

[1319] Step 8:

[1320] The server analyzes the collected feedback and improves the generative AI model and emotion engine. The server analyzes the feedback data and improves the accuracy of future suggestions.

[1321] Input: User feedback, emotional state

[1322] Data processing: analyzing feedback, improving generative AI models and emotion engines

[1323] Output: Improved generative AI models and emotion engines

[1324] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1325] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1326] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1327] [Fourth embodiment]

[1328] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1329] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1330] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1331] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1332] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1333] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1334] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1335] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1336] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1337] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1338] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1339] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1340] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1341] The present invention is a system that integrates the different preferences and needs of multiple users and makes optimal suggestions to the entire group. An embodiment of this system will be described in detail below.

[1342] System configuration and program processing

[1343] The system is mainly implemented by three main components: "server", "terminal", and "user".

[1344] 1. User authentication and information collection

[1345] A terminal is a device that a user uses to access the system, such as a smartphone or a PC. When a user logs in to the system using a terminal, the server authenticates the user. If authentication is successful, the server retrieves each user's profile information from the database. The profile information includes the user's preferences, past selection history, and customization settings.

[1346] 2. Selecting and combining generative AI models

[1347] The server selects the generative AI model that is most suitable for each user based on the acquired profile information. For example, "Italian Restaurant GPT" is selected for Person A, "Japanese Restaurant GPT" for Person B, and "Healthy Restaurant GPT" for Person C. The selected generative AI models are combined by the server to build an integrated model. This integrated model reflects each user's preferences and can perform analysis according to overall needs.

[1348] 3. Generate and send optimal proposals

[1349] The server uses the integrated model to analyze the needs and preferences of the entire group and generate optimal suggestions. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine" and describes in detail the benefits and features of each suggestion. The server then sends the generated suggestions to each user's device, and the device displays the received suggestions to the user. Users can provide feedback on the suggestions, which is then sent back to the server.

[1350] 4. Analyze feedback and improve the model

[1351] The server analyzes the collected feedback and uses it to improve the generative AI model. This process improves the accuracy of the system's suggestions, ensuring that future suggestions are more in line with the user's preferences.

[1352] Specific examples

[1353] For example, suppose three users (A, B, and C) want to choose a restaurant to have dinner together.

[1354] 1. Device: Persons A, B, and C each log in to the app on their own smartphones.

[1355] 2. Server: Retrieves each user's preference information (A prefers "Italian food," B prefers "Japanese food," and C prefers "healthy food") from the database.

[1356] 3. Server: Selects the optimal generative AI model for each user and combines the "Italian Restaurant GPT," "Japanese Restaurant GPT," and "Healthy Restaurant GPT."

[1357] 4. Server: Uses the combined model to search for restaurants that match everyone's preferences and generate multiple candidates.

[1358] 5. Server: Sends the generated proposals to each user's device, which displays them to the user.

[1359] 6. User: Each user enters feedback on the proposal, and the device sends it to the server.

[1360] 7. Server: Analyzes feedback and improves the generative AI model.

[1361] In this way, the system can effectively integrate the different preferences and needs of multiple users and provide optimal recommendations for the entire population.

[1362] The processing flow will be explained below.

[1363] Step 1:

[1364] The terminal displays an authentication screen for the user to access the system. The user enters their own authentication information (user ID and password).

[1365] Step 2:

[1366] The terminal transmits the input authentication information to the server.

[1367] Step 3:

[1368] The server compares the received authentication information with the database to authenticate the user. If authentication is successful, the session begins.

[1369] Step 4:

[1370] The server retrieves the authenticated user's profile information from the database, which includes the user's preferences, past behavior history, and customization settings.

[1371] Step 5:

[1372] The server selects the optimal generative AI model for each user based on the acquired profile information. For example, it might select "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[1373] Step 6:

[1374] The server combines the selected generative AI models to build an integrated model, which can integrate information from each combined generative AI model and analyze the needs and preferences of the entire population.

[1375] Step 7:

[1376] The server uses the integrated model to generate optimal recommendations that reflect the needs and preferences of the entire group, such as generating multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine."

[1377] Step 8:

[1378] The server transmits the generated proposals to each user's terminal.

[1379] Step 9:

[1380] The device displays the received suggestions to the user, including the restaurant name, location, menu details, and ratings.

[1381] Step 10:

[1382] Users select their preferred options from the suggestions and enter feedback, including their satisfaction level, additions, and corrections.

[1383] Step 11:

[1384] The terminal transmits the user's feedback to the server.

[1385] Step 12:

[1386] The server analyzes the received feedback and improves the generative AI model, so that future suggestions will better match the user's preferences.

[1387] In this way, the system effectively integrates the different preferences and needs of multiple users to provide optimal recommendations for the entire group.

[1388] Example 1

[1389] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1390] There is a need for a system that can integrate the different preferences and needs of multiple users to generate optimal proposals for the entire group. However, conventional methods have difficulty analyzing each user's preferences individually, making it difficult to efficiently generate proposals that meet the needs of the entire group. In addition, there is a lack of technology to effectively incorporate user feedback and improve the model. To solve this problem, a collective optimization method using an advanced generative AI model is required.

[1391] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1392] In this invention, the server includes means for authenticating users, means for collecting profile information for each user, means for selecting a generative AI model suitable for each user, means for combining the selected generative AI models to build an integrated model, means for analyzing the needs and preferences of the entire group using the integrated model, means for generating optimal proposals for the entire group, means for transmitting the generated proposals to each user's device, means for collecting user feedback, means for analyzing the collected feedback to improve the generative AI model, means for describing the optimal proposals in detail using prompt sentences, and means for notifying and displaying the proposals. This makes it possible to effectively integrate the preferences and needs of multiple users, generate optimal proposals for the entire group, and further improve the accuracy of the proposals by utilizing user feedback.

[1393] "Means for authenticating users" refers to a mechanism for verifying the user's login information and determining whether the user is legitimate.

[1394] The "means for collecting profile information for each user" is a function for obtaining information such as the user's preferences, past selection history, and customization settings from a database.

[1395] The "means for selecting a generative AI model that suits each user" is a mechanism for identifying the generative AI model that best suits the user's preferences and needs based on collected profile information.

[1396] "Means for combining selected generative AI models to construct an integrated model" refers to a method for integrating generative AI models selected for each user to create a common model that reflects the preferences and needs of multiple users.

[1397] "Means for analyzing needs and preferences across a population using an integrated model" means a process for utilizing an integrated generative AI model to comprehensively assess the needs and preferences of a group of multiple users.

[1398] The "means for generating optimal proposals for the entire group" is a function that creates optimal proposals that satisfy the preferences and needs of the entire group based on the analysis results of the entire group.

[1399] The "means for transmitting the generated proposal to each user's terminal" is a mechanism for distributing the generated proposal to each user's device via a network.

[1400] The "means for collecting user feedback" is a mechanism for collecting opinions and ratings provided by users on suggestions.

[1401] "Means for analyzing collected feedback to improve generative AI models" refers to methods for analyzing feedback collected from users and using the results to improve the accuracy and applicability of generative AI models.

[1402] The "means for describing the optimal proposal in detail using a prompt sentence" is a function for expressing the generated proposal in detail using a prompt sentence including specific sentences and instructions.

[1403] A "means for notifying and displaying suggestions" is a method for informing the user of and visually displaying the generated suggestions.

[1404] This invention is a system that integrates the different preferences and needs of multiple users and makes optimal suggestions to the entire group. How this system is implemented will be described in detail below.

[1405] System configuration

[1406] This system is mainly composed of three main components: "server," "terminal," and "user."

[1407] User authentication and information collection

[1408] A terminal is a device that a user uses to access the system, such as a smartphone or a PC. When a user logs in to the system using a terminal, the login information (user name and password) is sent from the terminal to the server. At this time, the communication is securely protected using the HTTPS protocol.

[1409] The server passes the received login information to the authentication processing module and performs user authentication. If authentication is successful, the server retrieves the user's profile information from the database. This profile information includes the user's preferences, past selection history, and customization settings. The database uses an RDBMS such as MySQL or PostgreSQL.

[1410] Selecting and combining generative AI models

[1411] The server selects the most suitable generative AI model for each user based on the acquired profile information. For example, "Italian Restaurant GPT" is selected for Person A, "Japanese Restaurant GPT" for Person B, and "Healthy Restaurant GPT" for Person C. These generative AI models are trained using training data specialized for each user's specific needs.

[1412] The server combines the selected generative AI models to construct an integrated model that comprehensively reflects the user's preferences. For this purpose, technologies such as reinforcement learning (RL) and ensemble learning may be used.

[1413] Generate and send optimal proposals

[1414] The server uses the integrated model to analyze the needs and preferences of the entire group and generate specific suggestions. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine," and includes detailed information about each suggestion (location, menu, recommended points, etc.). This process uses prompts from the generative AI model.

[1415] Here are some examples of prompts:

[1416] "Person A likes Italian restaurants, Person B likes Japanese food, and Person C likes healthy meals. Please provide restaurant suggestions that would be best for these three people."

[1417] The server sends the generated suggestions to each user's device, which then displays them to the user. The suggestions are displayed visually in rich text and image formats, and push notifications are also used if necessary.

[1418] Collecting and analyzing feedback

[1419] The user provides feedback on the displayed suggestions. The feedback is sent from the device to the server through a dedicated input form. The feedback includes the degree of satisfaction with the suggestions and specific comments.

[1420] The server analyzes this feedback and uses it to improve the generative AI model. The feedback data is fed into the machine learning algorithm, and the generative AI model is retrained to improve the accuracy of future suggestions.

[1421] Specific examples

[1422] For example, suppose three users (A, B, and C) want to choose a restaurant to have dinner together.

[1423] 1. Device: Persons A, B, and C each log in to the app on their own smartphones.

[1424] 2. Terminal: Once the login information is entered, the terminal sends it to the server.

[1425] 3. Server: The server receives the login information and performs authentication. If authentication is successful, it retrieves the profile information from the database.

[1426] 4. Server: Analyzes the profile information and selects "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[1427] 5. Server: Combines the selected generative AI models to create an integrated model.

[1428] 6. Server: Uses the integrated model to generate multiple restaurant options that match everyone's preferences, and generates detailed information using prompts.

[1429] 7. Server: Sends the generated proposals to each user's device.

[1430] 8. Device: The device displays the received suggestions to the user and optionally sends push notifications.

[1431] 9. User: Each user provides feedback on the proposal, which is sent to the server via their device.

[1432] 10. Server: Analyzes feedback and helps improve the generative AI model.

[1433] In this way, the system can effectively integrate the different preferences and needs of multiple users and provide optimal recommendations for the entire population.

[1434] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1435] Program processing flow

[1436] Step 1: User Login

[1437] Step 2: User authentication and profile information retrieval

[1438] Step 3: Selecting a generative AI model

[1439] Step 4: Combining generative AI models

[1440] Step 5: Generate optimal proposals

[1441] Step 6: Submit and view your proposal

[1442] Step 7: Gather feedback

[1443] Step 8: Analyze feedback and improve the generative AI model

[1444] Detailed explanation of each processing step

[1445] Step 1: User Login

[1446] A terminal is a device that users use to access the system. Users access the login screen using a smartphone or computer and enter their username and password. The terminal receives this login information and sends it to the server using the HTTPS protocol.

[1447] Input: Username, Password

[1448] Output: Login information sent to the server

[1449] Step 2: User authentication and profile information retrieval

[1450] The server passes the received login information to the authentication processing module and performs user authentication. If authentication is successful, the server retrieves the user's profile information (preferences, past selection history, customization settings, etc.) from the database. The database uses an RDBMS such as MySQL or PostgreSQL.

[1451] Input: Login information

[1452] Output: Authentication result (success / failure), profile information

[1453] Step 3: Selecting a generative AI model

[1454] The server analyzes the acquired profile information and selects the most suitable generative AI model for each user. For example, "Italian Restaurant GPT" is selected for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C. The selection criteria are based on the user's preferences and past selection history.

[1455] Input: Profile Information

[1456] Output: The selected generative AI model

[1457] Step 4: Combining generative AI models

[1458] The server combines the selected generative AI models to build an integrated model. This integrated model reflects each user's preferences and analyzes the overall needs. Specific methods include ensemble learning and reinforcement learning.

[1459] Input: A selected generative AI model

[1460] Output: Integrated model

[1461] Step 5: Generate optimal proposals

[1462] The server uses the integrated model to analyze the needs and preferences of the entire group and generate specific suggestions. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine," including detailed information (location, menu, recommended features, etc.). This process uses prompts from the generative AI model.

[1463] Input: Integration model

[1464] Output: Proposal

[1465] Step 6: Submit and view your proposal

[1466] The server sends the generated suggestions to each user's device, which displays the received suggestions to the user and, if necessary, sends push notifications. Suggestions are visually displayed in rich text and image formats.

[1467] Input: Proposal

[1468] Output: Proposal data sent to the user's device

[1469] Step 7: Gather feedback

[1470] The user provides feedback on the displayed suggestions. The feedback is sent from the device to the server via a dedicated input form. The feedback includes the degree of satisfaction with the suggestions and specific comments.

[1471] Input: Feedback information

[1472] Output: Feedback data sent to the server

[1473] Step 8: Analyze feedback and improve the generative AI model

[1474] The server analyzes the collected feedback and feeds the feedback data to a machine learning algorithm, which uses it as training data for the generative AI model. This improves the generative AI model so that future suggestions will better match the user's preferences.

[1475] Input: Feedback data

[1476] Output: An improved generative AI model

[1477] In this way, the system can effectively integrate the different preferences and needs of multiple users and provide optimal recommendations for the entire population.

[1478] (Application example 1)

[1479] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1480] Conventional recommendation systems face the challenge of effectively integrating the different preferences and needs of individual users and making optimal recommendations for the entire group. In particular, when multiple users have different food preferences, it is necessary to quickly and accurately recommend restaurants and dishes that will satisfy everyone. Furthermore, there is a lack of a way to continuously improve the system using user feedback, making it difficult to improve the accuracy of the recommendations.

[1481] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1482] In this invention, the server includes means for authenticating users, means for collecting profile information for each user, means for selecting a generative AI model suitable for each user, means for combining the selected generative AI models to build an integrated model, means for analyzing the needs and preferences of the entire group using the integrated model, means for generating optimal proposals for the entire group, means for sending the generated proposals to each user's device, means for collecting user feedback, and means for analyzing the collected feedback to improve the generative AI model. This makes it possible to effectively integrate the different preferences and needs of multiple users, quickly suggest restaurants and dishes that will satisfy everyone, and continuously improve the accuracy of the suggestions by reflecting user feedback.

[1483] "User authentication" is the process of verifying the identity and access rights of a user attempting to access a system and confirming that the user is a legitimate user.

[1484] "Profile information" is data that includes personal information such as each user's preferences, past selection history, and customization settings.

[1485] A "generative AI model" is an AI model that generates appropriate suggestions based on a user's preferences and needs, and is specialized for a specific genre or theme.

[1486] An "integrated model" is a model created by combining individual generative AI models, and is capable of integrating the preferences of multiple users to analyze and make suggestions based on overall needs.

[1487] "The needs and preferences of the entire group" refers to a comprehensive understanding of the different interests, concerns, and desired conditions of multiple users.

[1488] An "optimal suggestion" is one that provides the most satisfying options to the entire group and is generated based on the different preferences and needs of users.

[1489] A "terminal" is a device that a user uses to access the system, such as a smartphone, tablet, or PC.

[1490] "Feedback" refers to data such as ratings, opinions, and comments provided by users regarding proposed options, and is used to improve the system.

[1491] "Improvement" is the process of improving a generative AI model or the entire system based on feedback.

[1492] This invention is a system for integrating the different preferences and needs of multiple users and making optimal suggestions to the entire group. The system is implemented mainly by a "server," "terminals," and "users."

[1493] 1. User authentication and information collection

[1494] A terminal is a device that a user uses to access the system. Examples include smartphones, tablets, and PCs. When a user logs into the system using a terminal, the server authenticates the user. If authentication is successful, the server retrieves each user's profile information from the database. The profile information includes the user's preferences, past selection history, and customization settings.

[1495] 2. Selecting and combining generative AI models

[1496] The server selects the generative AI model that is most suitable for each user based on the acquired profile information. For example, "Italian GPT" is selected for User A, "Japanese Food GPT" for User B, and "Healthy GPT" for User C. The selected generative AI models are combined by the server to build an integrated model. This integrated model reflects each user's preferences and can perform analysis according to overall needs.

[1497] 3. Generate and send optimal proposals

[1498] The server uses the integrated model to analyze the needs and preferences of the entire population and generate optimal suggestions. For example, it generates multiple healthy restaurant options that combine Italian and Japanese cuisine, detailing the benefits and features of each suggestion. The server then sends the suggestions to each user's device, which displays them to the user. Users can provide feedback on the suggestions, which is then sent back to the server.

[1499] 4. Analyze feedback and improve the model

[1500] The server analyzes the collected feedback and uses it to improve the generative AI model. This process improves the accuracy of the system's suggestions, ensuring that future suggestions are more in line with the user's preferences.

[1501] Hardware and Software Used

[1502] Hardware: Smartphones, tablets, computers

[1503] Server: AWS Compute Services, Database (e.g., Amazon RDS)

[1504] Generative AI models: large-scale language models such as OpenAI GPT-3

[1505] Data analysis tools: Python, Pandas, Numpy

[1506] Examples:

[1507] For example, consider a scenario where three users want to have dinner together. If User A prefers Italian food, User B prefers Japanese food, and User C prefers healthy food, the server selects a generative AI model suited to each user. By integrating the selected models, restaurant and food recommendations that satisfy everyone's preferences are generated.

[1508] Example prompt sentence:

[1509] "User A likes Italian food, User B likes Japanese food, and User C prefers healthy food. Please combine these preferences and suggest restaurant candidates that will satisfy all of them."

[1510] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1511] Step 1:

[1512] The server authenticates the user. When a user logs in to the system using a terminal, the terminal sends the authentication information to the server. The server authenticates the user based on the authentication information, and if authentication is successful, returns the user ID.

[1513] Input: User credentials

[1514] Output: authenticated user ID

[1515] Step 2:

[1516] The server retrieves each user's profile information from the database based on the authenticated user ID, including the user's preferences, past selection history, and customization settings.

[1517] Input: authenticated user ID

[1518] Output: User profile information

[1519] Step 3:

[1520] The server selects a generative AI model that suits each user based on the acquired profile information. For example, "Italian GPT" is selected for User A, "Japanese Food GPT" for User B, and "Healthy GPT" for User C.

[1521] Input: User profile information

[1522] Output: A generative AI model suited to each user

[1523] Step 4:

[1524] The server combines the selected generative AI models to build an integrated model, which combines the capabilities of the individual models and is used to analyze the needs and preferences of the entire population.

[1525] Input: A generative AI model tailored to each user

[1526] Output: Integrated model

[1527] Step 5:

[1528] The server uses the integrated model to analyze the needs and preferences of the entire population and generate optimal recommendations, such as a list of healthy restaurants that combine Italian and Japanese cuisine, detailing the benefits and features of each recommendation.

[1529] Input: Integration model

[1530] Output: A list of the best suggestions

[1531] Step 6:

[1532] The server sends the generated proposals to each user's terminal, which then displays the received proposals to the user.

[1533] Input: List of best suggestions

[1534] Output: Proposal displayed on the user's device

[1535] Step 7:

[1536] The user provides feedback on the suggestion, and the terminal transmits the user-entered feedback to the server.

[1537] Input: User feedback

[1538] Output: Feedback sent to the server

[1539] Step 8:

[1540] The server analyzes the collected feedback, which is then used to improve the generative AI model, adjusting it so that future suggestions better match the user's preferences.

[1541] Input: User feedback

[1542] Output: An improved generative AI model

[1543] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1544] This invention is a system that authenticates users, collects profile information, selects and combines appropriate generative AI models, generates and shares suggestions, collects user feedback, and improves the generative AI models based on the feedback. Furthermore, by combining it with an emotion engine that recognizes user emotions, it provides more accurate and personalized suggestions.

[1545] System configuration and program processing

[1546] This system consists of the following main components: an "emotion engine" that recognizes the user's emotions and reflects them in feedback, a "server" that manages the user's profile information, and a "terminal."

[1547] 1. User authentication and information collection

[1548] The terminal displays an authentication screen for the user to access the system. The user enters authentication information (user ID and password) and sends the authentication information to the server. The server verifies the authentication information and, if authentication is successful, retrieves the user's profile information from the database.

[1549] 2. Selecting and combining generative AI models

[1550] The server selects the optimal generative AI model for each user based on their profile information, and also takes into account the user's emotional state using an emotion engine. For example, it selects "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C. The user's emotional state is reflected in the profile information, making it possible to select a more accurate generative AI model. The selected generative AI models are then combined to build an integrated model.

[1551] 3. Generate and send optimal proposals

[1552] The server uses the integrated model to generate optimal suggestions that reflect the needs, preferences, and emotional state of the entire group. For example, it generates multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine." The generated suggestions are sent to each user's device, which then displays the received suggestions to the user.

[1553] 4. User Emotion Recognition Feedback and Model Improvement

[1554] The user selects their preferred option from the suggested options and enters their feedback. The feedback includes not only their satisfaction level, desired additions, and corrections, but also reflects the user's emotional state as recognized by the emotion engine. The device sends the user's feedback to the server, which then analyzes the received feedback.

[1555] 5. Analyzing feedback and improving generative AI models

[1556] The server analyzes the feedback and continuously improves the generative AI model and emotion engine, thereby improving the accuracy of the system's suggestions so that future suggestions are more in line with the user's preferences and emotions.

[1557] Specific examples

[1558] For example, consider the case where three users (A, B, and C) are choosing a restaurant for dinner.

[1559] 1. Device: Persons A, B, and C log in to the app on their respective devices.

[1560] 2. Server: Each user's preference information (A-san prefers "Italian food," B-san prefers "Japanese food," and C-san prefers "healthy food") and their emotional state as recognized by the emotion engine at that time are obtained from the database.

[1561] 3. Server: Selects the optimal generative AI model for each user and combines the "Italian Restaurant GPT," "Japanese Restaurant GPT," and "Healthy Restaurant GPT" with the emotion engine.

[1562] 4. Server: Using the integrated model, it searches for restaurants that match the preferences and emotions of all users and generates multiple suggestions.

[1563] 5. Server: Sends the generated proposals to each user's device.

[1564] 6. Terminal: Displays the received suggestions to the user.

[1565] 7. User: Enters feedback on the suggestions and the emotional state is also reflected by the emotion engine.

[1566] 8. Device: Sends feedback to the server.

[1567] 9. Server: Analyzes feedback and improves generative AI models and emotion engines.

[1568] This system can effectively integrate the different preferences and emotions of multiple users and provide optimal suggestions to the entire group.

[1569] The processing flow will be explained below.

[1570] Step 1:

[1571] The terminal displays an authentication screen for the user to access the system. The user enters authentication information (user ID and password).

[1572] Step 2:

[1573] The terminal transmits the input authentication information to the server.

[1574] Step 3:

[1575] The server compares the received authentication information with the database to authenticate the user. If authentication is successful, the session begins.

[1576] Step 4:

[1577] The server retrieves profile information of the authenticated user from the database, including the user's preferences, past behavior history, customization settings, and emotional state data from the emotion engine.

[1578] Step 5:

[1579] The server selects the optimal generative AI model for each user based on the acquired profile information. Taking into account the user's emotional state, for example, it might select "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[1580] Step 6:

[1581] The server combines the selected generative AI models to build an integrated model, which integrates information from the combined generative AI models with emotional data from the emotion engine to analyze the needs, preferences, and emotions of the entire population.

[1582] Step 7:

[1583] The server uses the integrated model to generate optimal recommendations that reflect the needs, preferences, and emotional state of the entire population, such as generating multiple candidates for "healthy restaurants that combine Italian and Japanese cuisine."

[1584] Step 8:

[1585] The server transmits the generated proposals to each user's terminal.

[1586] Step 9:

[1587] The device displays the received suggestions to the user, including the restaurant name, location, menu details, and ratings.

[1588] Step 10:

[1589] Users select their preferred option from the options presented and enter their feedback, which can include their level of satisfaction, what they would like to add, or what needs to be corrected. The emotion engine also analyzes the user's emotional state in real time and reflects this in the feedback.

[1590] Step 11:

[1591] The terminal transmits the user's feedback and emotion data to the server.

[1592] Step 12:

[1593] The server analyzes the received feedback and emotion data to improve the generative AI model and emotion engine, so that future suggestions will better match the user's preferences and emotions.

[1594] Specific examples

[1595] Consider three users (A, B, and C) having dinner together.

[1596] Step 1:

[1597] Device: Persons A, B, and C each log in to the app on their own smartphones.

[1598] Step 2:

[1599] Terminal: Sends the entered authentication information (user ID and password) to the server.

[1600] Step 3:

[1601] Server: Compares the received authentication information with the database and performs authentication. If authentication is successful, a session begins.

[1602] Step 4:

[1603] Server: Retrieves profile information for each authenticated user from a database, including preferences, past behavioral history, customization settings, and current emotional state.

[1604] Step 5:

[1605] Server: Based on the profile information and emotional state, select "Italian Restaurant GPT" for person A, "Japanese Restaurant GPT" for person B, and "Healthy Restaurant GPT" for person C.

[1606] Step 6:

[1607] Server: Combines selected generative AI models to build an integrated model that integrates everyone's preferences and emotions.

[1608] Step 7:

[1609] Server: Using the integrated model, it generates multiple restaurant candidates that take into account everyone's preferences and emotions. For example, it generates candidates for "healthy restaurants that combine Italian and Japanese cuisine."

[1610] Step 8:

[1611] Server: Sends the generated proposals to the devices of A, B, and C.

[1612] Step 9:

[1613] Terminal: Displays the received suggestions to the user, including detailed restaurant information.

[1614] Step 10:

[1615] User: Selects the preferred option from the suggested options and enters feedback. The emotion engine analyzes the user's emotional state in real time and reflects that in the feedback.

[1616] Step 11:

[1617] Terminal: Sends feedback and emotion data to the server.

[1618] Step 12:

[1619] Server: Analyzes feedback and sentiment data to improve generative AI models and sentiment engines.

[1620] In this way, the system can effectively integrate the different preferences and emotions of multiple users and provide optimal suggestions for the entire population.

[1621] Example 2

[1622] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1623] Conventional suggestion systems make suggestions based solely on the user's preferences, resulting in a lack of personalized suggestions that reflect the user's real-time emotional state. Furthermore, the selection and integration of generative AI models was not centralized, making it difficult to improve the quality of the generated suggestions. This resulted in a decline in user satisfaction and an inability to fully utilize the system's effectiveness.

[1624] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1625] In this invention, the server includes means for authenticating users, means for collecting profile information for each user, means for selecting a generative AI model that suits each user, means for recognizing the user's emotional state, means for combining the selected generative AI model with an emotion engine to build an integrated model, means for analyzing the needs, preferences, and emotional state of the entire group using the integrated model, means for generating optimal proposals for the entire group, means for transmitting the generated proposals to each user's device, means for collecting user feedback and emotion recognition results, and means for analyzing the collected feedback and emotion recognition results to improve the generative AI model and emotion engine, thereby enabling more accurate and personalized proposals that reflect the user's real-time emotional state.

[1626] "Means of user authentication" refers to the method of having a user enter the necessary information (such as a user ID and password) to verify that they are a legitimate user when they access a system.

[1627] "Profile information" refers to detailed information about each user, such as personal data, preference information, and history information about the user.

[1628] The "means for selecting a generative AI model" is a method for selecting the optimal generative AI model based on the user's profile information and emotional state.

[1629] "Means for recognizing emotional state" refers to technologies (such as voice analysis and facial expression recognition) used to analyze a user's emotions in real time and identify their state.

[1630] "Means for constructing an integrated model by combining a generative AI model and an emotion engine" refers to a method for linking a selected generative AI model with an emotion engine to construct an integrated model.

[1631] The "means for analyzing the needs, preferences, and emotional states of an entire population" is a method for analyzing the needs, preferences, and emotional states of multiple users using an integrated model.

[1632] The "means for generating optimal proposals" is a method for creating the most suitable proposals for the user based on the analysis results.

[1633] The "means for transmitting the generated proposal to the terminal of each user" is a method for distributing the generated proposal to the terminal used by the user.

[1634] "Means for collecting feedback and emotion recognition results" refers to a method for collecting user evaluations and additional opinions on suggestions, as well as emotion analysis results from the emotion engine.

[1635] "Means for analyzing feedback and emotion recognition results to improve generative AI models and emotion engines" refers to methods for adjusting and improving generative AI models and emotion engines based on collected feedback and emotion data.

[1636] This invention is a system that authenticates users, collects profile information, selects and combines generative AI models, generates and shares suggestions, collects user feedback, and improves the generative AI models based on the feedback. Furthermore, by combining it with an emotion engine that recognizes user emotions, it provides more accurate and personalized suggestions.

[1637] The main components of the system are:

[1638] "Emotion Engine" that recognizes user emotions and reflects them in feedback

[1639] "Server" and "Terminal" that manage user profile information

[1640] 1. User authentication and information collection

[1641] The terminal displays an authentication screen for the user to access the system. The terminal collects the user ID and password entered by the user and sends this information to the server. The server verifies the received authentication information and retrieves the corresponding user profile information from a database. At this time, encrypted communication such as SSL is used for authentication to ensure the security of the information.

[1642] 2. Selecting and combining generative AI models

[1643] The server analyzes the acquired profile information. Data analysis tools such as Python are used for the analysis to select the optimal generative AI model based on the user's preferences and past behavioral history. At this time, the emotion engine analyzes the user's emotional state in real time, and this information is also added to the profile. The selected generative AI model and the emotion engine are then combined to create an integrated model. This integrated model is realized using machine learning frameworks such as TensorFlow and PyTorch.

[1644] 3. Generate and send optimal proposals

[1645] The server uses the integrated generative AI model to generate optimal suggestions based on the user's preferences and emotional state. For example, if a user prefers Italian restaurants with a relaxed atmosphere, the server generates multiple suggestions that reflect that information. The generated suggestions are sent to each user's device via a REST API. The device then displays the received suggestions on its screen, allowing the user to easily access them.

[1646] 4. User Emotion Recognition Feedback and Model Improvement

[1647] The user selects the preferred suggestion from the displayed suggestions and enters feedback regarding satisfaction, additions, and corrections. Specifically, the user uses a feedback form or rating slider. Furthermore, the emotion engine analyzes the user's emotional state and includes this information in the feedback. The device then sends this feedback information to the server.

[1648] 5. Analyzing feedback and improving generative AI models

[1649] The server performs a detailed analysis of the received feedback, using natural language processing and data mining techniques to extract patterns from the collected data. Based on the results, the generative AI model and emotion engine are refined. This continuous improvement ensures that future suggestions are more in line with the user's preferences and emotions.

[1650] Specific examples

[1651] For example, consider the case where three users (user A, user B, and user C) are choosing a restaurant for dinner.

[1652] 1. Device: User A, User B, and User C log in to the app on their respective devices.

[1653] 2. Server: Obtains each user's preference information (User A is "Italian food," User B is "Japanese food," and User C is "healthy food") and the emotional state recognized by the emotion engine at that time from the database.

[1654] 3. Server: Selects the optimal generative AI model for each user, for example, "Italian Restaurant GPT," "Japanese Restaurant GPT," and "Healthy Restaurant GPT," and combines them with an emotion engine.

[1655] 4. Server: Using the integrated model, it searches for restaurants that match the preferences and emotions of all users and generates multiple suggestions.

[1656] 5. Server: Sends the generated proposals to each user's device.

[1657] 6. Terminal: Displays the received suggestions to the user.

[1658] 7. User: Enters feedback on the suggestions and the emotional state is also reflected by the emotion engine.

[1659] 8. Device: Sends feedback to the server.

[1660] 9. Server: Analyzes feedback and improves generative AI models and emotion engines.

[1661] Prompt Sentence Examples

[1662] "User A currently likes Italian food, so please display several Italian restaurant suggestions for tonight's dinner. Please also consider the user's emotional state and prioritize restaurants with a relaxing atmosphere."

[1663] This system can effectively integrate the different preferences and emotions of multiple users and provide optimal suggestions to the entire group.

[1664] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1665] Step 1: User authentication and information gathering

[1666] The terminal displays an authentication screen for the user to access the system. The user enters their user ID and password and clicks the "Login button." Based on this input, the terminal sends the user ID and password to the server. The server searches for user information in a database, and if the authentication information is correct, retrieves profile information. At this time, encrypted communication such as SSL is used to ensure the security of the information. If authentication is successful, the user's profile information is sent from the server to the terminal.

[1667] Step 2: Selecting and combining generative AI models

[1668] The server analyzes the acquired profile information. This analysis uses data analysis tools such as Python to select the optimal generative AI model based on the user's preferences and past behavioral history. The emotion engine also becomes active, analyzing the user's real-time emotional state and adding this information to the profile. For example, if the user is looking for a "relaxed atmosphere," a generative AI model that reflects this is selected. The selected generative AI model is integrated with the emotion engine to build an integrated model. The input data is the profile information and emotional state, and the output is the generative AI model and the integrated model.

[1669] Step 3: Generate and submit optimal proposals

[1670] The server uses the integrated generative AI model to generate optimal suggestions based on the user's preferences and emotional state. For example, it generates multiple candidates for "relaxing Italian restaurants." The generative AI model uses data processing and machine learning algorithms to generate suggestions that match the user's preferences and emotions. The generated suggestions are sent to each user's device via a REST API. The input data is the integrated model, and the output is the generated suggestions. The device displays the received suggestions to the user.

[1671] Step 4: User emotion recognition feedback and model improvement

[1672] The user selects the suggestion they like best from the ones displayed and enters feedback regarding their satisfaction, additions they would like to make, and any corrections they would like to make. Specifically, they use a feedback form or a rating slider. Furthermore, the emotion engine analyzes the user's emotional state and includes this information in the feedback. For example, it may recognize from facial expression analysis that the user is "very satisfied" with the suggestion. The device then sends this feedback information to the server. The input data is the user's feedback and emotional state, and the output is the sent feedback information.

[1673] Step 5: Analyze feedback and improve the generative AI model

[1674] The server performs a detailed analysis of the received feedback. This analysis uses natural language processing and data mining techniques to extract patterns from the collected data. Based on the results, the generative AI model and emotion engine are improved. For example, the reason why a particular suggestion was not liked can be analyzed and the algorithm adjusted. This continuous improvement ensures that future suggestions will better match the user's preferences and emotions. The input data is feedback information and emotional state, and the output is an improved generative AI model and emotion engine.

[1675] (Application example 2)

[1676] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1677] Conventional recommendation systems have a problem in that they make suggestions based solely on the user's preference information. In particular, if the user's emotional state is not taken into account, the accuracy of the suggestions and user satisfaction may decrease. Furthermore, it is not easy to address the preferences and needs of an entire group, making it difficult to make appropriate suggestions to individual users. Therefore, there is a need for a system that takes into account the user's preference information and emotional state in an integrated manner and provides more personalized and appropriate suggestions.

[1678] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1679] In this invention, the server includes means for utilizing an emotion engine to reflect the user's emotional state, means for selecting a generative AI model suited to each user, and means for combining the selected generative AI models to construct an integrated model. This allows for the generation of suggestions that take into consideration the user's preference information and emotional state in an integrated manner, thereby improving user satisfaction.

[1680] "Means for authenticating users" refers to a function that allows a user to input authentication information required to access the system and verify that information.

[1681] The "means for collecting profile information of each user" is a function for collecting personal information, preference information, and history data provided by the user.

[1682] "Means for selecting a generative AI model that suits each user" is a function that selects the optimal generative AI model based on the user's profile information and emotional state.

[1683] "Means for combining selected generative AI models to construct an integrated model" refers to a function for combining multiple generative AI models to construct a single integrated model.

[1684] "Means for analyzing the needs and preferences of an entire group using an integrated model" is a function that utilizes an integrated model to analyze the needs and preferences of multiple users.

[1685] "Means for utilizing the emotion engine" refers to a function that recognizes the user's emotional state from facial expressions, voice, etc., and captures this as data.

[1686] The "means for generating optimal proposals" is a function that generates the most suitable proposals for the user based on all collected data.

[1687] The "means for transmitting the generated proposal to each user's terminal" is a function for transmitting the generated proposal from the server to the user's terminal.

[1688] The "means for collecting user feedback" is a function for obtaining reactions and evaluations made by users to suggestions.

[1689] "Means for analyzing collected feedback to improve the generative AI model and emotion engine" refers to a function for analyzing feedback obtained from users and continuously improving the generative AI model and emotion engine based on that feedback.

[1690] MODE FOR CARRYING OUT THE INVENTION

[1691] System Overview

[1692] This invention is a system that makes optimal suggestions based on a user's preferences and emotional state. The system consists of the following main components: an "emotion engine" that recognizes the user's emotions and reflects them in feedback, a "server" that manages the user's profile information, and a "terminal." Specifically, the system performs user authentication, profile collection, selection and integration of appropriate generative AI models, generation and sharing of suggestions, feedback collection, and model improvement based on the feedback.

[1693] Program processing overview

[1694] The program for this system is implemented as an application installed on a smartphone or smart glasses. Below, we will explain the processing overview of each component.

[1695] User authentication and information collection

[1696] A user logs in to the system using a smartphone or smart glasses. The device sends user authentication information to the server, which then verifies the authentication information. If authentication is successful, the server retrieves the user's profile information from the database.

[1697] Selecting and combining generative AI models

[1698] The server selects the optimal generative AI model based on the user's profile information and the emotion data recognized by the emotion engine. For example, it builds an integrated model by combining multiple generative AI models, such as "Chinese cuisine GPT," "Hamburger GPT," and "Vegan cuisine GPT."

[1699] Generate and send optimal proposals

[1700] The server uses the integrated model to generate optimal food delivery suggestions that reflect the user's preferences and emotional state. The suggestions are then sent to the user's device, which then displays the suggestions to the user.

[1701] User emotion recognition feedback and model improvement

[1702] The user selects from the presented suggestions and enters feedback. The emotion engine also recognizes the user's emotional state at the time of the feedback, and the device sends this feedback to the server. The server analyzes the received feedback and improves the generative AI model and emotion engine.

[1703] Hardware and software used

[1704] Hardware: Smartphones, smart glasses, servers

[1705] Software: Emotion recognition engine (EmotionEngine), AI model selection and combination library (ai_model_selector), database

[1706] Specific examples

[1707] For example, if a user is considering ordering dinner delivery, the system might:

[1708] 1. Authentication and profile information acquisition: The user logs in to the app on their smartphone, their authentication information is sent to the server, and their profile information is acquired.

[1709] 2. Recognizing emotional state: The emotion engine recognizes the user's emotional state, and the server receives the emotion data.

[1710] 3. Selection of generative AI model: The server selects the optimal generative AI model based on the profile information and emotional state, and constructs an integrated model.

[1711] 4. Proposal generation and transmission: The integrated model is used to generate optimal food delivery proposals and transmit them to the user's device.

[1712] 5. Feedback collection and model improvement: The user enters feedback and their emotional state is also recorded, which is sent to the server and used to improve the model.

[1713] Prompt Sentence Examples

[1714] "User Preferences: Chinese food"

[1715] "User's emotional state: Stressed"

[1716] "Proposal requirements: Please propose a quick and delicious Chinese food delivery menu."

[1717] This allows the system to make personalized suggestions that take into account the user's preferences and emotional state, increasing user satisfaction.

[1718] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1719] Step 1:

[1720] The terminal prompts the user to enter authentication information (user ID and password). The entered authentication information is sent to the server. The server compares the received authentication information with the database and determines whether the authentication was successful or not. If the authentication is successful, the user's profile information is retrieved from the database.

[1721] Input: User ID, Password

[1722] Data processing: Verification of authentication information

[1723] Output: User profile information, authentication result (success / failure)

[1724] Step 2:

[1725] The server uses an emotion engine to recognize the user's emotional state based on the acquired profile information. Sensors such as the device's camera and microphone are used to collect emotional data in real time, which is then analyzed by the emotion engine.

[1726] Input: User profile information, real-time emotional data (images, voice)

[1727] Data processing: Analysis of emotional data, recognition of emotional states

[1728] Output: User's emotional state

[1729] Step 3:

[1730] The server selects an appropriate generative AI model based on the user's profile information and emotional state. The server analyzes the profile information and emotional data and selects an appropriate generative AI model, such as "Chinese Food GPT" or "Hamburger GPT."

[1731] Input: User profile information, emotional state

[1732] Data processing: Selection of generative AI model

[1733] Output: A list of selected generative AI models

[1734] Step 4:

[1735] The server combines the selected generative AI models to build an integrated model. For example, it combines the selected "Chinese food GPT" and "Hamburger GPT" to create an integrated model.

[1736] Input: A list of selected generative AI models

[1737] Data processing: Combining generative AI models

[1738] Output: Integrated model

[1739] Step 5:

[1740] The server uses the integrated model to generate food delivery suggestions that reflect the user's preferences and emotional state. The server inputs the prompt sentence into the integrated model to generate suggestions.

[1741] Input: Integrated model, prompt statement

[1742] Data processing: Proposal generation

[1743] Output: Food delivery suggestions

[1744] Step 6:

[1745] The server sends the generated proposals to each user's terminal, which then displays the received proposals on the user's screen.

[1746] Input: Food delivery suggestions

[1747] Data Processing: Submit Proposal

[1748] Output: The proposal displayed on the user's device

[1749] Step 7:

[1750] The user selects from the presented suggestions and inputs feedback, and the terminal transmits the user's feedback and current emotional state to the server.

[1751] Input: User feedback, emotional state

[1752] Data processing: Feedback collection

[1753] Output: Feedback sent to the server

[1754] Step 8:

[1755] The server analyzes the collected feedback and improves the generative AI model and emotion engine. The server analyzes the feedback data and improves the accuracy of future suggestions.

[1756] Input: User feedback, emotional state

[1757] Data processing: analyzing feedback, improving generative AI models and emotion engines

[1758] Output: Improved generative AI models and emotion engines

[1759] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1760] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1761] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1762] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1763] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1764] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1765] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1766] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1767] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1768] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1769] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1770] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1771] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1772] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1773] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1774] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1775] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1776] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1777] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1778] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1779] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1780] The following is further disclosed regarding the above embodiment.

[1781] (Claim 1)

[1782] a means for authenticating a user;

[1783] a means for collecting profile information for each user;

[1784] A means for selecting a generative AI model that is suitable for each user;

[1785] A means for combining the selected generative AI models to construct an integrated model; and

[1786] a means of analyzing the needs and preferences of the entire population using integrated models;

[1787] a means for generating optimal proposals for the entire population;

[1788] means for transmitting the generated proposals to each user's terminal;

[1789] a means for collecting user feedback;

[1790] A means to analyze collected feedback and improve generative AI models

[1791] A system including:

[1792] (Claim 2)

[1793] 10. The system of claim 1, wherein the system generates optimal restaurant suggestions using user preference information.

[1794] (Claim 3)

[1795] The system of claim 1 combines different types of generative AI models to make recommendations tailored to the needs of the entire population.

[1796] "Example 1"

[1797] (Claim 1)

[1798] a means for authenticating a user;

[1799] a means for collecting profile information for each user;

[1800] A means for selecting a generative AI model that is suitable for each user;

[1801] A means for combining the selected generative AI models to construct an integrated model; and

[1802] a means of analyzing the needs and preferences of the entire population using integrated models;

[1803] a means for generating optimal proposals for the entire population;

[1804] means for transmitting the generated proposals to each user's terminal;

[1805] a means for collecting user feedback;

[1806] A means to analyze the collected feedback to improve the generative AI model; and

[1807] a means for detailing the optimal suggestion using a prompt sentence;

[1808] Means of notifying and displaying the proposal

[1809] A system including:

[1810] (Claim 2)

[1811] 10. The system of claim 1, wherein the system uses user preference information to generate optimal location suggestions.

[1812] (Claim 3)

[1813] The system of claim 1 combines different types of generative AI models to make recommendations tailored to the needs of the entire population.

[1814] "Application Example 1"

[1815] (Claim 1)

[1816] a means for authenticating a user;

[1817] a means for collecting profile information for each user;

[1818] A means for selecting a generative AI model that is suitable for each user;

[1819] A means for combining the selected generative AI models to construct an integrated model; and

[1820] a means of analyzing the needs and preferences of the entire population using integrated models;

[1821] a means for generating optimal proposals for the entire population;

[1822] means for transmitting the generated proposals to each user's terminal;

[1823] a means for collecting user feedback;

[1824] A means to analyze collected feedback and improve generative AI models

[1825] A system including:

[1826] (Claim 2)

[1827] 2. The system according to claim 1, wherein the system generates optimal restaurant suggestions using user preference information.

[1828] (Claim 3)

[1829] The system of claim 1 combines different types of generative AI models to make recommendations tailored to the needs of the entire population.

[1830] "Example 2: Combining Emotion Engines"

[1831] (Claim 1)

[1832] a means for authenticating a user;

[1833] a means for collecting profile information for each user;

[1834] A means for selecting a generative AI model that is suitable for each user;

[1835] means for recognizing the emotional state of a user;

[1836] A means for combining the selected generative AI model and the emotion engine to build an integrated model; and

[1837] a means of analyzing the needs, preferences, and emotional states of entire populations using integrative models;

[1838] a means for generating optimal proposals for the entire population;

[1839] means for transmitting the generated proposals to each user's terminal;

[1840] a means for collecting user feedback and emotion recognition results;

[1841] A means to analyze collected feedback and emotion recognition results to improve generative AI models and emotion engines

[1842] A system including:

[1843] (Claim 2)

[1844] 10. The system of claim 1, wherein the system uses the user's preference information and emotional state to generate optimal restaurant suggestions.

[1845] (Claim 3)

[1846] The system of claim 1, which combines different types of generative AI models to make suggestions tailored to the needs and emotional state of the entire population.

[1847] "Application example 2 when combining emotion engines"

[1848] (Claim 1)

[1849] a means for authenticating a user;

[1850] a means for collecting profile information for each user;

[1851] A means for selecting a generative AI model that is suitable for each user;

[1852] A means for combining the selected generative AI models to construct an integrated model; and

[1853] a means of analyzing the needs and preferences of the entire population using integrated models;

[1854] a means for utilizing an emotion engine to reflect the user's emotional state in the generated suggestions;

[1855] a means for generating optimal proposals for the entire population;

[1856] means for transmitting the generated proposals to each user's terminal;

[1857] a means for collecting user feedback;

[1858] A means to analyze collected feedback to improve generative AI models and emotion engines

[1859] A system including:

[1860] (Claim 2)

[1861] 10. The system of claim 1, wherein the system uses the user's preference information and emotional state to generate optimal food delivery suggestions.

[1862] (Claim 3)

[1863] The system of claim 1 combines different types of generative AI models and an emotion engine to make suggestions tailored to the needs and emotional state of the entire population. [Explanation of symbols]

[1864] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for authenticating a user; a means for collecting profile information for each user; A means for selecting a generative AI model that is suitable for each user; A means for combining the selected generative AI models to construct an integrated model; and a means of analyzing the needs and preferences of the entire population using integrated models; a means for generating optimal proposals for the entire population; means for transmitting the generated proposals to each user's terminal; a means for collecting user feedback; A means to analyze collected feedback and improve generative AI models A system including:

2. The system of claim 1 , wherein the system generates optimal restaurant suggestions using user preference information.

3. The system of claim 1, which combines different types of generative AI models to make suggestions tailored to the needs of the entire population.

Images