system

Abstract

We provide the system. [Solution] Means for providing a computing device that includes individually adjusted generative models, A means for aggregating generative models from multiple computing devices and constructing an integrated model, A means for generating and outputting group-related information based on an integrated model, A means to save the usage history of collected information and reflect it in future optimizations, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] [In a conventionally individually adjusted generation model, when multiple users make a joint decision, it has been difficult to provide an optimal proposal that meets the preferences and requirements of all. For this reason, there is a need for an efficient and flexible system that can improve the satisfaction of the entire group while also meeting individual requirements.]

Means for Solving the Problems

[0005] [This invention provides a computing device that includes individually adjusted generative models, and generates suggestions tailored to the preferences of an entire group by aggregating generative models from multiple computing devices to construct an integrated model. Furthermore, it has a feedback function that acquires preference information from users and uses it to adjust the generative models, and evaluates the user's selection results, thereby improving the accuracy of suggestions in subsequent instances.]

[0006] A "personally tailored generative model" is an artificial intelligence model that is customized to reflect specific patterns and characteristics based on each user's preferences and history.

[0007] "Computing device" refers to [a machine or device for processing data and performing specific calculations, particularly a computer system for running generative models].

[0008] An "integrated model" is an AI model that combines multiple individually tailored generative models to enable analysis and proposals for an entire group.

[0009] A "group" refers to [a collection of multiple users or individuals participating in decision-making, and is the object of optimization of processing based on their combined needs and preferences].

[0010] "Preference information" refers to data about a user's interests and preferences, including, for example, food preferences and past selection history.

[0011] A "feedback function" is a function that evaluates and improves the accuracy and usefulness of a model based on the user's choices and actions.

[0012] A "proposal" is [information and options generated from a model that meet the needs of users or groups, and are provided to support decision-making]. [Brief explanation of the drawing]

[0013] [Figure 1]This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] The present invention can be implemented as a system that integrates individually tailored generative models to provide suggestions tailored to the preferences and needs of a group. Typically, this system is applied in situations involving multiple users, generating optimal suggestions based on each user's information. The main components and operation of this system are described below.

[0035] Server role:

[0036] The server receives preference information sent from each user. This includes user preferences, past selection history, and allergy information. Based on this information, the server calculates a generative model specific to each user and stores it in the database. This model is tailored to the individual requests of each user.

[0037] The role of the integrated model:

[0038] The server integrates the generative models of multiple users as needed to generate a new integrated model. This integrated model aims to optimize decision-making for the entire group, and is used to generate suggestions that match the requirements of each user. For example, if multiple people need to choose a restaurant, the server can generate options that everyone will agree on.

[0039] Proposal generation and feedback:

[0040] The generated suggestions are delivered to the user's device and managed by the server. Users provide feedback by reviewing and selecting suggestions. This feedback is analyzed by the server and used to optimize the system for future suggestions.

[0041] Specific example:

[0042] For example, consider a scenario where friends plan to watch a movie together. Each user sends their favorite movie genres and movies they want to see to the server. The server collects each user's information and creates individual generative models based on it. Then, it integrates multiple generative models to generate a list that includes everyone's favorite genres and movies of interest. This list is displayed on the users' devices, and feedback from everyone can be used to further optimize future suggestions.

[0043] Thus, by using this invention, it becomes possible to make optimal decisions as a group while taking individual preferences into account. This achieves a level of personalization and comprehensiveness not found in conventional individual models.

[0044] The following describes the processing flow.

[0045] Step 1:

[0046] User: Enters their preferences into the device. This includes favorite genres, allergy information, topics of interest, etc.

[0047] Step 2:

[0048] Terminal: Formats the entered information into a data package and sends it to the server. The information is encrypted using a secure protocol.

[0049] Step 3:

[0050] Server: Analyzes preference information received from terminals and extracts features to create personalized generative models tailored to each user.

[0051] Step 4:

[0052] Server: Builds generative models tailored to each user and stores them in a database. These models are optimized using machine learning algorithms.

[0053] Step 5:

[0054] Server: If necessary, aggregates the generative models of multiple users to build an integrated model. The integrated model enables comprehensive recommendations based on the preferences of the entire group.

[0055] Step 6:

[0056] Server: Generates optimal suggestions based on the integrated model and creates specific options. These options are situation-dependent, such as a list of restaurants or movie choices.

[0057] Step 7:

[0058] Server: Distributes suggestions to the user's device. Suggestions are presented as notifications or in-app messages.

[0059] Step 8:

[0060] User: Review the options presented on the device and select the one they deem most preferable. The selection can be made with simple operations.

[0061] Step 9:

[0062] Terminal: Sends user selections to the server and provides feedback data.

[0063] Step 10:

[0064] Server: Analyzes received feedback and evaluates the quality of the selections. Based on this, the generative model is readjusted and used for future suggestions.

[0065] Each of the steps described above is designed to ensure smooth and effective information processing across the entire system.

[0066] (Example 1)

[0067] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0068] While existing information processing systems can provide suggestions based on individual user preferences, they struggle to create optimal suggestions for the group as a whole. Therefore, there is a growing need for a system that can provide suggestions that reflect the group's opinions while also satisfying each individual user, especially when multiple users share common goals and interests.

[0069] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0070] In this invention, the server includes means for providing an information processing device that includes individually tailored generative models, means for aggregating generative models from multiple information processing devices and constructing an integrated model, and means for generating and outputting relevant group information based on the integrated model. This makes it possible to automatically generate optimal proposals for the group as a whole while respecting the needs of individual users, and to encourage appropriate decision-making among members.

[0071] An "information processing device" refers to a mechanical or electronic device that can input, process, and output data.

[0072] A "generative model" refers to a set of algorithms trained to produce an expected output based on a specific input.

[0073] An "integrated model" refers to a unified algorithm that aggregates multiple individual generative models to provide the optimal overall output.

[0074] "Group-related information" refers to information that reflects the opinions and preferences of all users and is useful for decision-making for the group as a whole.

[0075] "Usage history" refers to a record of information about choices a user has made in the past and the results thereof.

[0076] "Preference information" refers to information that indicates an individual user's preferences and interests.

[0077] "Analysis function" refers to a process that has the ability to analyze data in detail and contributes to evaluating results and improving models.

[0078] "Suggestions" refer to recommendations that encourage users to make choices or take actions that are appropriate to their needs and circumstances.

[0079] This invention is an information processing system that uses individually tailored generative models to provide suggestions based on the preferences and needs of a group.

[0080] The server receives data such as preference information and past selection history from each user as input. This data is processed using a generative AI model to create individual generative models for each user. These generative models reflect each user's preferences and are stored in a database. If necessary, the server integrates these individual generative models to build an integrated model that provides optimal suggestions for the group.

[0081] The terminal displays server-generated suggestions to the user. These suggestions are delivered via an interface to facilitate user selection. The user selects a suggestion through the terminal and sends feedback to the server. The server analyzes the feedback received and uses it to further optimize future suggestions.

[0082] For example, when friends plan a trip together, their preferences for destinations and budget constraints are entered into the server. The server creates a generative model for each user, integrates them, and proposes a travel plan that satisfies everyone. This proposal is delivered to the user's device, and the final selection is sent back to the server as user feedback.

[0083] An example of a prompt to a generative AI model might be: "Please suggest the best travel destination for our group. Here are each member's preferences: Member A wants a beach, Member B wants historical sites, and Member C wants to go shopping."

[0084] In this way, the system can optimize the group as a whole while also providing personalized suggestions to each user.

[0085] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0086] Step 1:

[0087] Users use their devices to send preference information and past selection history to the server. This input includes categories of interest and past behavioral data. The server receives this data and stores it in a database specific to each user.

[0088] Step 2:

[0089] The server uses a generative AI model to create personalized generative models based on user preference information. Using the preference information received as input, it analyzes user preference patterns in a data-driven approach. Based on this analysis, a personalized generative model is output and saved.

[0090] Step 3:

[0091] If necessary, the server integrates multiple individual generative models to generate a new integrated model. The input data consists of generative models from multiple users, and the output is an integrated model that optimizes the opinions of the entire group. The generated integrated model comprehensively considers the preferences of different users to produce the optimal proposal for the group.

[0092] Step 4:

[0093] The server uses an integrated model to generate proposals and delivers them to the user's terminal. The input is the generated integrated model, and the output is the new proposal content displayed on the user's terminal. The terminal receives these proposals and presents them visually in an easy-to-use interface for the user to select from.

[0094] Step 5:

[0095] Users review suggestions via their terminal and send feedback on their selected items to the server. The input data is the selection result for the suggestions, which the server receives and analyzes. As output, the feedback information is stored and used to improve the accuracy of future suggestions.

[0096] In this way, the system constructs individual and integrated generative models from input data and provides users with personalized and collectively optimized suggestions.

[0097] (Application Example 1)

[0098] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] In modern information processing, it is crucial to consider the preferences and needs of multiple individuals simultaneously and optimize group decision-making. However, conventional systems often fail to adequately reflect individual preferences, making it difficult to propose solutions that satisfy the entire group. This invention aims to solve this problem and provide optimal proposals that reflect the preferences of each individual, particularly in scenarios involving multiple people.

[0100] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0101] In this invention, the server includes means for providing an information processing device that includes individually adjusted generation models; means for aggregating generation models from multiple information processing devices and constructing an integrated model; means for generating and outputting suggestion information based on the group's requests; means for saving the usage history of recorded information and reflecting it in subsequent optimizations; and means for having a suggestion function that optimizes dish selection based on the individual preferences of multiple people. This makes it possible to make optimal suggestions that take into account the preferences of multiple individuals.

[0102] A "personally tailored generative model" is an information processing model that reflects the preferences and requirements of each user and is optimized for a specific purpose.

[0103] An "information processing device" is a device or system used to process data and perform calculations.

[0104] "Aggregating generative models and constructing an integrated model" refers to the process of gathering and integrating multiple generative models to construct a comprehensive model that reflects the needs and preferences of the entire group.

[0105] "Generating and outputting proposal information" means creating proposals suitable for a group based on an integrated model and providing them to users.

[0106] "Recorded information usage history" refers to data such as past user choices and responses to suggestions, which is stored to optimize future suggestions.

[0107] The "suggestion function that optimizes dish selection based on the individual preferences of multiple people" is a function that takes into account each user's preferences and limitations to determine the optimal dish selection for multiple people.

[0108] The server, acting as an information processing device, receives preference information from multiple users and creates individually tailored generative models based on this information. These generative models reflect the users' preferences and past selection history. The created generative models are stored in a database, and if necessary, the generative models of multiple users are aggregated to build an integrated model. This integrated model is used to optimize decision-making for the entire group.

[0109] Based on this integrated model, the server generates suggestion information that reflects the group's needs and outputs it to each user's terminal. The terminal operates as a smartphone application and displays this suggestion information to the user. When a user provides feedback on the suggestion, that information is also sent to the server and stored as a record. This recorded information is used to optimize future suggestions.

[0110] The hardware used includes smartphones and cloud servers (e.g., AWS®, Google® Cloud), while the software utilizes React Native for mobile applications, Node.js for server systems, and MongoDB for databases. For data processing, user-entered preference data is sent to the server in JSON format, and an integrated model is generated using Python machine learning libraries (e.g., TENSORFLOW®, PyTorch) for data calculations.

[0111] For example, when suggesting the best lunch menu for several employees, using a prompt like, "Please suggest lunch menus for 5 employees. Their preferences and allergy information are as follows: Person A likes sushi, Person B likes pizza, and Person C has a wheat allergy," makes it possible to suggest meals that will satisfy everyone.

[0112] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0113] Step 1:

[0114] Users input their preferences and restrictions into a smartphone application. This information is sent to the server as JSON data. This data provides detailed information about each user's preferences.

[0115] Step 2:

[0116] The server parses the received JSON data and extracts preference information for each user. Based on the extracted information, it generates individually tailored generative models using Python's machine learning libraries. The generated models are stored in a database, ready for use in the next step.

[0117] Step 3:

[0118] The server aggregates generative models created by multiple users to build an integrated model. The integrated model aims to optimize the requirements of the entire group. It compares and integrates data between models to create an information structure that supports optimal decision-making.

[0119] Step 4:

[0120] The server generates suggestion information that matches the group's requests based on an integrated model. This information includes optimal dish selections that reflect each user's preferences and allergy information. The generated suggestions are sent to the user's device as push notifications.

[0121] Step 5:

[0122] Users review the received suggestion information and provide feedback to the server through the application. This feedback is data indicating their evaluation and satisfaction with the suggestion, and is collected on the server in real time.

[0123] Step 6:

[0124] The server analyzes the collected feedback and uses it to improve the accuracy of the generative and integrated models. It uses the feedback data to adjust parameters to improve the accuracy of future proposals.

[0125] This processing flow ensures that users receive efficient and optimal suggestions that take their individual preferences into account.

[0126] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0127] This invention is a system that generates suggestions that take the user's emotions into account by combining an emotion engine with individually tuned generative models. This system utilizes emotion data along with user preference information to achieve advanced personalization that reflects the individual's emotional state.

[0128] Server role:

[0129] The server creates a generative model based on user preference information and emotion data received from the terminal. The emotion engine analyzes each user's emotional state from their speech, facial expressions, and input text, and generates emotion data. Based on this, it builds a personalized generative model for each user and stores it in the database.

[0130] Emotional engine function:

[0131] The emotion engine analyzes user emotions in real time during interaction, identifying emotional states such as positive, negative, and neutral. This allows the server to add emotional information to the generative model, enabling more refined personalization. This emotional information is considered a crucial element when generating suggestions.

[0132] Generation of an integrated model:

[0133] The server integrates individual generative models to generate an integrated model that reflects the emotional state of the entire group. This enables optimal suggestions for enhancing positive collective experiences.

[0134] Suggestions and feedback:

[0135] Suggestions generated using the integrated model are delivered to the user's device. The user evaluates the presented options and makes a selection. The device sends feedback data about this selection to the server. This feedback includes changes in emotional state as determined by the emotion engine, which the server uses to improve the accuracy of future suggestions.

[0136] Specific example:

[0137] For example, consider a scenario where a family decides how to spend a holiday. Each family member inputs their desired activity and the emotions they feel during that activity (excited, wanting to relax, etc.) into a device. An emotion engine analyzes this information, and the server creates individual generative models and builds an integrated model. Based on this, holiday suggestions (e.g., theme park or hot spring trip) are considered and sent to the user. Through feedback, the system is adjusted to provide the best possible suggestions that the whole family can enjoy.

[0138] Through the above process, the present invention is a system that can improve the quality of the user experience through advanced emotion-based customization.

[0139] The following describes the processing flow.

[0140] Step 1:

[0141] User: Enters information about their emotional state and preferences into the device. This information includes desired activities and the emotions associated with them (e.g., excitement, relaxation).

[0142] Step 2:

[0143] Terminal: Sends information entered by the user to the emotion engine and analyzes the emotional state in real time. The analyzed emotional information and preference data are sent together to the server.

[0144] Step 3:

[0145] Server: Based on preference and sentiment data received from terminals, it creates a generative model tailored to each user. The generative model is optimized using machine learning algorithms and incorporates individual sentiment data.

[0146] Step 4:

[0147] Server: Aggregates generative models from multiple users to build an integrated model that reflects the emotional state of the entire group. This integrated model considers numerous emotional patterns and serves as the foundation for generating optimal suggestions.

[0148] Step 5:

[0149] Server: Based on an integrated model, it generates suggestions that align with the collective sentiment. The suggestions are adjusted to balance user preferences and emotions, forming appropriate options.

[0150] Step 6:

[0151] Server: Sends generated suggestions to users' terminals and distributes them to individual users. Suggestions are displayed as options and presented in a way that allows for individual feedback.

[0152] Step 7:

[0153] User: Review the suggestions displayed on the device and make a selection. When making a selection, evaluate them based on your emotional state and initial preferences.

[0154] Step 8:

[0155] Terminal: The emotion engine re-analyzes the user's selection results and the resulting changes in their emotional state, and sends that data to the server.

[0156] Step 9:

[0157] Server: Based on the feedback received, the server adjusts the generative and integrated models. This allows for learning and optimization to improve the accuracy of future proposals.

[0158] This process enables personalization that comprehensively considers emotions and preferences, allowing us to provide the most appropriate suggestions for the user.

[0159] (Example 2)

[0160] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0161] Conventional generative model-based suggestion systems could consider user preference information, but they were insufficient for personalization that reflected the user's emotional state. As a result, suggestions aimed at improving the user experience were inaccurate, sometimes leading to decreased satisfaction. Furthermore, it was difficult to make suggestions that considered the emotional state of the entire group, resulting in the challenge of not being able to make effective suggestions for large groups.

[0162] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0163] In this invention, the server includes means for constructing individually tailored generative models based on user emotional information and preference information; means for collecting multiple generative models and generating an integrated model that reflects the emotional state of the entire group; and means for transmitting the suggested content generated based on the integrated model to an information output device. This makes it possible to generate detailed suggestions that reflect the emotional state of individual users and suggested content that promotes a positive experience for the group.

[0164] An "information processing device" is a device that processes data input by a user and has the function of creating a generative model based on emotional information and preference information.

[0165] A "generative model" is a model that is adjusted to reflect the user's individual emotional and preference information, and serves as the foundation for generating personalized suggestions.

[0166] An "integrated model" is a model created by aggregating multiple generative models and taking into account the emotional state of the entire group, and is used to generate proposals for large groups.

[0167] "Emotional information" refers to data that indicates the user's current emotional state, and is obtained from information such as voice, text, and facial expression analysis.

[0168] "Preference information" refers to data about users' long-term preferences and tastes, and serves as the basis for making personalized recommendations to individual users.

[0169] An "information output device" is a device that has an output function to provide the user with the suggested content generated from the server.

[0170] "Proposed content" refers to options or activity proposals generated from a generative or integrated model and presented to the user or group.

[0171] "Feedback" refers to the evaluation and selection results of user suggestions, and is data used to improve the accuracy of the system.

[0172] The system of this invention utilizes user emotional and preference information and achieves advanced personalization using individually tailored generative AI models. The server generates suggestions in real time based on data collected from each user's terminal.

[0173] 1. Hardware and software:

[0174] User terminal: Equipped with an interface for receiving voice and text input, and performs data processing using an energy-saving processor.

[0175] Server: A high-performance computing system for large-scale data processing, which performs emotion analysis using "EmotionAnalyzer" software. It also uses "AIModelBuilder" to build generative AI models.

[0176] Information output device: A device that presents suggestions generated from the server to the user.

[0177] 2. Data processing and data calculation:

[0178] The user's device converts voice input into text using "SpeechToTextConverter" and prepares for sentiment analysis.

[0179] The server uses "EmotionAnalyzer" to extract emotional information from user input and adjusts the generated AI model based on this information.

[0180] "AIModelBuilder" creates generative models for each user, and then generates an integrated model for the group.

[0181] 3. Specific examples and examples of prompt statements:

[0182] For example, consider a scenario where a family decides how to spend their holiday. Each family member inputs their desired activity and their corresponding feelings (e.g., "exciting," "relaxing") into a terminal. This information is analyzed by an emotion engine, and the server builds a generative AI model to generate suggestions that the whole family can enjoy. The optimal activity (e.g., visiting a theme park, going to a hot spring) is then suggested.

[0183] An example of a prompt message is, "What kind of activity would you like to do today? Please tell us how you feel about that activity (e.g., I'm excited, I want to relax)."

[0184] This system aims to improve the user experience by providing suggestions that take into account the user's emotional state.

[0185] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0186] Step 1:

[0187] The user inputs their daily activities into the device. If voice input is used, the device uses "SpeechToTextConverter" to convert the voice data into text. This input data becomes the basis for subsequent sentiment analysis.

[0188] Step 2:

[0189] The device sends the converted text data to the server. The server sends the received text data to "EmotionAnalyzer," which analyzes the user's emotional state. "EmotionAnalyzer" analyzes keywords and context in the text and generates emotional information such as positive, negative, or neutral. This is output as emotional data.

[0190] Step 3:

[0191] The server integrates the generated sentiment data with user preference information and inputs it into "AIModelBuilder." "AIModelBuilder" then constructs individually optimized generative AI models based on this information. These generative models form the basis for providing user-specific suggestions and are stored on the server.

[0192] Step 4:

[0193] The server aggregates the generative models of multiple users and generates an integrated model using a "group modeling tool" that takes into account the emotional state and common preferences of the entire group. This integrated model is used to derive the most suitable suggestions for the group.

[0194] Step 5:

[0195] The server generates group-oriented suggestions based on an integrated model and distributes them to users through an information output device. The suggestions are presented as specific activities and options, prompting users to make choices.

[0196] Step 6:

[0197] The user evaluates the presented suggestions and makes a selection. The device records feedback on the selection and subsequent changes in emotional state. This feedback is sent to the server and used to improve the accuracy of future suggestions.

[0198] The above outlines the processing flow of this system's program. Each step involves specific data processing and model generation, aiming to improve the user experience.

[0199] (Application Example 2)

[0200] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0201] This invention aims to solve the problem of improving the experience of individual users and groups by providing a system that takes into account the emotional state and preference information of users and provides appropriate information and suggestions in real time. Conventional systems sometimes fail to capture users' interest and satisfaction because they provide information without adequately considering the emotions of the users.

[0202] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0203] In this invention, the server includes means for providing a computing device that analyzes emotions and generates information based on the user's emotional state, means for providing a computing device that includes an individually adjusted generative model, and means for providing information that optimizes selected measures based on real-time emotion analysis results. This enables the provision of personalized information according to the user's emotional state.

[0204] A "computational device that analyzes emotions and generates information based on the user's emotional state" is a device that determines emotions from the user's facial expressions, speech, and input data, and generates information that matches those emotions.

[0205] A "computational device including individually tailored generative models" is a device equipped with a specific generative model based on each user's preference information and historical data.

[0206] "Means for aggregating generative models and constructing an integrated model" refers to the function responsible for the process of aggregating multiple individual generative models and creating a collective model.

[0207] "A means of providing information to optimize selected measures based on real-time sentiment analysis results" refers to a function that instantly analyzes users' sentiment data and uses the results to dynamically adjust the content of suggestions and information provided.

[0208] "Means for saving the usage history of collected information and reflecting it in future optimizations" refers to a function that saves user behavior information collected in the past and uses it to improve future suggestions and information provision.

[0209] "Having a feedback function" means having a function that receives responses and results from users and uses them to improve the system's operation.

[0210] The system for implementing the present invention utilizes emotional data and preference information to provide users with optimal information and suggestions. The system consists of a server, a terminal, an emotional analysis engine, and a computing device including a generative model.

[0211] First, the user's facial expressions, speech, and text input are collected from the device. The emotion analysis engine analyzes this data in real time to identify emotional states such as positive, negative, and neutral. For example, the user's facial expressions are captured with a camera and analyzed using OpenCV. In addition, the user's voice input is converted into text using speech recognition technology, and emotions are inferred based on this.

[0212] The server integrates analyzed sentiment data with past preference information to create a personalized generative model. Using TensorFlow, this generative model is newly generated and adjusted for each user. As a result, the server generates the most suitable information and optimizes selection strategies. This can be achieved, for example, by recommending products or content that might interest a user if they appear to be enjoying themselves.

[0213] Furthermore, this generated information and suggestions are delivered to the user's device. When suggestions are presented, feedback is collected and sent to the server. This feedback allows the integrated model to continuously improve its accuracy.

[0214] A concrete example is the shopping experience in a virtual store. If a user smiles while looking at a product, recommendations for related items will be enhanced. As an example of a prompt, inputting the instruction "When the user shows a relaxed expression, present options with a relaxing effect" into the generating AI model will facilitate the provision of appropriate information.

[0215] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0216] Step 1:

[0217] The device collects input from the user. Specifically, it captures the user's facial expressions with a camera and records their speech with a microphone. This input data is treated as important information for identifying the user's current emotional state. Facial expression data is sent to an emotion analysis engine through image processing, and speech data is converted into text using speech recognition technology.

[0218] Step 2:

[0219] The server analyzes input data sent from the terminal using an emotion analysis engine. Specifically, it uses OpenCV to identify emotions (positive, negative, neutral, etc.) from facial expression data. In addition, data converted from speech to text is used as preference information through natural language processing. By integrating these analysis results, the server determines the user's current emotional state.

[0220] Step 3:

[0221] The server creates an individualized generative model based on the determined emotional state and past preference information. TensorFlow is used to process this data and generate a user-specific generative model. This generated model functions as a foundation for generating optimal information and suggestions that match the user's emotions and preferences.

[0222] Step 4:

[0223] Using the generated model, the server creates optimal information and suggestions for the user and sends them to the terminal. In this process, the generating AI model uses instructions such as, "When the user shows a relaxed expression, present options with a relaxing effect." The generated suggestions are then presented to the user, for example, as product recommendations while shopping in a virtual store.

[0224] Step 5:

[0225] The device then collects user responses to the presented suggestions. Specifically, it acquires data on the user's selections and changes in their emotions during those selections. This feedback data is sent to a server and used for further analysis and to improve the accuracy of the generative model.

[0226] Step 6:

[0227] The server uses the collected feedback data to improve the accuracy of the integrated model. This process involves adjusting the model based on the feedback, improving the accuracy of subsequent information provision. Through this iterative process, the entire system enables more refined personalization tailored to the user's preferences and emotions.

[0228] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0229] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0230] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0231] [Second Embodiment]

[0232] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0233] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0234] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0235] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0236] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0237] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0238] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0239] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0240] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0241] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0242] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0243] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0244] The present invention can be implemented as a system that integrates individually tailored generative models to provide suggestions tailored to the preferences and needs of a group. Typically, this system is applied in situations involving multiple users, generating optimal suggestions based on each user's information. The main components and operation of this system are described below.

[0245] Server role:

[0246] The server receives preference information sent from each user. This includes user preferences, past selection history, and allergy information. Based on this information, the server calculates a generative model specific to each user and stores it in the database. This model is tailored to the individual requests of each user.

[0247] The role of the integrated model:

[0248] The server integrates the generative models of multiple users as needed to generate a new integrated model. This integrated model aims to optimize decision-making for the entire group, and is used to generate suggestions that match the requirements of each user. For example, if multiple people need to choose a restaurant, the server can generate options that everyone will agree on.

[0249] Proposal generation and feedback:

[0250] The generated suggestions are delivered to the user's device and managed by the server. Users provide feedback by reviewing and selecting suggestions. This feedback is analyzed by the server and used to optimize the system for future suggestions.

[0251] Specific example:

[0252] For example, consider a scenario where friends plan to watch a movie together. Each user sends their favorite movie genres and movies they want to see to the server. The server collects each user's information and creates individual generative models based on it. Then, it integrates multiple generative models to generate a list that includes everyone's favorite genres and movies of interest. This list is displayed on the users' devices, and feedback from everyone can be used to further optimize future suggestions.

[0253] Thus, by using this invention, it becomes possible to make optimal decisions as a group while taking individual preferences into account. This achieves a level of personalization and comprehensiveness not found in conventional individual models.

[0254] The following describes the processing flow.

[0255] Step 1:

[0256] User: Enters their preferences into the device. This includes favorite genres, allergy information, topics of interest, etc.

[0257] Step 2:

[0258] Terminal: Formats the entered information into a data package and sends it to the server. The information is encrypted using a secure protocol.

[0259] Step 3:

[0260] Server: Analyzes preference information received from terminals and extracts features to create personalized generative models tailored to each user.

[0261] Step 4:

[0262] Server: Builds generative models tailored to each user and stores them in a database. These models are optimized using machine learning algorithms.

[0263] Step 5:

[0264] Server: If necessary, aggregates the generative models of multiple users to build an integrated model. The integrated model enables comprehensive recommendations based on the preferences of the entire group.

[0265] Step 6:

[0266] Server: Generates optimal suggestions based on the integrated model and creates specific options. These options are situation-dependent, such as a list of restaurants or movie choices.

[0267] Step 7:

[0268] Server: Distributes suggestions to the user's device. Suggestions are presented as notifications or in-app messages.

[0269] Step 8:

[0270] User: Review the options presented on the device and select the one they deem most preferable. The selection can be made with simple operations.

[0271] Step 9:

[0272] Terminal: Sends user selections to the server and provides feedback data.

[0273] Step 10:

[0274] Server: Analyzes received feedback and evaluates the quality of the selections. Based on this, the generative model is readjusted and used for future suggestions.

[0275] Each of the steps described above is designed to ensure smooth and effective information processing across the entire system.

[0276] (Example 1)

[0277] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0278] While existing information processing systems can provide suggestions based on individual user preferences, they struggle to create optimal suggestions for the group as a whole. Therefore, there is a growing need for a system that can provide suggestions that reflect the group's opinions while also satisfying each individual user, especially when multiple users share common goals and interests.

[0279] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0280] In this invention, the server includes: means for providing an information processing apparatus including an individually adjusted generation model; means for aggregating generation models from a plurality of information processing apparatuses to construct an integrated model; and means for generating and outputting relevant information of the group based on the integrated model. Thereby, while respecting the needs of individual users, it becomes possible to automatically generate an optimal proposal for the entire group and prompt each member to make appropriate decisions.

[0281] The "information processing apparatus" refers to a mechanical or electronic device that can input, process, and output data.

[0282] The "generation model" refers to a set of algorithms trained to generate an expected output based on a specific input.

[0283] The "integrated model" refers to a unified algorithm for aggregating a plurality of individual generation models and providing an optimal output as a whole.

[0284] The "relevant information of the group" refers to information useful for making decisions for the entire group that reflects the opinions and preferences of all users.

[0285] The "usage history" refers to a record of information regarding the selections made by the user in the past and their results.

[0286] The "preference information" refers to information indicating the individual preferences and interests of the user.

[0287] The "analysis function" refers to a process that has the function of analyzing data in detail and contributes to the evaluation of results and the improvement of models.

[0288] The "proposal" refers to recommendations for prompting options and actions according to the needs and situations of the user.

[0289] This invention is an information processing system that provides proposals based on the preferences and requirements of a group using an individually adjusted generation model.

[0290] The server receives data such as preference information and past selection history from each user as input. This data is processed using a generative AI model to create individual generative models for each user. These generative models reflect each user's preferences and are stored in a database. If necessary, the server integrates these individual generative models to build an integrated model that provides optimal suggestions for the group.

[0291] The terminal displays server-generated suggestions to the user. These suggestions are delivered via an interface to facilitate user selection. The user selects a suggestion through the terminal and sends feedback to the server. The server analyzes the feedback received and uses it to further optimize future suggestions.

[0292] For example, when friends plan a trip together, their preferences for destinations and budget constraints are entered into the server. The server creates a generative model for each user, integrates them, and proposes a travel plan that satisfies everyone. This proposal is delivered to the user's device, and the final selection is sent back to the server as user feedback.

[0293] An example of a prompt to a generative AI model might be: "Please suggest the best travel destination for our group. Here are each member's preferences: Member A wants a beach, Member B wants historical sites, and Member C wants to go shopping."

[0294] In this way, the system can optimize the group as a whole while also providing personalized suggestions to each user.

[0295] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0296] Step 1:

[0297] Users use their devices to send preference information and past selection history to the server. This input includes categories of interest and past behavioral data. The server receives this data and stores it in a database specific to each user.

[0298] Step 2:

[0299] The server uses a generative AI model to create personalized generative models based on user preference information. Using the preference information received as input, it analyzes user preference patterns in a data-driven approach. Based on this analysis, a personalized generative model is output and saved.

[0300] Step 3:

[0301] If necessary, the server integrates multiple individual generative models to generate a new integrated model. The input data consists of generative models from multiple users, and the output is an integrated model that optimizes the opinions of the entire group. The generated integrated model comprehensively considers the preferences of different users to produce the optimal proposal for the group.

[0302] Step 4:

[0303] The server uses an integrated model to generate proposals and delivers them to the user's terminal. The input is the generated integrated model, and the output is the new proposal content displayed on the user's terminal. The terminal receives these proposals and presents them visually in an easy-to-use interface for the user to select from.

[0304] Step 5:

[0305] Users review suggestions via their terminal and send feedback on their selected items to the server. The input data is the selection result for the suggestions, which the server receives and analyzes. As output, the feedback information is stored and used to improve the accuracy of future suggestions.

[0306] In this way, the system constructs individual and integrated generation models from the input data and provides personalized and group-optimized proposals to the user.

[0307] (Application Example 1)

[0308] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as "terminals".

[0309] In modern information processing, it is very important to consider the preferences and requirements of multiple individuals at once and optimize group decision-making. However, in conventional systems, individual preferences are not appropriately reflected, and it may be difficult to make proposals that satisfy the entire group. The present invention solves this problem and aims to provide optimal proposals that reflect the preferences of each individual, particularly in a scene where multiple people are involved.

[0310] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0311] In this invention, the server includes: means for providing an information processing device including an individually adjusted generation model; means for aggregating generation models from a plurality of information processing devices and constructing an integrated model; means for generating and outputting proposal information based on the requirements of the group; means for storing the usage history of the recorded information and reflecting it in subsequent optimizations; and means having a proposal function for optimizing the selection of dishes based on the individual preferences of multiple people. This enables optimal proposals considering the preferences of multiple individuals.

[0312] The "individually adjusted generation model" is an information processing model that reflects the preferences and requirements of each user and is optimized for a specific purpose.

[0313] The "information processing device" refers to a device or system for processing data and performing calculations.

[0314] "Aggregating generative models and constructing an integrated model" refers to the process of gathering and integrating multiple generative models to construct a comprehensive model that reflects the needs and preferences of the entire group.

[0315] "Generating and outputting proposal information" means creating proposals suitable for a group based on an integrated model and providing them to users.

[0316] "Recorded information usage history" refers to data such as past user choices and responses to suggestions, which is stored to optimize future suggestions.

[0317] The "suggestion function that optimizes dish selection based on the individual preferences of multiple people" is a function that takes into account each user's preferences and limitations to determine the optimal dish selection for multiple people.

[0318] The server, acting as an information processing device, receives preference information from multiple users and creates individually tailored generative models based on this information. These generative models reflect the users' preferences and past selection history. The created generative models are stored in a database, and if necessary, the generative models of multiple users are aggregated to build an integrated model. This integrated model is used to optimize decision-making for the entire group.

[0319] Based on this integrated model, the server generates suggestion information that reflects the group's needs and outputs it to each user's terminal. The terminal operates as a smartphone application and displays this suggestion information to the user. When a user provides feedback on the suggestion, that information is also sent to the server and stored as a record. This recorded information is used to optimize future suggestions.

[0320] The hardware used includes smartphones and cloud servers (e.g., AWS, Google Cloud), while the software utilizes React Native for mobile applications, Node.js for server systems, and MongoDB for databases. For data processing, user-entered preference data is sent to the server in JSON format, and Python machine learning libraries (e.g., TensorFlow, PyTorch) are used for data calculations to generate an integrated model.

[0321] For example, when suggesting the best lunch menu for several employees, using a prompt like, "Please suggest lunch menus for 5 employees. Their preferences and allergy information are as follows: Person A likes sushi, Person B likes pizza, and Person C has a wheat allergy," makes it possible to suggest meals that will satisfy everyone.

[0322] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0323] Step 1:

[0324] Users input their preferences and restrictions into a smartphone application. This information is sent to the server as JSON data. This data provides detailed information about each user's preferences.

[0325] Step 2:

[0326] The server parses the received JSON data and extracts preference information for each user. Based on the extracted information, it generates individually tailored generative models using Python's machine learning libraries. The generated models are stored in a database, ready for use in the next step.

[0327] Step 3:

[0328] The server aggregates generative models created by multiple users to build an integrated model. The integrated model aims to optimize the requirements of the entire group. It compares and integrates data between models to create an information structure that supports optimal decision-making.

[0329] Step 4:

[0330] The server generates suggestion information that matches the group's requests based on an integrated model. This information includes optimal dish selections that reflect each user's preferences and allergy information. The generated suggestions are sent to the user's device as push notifications.

[0331] Step 5:

[0332] Users review the received suggestion information and provide feedback to the server through the application. This feedback is data indicating their evaluation and satisfaction with the suggestion, and is collected on the server in real time.

[0333] Step 6:

[0334] The server analyzes the collected feedback and uses it to improve the accuracy of the generative and integrated models. It uses the feedback data to adjust parameters to improve the accuracy of future proposals.

[0335] This processing flow ensures that users receive efficient and optimal suggestions that take their individual preferences into account.

[0336] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0337] This invention is a system that generates suggestions that take the user's emotions into account by combining an emotion engine with individually tuned generative models. This system utilizes emotion data along with user preference information to achieve advanced personalization that reflects the individual's emotional state.

[0338] Server role:

[0339] The server creates a generative model based on user preference information and emotion data received from the terminal. The emotion engine analyzes each user's emotional state from their speech, facial expressions, and input text, and generates emotion data. Based on this, it builds a personalized generative model for each user and stores it in the database.

[0340] Emotional engine function:

[0341] The emotion engine analyzes user emotions in real time during interaction, identifying emotional states such as positive, negative, and neutral. This allows the server to add emotional information to the generative model, enabling more refined personalization. This emotional information is considered a crucial element when generating suggestions.

[0342] Generation of an integrated model:

[0343] The server integrates individual generative models to generate an integrated model that reflects the emotional state of the entire group. This enables optimal suggestions for enhancing positive collective experiences.

[0344] Suggestions and feedback:

[0345] Suggestions generated using the integrated model are delivered to the user's device. The user evaluates the presented options and makes a selection. The device sends feedback data about this selection to the server. This feedback includes changes in emotional state as determined by the emotion engine, which the server uses to improve the accuracy of future suggestions.

[0346] Specific example:

[0347] For example, consider a scenario where a family decides how to spend a holiday. Each family member inputs their desired activity and the emotions they feel during that activity (excited, wanting to relax, etc.) into a device. An emotion engine analyzes this information, and the server creates individual generative models and builds an integrated model. Based on this, holiday suggestions (e.g., theme park or hot spring trip) are considered and sent to the user. Through feedback, the system is adjusted to provide the best possible suggestions that the whole family can enjoy.

[0348] Through the above process, the present invention is a system that can improve the quality of the user experience through advanced emotion-based customization.

[0349] The following describes the processing flow.

[0350] Step 1:

[0351] User: Enters information about their emotional state and preferences into the device. This information includes desired activities and the emotions associated with them (e.g., excitement, relaxation).

[0352] Step 2:

[0353] Terminal: Sends information entered by the user to the emotion engine and analyzes the emotional state in real time. The analyzed emotional information and preference data are sent together to the server.

[0354] Step 3:

[0355] Server: Based on preference and sentiment data received from terminals, it creates a generative model tailored to each user. The generative model is optimized using machine learning algorithms and incorporates individual sentiment data.

[0356] Step 4:

[0357] Server: Aggregates generative models from multiple users to build an integrated model that reflects the emotional state of the entire group. This integrated model considers numerous emotional patterns and serves as the foundation for generating optimal suggestions.

[0358] Step 5:

[0359] Server: Based on an integrated model, it generates suggestions that align with the collective sentiment. The suggestions are adjusted to balance user preferences and emotions, forming appropriate options.

[0360] Step 6:

[0361] Server: Sends generated suggestions to users' terminals and distributes them to individual users. Suggestions are displayed as options and presented in a way that allows for individual feedback.

[0362] Step 7:

[0363] User: Review the suggestions displayed on the device and make a selection. When making a selection, evaluate them based on your emotional state and initial preferences.

[0364] Step 8:

[0365] Terminal: The emotion engine re-analyzes the user's selection results and the resulting changes in their emotional state, and sends that data to the server.

[0366] Step 9:

[0367] Server: Based on the feedback received, the server adjusts the generative and integrated models. This allows for learning and optimization to improve the accuracy of future proposals.

[0368] This process enables personalization that comprehensively considers emotions and preferences, allowing us to provide the most appropriate suggestions for the user.

[0369] (Example 2)

[0370] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0371] Conventional generative model-based suggestion systems could consider user preference information, but they were insufficient for personalization that reflected the user's emotional state. As a result, suggestions aimed at improving the user experience were inaccurate, sometimes leading to decreased satisfaction. Furthermore, it was difficult to make suggestions that considered the emotional state of the entire group, resulting in the challenge of not being able to make effective suggestions for large groups.

[0372] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0373] In this invention, the server includes means for constructing individually tailored generative models based on user emotional information and preference information; means for collecting multiple generative models and generating an integrated model that reflects the emotional state of the entire group; and means for transmitting the suggested content generated based on the integrated model to an information output device. This makes it possible to generate detailed suggestions that reflect the emotional state of individual users and suggested content that promotes a positive experience for the group.

[0374] An "information processing device" is a device that processes data input by a user and has the function of creating a generative model based on emotional information and preference information.

[0375] A "generative model" is a model that is adjusted to reflect the user's individual emotional and preference information, and serves as the foundation for generating personalized suggestions.

[0376] An "integrated model" is a model created by aggregating multiple generative models and taking into account the emotional state of the entire group, and is used to generate proposals for large groups.

[0377] "Emotional information" refers to data that indicates the user's current emotional state, and is obtained from information such as voice, text, and facial expression analysis.

[0378] "Preference information" refers to data about users' long-term preferences and tastes, and serves as the basis for making personalized recommendations to individual users.

[0379] An "information output device" is a device that has an output function to provide the user with the suggested content generated from the server.

[0380] "Proposed content" refers to options or activity proposals generated from a generative or integrated model and presented to the user or group.

[0381] "Feedback" refers to the evaluation and selection results of user suggestions, and is data used to improve the accuracy of the system.

[0382] The system of this invention utilizes user emotional and preference information and achieves advanced personalization using individually tailored generative AI models. The server generates suggestions in real time based on data collected from each user's terminal.

[0383] 1. Hardware and software:

[0384] User terminal: Equipped with an interface for receiving voice and text input, and performs data processing using an energy-saving processor.

[0385] Server: A high-performance computing system for large-scale data processing, which performs emotion analysis using "EmotionAnalyzer" software. It also uses "AIModelBuilder" to build generative AI models.

[0386] Information output device: A device that presents suggestions generated from the server to the user.

[0387] 2. Data processing and data calculation:

[0388] The user's device converts voice input into text using "SpeechToTextConverter" and prepares for sentiment analysis.

[0389] The server uses "EmotionAnalyzer" to extract emotional information from user input and adjusts the generated AI model based on this information.

[0390] "AIModelBuilder" creates generative models for each user, and then generates an integrated model for the group.

[0391] 3. Specific examples and examples of prompt statements:

[0392] For example, consider a scenario where a family decides how to spend their holiday. Each family member inputs their desired activity and their corresponding feelings (e.g., "exciting," "relaxing") into a terminal. This information is analyzed by an emotion engine, and the server builds a generative AI model to generate suggestions that the whole family can enjoy. The optimal activity (e.g., visiting a theme park, going to a hot spring) is then suggested.

[0393] An example of a prompt message is, "What kind of activity would you like to do today? Please tell us how you feel about that activity (e.g., I'm excited, I want to relax)."

[0394] This system aims to improve the user experience by providing suggestions that take into account the user's emotional state.

[0395] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0396] Step 1:

[0397] The user inputs their daily activities into the device. If voice input is used, the device uses "SpeechToTextConverter" to convert the voice data into text. This input data becomes the basis for subsequent sentiment analysis.

[0398] Step 2:

[0399] The device sends the converted text data to the server. The server sends the received text data to "EmotionAnalyzer," which analyzes the user's emotional state. "EmotionAnalyzer" analyzes keywords and context in the text and generates emotional information such as positive, negative, or neutral. This is output as emotional data.

[0400] Step 3:

[0401] The server integrates the generated sentiment data with user preference information and inputs it into "AIModelBuilder." "AIModelBuilder" then constructs individually optimized generative AI models based on this information. These generative models form the basis for providing user-specific suggestions and are stored on the server.

[0402] Step 4:

[0403] The server aggregates the generative models of multiple users and generates an integrated model using a "group modeling tool" that takes into account the emotional state and common preferences of the entire group. This integrated model is used to derive the most suitable suggestions for the group.

[0404] Step 5:

[0405] The server generates group-oriented suggestions based on an integrated model and distributes them to users through an information output device. The suggestions are presented as specific activities and options, prompting users to make choices.

[0406] Step 6:

[0407] The user evaluates the presented suggestions and makes a selection. The device records feedback on the selection and subsequent changes in emotional state. This feedback is sent to the server and used to improve the accuracy of future suggestions.

[0408] The above outlines the processing flow of this system's program. Each step involves specific data processing and model generation, aiming to improve the user experience.

[0409] (Application Example 2)

[0410] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0411] This invention aims to solve the problem of improving the experience of individual users and groups by providing a system that takes into account the emotional state and preference information of users and provides appropriate information and suggestions in real time. Conventional systems sometimes fail to capture users' interest and satisfaction because they provide information without adequately considering the emotions of the users.

[0412] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0413] In this invention, the server includes means for providing a computing device that analyzes emotions and generates information based on the user's emotional state, means for providing a computing device that includes an individually adjusted generative model, and means for providing information that optimizes selected measures based on real-time emotion analysis results. This enables the provision of personalized information according to the user's emotional state.

[0414] A "computational device that analyzes emotions and generates information based on the user's emotional state" is a device that determines emotions from the user's facial expressions, speech, and input data, and generates information that matches those emotions.

[0415] A "computational device including individually tailored generative models" is a device equipped with a specific generative model based on each user's preference information and historical data.

[0416] "Means for aggregating generative models and constructing an integrated model" refers to the function responsible for the process of aggregating multiple individual generative models and creating a collective model.

[0417] "A means of providing information to optimize selected measures based on real-time sentiment analysis results" refers to a function that instantly analyzes users' sentiment data and uses the results to dynamically adjust the content of suggestions and information provided.

[0418] "Means for saving the usage history of collected information and reflecting it in future optimizations" refers to a function that saves user behavior information collected in the past and uses it to improve future suggestions and information provision.

[0419] "Having a feedback function" means having a function that receives responses and results from users and uses them to improve the system's operation.

[0420] The system for implementing the present invention utilizes emotional data and preference information to provide users with optimal information and suggestions. The system consists of a server, a terminal, an emotional analysis engine, and a computing device including a generative model.

[0421] First, the user's facial expressions, speech, and text input are collected from the device. The emotion analysis engine analyzes this data in real time to identify emotional states such as positive, negative, and neutral. For example, the user's facial expressions are captured with a camera and analyzed using OpenCV. In addition, the user's voice input is converted into text using speech recognition technology, and emotions are inferred based on this.

[0422] The server integrates analyzed sentiment data with past preference information to create a personalized generative model. Using TensorFlow, this generative model is newly generated and adjusted for each user. As a result, the server generates the most suitable information and optimizes selection strategies. This can be achieved, for example, by recommending products or content that might interest a user if they appear to be enjoying themselves.

[0423] Furthermore, this generated information and suggestions are delivered to the user's device. When suggestions are presented, feedback is collected and sent to the server. This feedback allows the integrated model to continuously improve its accuracy.

[0424] A concrete example is the shopping experience in a virtual store. If a user smiles while looking at a product, recommendations for related items will be enhanced. As an example of a prompt, inputting the instruction "When the user shows a relaxed expression, present options with a relaxing effect" into the generating AI model will facilitate the provision of appropriate information.

[0425] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0426] Step 1:

[0427] The device collects input from the user. Specifically, it captures the user's facial expressions with a camera and records their speech with a microphone. This input data is treated as important information for identifying the user's current emotional state. Facial expression data is sent to an emotion analysis engine through image processing, and speech data is converted into text using speech recognition technology.

[0428] Step 2:

[0429] The server analyzes input data sent from the terminal using an emotion analysis engine. Specifically, it uses OpenCV to identify emotions (positive, negative, neutral, etc.) from facial expression data. In addition, data converted from speech to text is used as preference information through natural language processing. By integrating these analysis results, the server determines the user's current emotional state.

[0430] Step 3:

[0431] The server creates an individualized generative model based on the determined emotional state and past preference information. TensorFlow is used to process this data and generate a user-specific generative model. This generated model functions as a foundation for generating optimal information and suggestions that match the user's emotions and preferences.

[0432] Step 4:

[0433] Using the generated model, the server creates optimal information and suggestions for the user and sends them to the terminal. In this process, the generating AI model uses instructions such as, "When the user shows a relaxed expression, present options with a relaxing effect." The generated suggestions are then presented to the user, for example, as product recommendations while shopping in a virtual store.

[0434] Step 5:

[0435] The device then collects user responses to the presented suggestions. Specifically, it acquires data on the user's selections and changes in their emotions during those selections. This feedback data is sent to a server and used for further analysis and to improve the accuracy of the generative model.

[0436] Step 6:

[0437] The server uses the collected feedback data to improve the accuracy of the integrated model. This process involves adjusting the model based on the feedback, improving the accuracy of subsequent information provision. Through this iterative process, the entire system enables more refined personalization tailored to the user's preferences and emotions.

[0438] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0439] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0440] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0441] [Third Embodiment]

[0442] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0443] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0444] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0445] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0446] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0447] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0448] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0449] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0450] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0451] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0452] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0453] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0454] The present invention can be implemented as a system that integrates individually tailored generative models to provide suggestions tailored to the preferences and needs of a group. Typically, this system is applied in situations involving multiple users, generating optimal suggestions based on each user's information. The main components and operation of this system are described below.

[0455] Server role:

[0456] The server receives preference information sent from each user. This includes user preferences, past selection history, and allergy information. Based on this information, the server calculates a generative model specific to each user and stores it in the database. This model is tailored to the individual requests of each user.

[0457] The role of the integrated model:

[0458] The server integrates the generative models of multiple users as needed to generate a new integrated model. This integrated model aims to optimize decision-making for the entire group, and is used to generate suggestions that match the requirements of each user. For example, if multiple people need to choose a restaurant, the server can generate options that everyone will agree on.

[0459] Proposal generation and feedback:

[0460] The generated suggestions are delivered to the user's device and managed by the server. Users provide feedback by reviewing and selecting suggestions. This feedback is analyzed by the server and used to optimize the system for future suggestions.

[0461] Specific example:

[0462] For example, consider a scenario where friends plan to watch a movie together. Each user sends their favorite movie genres and movies they want to see to the server. The server collects each user's information and creates individual generative models based on it. Then, it integrates multiple generative models to generate a list that includes everyone's favorite genres and movies of interest. This list is displayed on the users' devices, and feedback from everyone can be used to further optimize future suggestions.

[0463] Thus, by using this invention, it becomes possible to make optimal decisions as a group while taking individual preferences into account. This achieves a level of personalization and comprehensiveness not found in conventional individual models.

[0464] The following describes the processing flow.

[0465] Step 1:

[0466] User: Enters their preferences into the device. This includes favorite genres, allergy information, topics of interest, etc.

[0467] Step 2:

[0468] Terminal: Formats the entered information into a data package and sends it to the server. The information is encrypted using a secure protocol.

[0469] Step 3:

[0470] Server: Analyzes preference information received from terminals and extracts features to create personalized generative models tailored to each user.

[0471] Step 4:

[0472] Server: Builds generative models tailored to each user and stores them in a database. These models are optimized using machine learning algorithms.

[0473] Step 5:

[0474] Server: If necessary, aggregates the generative models of multiple users to build an integrated model. The integrated model enables comprehensive recommendations based on the preferences of the entire group.

[0475] Step 6:

[0476] Server: Generates optimal suggestions based on the integrated model and creates specific options. These options are situation-dependent, such as a list of restaurants or movie choices.

[0477] Step 7:

[0478] Server: Distributes suggestions to the user's device. Suggestions are presented as notifications or in-app messages.

[0479] Step 8:

[0480] User: Review the options presented on the device and select the one they deem most preferable. The selection can be made with simple operations.

[0481] Step 9:

[0482] Terminal: Sends user selections to the server and provides feedback data.

[0483] Step 10:

[0484] Server: Analyzes received feedback and evaluates the quality of the selections. Based on this, the generative model is readjusted and used for future suggestions.

[0485] Each of the steps described above is designed to ensure smooth and effective information processing across the entire system.

[0486] (Example 1)

[0487] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0488] While existing information processing systems can provide suggestions based on individual user preferences, they struggle to create optimal suggestions for the group as a whole. Therefore, there is a growing need for a system that can provide suggestions that reflect the group's opinions while also satisfying each individual user, especially when multiple users share common goals and interests.

[0489] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0490] In this invention, the server includes means for providing an information processing device that includes individually tailored generative models, means for aggregating generative models from multiple information processing devices and constructing an integrated model, and means for generating and outputting relevant group information based on the integrated model. This makes it possible to automatically generate optimal proposals for the group as a whole while respecting the needs of individual users, and to encourage appropriate decision-making among members.

[0491] An "information processing device" refers to a mechanical or electronic device that can input, process, and output data.

[0492] A "generative model" refers to a set of algorithms trained to produce an expected output based on a specific input.

[0493] An "integrated model" refers to a unified algorithm that aggregates multiple individual generative models to provide the optimal overall output.

[0494] "Group-related information" refers to information that reflects the opinions and preferences of all users and is useful for decision-making for the group as a whole.

[0495] "Usage history" refers to a record of information about choices a user has made in the past and the results thereof.

[0496] "Preference information" refers to information that indicates an individual user's preferences and interests.

[0497] "Analysis function" refers to a process that has the ability to analyze data in detail and contributes to evaluating results and improving models.

[0498] "Suggestions" refer to recommendations that encourage users to make choices or take actions that are appropriate to their needs and circumstances.

[0499] This invention is an information processing system that uses individually tailored generative models to provide suggestions based on the preferences and needs of a group.

[0500] The server receives data such as preference information and past selection history from each user as input. This data is processed using a generative AI model to create individual generative models for each user. These generative models reflect each user's preferences and are stored in a database. If necessary, the server integrates these individual generative models to build an integrated model that provides optimal suggestions for the group.

[0501] The terminal displays server-generated suggestions to the user. These suggestions are delivered via an interface to facilitate user selection. The user selects a suggestion through the terminal and sends feedback to the server. The server analyzes the feedback received and uses it to further optimize future suggestions.

[0502] For example, when friends plan a trip together, their preferences for destinations and budget constraints are entered into the server. The server creates a generative model for each user, integrates them, and proposes a travel plan that satisfies everyone. This proposal is delivered to the user's device, and the final selection is sent back to the server as user feedback.

[0503] An example of a prompt to a generative AI model might be: "Please suggest the best travel destination for our group. Here are each member's preferences: Member A wants a beach, Member B wants historical sites, and Member C wants to go shopping."

[0504] In this way, the system can optimize the group as a whole while also providing personalized suggestions to each user.

[0505] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0506] Step 1:

[0507] Users use their devices to send preference information and past selection history to the server. This input includes categories of interest and past behavioral data. The server receives this data and stores it in a database specific to each user.

[0508] Step 2:

[0509] The server uses a generative AI model to create personalized generative models based on user preference information. Using the preference information received as input, it analyzes user preference patterns in a data-driven approach. Based on this analysis, a personalized generative model is output and saved.

[0510] Step 3:

[0511] If necessary, the server integrates multiple individual generative models to generate a new integrated model. The input data consists of generative models from multiple users, and the output is an integrated model that optimizes the opinions of the entire group. The generated integrated model comprehensively considers the preferences of different users to produce the optimal proposal for the group.

[0512] Step 4:

[0513] The server uses an integrated model to generate proposals and delivers them to the user's terminal. The input is the generated integrated model, and the output is the new proposal content displayed on the user's terminal. The terminal receives these proposals and presents them visually in an easy-to-use interface for the user to select from.

[0514] Step 5:

[0515] Users review suggestions via their terminal and send feedback on their selected items to the server. The input data is the selection result for the suggestions, which the server receives and analyzes. As output, the feedback information is stored and used to improve the accuracy of future suggestions.

[0516] In this way, the system constructs individual and integrated generative models from input data and provides users with personalized and collectively optimized suggestions.

[0517] (Application Example 1)

[0518] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0519] In modern information processing, it is crucial to consider the preferences and needs of multiple individuals simultaneously and optimize group decision-making. However, conventional systems often fail to adequately reflect individual preferences, making it difficult to propose solutions that satisfy the entire group. This invention aims to solve this problem and provide optimal proposals that reflect the preferences of each individual, particularly in scenarios involving multiple people.

[0520] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0521] In this invention, the server includes means for providing an information processing device that includes individually adjusted generation models; means for aggregating generation models from multiple information processing devices and constructing an integrated model; means for generating and outputting suggestion information based on the group's requests; means for saving the usage history of recorded information and reflecting it in subsequent optimizations; and means for having a suggestion function that optimizes dish selection based on the individual preferences of multiple people. This makes it possible to make optimal suggestions that take into account the preferences of multiple individuals.

[0522] A "personally tailored generative model" is an information processing model that reflects the preferences and requirements of each user and is optimized for a specific purpose.

[0523] An "information processing device" is a device or system used to process data and perform calculations.

[0524] "Aggregating generative models and constructing an integrated model" refers to the process of gathering and integrating multiple generative models to construct a comprehensive model that reflects the needs and preferences of the entire group.

[0525] "Generating and outputting proposal information" means creating proposals suitable for a group based on an integrated model and providing them to users.

[0526] "Recorded information usage history" refers to data such as past user choices and responses to suggestions, which is stored to optimize future suggestions.

[0527] The "suggestion function that optimizes dish selection based on the individual preferences of multiple people" is a function that takes into account each user's preferences and limitations to determine the optimal dish selection for multiple people.

[0528] The server, acting as an information processing device, receives preference information from multiple users and creates individually tailored generative models based on this information. These generative models reflect the users' preferences and past selection history. The created generative models are stored in a database, and if necessary, the generative models of multiple users are aggregated to build an integrated model. This integrated model is used to optimize decision-making for the entire group.

[0529] Based on this integrated model, the server generates suggestion information that reflects the group's needs and outputs it to each user's terminal. The terminal operates as a smartphone application and displays this suggestion information to the user. When a user provides feedback on the suggestion, that information is also sent to the server and stored as a record. This recorded information is used to optimize future suggestions.

[0530] The hardware used includes smartphones and cloud servers (e.g., AWS, Google Cloud), while the software utilizes React Native for mobile applications, Node.js for server systems, and MongoDB for databases. For data processing, user-entered preference data is sent to the server in JSON format, and Python machine learning libraries (e.g., TensorFlow, PyTorch) are used for data calculations to generate an integrated model.

[0531] For example, when suggesting the best lunch menu for several employees, using a prompt like, "Please suggest lunch menus for 5 employees. Their preferences and allergy information are as follows: Person A likes sushi, Person B likes pizza, and Person C has a wheat allergy," makes it possible to suggest meals that will satisfy everyone.

[0532] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0533] Step 1:

[0534] Users input their preferences and restrictions into a smartphone application. This information is sent to the server as JSON data. This data provides detailed information about each user's preferences.

[0535] Step 2:

[0536] The server parses the received JSON data and extracts preference information for each user. Based on the extracted information, it generates individually tailored generative models using Python's machine learning libraries. The generated models are stored in a database, ready for use in the next step.

[0537] Step 3:

[0538] The server aggregates generative models created by multiple users to build an integrated model. The integrated model aims to optimize the requirements of the entire group. It compares and integrates data between models to create an information structure that supports optimal decision-making.

[0539] Step 4:

[0540] The server generates suggestion information that matches the group's requests based on an integrated model. This information includes optimal dish selections that reflect each user's preferences and allergy information. The generated suggestions are sent to the user's device as push notifications.

[0541] Step 5:

[0542] Users review the received suggestion information and provide feedback to the server through the application. This feedback is data indicating their evaluation and satisfaction with the suggestion, and is collected on the server in real time.

[0543] Step 6:

[0544] The server analyzes the collected feedback and uses it to improve the accuracy of the generative and integrated models. It uses the feedback data to adjust parameters to improve the accuracy of future proposals.

[0545] This processing flow ensures that users receive efficient and optimal suggestions that take their individual preferences into account.

[0546] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0547] This invention is a system that generates suggestions that take the user's emotions into account by combining an emotion engine with individually tuned generative models. This system utilizes emotion data along with user preference information to achieve advanced personalization that reflects the individual's emotional state.

[0548] Server role:

[0549] The server creates a generative model based on user preference information and emotion data received from the terminal. The emotion engine analyzes each user's emotional state from their speech, facial expressions, and input text, and generates emotion data. Based on this, it builds a personalized generative model for each user and stores it in the database.

[0550] Emotional engine function:

[0551] The emotion engine analyzes user emotions in real time during interaction, identifying emotional states such as positive, negative, and neutral. This allows the server to add emotional information to the generative model, enabling more refined personalization. This emotional information is considered a crucial element when generating suggestions.

[0552] Generation of an integrated model:

[0553] The server integrates individual generative models to generate an integrated model that reflects the emotional state of the entire group. This enables optimal suggestions for enhancing positive collective experiences.

[0554] Suggestions and feedback:

[0555] Suggestions generated using the integrated model are delivered to the user's device. The user evaluates the presented options and makes a selection. The device sends feedback data about this selection to the server. This feedback includes changes in emotional state as determined by the emotion engine, which the server uses to improve the accuracy of future suggestions.

[0556] Specific example:

[0557] For example, consider a scenario where a family decides how to spend a holiday. Each family member inputs their desired activity and the emotions they feel during that activity (excited, wanting to relax, etc.) into a device. An emotion engine analyzes this information, and the server creates individual generative models and builds an integrated model. Based on this, holiday suggestions (e.g., theme park or hot spring trip) are considered and sent to the user. Through feedback, the system is adjusted to provide the best possible suggestions that the whole family can enjoy.

[0558] Through the above process, the present invention is a system that can improve the quality of the user experience through advanced emotion-based customization.

[0559] The following describes the processing flow.

[0560] Step 1:

[0561] User: Enters information about their emotional state and preferences into the device. This information includes desired activities and the emotions associated with them (e.g., excitement, relaxation).

[0562] Step 2:

[0563] Terminal: Sends information entered by the user to the emotion engine and analyzes the emotional state in real time. The analyzed emotional information and preference data are sent together to the server.

[0564] Step 3:

[0565] Server: Based on preference and sentiment data received from terminals, it creates a generative model tailored to each user. The generative model is optimized using machine learning algorithms and incorporates individual sentiment data.

[0566] Step 4:

[0567] Server: Aggregates generative models from multiple users to build an integrated model that reflects the emotional state of the entire group. This integrated model considers numerous emotional patterns and serves as the foundation for generating optimal suggestions.

[0568] Step 5:

[0569] Server: Based on an integrated model, it generates suggestions that align with the collective sentiment. The suggestions are adjusted to balance user preferences and emotions, forming appropriate options.

[0570] Step 6:

[0571] Server: Sends generated suggestions to users' terminals and distributes them to individual users. Suggestions are displayed as options and presented in a way that allows for individual feedback.

[0572] Step 7:

[0573] User: Review the suggestions displayed on the device and make a selection. When making a selection, evaluate them based on your emotional state and initial preferences.

[0574] Step 8:

[0575] Terminal: The emotion engine re-analyzes the user's selection results and the resulting changes in their emotional state, and sends that data to the server.

[0576] Step 9:

[0577] Server: Based on the feedback received, the server adjusts the generative and integrated models. This allows for learning and optimization to improve the accuracy of future proposals.

[0578] This process enables personalization that comprehensively considers emotions and preferences, allowing us to provide the most appropriate suggestions for the user.

[0579] (Example 2)

[0580] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0581] Conventional generative model-based suggestion systems could consider user preference information, but they were insufficient for personalization that reflected the user's emotional state. As a result, suggestions aimed at improving the user experience were inaccurate, sometimes leading to decreased satisfaction. Furthermore, it was difficult to make suggestions that considered the emotional state of the entire group, resulting in the challenge of not being able to make effective suggestions for large groups.

[0582] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0583] In this invention, the server includes means for constructing individually tailored generative models based on user emotional information and preference information; means for collecting multiple generative models and generating an integrated model that reflects the emotional state of the entire group; and means for transmitting the suggested content generated based on the integrated model to an information output device. This makes it possible to generate detailed suggestions that reflect the emotional state of individual users and suggested content that promotes a positive experience for the group.

[0584] An "information processing device" is a device that processes data input by a user and has the function of creating a generative model based on emotional information and preference information.

[0585] A "generative model" is a model that is adjusted to reflect the user's individual emotional and preference information, and serves as the foundation for generating personalized suggestions.

[0586] An "integrated model" is a model created by aggregating multiple generative models and taking into account the emotional state of the entire group, and is used to generate proposals for large groups.

[0587] "Emotional information" refers to data that indicates the user's current emotional state, and is obtained from information such as voice, text, and facial expression analysis.

[0588] "Preference information" refers to data about users' long-term preferences and tastes, and serves as the basis for making personalized recommendations to individual users.

[0589] An "information output device" is a device that has an output function to provide the user with the suggested content generated from the server.

[0590] "Proposed content" refers to options or activity proposals generated from a generative or integrated model and presented to the user or group.

[0591] "Feedback" refers to the evaluation and selection results of user suggestions, and is data used to improve the accuracy of the system.

[0592] The system of this invention utilizes user emotional and preference information and achieves advanced personalization using individually tailored generative AI models. The server generates suggestions in real time based on data collected from each user's terminal.

[0593] 1. Hardware and software:

[0594] User terminal: Equipped with an interface for receiving voice and text input, and performs data processing using an energy-saving processor.

[0595] Server: A high-performance computing system for large-scale data processing, which performs emotion analysis using "EmotionAnalyzer" software. It also uses "AIModelBuilder" to build generative AI models.

[0596] Information output device: A device that presents suggestions generated from the server to the user.

[0597] 2. Data processing and data calculation:

[0598] The user's device converts voice input into text using "SpeechToTextConverter" and prepares for sentiment analysis.

[0599] The server uses "EmotionAnalyzer" to extract emotional information from user input and adjusts the generated AI model based on this information.

[0600] "AIModelBuilder" creates generative models for each user, and then generates an integrated model for the group.

[0601] 3. Specific examples and examples of prompt statements:

[0602] For example, consider a scenario where a family decides how to spend their holiday. Each family member inputs their desired activity and their corresponding feelings (e.g., "exciting," "relaxing") into a terminal. This information is analyzed by an emotion engine, and the server builds a generative AI model to generate suggestions that the whole family can enjoy. The optimal activity (e.g., visiting a theme park, going to a hot spring) is then suggested.

[0603] An example of a prompt message is, "What kind of activity would you like to do today? Please tell us how you feel about that activity (e.g., I'm excited, I want to relax)."

[0604] This system aims to improve the user experience by providing suggestions that take into account the user's emotional state.

[0605] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0606] Step 1:

[0607] The user inputs their daily activities into the device. If voice input is used, the device uses "SpeechToTextConverter" to convert the voice data into text. This input data becomes the basis for subsequent sentiment analysis.

[0608] Step 2:

[0609] The device sends the converted text data to the server. The server sends the received text data to "EmotionAnalyzer," which analyzes the user's emotional state. "EmotionAnalyzer" analyzes keywords and context in the text and generates emotional information such as positive, negative, or neutral. This is output as emotional data.

[0610] Step 3:

[0611] The server integrates the generated sentiment data with user preference information and inputs it into "AIModelBuilder." "AIModelBuilder" then constructs individually optimized generative AI models based on this information. These generative models form the basis for providing user-specific suggestions and are stored on the server.

[0612] Step 4:

[0613] The server aggregates the generative models of multiple users and generates an integrated model using a "group modeling tool" that takes into account the emotional state and common preferences of the entire group. This integrated model is used to derive the most suitable suggestions for the group.

[0614] Step 5:

[0615] The server generates group-oriented suggestions based on an integrated model and distributes them to users through an information output device. The suggestions are presented as specific activities and options, prompting users to make choices.

[0616] Step 6:

[0617] The user evaluates the presented suggestions and makes a selection. The device records feedback on the selection and subsequent changes in emotional state. This feedback is sent to the server and used to improve the accuracy of future suggestions.

[0618] The above outlines the processing flow of this system's program. Each step involves specific data processing and model generation, aiming to improve the user experience.

[0619] (Application Example 2)

[0620] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0621] This invention aims to solve the problem of improving the experience of individual users and groups by providing a system that takes into account the emotional state and preference information of users and provides appropriate information and suggestions in real time. Conventional systems sometimes fail to capture users' interest and satisfaction because they provide information without adequately considering the emotions of the users.

[0622] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0623] In this invention, the server includes means for providing a computing device that analyzes emotions and generates information based on the user's emotional state, means for providing a computing device that includes an individually adjusted generative model, and means for providing information that optimizes selected measures based on real-time emotion analysis results. This enables the provision of personalized information according to the user's emotional state.

[0624] A "computational device that analyzes emotions and generates information based on the user's emotional state" is a device that determines emotions from the user's facial expressions, speech, and input data, and generates information that matches those emotions.

[0625] A "computational device including individually tailored generative models" is a device equipped with a specific generative model based on each user's preference information and historical data.

[0626] "Means for aggregating generative models and constructing an integrated model" refers to the function responsible for the process of aggregating multiple individual generative models and creating a collective model.

[0627] "A means of providing information to optimize selected measures based on real-time sentiment analysis results" refers to a function that instantly analyzes users' sentiment data and uses the results to dynamically adjust the content of suggestions and information provided.

[0628] "Means for saving the usage history of collected information and reflecting it in future optimizations" refers to a function that saves user behavior information collected in the past and uses it to improve future suggestions and information provision.

[0629] "Having a feedback function" means having a function that receives responses and results from users and uses them to improve the system's operation.

[0630] The system for implementing the present invention utilizes emotional data and preference information to provide users with optimal information and suggestions. The system consists of a server, a terminal, an emotional analysis engine, and a computing device including a generative model.

[0631] First, the user's facial expressions, speech, and text input are collected from the device. The emotion analysis engine analyzes this data in real time to identify emotional states such as positive, negative, and neutral. For example, the user's facial expressions are captured with a camera and analyzed using OpenCV. In addition, the user's voice input is converted into text using speech recognition technology, and emotions are inferred based on this.

[0632] The server integrates analyzed sentiment data with past preference information to create a personalized generative model. Using TensorFlow, this generative model is newly generated and adjusted for each user. As a result, the server generates the most suitable information and optimizes selection strategies. This can be achieved, for example, by recommending products or content that might interest a user if they appear to be enjoying themselves.

[0633] Furthermore, this generated information and suggestions are delivered to the user's device. When suggestions are presented, feedback is collected and sent to the server. This feedback allows the integrated model to continuously improve its accuracy.

[0634] A concrete example is the shopping experience in a virtual store. If a user smiles while looking at a product, recommendations for related items will be enhanced. As an example of a prompt, inputting the instruction "When the user shows a relaxed expression, present options with a relaxing effect" into the generating AI model will facilitate the provision of appropriate information.

[0635] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0636] Step 1:

[0637] The device collects input from the user. Specifically, it captures the user's facial expressions with a camera and records their speech with a microphone. This input data is treated as important information for identifying the user's current emotional state. Facial expression data is sent to an emotion analysis engine through image processing, and speech data is converted into text using speech recognition technology.

[0638] Step 2:

[0639] The server analyzes input data sent from the terminal using an emotion analysis engine. Specifically, it uses OpenCV to identify emotions (positive, negative, neutral, etc.) from facial expression data. In addition, data converted from speech to text is used as preference information through natural language processing. By integrating these analysis results, the server determines the user's current emotional state.

[0640] Step 3:

[0641] The server creates an individualized generative model based on the determined emotional state and past preference information. TensorFlow is used to process this data and generate a user-specific generative model. This generated model functions as a foundation for generating optimal information and suggestions that match the user's emotions and preferences.

[0642] Step 4:

[0643] Using the generated model, the server creates optimal information and suggestions for the user and sends them to the terminal. In this process, the generating AI model uses instructions such as, "When the user shows a relaxed expression, present options with a relaxing effect." The generated suggestions are then presented to the user, for example, as product recommendations while shopping in a virtual store.

[0644] Step 5:

[0645] The device then collects user responses to the presented suggestions. Specifically, it acquires data on the user's selections and changes in their emotions during those selections. This feedback data is sent to a server and used for further analysis and to improve the accuracy of the generative model.

[0646] Step 6:

[0647] The server uses the collected feedback data to improve the accuracy of the integrated model. This process involves adjusting the model based on the feedback, improving the accuracy of subsequent information provision. Through this iterative process, the entire system enables more refined personalization tailored to the user's preferences and emotions.

[0648] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0649] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0650] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0651] [Fourth Embodiment]

[0652] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0653] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0654] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0655] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0656] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0657] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0658] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0659] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0660] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0661] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0662] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0663] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0664] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0665] The present invention can be implemented as a system that integrates individually tailored generative models to provide suggestions tailored to the preferences and needs of a group. Typically, this system is applied in situations involving multiple users, generating optimal suggestions based on each user's information. The main components and operation of this system are described below.

[0666] Server role:

[0667] The server receives preference information sent from each user. This includes user preferences, past selection history, and allergy information. Based on this information, the server calculates a generative model specific to each user and stores it in the database. This model is tailored to the individual requests of each user.

[0668] The role of the integrated model:

[0669] The server integrates the generative models of multiple users as needed to generate a new integrated model. This integrated model aims to optimize decision-making for the entire group, and is used to generate suggestions that match the requirements of each user. For example, if multiple people need to choose a restaurant, the server can generate options that everyone will agree on.

[0670] Proposal generation and feedback:

[0671] The generated suggestions are delivered to the user's device and managed by the server. Users provide feedback by reviewing and selecting suggestions. This feedback is analyzed by the server and used to optimize the system for future suggestions.

[0672] Specific example:

[0673] For example, consider a scenario where friends plan to watch a movie together. Each user sends their favorite movie genres and movies they want to see to the server. The server collects each user's information and creates individual generative models based on it. Then, it integrates multiple generative models to generate a list that includes everyone's favorite genres and movies of interest. This list is displayed on the users' devices, and feedback from everyone can be used to further optimize future suggestions.

[0674] Thus, by using this invention, it becomes possible to make optimal decisions as a group while taking individual preferences into account. This achieves a level of personalization and comprehensiveness not found in conventional individual models.

[0675] The following describes the processing flow.

[0676] Step 1:

[0677] User: Enters their preferences into the device. This includes favorite genres, allergy information, topics of interest, etc.

[0678] Step 2:

[0679] Terminal: Formats the entered information into a data package and sends it to the server. The information is encrypted using a secure protocol.

[0680] Step 3:

[0681] Server: Analyzes preference information received from terminals and extracts features to create personalized generative models tailored to each user.

[0682] Step 4:

[0683] Server: Builds generative models tailored to each user and stores them in a database. These models are optimized using machine learning algorithms.

[0684] Step 5:

[0685] Server: If necessary, aggregates the generative models of multiple users to build an integrated model. The integrated model enables comprehensive recommendations based on the preferences of the entire group.

[0686] Step 6:

[0687] Server: Generates optimal suggestions based on the integrated model and creates specific options. These options are situation-dependent, such as a list of restaurants or movie choices.

[0688] Step 7:

[0689] Server: Distributes suggestions to the user's device. Suggestions are presented as notifications or in-app messages.

[0690] Step 8:

[0691] User: Review the options presented on the device and select the one they deem most preferable. The selection can be made with simple operations.

[0692] Step 9:

[0693] Terminal: Sends user selections to the server and provides feedback data.

[0694] Step 10:

[0695] Server: Analyzes received feedback and evaluates the quality of the selections. Based on this, the generative model is readjusted and used for future suggestions.

[0696] Each of the steps described above is designed to ensure smooth and effective information processing across the entire system.

[0697] (Example 1)

[0698] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0699] While existing information processing systems can provide suggestions based on individual user preferences, they struggle to create optimal suggestions for the group as a whole. Therefore, there is a growing need for a system that can provide suggestions that reflect the group's opinions while also satisfying each individual user, especially when multiple users share common goals and interests.

[0700] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0701] In this invention, the server includes means for providing an information processing device that includes individually tailored generative models, means for aggregating generative models from multiple information processing devices and constructing an integrated model, and means for generating and outputting relevant group information based on the integrated model. This makes it possible to automatically generate optimal proposals for the group as a whole while respecting the needs of individual users, and to encourage appropriate decision-making among members.

[0702] An "information processing device" refers to a mechanical or electronic device that can input, process, and output data.

[0703] A "generative model" refers to a set of algorithms trained to produce an expected output based on a specific input.

[0704] An "integrated model" refers to a unified algorithm that aggregates multiple individual generative models to provide the optimal overall output.

[0705] "Group-related information" refers to information that reflects the opinions and preferences of all users and is useful for decision-making for the group as a whole.

[0706] "Usage history" refers to a record of information about choices a user has made in the past and the results thereof.

[0707] "Preference information" refers to information that indicates an individual user's preferences and interests.

[0708] "Analysis function" refers to a process that has the ability to analyze data in detail and contributes to evaluating results and improving models.

[0709] "Suggestions" refer to recommendations that encourage users to make choices or take actions that are appropriate to their needs and circumstances.

[0710] This invention is an information processing system that uses individually tailored generative models to provide suggestions based on the preferences and needs of a group.

[0711] The server receives data such as preference information and past selection history from each user as input. This data is processed using a generative AI model to create individual generative models for each user. These generative models reflect each user's preferences and are stored in a database. If necessary, the server integrates these individual generative models to build an integrated model that provides optimal suggestions for the group.

[0712] The terminal displays server-generated suggestions to the user. These suggestions are delivered via an interface to facilitate user selection. The user selects a suggestion through the terminal and sends feedback to the server. The server analyzes the feedback received and uses it to further optimize future suggestions.

[0713] For example, when friends plan a trip together, their preferences for destinations and budget constraints are entered into the server. The server creates a generative model for each user, integrates them, and proposes a travel plan that satisfies everyone. This proposal is delivered to the user's device, and the final selection is sent back to the server as user feedback.

[0714] An example of a prompt to a generative AI model might be: "Please suggest the best travel destination for our group. Here are each member's preferences: Member A wants a beach, Member B wants historical sites, and Member C wants to go shopping."

[0715] In this way, the system can optimize the group as a whole while also providing personalized suggestions to each user.

[0716] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0717] Step 1:

[0718] Users use their devices to send preference information and past selection history to the server. This input includes categories of interest and past behavioral data. The server receives this data and stores it in a database specific to each user.

[0719] Step 2:

[0720] The server uses a generative AI model to create personalized generative models based on user preference information. Using the preference information received as input, it analyzes user preference patterns in a data-driven approach. Based on this analysis, a personalized generative model is output and saved.

[0721] Step 3:

[0722] If necessary, the server integrates multiple individual generative models to generate a new integrated model. The input data consists of generative models from multiple users, and the output is an integrated model that optimizes the opinions of the entire group. The generated integrated model comprehensively considers the preferences of different users to produce the optimal proposal for the group.

[0723] Step 4:

[0724] The server uses an integrated model to generate proposals and delivers them to the user's terminal. The input is the generated integrated model, and the output is the new proposal content displayed on the user's terminal. The terminal receives these proposals and presents them visually in an easy-to-use interface for the user to select from.

[0725] Step 5:

[0726] Users review suggestions via their terminal and send feedback on their selected items to the server. The input data is the selection result for the suggestions, which the server receives and analyzes. As output, the feedback information is stored and used to improve the accuracy of future suggestions.

[0727] In this way, the system constructs individual and integrated generative models from input data and provides users with personalized and collectively optimized suggestions.

[0728] (Application Example 1)

[0729] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0730] In modern information processing, it is crucial to consider the preferences and needs of multiple individuals simultaneously and optimize group decision-making. However, conventional systems often fail to adequately reflect individual preferences, making it difficult to propose solutions that satisfy the entire group. This invention aims to solve this problem and provide optimal proposals that reflect the preferences of each individual, particularly in scenarios involving multiple people.

[0731] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0732] In this invention, the server includes means for providing an information processing device that includes individually adjusted generation models; means for aggregating generation models from multiple information processing devices and constructing an integrated model; means for generating and outputting suggestion information based on the group's requests; means for saving the usage history of recorded information and reflecting it in subsequent optimizations; and means for having a suggestion function that optimizes dish selection based on the individual preferences of multiple people. This makes it possible to make optimal suggestions that take into account the preferences of multiple individuals.

[0733] A "personally tailored generative model" is an information processing model that reflects the preferences and requirements of each user and is optimized for a specific purpose.

[0734] An "information processing device" is a device or system used to process data and perform calculations.

[0735] "Aggregating generative models and constructing an integrated model" refers to the process of gathering and integrating multiple generative models to construct a comprehensive model that reflects the needs and preferences of the entire group.

[0736] "Generating and outputting proposal information" means creating proposals suitable for a group based on an integrated model and providing them to users.

[0737] "Recorded information usage history" refers to data such as past user choices and responses to suggestions, which is stored to optimize future suggestions.

[0738] The "suggestion function that optimizes dish selection based on the individual preferences of multiple people" is a function that takes into account each user's preferences and limitations to determine the optimal dish selection for multiple people.

[0739] The server, acting as an information processing device, receives preference information from multiple users and creates individually tailored generative models based on this information. These generative models reflect the users' preferences and past selection history. The created generative models are stored in a database, and if necessary, the generative models of multiple users are aggregated to build an integrated model. This integrated model is used to optimize decision-making for the entire group.

[0740] Based on this integrated model, the server generates suggestion information that reflects the group's needs and outputs it to each user's terminal. The terminal operates as a smartphone application and displays this suggestion information to the user. When a user provides feedback on the suggestion, that information is also sent to the server and stored as a record. This recorded information is used to optimize future suggestions.

[0741] The hardware used includes smartphones and cloud servers (e.g., AWS, Google Cloud), while the software utilizes React Native for mobile applications, Node.js for server systems, and MongoDB for databases. For data processing, user-entered preference data is sent to the server in JSON format, and Python machine learning libraries (e.g., TensorFlow, PyTorch) are used for data calculations to generate an integrated model.

[0742] For example, when suggesting the best lunch menu for several employees, using a prompt like, "Please suggest lunch menus for 5 employees. Their preferences and allergy information are as follows: Person A likes sushi, Person B likes pizza, and Person C has a wheat allergy," makes it possible to suggest meals that will satisfy everyone.

[0743] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0744] Step 1:

[0745] Users input their preferences and restrictions into a smartphone application. This information is sent to the server as JSON data. This data provides detailed information about each user's preferences.

[0746] Step 2:

[0747] The server parses the received JSON data and extracts preference information for each user. Based on the extracted information, it generates individually tailored generative models using Python's machine learning libraries. The generated models are stored in a database, ready for use in the next step.

[0748] Step 3:

[0749] The server aggregates generative models created by multiple users to build an integrated model. The integrated model aims to optimize the requirements of the entire group. It compares and integrates data between models to create an information structure that supports optimal decision-making.

[0750] Step 4:

[0751] The server generates suggestion information that matches the group's requests based on an integrated model. This information includes optimal dish selections that reflect each user's preferences and allergy information. The generated suggestions are sent to the user's device as push notifications.

[0752] Step 5:

[0753] Users review the received suggestion information and provide feedback to the server through the application. This feedback is data indicating their evaluation and satisfaction with the suggestion, and is collected on the server in real time.

[0754] Step 6:

[0755] The server analyzes the collected feedback and uses it to improve the accuracy of the generative and integrated models. It uses the feedback data to adjust parameters to improve the accuracy of future proposals.

[0756] This processing flow ensures that users receive efficient and optimal suggestions that take their individual preferences into account.

[0757] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0758] This invention is a system that generates suggestions that take the user's emotions into account by combining an emotion engine with individually tuned generative models. This system utilizes emotion data along with user preference information to achieve advanced personalization that reflects the individual's emotional state.

[0759] Server role:

[0760] The server creates a generative model based on user preference information and emotion data received from the terminal. The emotion engine analyzes each user's emotional state from their speech, facial expressions, and input text, and generates emotion data. Based on this, it builds a personalized generative model for each user and stores it in the database.

[0761] Emotional engine function:

[0762] The emotion engine analyzes user emotions in real time during interaction, identifying emotional states such as positive, negative, and neutral. This allows the server to add emotional information to the generative model, enabling more refined personalization. This emotional information is considered a crucial element when generating suggestions.

[0763] Generation of an integrated model:

[0764] The server integrates individual generative models to generate an integrated model that reflects the emotional state of the entire group. This enables optimal suggestions for enhancing positive collective experiences.

[0765] Suggestions and feedback:

[0766] Suggestions generated using the integrated model are delivered to the user's device. The user evaluates the presented options and makes a selection. The device sends feedback data about this selection to the server. This feedback includes changes in emotional state as determined by the emotion engine, which the server uses to improve the accuracy of future suggestions.

[0767] Specific example:

[0768] For example, consider a scenario where a family decides how to spend a holiday. Each family member inputs their desired activity and the emotions they feel during that activity (excited, wanting to relax, etc.) into a device. An emotion engine analyzes this information, and the server creates individual generative models and builds an integrated model. Based on this, holiday suggestions (e.g., theme park or hot spring trip) are considered and sent to the user. Through feedback, the system is adjusted to provide the best possible suggestions that the whole family can enjoy.

[0769] Through the above process, the present invention is a system that can improve the quality of the user experience through advanced emotion-based customization.

[0770] The following describes the processing flow.

[0771] Step 1:

[0772] User: Enters information about their emotional state and preferences into the device. This information includes desired activities and the emotions associated with them (e.g., excitement, relaxation).

[0773] Step 2:

[0774] Terminal: Sends information entered by the user to the emotion engine and analyzes the emotional state in real time. The analyzed emotional information and preference data are sent together to the server.

[0775] Step 3:

[0776] Server: Based on preference and sentiment data received from terminals, it creates a generative model tailored to each user. The generative model is optimized using machine learning algorithms and incorporates individual sentiment data.

[0777] Step 4:

[0778] Server: Aggregates generative models from multiple users to build an integrated model that reflects the emotional state of the entire group. This integrated model considers numerous emotional patterns and serves as the foundation for generating optimal suggestions.

[0779] Step 5:

[0780] Server: Based on an integrated model, it generates suggestions that align with the collective sentiment. The suggestions are adjusted to balance user preferences and emotions, forming appropriate options.

[0781] Step 6:

[0782] Server: Sends generated suggestions to users' terminals and distributes them to individual users. Suggestions are displayed as options and presented in a way that allows for individual feedback.

[0783] Step 7:

[0784] User: Review the suggestions displayed on the device and make a selection. When making a selection, evaluate them based on your emotional state and initial preferences.

[0785] Step 8:

[0786] Terminal: The emotion engine re-analyzes the user's selection results and the resulting changes in their emotional state, and sends that data to the server.

[0787] Step 9:

[0788] Server: Based on the feedback received, the server adjusts the generative and integrated models. This allows for learning and optimization to improve the accuracy of future proposals.

[0789] This process enables personalization that comprehensively considers emotions and preferences, allowing us to provide the most appropriate suggestions for the user.

[0790] (Example 2)

[0791] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0792] Conventional generative model-based suggestion systems could consider user preference information, but they were insufficient for personalization that reflected the user's emotional state. As a result, suggestions aimed at improving the user experience were inaccurate, sometimes leading to decreased satisfaction. Furthermore, it was difficult to make suggestions that considered the emotional state of the entire group, resulting in the challenge of not being able to make effective suggestions for large groups.

[0793] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0794] In this invention, the server includes means for constructing individually tailored generative models based on user emotional information and preference information; means for collecting multiple generative models and generating an integrated model that reflects the emotional state of the entire group; and means for transmitting the suggested content generated based on the integrated model to an information output device. This makes it possible to generate detailed suggestions that reflect the emotional state of individual users and suggested content that promotes a positive experience for the group.

[0795] An "information processing device" is a device that processes data input by a user and has the function of creating a generative model based on emotional information and preference information.

[0796] A "generative model" is a model that is adjusted to reflect the user's individual emotional and preference information, and serves as the foundation for generating personalized suggestions.

[0797] An "integrated model" is a model created by aggregating multiple generative models and taking into account the emotional state of the entire group, and is used to generate proposals for large groups.

[0798] "Emotional information" refers to data that indicates the user's current emotional state, and is obtained from information such as voice, text, and facial expression analysis.

[0799] "Preference information" refers to data about users' long-term preferences and tastes, and serves as the basis for making personalized recommendations to individual users.

[0800] An "information output device" is a device that has an output function to provide the user with the suggested content generated from the server.

[0801] "Proposed content" refers to options or activity proposals generated from a generative or integrated model and presented to the user or group.

[0802] "Feedback" refers to the evaluation and selection results of user suggestions, and is data used to improve the accuracy of the system.

[0803] The system of this invention utilizes user emotional and preference information and achieves advanced personalization using individually tailored generative AI models. The server generates suggestions in real time based on data collected from each user's terminal.

[0804] 1. Hardware and software:

[0805] User terminal: Equipped with an interface for receiving voice and text input, and performs data processing using an energy-saving processor.

[0806] Server: A high-performance computing system for large-scale data processing, which performs emotion analysis using "EmotionAnalyzer" software. It also uses "AIModelBuilder" to build generative AI models.

[0807] Information output device: A device that presents suggestions generated from the server to the user.

[0808] 2. Data processing and data calculation:

[0809] The user's device converts voice input into text using "SpeechToTextConverter" and prepares for sentiment analysis.

[0810] The server uses "EmotionAnalyzer" to extract emotional information from user input and adjusts the generated AI model based on this information.

[0811] "AIModelBuilder" creates generative models for each user, and then generates an integrated model for the group.

[0812] 3. Specific examples and examples of prompt statements:

[0813] For example, consider a scenario where a family decides how to spend their holiday. Each family member inputs their desired activity and their corresponding feelings (e.g., "exciting," "relaxing") into a terminal. This information is analyzed by an emotion engine, and the server builds a generative AI model to generate suggestions that the whole family can enjoy. The optimal activity (e.g., visiting a theme park, going to a hot spring) is then suggested.

[0814] An example of a prompt message is, "What kind of activity would you like to do today? Please tell us how you feel about that activity (e.g., I'm excited, I want to relax)."

[0815] This system aims to improve the user experience by providing suggestions that take into account the user's emotional state.

[0816] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0817] Step 1:

[0818] The user inputs their daily activities into the device. If voice input is used, the device uses "SpeechToTextConverter" to convert the voice data into text. This input data becomes the basis for subsequent sentiment analysis.

[0819] Step 2:

[0820] The device sends the converted text data to the server. The server sends the received text data to "EmotionAnalyzer," which analyzes the user's emotional state. "EmotionAnalyzer" analyzes keywords and context in the text and generates emotional information such as positive, negative, or neutral. This is output as emotional data.

[0821] Step 3:

[0822] The server integrates the generated sentiment data with user preference information and inputs it into "AIModelBuilder." "AIModelBuilder" then constructs individually optimized generative AI models based on this information. These generative models form the basis for providing user-specific suggestions and are stored on the server.

[0823] Step 4:

[0824] The server aggregates the generative models of multiple users and generates an integrated model using a "group modeling tool" that takes into account the emotional state and common preferences of the entire group. This integrated model is used to derive the most suitable suggestions for the group.

[0825] Step 5:

[0826] The server generates group-oriented suggestions based on an integrated model and distributes them to users through an information output device. The suggestions are presented as specific activities and options, prompting users to make choices.

[0827] Step 6:

[0828] The user evaluates the presented suggestions and makes a selection. The device records feedback on the selection and subsequent changes in emotional state. This feedback is sent to the server and used to improve the accuracy of future suggestions.

[0829] The above outlines the processing flow of this system's program. Each step involves specific data processing and model generation, aiming to improve the user experience.

[0830] (Application Example 2)

[0831] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0832] This invention aims to solve the problem of improving the experience of individual users and groups by providing a system that takes into account the emotional state and preference information of users and provides appropriate information and suggestions in real time. Conventional systems sometimes fail to capture users' interest and satisfaction because they provide information without adequately considering the emotions of the users.

[0833] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0834] In this invention, the server includes means for providing a computing device that analyzes emotions and generates information based on the user's emotional state, means for providing a computing device that includes an individually adjusted generative model, and means for providing information that optimizes selected measures based on real-time emotion analysis results. This enables the provision of personalized information according to the user's emotional state.

[0835] A "computational device that analyzes emotions and generates information based on the user's emotional state" is a device that determines emotions from the user's facial expressions, speech, and input data, and generates information that matches those emotions.

[0836] A "computational device including individually tailored generative models" is a device equipped with a specific generative model based on each user's preference information and historical data.

[0837] "Means for aggregating generative models and constructing an integrated model" refers to the function responsible for the process of aggregating multiple individual generative models and creating a collective model.

[0838] "A means of providing information to optimize selected measures based on real-time sentiment analysis results" refers to a function that instantly analyzes users' sentiment data and uses the results to dynamically adjust the content of suggestions and information provided.

[0839] "Means for saving the usage history of collected information and reflecting it in future optimizations" refers to a function that saves user behavior information collected in the past and uses it to improve future suggestions and information provision.

[0840] "Having a feedback function" means having a function that receives responses and results from users and uses them to improve the system's operation.

[0841] The system for implementing the present invention utilizes emotional data and preference information to provide users with optimal information and suggestions. The system consists of a server, a terminal, an emotional analysis engine, and a computing device including a generative model.

[0842] First, the user's facial expressions, speech, and text input are collected from the device. The emotion analysis engine analyzes this data in real time to identify emotional states such as positive, negative, and neutral. For example, the user's facial expressions are captured with a camera and analyzed using OpenCV. In addition, the user's voice input is converted into text using speech recognition technology, and emotions are inferred based on this.

[0843] The server integrates analyzed sentiment data with past preference information to create a personalized generative model. Using TensorFlow, this generative model is newly generated and adjusted for each user. As a result, the server generates the most suitable information and optimizes selection strategies. This can be achieved, for example, by recommending products or content that might interest a user if they appear to be enjoying themselves.

[0844] Furthermore, this generated information and suggestions are delivered to the user's device. When suggestions are presented, feedback is collected and sent to the server. This feedback allows the integrated model to continuously improve its accuracy.

[0845] A concrete example is the shopping experience in a virtual store. If a user smiles while looking at a product, recommendations for related items will be enhanced. As an example of a prompt, inputting the instruction "When the user shows a relaxed expression, present options with a relaxing effect" into the generating AI model will facilitate the provision of appropriate information.

[0846] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0847] Step 1:

[0848] The device collects input from the user. Specifically, it captures the user's facial expressions with a camera and records their speech with a microphone. This input data is treated as important information for identifying the user's current emotional state. Facial expression data is sent to an emotion analysis engine through image processing, and speech data is converted into text using speech recognition technology.

[0849] Step 2:

[0850] The server analyzes input data sent from the terminal using an emotion analysis engine. Specifically, it uses OpenCV to identify emotions (positive, negative, neutral, etc.) from facial expression data. In addition, data converted from speech to text is used as preference information through natural language processing. By integrating these analysis results, the server determines the user's current emotional state.

[0851] Step 3:

[0852] The server creates an individualized generative model based on the determined emotional state and past preference information. TensorFlow is used to process this data and generate a user-specific generative model. This generated model functions as a foundation for generating optimal information and suggestions that match the user's emotions and preferences.

[0853] Step 4:

[0854] Using the generated model, the server creates optimal information and suggestions for the user and sends them to the terminal. In this process, the generating AI model uses instructions such as, "When the user shows a relaxed expression, present options with a relaxing effect." The generated suggestions are then presented to the user, for example, as product recommendations while shopping in a virtual store.

[0855] Step 5:

[0856] The device then collects user responses to the presented suggestions. Specifically, it acquires data on the user's selections and changes in their emotions during those selections. This feedback data is sent to a server and used for further analysis and to improve the accuracy of the generative model.

[0857] Step 6:

[0858] The server uses the collected feedback data to improve the accuracy of the integrated model. This process involves adjusting the model based on the feedback, improving the accuracy of subsequent information provision. Through this iterative process, the entire system enables more refined personalization tailored to the user's preferences and emotions.

[0859] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0860] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0861] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0862] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0863] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0864] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0865] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0866] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0867] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0868] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0869] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0870] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0871] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0872] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0873] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0874] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0875] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0876] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0877] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0878] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0879] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0880] The following is further disclosed regarding the embodiments described above.

[0881] (Claim 1)

[0882] [Means for providing a computing device including individually adjusted generative models,

[0883] [Means for aggregating generative models from multiple computing devices and constructing an integrated model,

[0884] [Means for generating and outputting group-related information based on an integrated model,

[0885] [Means for saving the usage history of collected information and reflecting it in future optimizations,

[0886] A system that includes this.

[0887] (Claim 2)

[0888] [The system according to claim 1, which obtains user preference information from a computing device and uses it to adjust the generative model.

[0889] (Claim 3)

[0890] The system according to claim 1, which has a feedback function for evaluating the user's selection results and improving the accuracy of the integrated model.

[0891] "Example 1"

[0892] (Claim 1)

[0893] [Means for providing an information processing device including individually adjusted generative models,

[0894] [Means for aggregating generative models from multiple information processing devices and constructing an integrated model,

[0895] [Means for generating and outputting group-related information based on an integrated model,

[0896] [Means for saving the usage history of collected instructions and reflecting it in future optimizations,

[0897] [A means for generating a common proposal for a group based on individual information provided by an information processing device,

[0898] A system that includes this.

[0899] (Claim 2)

[0900] [The system according to claim 1, which obtains user preference information from an information processing device and uses it to adjust the generation model.

[0901] (Claim 3)

[0902] The system according to claim 1, which has an analysis function for evaluating the user's selection results and improving the accuracy of the integrated model.

[0903] "Application Example 1"

[0904] (Claim 1)

[0905] [Means for providing an information processing device including individually adjusted generative models,

[0906] [Means for aggregating generative models from multiple information processing devices and constructing an integrated model,

[0907] [Means for generating and outputting proposal information based on the group's requirements,

[0908] [Means for saving the usage history of recorded information and reflecting it in future optimizations,

[0909] [Means having a suggestion function that optimizes the selection of dishes based on the individual preferences of multiple people,

[0910] A system that includes this.

[0911] (Claim 2)

[0912] [The system according to claim 1, which obtains user preference information from an information processing device and uses it to adjust the generation model.

[0913] (Claim 3)

[0914] The system according to claim 1, which has a feedback function for evaluating the user's selection results and improving the accuracy of the integrated model.

[0915] "Example 2 of combining an emotion engine"

[0916] (Claim 1)

[0917] [Means for providing an information processing device that constructs a generative model individually adjusted based on the user's emotional information and preference information,

[0918] [Methods for collecting generative models from multiple information processing devices and generating an integrated model that reflects the emotional state of the entire group,

[0919] [Means for generating proposals for a group based on an integrated model and transmitting them to an information output device,

[0920] [A means of saving data to improve the accuracy of suggestions based on the user's selection results and using it for optimization in the future,

[0921] [Methods for analyzing emotional states in real time and integrating emotional data into generative models,

[0922] A system that includes this.

[0923] (Claim 2)

[0924] [The system according to claim 1, which uses user emotion information and preference information obtained from an information processing device to adjust the generation model.

[0925] (Claim 3)

[0926] The system according to claim 1, which has a function to evaluate the user's selections and emotional changes after the proposed content has been delivered to the user, and to aim for improvement in the accuracy of the next integrated model.

[0927] "Application example 2 of combining emotional engines"

[0928] (Claim 1)

[0929] [Means for providing a computing device that analyzes emotions and generates information based on the user's emotional state,

[0930] [Means for providing a computing device including individually adjusted generative models,

[0931] [Means for aggregating generative models from multiple computing devices and constructing an integrated model,

[0932] [Means for generating and outputting group-related information based on an integrated model,

[0933] [A means of providing information to optimize selected measures based on real-time sentiment analysis results,

[0934] [Means for saving the usage history of collected information and reflecting it in future optimizations,

[0935] A system that includes this.

[0936] (Claim 2)

[0937] [The system according to claim 1, which acquires user preference information and emotional data from a computing device and uses them to adjust the generative model.

[0938] (Claim 3)

[0939] The system according to claim 1, which has a feedback function to evaluate the user's selection results and emotional changes in order to improve the accuracy of the integrated model. [Explanation of Symbols]

[0940] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for providing a computing device that includes individually adjusted generative models, A means for aggregating generative models from multiple computing devices and constructing an integrated model, A means for generating and outputting group-related information based on an integrated model, A means to save the usage history of collected information and reflect it in future optimizations, A system that includes this.

2. The system according to claim 1, which obtains user preference information from a computing device and uses it to adjust the generative model.

3. The system according to claim 1, which has a feedback function for evaluating the user's selection results and improving the accuracy of the integrated model.

Images