System

Abstract

A system is provided.SOLUTION: A system comprising: means for registering a generative artificial intelligence model for a user to set his / her own preferences and requirements; means for collecting data of generative artificial intelligence models from a plurality of users; means for preprocessing the collected data and creating a shared generative artificial intelligence model that reflects preferences and requirements of each user in a balanced manner by a fusion algorithm; means for generating an optimal recommendation using the generated shared generative artificial intelligence model; and means for presenting the generated optimal recommendation to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, there are generative AI models that consider each user's preferences and requirements individually, but no system exists that simultaneously considers the preferences and requirements of multiple users and provides optimal recommendations that satisfy everyone. For example, when handling multiple proposals simultaneously, each user's generative AI model operates independently, making it difficult to generate proposals that reflect everyone's needs in a balanced manner. Another challenge is providing different interfaces and recommendations for each user. Therefore, the present invention aims to provide a system that combines the preferences and requirements of multiple users to provide optimal recommendations. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system including the following means: a means for registering a generative AI model that allows a user to set their own preferences and requirements; a means for collecting generative AI model data from multiple users; a means for preprocessing the collected data and creating a shared generative AI model that reflects each user's preferences and requirements in a balanced manner using a fusion algorithm; a means for generating optimal recommendations using the generated shared generative AI model; and a means for presenting the generated optimal recommendations to the user. The system also includes a means for generating an optimal interface based on the user's request content and situation and displaying it to the user, and a means for performing error handling. This makes it possible to provide optimal recommendations that take into account everyone's preferences and requirements.

[0006] "User" means an individual or group that uses the system and sets their own preferences and requirements.

[0007] "Preferences" refers to a user's preferred categories, genres, or specific preferences.

[0008] "Requirements" indicate the desired conditions and constraints that a user provides to a system.

[0009] A "generative AI model" is an artificial intelligence model that learns user preferences and requests and generates suggestions and recommendations based on that information.

[0010] "Means of registration" refers to the method or functionality by which a user stores their generated AI model in the system and makes that data available.

[0011] "Means of collection" refers to the method or function of obtaining data for the generative AI model from multiple users and collecting it in the system.

[0012] "Preprocessing" refers to the process of standardizing the format of collected data to make it easier to analyze and process.

[0013] A "fusion algorithm" is a calculation method that integrates multiple collected generative AI models to create a shared generative AI model that reflects each user's data in a balanced manner.

[0014] A "shared generative AI model" is an integrated generative AI model that takes into account the preferences and requirements of multiple users and is used to provide optimal recommendations.

[0015] "Recommendation" means a suggestion, candidate, or recommendation to a user.

[0016] "Means for generating optimal recommendations" means a method or function for generating the most appropriate suggestions for a user using a shared generative AI model.

[0017] A "means for presenting recommendations" is a method or function for visually or otherwise presenting the generated recommendations to the user.

[0018] "Interface" refers to the screen, input means, and operating means that users use to interact with the system.

[0019] "Error handling" refers to methods and functions for detecting errors and inconsistencies that occur in a system and dealing with them appropriately. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] System configuration:

[0042] The system of the present invention includes a server, a terminal, and a user. The user sets their preferences and requirements and registers a generative AI model. The server collects generative AI model data from multiple users and combines it to create a shared generative AI model. The terminal presents the generated recommendations and optimized interface to the user.

[0043] System Action:

[0044] First, the user inputs their preferences and requests, and a generative AI model is created based on them. The user accesses the system and uses a device to register this generative AI model. The device then sends the user's input information to the server, which then stores this data in a database.

[0045] When multiple users want to use a service together, for example when a group of friends decides on a restaurant, each user's generative AI model is requested from the server. The server collects these models and preprocesses the data, specifically standardizing the format and imputing missing values. A fusion algorithm is then applied to create a shared generative AI model that balances each user's preferences and needs.

[0046] The server uses this shared generative AI model to generate optimal recommendations for the user. For example, if multiple friends like Japanese and Italian food, the server might list fusion restaurants as candidates. These recommendations are then sent to the device and presented to the user.

[0047] Furthermore, the server generates the optimal interface for the use case based on the user's request and the current situation. For example, in the case of a ticket vending machine, a menu optimized for the user is automatically displayed. This interface is also displayed on the terminal, making it easy for the user to operate.

[0048] Examples:

[0049] User A registers his / her preference as "Japanese food," his / her budget as "under 3,000 yen," and his / her meal time as "evening." User B registers his / her preference as "Italian food," his / her budget as "under 4,000 yen," and his / her meal time as "evening." If both users want to choose a restaurant in the same group, each user's generative AI model is sent to the server. The server collects this data and performs preprocessing. A fusion algorithm then creates a shared generative AI model that matches each user's preference, such as a restaurant that serves Japanese and Italian fusion cuisine.

[0050] The server uses a shared generative AI model to generate optimal recommendations and create a candidate list. This list is sent to the device and displayed to User A and User B. User A and User B can then choose a restaurant that satisfies both of them from this list. In addition, the device displays an interface that is tailored to the situation, making operation intuitive and easy.

[0051] This system not only makes it possible to provide optimal information that takes into account the preferences and needs of multiple users in a balanced manner, but also provides a flexible interface that suits the situation.

[0052] The processing flow will be explained below.

[0053] Step 1:

[0054] Users set their own preferences and requests by entering information such as their preferred genre, budget, and desired time slot through the device interface.

[0055] Step 2:

[0056] The device receives the input information and creates a generative AI model, which learns the user's preferences and requests and converts them into the required data structure.

[0057] Step 3:

[0058] The device sends the generated AI model data to the server, which receives the data and stores it in a database.

[0059] Step 4:

[0060] When multiple users use the service together, each user's generated AI model is requested from the server, such as when a group of friends want to decide on a restaurant together.

[0061] Step 5:

[0062] The server collects the data for the requested generative AI model and performs preprocessing, which includes standardizing the data format and filling in missing values.

[0063] Step 6:

[0064] The server then runs a fusion algorithm based on the pre-processed data, creating a shared generative AI model that balances each user's preferences and needs.

[0065] Step 7:

[0066] The server uses a shared generative AI model to generate optimal recommendations, including options that balance the preferences of multiple users, such as "restaurants that serve Japanese and Italian fusion cuisine."

[0067] Step 8:

[0068] The server sends the generated recommendations to the terminal, which receives the recommendations and presents them to the user. The user can then select from the displayed list.

[0069] Step 9:

[0070] The server generates the optimal interface based on the user's request and the current situation, and sends it to the device, providing a display tailored to a specific use case, such as the interface of a ticket vending machine.

[0071] Step 10:

[0072] The terminal displays the optimal interface sent from the server, allowing the user to use the system in an easy-to-use environment.

[0073] In this way, by performing specific processing at each step, a system is realized that provides optimal recommendations that reflect the preferences and requests of multiple users in a balanced manner.

[0074] Example 1

[0075] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0076] In modern information provision systems, it is extremely difficult to generate recommendations that reflect the preferences and needs of multiple users in a balanced manner and provide an appropriate interface. In particular, when user preferences vary greatly or when an interface that adapts to real-time situations is required, it is difficult for general systems to respond flexibly and appropriately. This leads to issues such as a poor user experience and difficulty in achieving satisfaction.

[0077] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0078] In this invention, the server includes means for registering a generation program that allows a user to set their own preferences and requirements, means for collecting generation program data from multiple users, means for preprocessing the collected data, standardizing formats, and completing missing values, means for creating a shared generation program that uses a fusion algorithm to reflect the preferences and requirements of each user in a balanced manner, means for generating optimal recommendations using the generated shared generation program, means for presenting the generated optimal recommendations to the user, means for generating an optimal interface based on the user's request content and situation and displaying it to the user, and means for error handling. This makes it possible to provide flexible and appropriate recommendations and interfaces in real time, taking into account the preferences and requirements of multiple users in a balanced manner.

[0079] A "generator" is a program that is created based on data entered by a user to reflect their preferences and requirements.

[0080] "Means for collecting" refers to a method or device for collecting generator data from multiple users.

[0081] "Preprocessing" refers to the process of analyzing collected data, standardizing the format, and filling in missing values.

[0082] A "fusion algorithm" is a calculation method for unifying multiple generation programs and creating a shared generation program that reflects each user's preferences and requirements in a balanced manner.

[0083] A "shared generation program" is an integrated generation program created to reflect each user's preferences and requests in a balanced manner.

[0084] "Means for generating optimal recommendations" refers to a method or apparatus for generating the best suggestions or candidates for each user using a shared generator.

[0085] "Means for generating an optimal interface" refers to a method or device for creating a screen display or operating means that allows the user to operate intuitively and easily based on the user's request and situation.

[0086] "Error handling" refers to a method or device for appropriately handling unexpected errors or abnormalities that occur in a system, thereby maintaining normal system operation.

[0087] The system according to the present invention is composed of a user, a server, and a terminal, and each component operates in cooperation with each other. Specific embodiments will be described below.

[0088] overview:

[0089] In this system, users set their own preferences and requirements and register a generator program based on those preferences. The server collects generator data from multiple users, preprocesses it, and then applies a fusion algorithm to create a shared generator program that reflects each user's preferences and requirements in a balanced manner. Finally, the shared generator program is used to generate optimal recommendations and provide them to the user.

[0090] Equipment and software used:

[0091] Server: Database management system (e.g., MySQL, PostgreSQL), generative AI model (e.g., OpenAI GPT-4)

[0092] Devices: Smartphones, PCs

[0093] Software: Data preprocessing programs, fusion algorithms

[0094] System Action:

[0095] First, the user inputs their preferences and requests into the terminal. For example, the user may register information such as a preference for Japanese food, a budget of 3,000 yen or less, and a preference for dinner time. Based on this, a generation program is created and sent to the server via the terminal.

[0096] The server stores the received generation programs in a database. Next, the server collects data from the generation programs collected by multiple users and preprocesses the data. Specifically, it standardizes the data format and performs processes such as filling in missing values.

[0097] After the preprocessing is complete, the server applies a fusion algorithm to create a shared generator that balances each user's preferences and needs, enabling it to recommend recommendations that will satisfy both a group of friends who like Japanese and Italian food, for example.

[0098] Using the created shared generation program, the server generates optimal recommendations. These recommendations are sent to the device and presented to the user. The user can review the recommendations and take action as necessary. The server also generates an optimal interface based on the user's request and the current situation, and displays it on the device.

[0099] Examples:

[0100] Example prompt sentence:

[0101] "I like Japanese food, my budget is under 3,000 yen, and I'd like to eat in the evening. Can you recommend a suitable restaurant?"

[0102] User A registers his / her preference as "Japanese food," his / her budget as "under 3,000 yen," and his / her meal time as "evening." User B registers his / her preference as "Italian food," his / her budget as "under 4,000 yen," and his / her meal time as "evening." If both users want to choose a restaurant in the same group, each user's generator program is sent to the server. The server collects this data and performs preprocessing. The fusion algorithm then creates a shared generator program that matches each user's preference, such as a restaurant that serves Japanese-Italian fusion cuisine.

[0103] The server uses a shared generation program to generate optimal recommendations and create a candidate list. This list is sent to the device and displayed to User A and User B. User A and User B can then choose a restaurant that satisfies both of them from this list. In addition, the device displays an interface that is tailored to the situation, making operation intuitive and easy.

[0104] This system not only makes it possible to provide optimal information that takes into account the preferences and needs of multiple users in a balanced manner, but also provides a flexible interface that suits the situation.

[0105] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0106] Step 1:

[0107] The user inputs their preferences and requests into the terminal. For example, the user may input information such as a preference for Japanese food, a budget of 3,000 yen or less, and a preference for dinner time. The input information becomes data for creating a generation program within the terminal.

[0108] Input: User preference (Japanese food), budget (under 3000 yen), meal time (evening)

[0109] Output: Data format for generator

[0110] Specific behavior:

[0111] The user uses a smartphone or PC interface to input information such as preferences, budget, and time. This input is then appropriately converted within the device and a data format is created for the generation program.

[0112] Step 2:

[0113] The terminal sends the generation program and the user's input data to the server, which stores the received data in a database.

[0114] Input: Data format for generator

[0115] Output: Data stored in the server database

[0116] Specific behavior:

[0117] The terminal generates a data packet containing the generation program and transmits it over the network to the server, which then analyzes the received data and stores it in a database.

[0118] Step 3:

[0119] The server collects multiple user generated programs from the database and preprocesses them, including standardizing data formats and filling in missing values.

[0120] Input: Generator for multiple users in a database

[0121] Output: Preprocessed generator data

[0122] Specific behavior:

[0123] The server retrieves the generation program from the database, unifies the data formats, performs imputation processing if there are missing values, and generates preprocessed data.

[0124] Step 4:

[0125] The server uses the preprocessed data to apply a fusion algorithm to create a shared generator that balances each user's preferences and requirements.

[0126] Input: Preprocessed generator data

[0127] Output: Shared generator

[0128] Specific behavior:

[0129] The server applies a fusion algorithm (e.g., weighted average method, clustering) to the preprocessed data to generate a shared generator that reflects the preferences and requirements of multiple users.

[0130] Step 5:

[0131] The server uses a shared generator to generate optimal recommendations for each user, which are then sent to the terminal and presented to the user.

[0132] Input: Shared Generator

[0133] Output: Best recommendation and recommendation list

[0134] Specific behavior:

[0135] The server creates the optimal recommendations for each user from the generated shared generation program and generates a recommendation list. This list is sent to the terminal and displayed on the user's smartphone or PC.

[0136] Step 6:

[0137] The server generates an optimal interface based on the user's request and the current situation, and the terminal displays this interface to the user.

[0138] Input: User request details and status

[0139] Output: Optimized interface

[0140] Specific behavior:

[0141] The server selects an appropriate interface template based on the user's context (such as location and time of use), renders the selected template, and sends it to the device. The device displays the received template, allowing the user to operate it intuitively.

[0142] In this way, the systems work together to balance user preferences and requirements and provide optimal recommendations and interfaces.

[0143] (Application example 1)

[0144] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0145] While conventional content distribution services can reflect the preferences of individual users, it is difficult to recommend content that reflects the preferences and needs of all users in a balanced manner when multiple users are watching together. In addition, there are limited ways for multiple users to share their viewing experience in real time, which can lead to a decrease in satisfaction with the shared viewing experience.

[0146] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0147] In this invention, the server includes means for registering a generative artificial intelligence model for a user to set his or her preferences and requirements, means for collecting data on the generative artificial intelligence model from multiple users, means for preprocessing the collected data and using a fusion algorithm to create a shared generative artificial intelligence model that reflects the preferences and requirements of each user in a balanced manner, means for recommending optimal content using the generated shared generative artificial intelligence model, means for presenting the recommended optimal content to the user, and means for providing a real-time function for multiple users to share their viewing experience, thereby enabling the recommendation of optimal content that takes into account the preferences and requirements of multiple users in a balanced manner and joint viewing in real time.

[0148] "User" refers to an individual or group of people who access the System and use it to set their own preferences and requirements.

[0149] A "generative artificial intelligence model" refers to an algorithm or dataset that is generated based on user preferences and requests.

[0150] "Means of registration" refers to the method by which a user inputs their preferences and requests and stores a generative artificial intelligence model based on them in the system.

[0151] "Means of collection" refers to the method of aggregating data for the generative AI model provided by multiple users.

[0152] "Preprocessing" refers to the process of standardizing the format of collected data and filling in missing values.

[0153] A "fusion algorithm" refers to a technical method for balancing the preferences and needs of multiple users to create a new shared generative model.

[0154] A "shared generative AI model" is a model created to reflect the preferences and requirements of multiple users, meeting everyone's needs in a balanced manner.

[0155] "Recommendation means" refers to a method of selecting and presenting optimal content using a shared generative artificial intelligence model.

[0156] "Means of presentation" refers to the method of displaying the generated recommendation content to the user.

[0157] "Real-time functionality" refers to functionality that allows multiple users to share a viewing experience and communicate simultaneously.

[0158] System configuration

[0159] The system according to the present invention includes a server, a terminal, and a user. The user sets his or her preferences and requests and registers a generative AI model. The server collects generative AI model data from multiple users and combines it to create a shared generative AI model. The terminal presents the generated recommendations and optimized interface to the user.

[0160] System action

[0161] 1. User preference data input:

[0162] Users use a terminal to input their preferences and requests into the system, such as movie genres, favorite actors, and viewing times.

[0163] 2. Registering a generative AI model:

[0164] Based on the information entered by the user, the terminal generates a generative artificial intelligence model and sends it to the server, which stores the received data in a database (e.g., MySQL).

[0165] 3. Data collection and preprocessing:

[0166] Generative AI models collected from multiple users are preprocessed by the server, which includes standardizing formats and filling in missing values. This process uses machine learning frameworks such as TensorFlow and PyTorch that run on the server.

[0167] 4. Applying the fusion algorithm:

[0168] After the preprocessing is complete, the server applies a fusion algorithm to the data to create a shared generative AI model that reflects the preferences and needs of multiple users in a balanced manner. This shared generative model takes each user's preferences into full consideration.

[0169] 5. Content Recommendations:

[0170] The server uses a shared live AI model to recommend optimal content, for example, movies containing elements of comedy and suspense for multiple users to watch.

[0171] 6. Recommendations:

[0172] The generated list of optimal content is sent to the terminal and presented to the user, who can then select the content they prefer.

[0173] 7. Real-time function:

[0174] The server uses Firebase Realtime Database to provide real-time functionality for multiple users to share the viewing experience, allowing users to watch content together while chatting in real time.

[0175] Specific examples

[0176] User A: "Action movies", "Comedy dramas"

[0177] User B: "Romance movies", "Suspense dramas"

[0178] Based on this information, the server recommends content that reflects the preferences of both parties in a balanced manner.

[0179] Example prompt for a generative AI model:

[0180] User A: Action movies, comedy dramas

[0181] User B: Romance movies, suspense dramas

[0182] Requirements for a shared generative AI model: Recommending optimal video content to satisfy multiple preferences in the evening

[0183] This system enables optimal content recommendations that balance the preferences and needs of multiple users, and enables real-time collaborative viewing.

[0184] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0185] Step 1: Enter user preference data

[0186] Users use a device to input their preferences and requests (e.g., movie genres, favorite actors, viewing times). The device receives these inputs and formats the data for later transmission to the server. The input data includes genres and keywords selected by the user, and is used by the device as basic information for generating a generative artificial intelligence model.

[0187] Input: User preferences and request data

[0188] Output: Formatted user preference data

[0189] Step 2: Registering a Generative AI Model

[0190] The device generates a generative AI model based on the preference data acquired from the user. The generated model is sent to the server via API and stored in a server-side database (e.g., MySQL).

[0191] Input: Formatted user preference data

[0192] Output: Generative AI model

[0193] Step 3: Data collection and preprocessing

[0194] The server collects generative AI models sent by multiple users. The collected data undergoes preprocessing, such as standardizing the format and filling in missing values. During this process, the data is cleaned using frameworks such as TensorFlow.

[0195] Input: Generative AI model

[0196] Output: A preprocessed generative AI model

[0197] Step 4: Applying the fusion algorithm

[0198] The server applies a fusion algorithm to the preprocessed generative AI model, creating a shared generative AI model that balances the preferences and needs of multiple users. This fusion operation is performed using a machine learning framework such as PyTorch.

[0199] Input: A preprocessed generative AI model

[0200] Output: Shared generative AI model

[0201] Step 5: Content Recommendations

[0202] The server uses a shared generative artificial intelligence model to recommend the best content, taking into account the user's preferences and past viewing history to generate a list of the best content. This recommendation is made in real time.

[0203] Input: Shared generative artificial intelligence model

[0204] Output: Recommended content list

[0205] Step 6: Present recommended content

[0206] The server sends a list of recommended content to the device, which receives it and displays it in an interface optimized for the user. The user can then select and watch the content of their interest from the list.

[0207] Input: Recommended content list

[0208] Output: Optimized content list displayed on user device

[0209] Step 7: Providing real-time functionality

[0210] The server uses Firebase Realtime Database to provide functionality for multiple users to share their viewing experiences in real time, specifically allowing users to chat with each other in real time and share their viewing status.

[0211] Input: Viewer information and chat messages

[0212] Output: Real-time synchronized viewing information and messages

[0213] The above processing steps enable the preferences and needs of multiple users to be considered in a balanced manner, and optimal content recommendations and real-time joint viewing can be realized.

[0214] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0215] System configuration

[0216] The system according to the present invention includes a server, a terminal, a user, and an emotion engine. The user sets his or her preferences and requests and registers a generative AI model. The server collects generative AI model data from multiple users and combines it to create a shared generative AI model. Furthermore, the emotion engine recognizes the user's emotions and reflects them in the system. The terminal presents the generated recommendations and optimized interface to the user.

[0217] System action

[0218] First, the user inputs their preferences and requests, and a generative AI model is created based on them. The user accesses the system and uses a device to register this generative AI model. The device then sends the user's input information to the server, which then stores this data in a database.

[0219] When multiple users want to use a service together, for example when deciding on a restaurant for a group of friends, each user's generative AI model is requested from the server. The server collects these models and preprocesses the data, specifically standardizing the format and imputing missing values. A fusion algorithm is then applied to create a shared generative AI model that balances each user's preferences and needs.

[0220] The server uses the results of this shared generative AI model and the emotion engine to generate optimal recommendations for the user. For example, if multiple friends like Japanese and Italian food, the server might list fusion restaurants as candidates. These recommendations are then sent to the device and presented to the user.

[0221] Furthermore, the server generates the optimal interface for the use case based on the user's request and the current situation. For example, in the case of a ticket vending machine, a menu optimized for the user is automatically displayed. This interface is also displayed on the terminal, making it easy for the user to operate.

[0222] Emotion engine processing

[0223] The emotion engine uses sensors and cameras to recognize the user's emotions in real time. While the user is using the device, it analyzes their facial expressions, voice, and text input to determine their emotional state. The server takes in the data from this emotion engine and uses it to generate generative AI models and provide recommendations. This makes it possible to make suggestions tailored to the situation, such as recommending places where a user who is feeling stressed can relax.

[0224] Specific examples

[0225] User A registers his / her preference as "Japanese food," his / her budget as "under 3,000 yen," and his / her meal time as "evening." User B registers his / her preference as "Italian food," his / her budget as "under 4,000 yen," and his / her meal time as "evening." If both users want to choose a restaurant in the same group, each user's generative AI model is sent to the server. The server collects this data and performs preprocessing. A fusion algorithm then creates a shared generative AI model that matches each user's preference, such as a restaurant that serves Japanese and Italian fusion cuisine.

[0226] Furthermore, if User A is feeling stressed, the emotion engine recognizes this state and notifies the server. Based on this, the server adjusts the recommendation content to suggest relaxing environments and services. The server uses the results of the shared generative AI model and data from the emotion engine to generate optimal recommendations and create a candidate list. This list is sent to the device and displayed to User A and User B.

[0227] The server then generates the optimal interface for each use case and displays it on the device. For example, if User A is feeling stressed, it displays an interface with a menu that helps them relax.

[0228] This system not only makes it possible to provide optimal information that reflects the preferences and needs of multiple users in a balanced manner, but also provides a flexible interface that responds to the user's emotional state and usage situation.

[0229] The processing flow will be explained below.

[0230] Step 1:

[0231] Users set their own preferences and requests. Specifically, they input information such as their preferred genre, budget, and desired time slot through the interface on their device.

[0232] Step 2:

[0233] The device receives the input information and creates a generative AI model, which learns the user's preferences and requests and structures the data based on that information.

[0234] Step 3:

[0235] The device sends the generated AI model data to the server, which receives the data and stores it in a database.

[0236] Step 4:

[0237] When multiple users use the service together, for example when a group of friends want to choose a restaurant, each user's generated AI model is requested from the server.

[0238] Step 5:

[0239] The server collects the requested data for the generative AI model and performs preprocessing, which includes standardizing the data format and filling in missing values.

[0240] Step 6:

[0241] The server runs a fusion algorithm based on the preprocessed data to create a shared generative AI model that reflects each user's preferences and requirements in a balanced manner.

[0242] Step 7:

[0243] The server receives emotion data generated by the emotion engine from the user's device. Specifically, sensors and cameras installed on the device analyze the user's facial expressions, voice, and text input to determine their emotional state.

[0244] Step 8:

[0245] The server integrates the generated shared generative AI model with the emotion data to generate optimal recommendations, taking into account data from the emotion engine and adjusting the recommendations accordingly.

[0246] Step 9:

[0247] The server sends the generated optimal recommendations to the device, which receives the recommendations and presents them to the user, who can then select from the displayed list.

[0248] Step 10:

[0249] The server generates the most appropriate interface based on the user's request, emotional state, and the situation at hand. For example, a user feeling stressed will be shown a menu that helps them relax.

[0250] Step 11:

[0251] The terminal displays the optimal interface sent from the server, allowing the user to use the system in an easy-to-use environment.

[0252] Examples:

[0253] User A sets his preference as "Japanese food," his budget as "under 3,000 yen," and his meal time as "evening," and creates a generative AI model. User B sets his preference as "Italian food," his budget as "under 4,000 yen," and his meal time as "evening," and creates a generative AI model. If both users want to choose a restaurant in the same group, each user's generative AI model is sent to the server. The server collects this data and performs preprocessing. A fusion algorithm then creates a shared generative AI model that reflects each user's preferences, such as a restaurant that serves Japanese and Italian fusion cuisine.

[0254] Next, if User A is feeling stressed, the emotion engine recognizes this state and sends the data to the server. The server then adjusts the recommendation content taking this emotion data into account, suggesting, for example, a relaxing restaurant that serves Japanese and Italian fusion cuisine. The optimal recommendation is generated, and a candidate list is sent to the device and displayed to the user.

[0255] Finally, the server generates an optimal interface and adjusts the display content according to the user's state based on the emotional data. A relaxing menu interface is displayed on the device, providing an environment that is easy for the user to operate. In this way, the system provides optimal information by taking into account the preferences and emotional states of multiple users in a balanced manner.

[0256] Example 2

[0257] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0258] Conventional recommendation systems have limitations in reflecting user preferences and requests, making it difficult to incorporate the intentions of multiple users in a balanced manner. Furthermore, they lack the ability to provide recommendations that take into account the user's emotional state and optimal operation screens that are tailored to the situation.

[0259] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for registering a generative machine learning model for a user to set his or her preferences and requirements; means for collecting data of the generative machine learning model from multiple users; means for preprocessing the collected data and creating a shared generative machine learning model that reflects the preferences and requirements of each user in a balanced manner using a fusion algorithm; means for generating optimal recommendations using the generated shared generative machine learning model; emotion recognition means for recognizing the emotional state of the user in real time; means for providing the generated optimal recommendation to the user using the result of combining the data obtained from the emotion recognition means; and means for presenting the generated optimal recommendation to the user. This makes it possible to not only precisely reflect the preferences and requirements of users and incorporate the intentions of multiple users in a balanced manner, but also to provide optimal recommendations and operation screens according to the emotional state and situation.

[0260] A "generative machine learning model" is a data model created using machine learning algorithms based on user preferences and requirements.

[0261] "Means of registration" refers to the function for storing and managing user input information and generated machine learning models in the system's database or cloud service.

[0262] The "means of collection" is a function for centrally collecting and managing generative machine learning models and related data provided by multiple users.

[0263] "Preprocessing" refers to processes for improving data quality, such as standardizing the format of collected data and filling in missing values.

[0264] A "fusion algorithm" is a computational method for creating a new model that reflects each user's preferences and requirements in a balanced manner, based on data from a generative machine learning model collected from multiple users.

[0265] A "shared generative machine learning model" is a generative machine learning model that reflects the preferences and requirements of multiple users and combines them in a balanced manner.

[0266] The "means for generating recommendations" is a function for providing appropriate suggestions and options to the user using the generated shared generative machine learning model.

[0267] The "emotion recognition means" is a function that analyzes the user's facial expressions, voice, text input, etc. in real time to determine the user's emotional state.

[0268] The "means for providing" is a function for displaying the generated recommendations to the user and making them actually usable.

[0269] The "presentation means" is a function for visually or audibly presenting the recommendation content to the user.

[0270] Describe in detail the implementation of the invention

[0271] The present invention relates to a system that utilizes a generative machine learning model that reflects user preferences and requirements, creates a shared generative machine learning model that incorporates the requirements of multiple users in a balanced manner, and provides optimal recommendations. The system is also characterized by recognizing the user's emotional state in real time and providing recommendations and operation screens that correspond to that state.

[0272] First, a user inputs their preferences and requests using a device. The device formats this information and creates a generative machine learning model. This generative machine learning model is sent from the device to a server and stored in a database on the server. Next, the server collects the generative machine learning model data sent by multiple users and performs preprocessing. This preprocessing includes standardizing the format and filling in missing values. The collected data is then subjected to a fusion algorithm to create a shared generative machine learning model that reflects each user's preferences and requests in a balanced manner.

[0273] Using this shared generative machine learning model, the server generates optimal recommendations for users. For example, for a user group that likes "Japanese" and "Italian," it might list fusion restaurants as candidates. This optimized recommendation is sent to the user's device and presented visually or audibly.

[0274] Furthermore, the system uses emotion recognition to recognize the user's emotional state in real time. While the user is operating the device, the emotion engine analyzes the user's facial expressions, voice, and text through the camera, microphone, and text input to determine their emotional state. The data obtained from the emotion recognition is sent to the server and reflected in the generated recommendations. For example, if the user is feeling stressed, recommendations will be generated that provide a relaxing environment or service.

[0275] This system also dynamically generates the optimal operation screen based on the user's request and situation. For example, a user feeling stressed will be shown an easy-to-operate relaxation menu. This interface is displayed on the user's device and is designed to be intuitive for the user to operate.

[0276] Specific examples

[0277] User A likes "Japanese food," has a budget of "less than 3,000 yen," and registers his / her meal time as "evening." User B likes "Italian food," has a budget of "less than 4,000 yen," and also registers his / her meal time as "evening." If both users want to choose a restaurant in the same group, the generative machine learning models of each user are sent to the server. The server collects this data and performs preprocessing. After that, a fusion algorithm recommends restaurants that serve Japanese and Italian fusion cuisine.

[0278] Examples of prompts include:

[0279] "I like Japanese food, my budget is under 3,000 yen, and I eat in the evening. My friend likes Italian food, my budget is under 4,000 yen, and we also eat in the evening. What kind of restaurant would you recommend?"

[0280] This not only allows the system to precisely reflect users' preferences and requests and incorporate the intentions of multiple users in a balanced manner, but also makes it possible to provide optimal recommendations and operation screens that are tailored to the user's emotional state and situation.

[0281] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0282] Step 1:

[0283] The user inputs their preferences and requests. Using a terminal, the user inputs detailed information such as preferences, budget, and meal times. Input information may include "Japanese food," "budget 3,000 yen," and "dinner." This information is formatted by the terminal and converted into a unified format such as JSON.

[0284] Step 2:

[0285] A generative machine learning model is created and registered. The device creates a generative machine learning model based on the input data. The generated model is sent to the server, which registers it in a database. As an output, the generative machine learning model is saved on the server.

[0286] Step 3:

[0287] Collecting data for generative machine learning models. The server collects data for generative machine learning models sent by multiple users. The collected data is compiled for preprocessing. Multiple generative machine learning models are included as input.

[0288] Step 4:

[0289] The server preprocesses the collected data. The server performs preprocessing such as standardizing the format of the collected data and completing missing values. For example, it standardizes the data format for each user and estimates and completes missing data. The output is a preprocessed dataset.

[0290] Step 5:

[0291] A fusion algorithm is applied to create a shared generative machine learning model. The server uses the preprocessed data to create a shared generative machine learning model that balances the preferences and needs of multiple users. For example, data from users who like "Japanese food" and "Italian food" is integrated to generate a recommendation model for fusion cuisine. A shared generative machine learning model is generated as the output.

[0292] Step 6:

[0293] Emotion recognition is performed. While the user is using the device, the emotion engine analyzes the user's facial expressions and voice using sensors and cameras to determine their emotional state. The server collects this data and understands the user's emotional state. Real-time emotional data is included as input.

[0294] Step 7:

[0295] Generate optimal recommendations. The server integrates the shared generative machine learning model with emotion recognition data to generate optimal recommendations. For example, if the user is feeling stressed, it will recommend restaurants where they can relax. The output is an optimized recommendation list.

[0296] Step 8:

[0297] Present the recommendations to the user. The server sends the generated recommendation list to the terminal and presents it to the user. For example, a list of fusion restaurants is displayed on the user's terminal. The input contains the optimal recommendation list, and the output is a screen that the user can visually confirm.

[0298] Step 9:

[0299] The optimal interface is generated and displayed. The server generates the optimal operation screen based on the user's request and emotional state. For example, a user who is feeling stressed will be shown a simple and easy-to-understand relaxation menu. The optimal interface is then displayed on the terminal as output.

[0300] (Application example 2)

[0301] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0302] While there are existing recommendation systems based on user preferences and requests, it is difficult to provide flexible recommendations that take into account each individual's emotional state. It is also difficult to integrate the preferences of multiple users and make recommendations that satisfy everyone. Furthermore, there are few systems in physical stores that can provide an optimal interface based on the user's emotions.

[0303] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0304] In this invention, the server includes means for registering a generative artificial intelligence model for a user to set their own preferences and requirements, means for collecting data on the generative artificial intelligence model from multiple users, means for preprocessing the collected data and creating a shared generative artificial intelligence model that reflects each user's preferences and requirements in a balanced manner using a fusion algorithm, means for generating optimal recommendations using the generated shared generative artificial intelligence model, means for presenting the generated optimal recommendations to the user, means for recognizing the user's emotions in real time using an emotion recognition engine, and means for adjusting the recommendation content based on the recognized emotions and automatically generating an optimal interface. This makes it possible to provide more optimal recommendations and interfaces that take both the user's preferences and emotions into consideration.

[0305] A "user" is someone who uses the system to register their preferences and needs and receive recommendations.

[0306] "Preferences" is a concept that refers to the things and conditions that a user prefers.

[0307] "Requirements" refer to the specific requests and conditions that users have for the system.

[0308] A "generative artificial intelligence model" is an artificial intelligence-based data model that is generated based on user preferences and requirements.

[0309] "Means of registration" refers to the functions and methods by which users can input and save their preferences and requests in the system.

[0310] "Means of collection" refers to the functions and methods for collecting data for the generative artificial intelligence model sent by multiple users.

[0311] "Preprocessing" is the process of processing collected data, such as filling in missing values ​​and standardizing the format.

[0312] A "fusion algorithm" is an algorithm that balances the preferences and needs of multiple users to create a single shared generative artificial intelligence model.

[0313] A "shared generative artificial intelligence model" is a common generative artificial intelligence model created by combining the preferences and requests of multiple users.

[0314] "Means for generating" refers to methods and functions for generating optimal recommendations using a shared generative artificial intelligence model.

[0315] "Recommendation" refers to recommended information and services provided by a system based on a user's preferences and requests.

[0316] The "presentation means" refers to a method or function for showing the generated recommended information or services to the user.

[0317] An "emotion recognition engine" is a system or device that analyzes a user's facial expressions, voice, text, etc. to recognize their emotional state in real time.

[0318] "Means of recognition" refers to functions and methods for capturing the user's emotions in real time, analyzing them, and making judgments.

[0319] "Tuning" means methods or functions for modifying and optimizing recommendations based on perceived sentiment.

[0320] "Interface" refers to the display screen and operation method for exchanging information between the user and the system.

[0321] "Automatic generation means" refers to methods or functions that allow the system to operate on its own based on specific conditions and create an appropriate interface.

[0322] The system for implementing this invention includes a server, a terminal, a user, and an emotion engine. The user sets his or her preferences and requests, and generates a generative artificial intelligence model based on them. The generated model is sent from the user's terminal to the server and stored in the server's database.

[0323] System configuration and program processing

[0324] server

[0325] The server collects data for generative AI models sent by multiple users and preprocesses it. This preprocessing involves standardizing the format and filling in missing values. Next, a fusion algorithm is applied to create a shared generative AI model that reflects each user's preferences and requests in a balanced manner. Optimal recommendations are generated based on this created model. Furthermore, emotional data from an emotion recognition engine is incorporated, the recommendation content is adjusted, and an optimal interface is generated according to the situation.

[0326] Specific software used includes scikit-learn for data preprocessing and fusion algorithms and the face_recognition library for emotion recognition.

[0327] Terminal

[0328] The device provides a means for users to input their preferences and requests and register them on the server. It also displays recommended information and optimized interfaces received from the server to the user. The device is equipped with sensors and cameras to transmit the user's facial expressions and voice to an emotion recognition engine in real time.

[0329] As a specific example, smartphones and tablets can be used as devices, as they have built-in cameras and microphones and can be used to collect data for emotion recognition.

[0330] User

[0331] Users input their preferences and requests through their devices, and a generative AI model is generated based on this information and registered on the server. Furthermore, the user's emotional data is also collected in real time and sent to the server. This allows the system to provide optimal recommendations tailored to the user's situation.

[0332] Specific examples

[0333] For example, if User A specifies his / her preference and requirement that he / she prefers Japanese food and is only available in the evening, this information is sent to the server as a generative AI model. Similarly, if User B specifies his / her preference and requirement that he / she prefers Italian food and has a budget of 4,000 yen or less, the same applies. By combining this information, the server can recommend restaurants that offer Japanese-Italian fusion cuisine.

[0334] If User A is feeling stressed, the emotion recognition engine will recognize that emotion in real time and notify the server. Based on this, the server will adjust its recommendations to suggest relaxing environments and services.

[0335] Prompt Sentence Examples

[0336] "Please recommend restaurants based on User A's preference for Japanese food and User B's preference for Italian food. Also, if User A is feeling stressed, please generate appropriate recommendations that will help them relax."

[0337] In this way, the system can simultaneously consider user preferences and emotions to provide optimal recommendations and interfaces for physical stores.

[0338] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0339] Step 1:

[0340] Users input their preferences and requests to create a generative AI model.

[0341] The user uses a device to input their preferences and requests, such as their preferred cuisine genre, budget, and time of day. The input data is converted into a generative AI model on the device. Inputs include preferences and requests such as "Japanese food," "under 3,000 yen," and "evening." The output is a generative AI model that reflects the user's preferences and requests.

[0342] Step 2:

[0343] Send the generative AI model from the device to the server

[0344] The generative AI model entered by the user is sent from the device to the server. The device sends the input data in packets. The generative AI model is used as input, and the model is saved on the server side as output.

[0345] Step 3:

[0346] The server collects and preprocesses the generated AI model data from multiple users.

[0347] The server receives generative AI model data sent by multiple users. The received data undergoes preprocessing, such as format standardization and missing value completion. The input is the generative AI models from multiple users, and the output is the preprocessed data.

[0348] Step 4:

[0349] The pre-processed data is processed through a fusion algorithm to create a shared generative AI model.

[0350] The server inputs the preprocessed data into a fusion algorithm to create a shared generative AI model that reflects each user's preferences and requirements in a balanced manner. The input is the preprocessed data, and the output is the shared generative AI model.

[0351] Step 5:

[0352] The server uses an emotion recognition engine to recognize the user's emotions in real time and captures the data.

[0353] The device sends the user's facial image and voice to the emotion recognition engine in real time. The server captures the recognized emotion data and stores it in a database. The input is the user's facial image and voice, and the output is the recognized emotional state data.

[0354] Step 6:

[0355] The server adjusts the recommendation content based on the emotional data and generates the optimal interface.

[0356] The server analyzes the acquired emotional data and adjusts the recommendation content based on that data. It also automatically generates an optimal interface for the user. The input is emotional state data, and the output is the adjusted recommendation content and the optimal interface.

[0357] Step 7:

[0358] The server sends the recommendations and the optimal interface to the user's device.

[0359] The generated recommendation content and interface are sent from the server to the user's device, which receives them and displays them to the user. The input is the recommendation content and interface data, and the output is the information presented to the user.

[0360] Step 8:

[0361] The user interacts with the recommendations and interface and sends feedback to the server.

[0362] The user operates the recommended content and interface provided through the device, and sends feedback from the device to the server as needed. The input is the user's operations and feedback data, and the output is the feedback information accumulated on the server.

[0363] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0364] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0365] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0366] [Second embodiment]

[0367] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0368] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0369] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0370] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0371] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0372] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0373] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0374] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0375] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0376] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0377] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0378] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0379] System configuration:

[0380] The system of the present invention includes a server, a terminal, and a user. The user sets their preferences and requirements and registers a generative AI model. The server collects generative AI model data from multiple users and combines it to create a shared generative AI model. The terminal presents the generated recommendations and optimized interface to the user.

[0381] System Action:

[0382] First, the user inputs their preferences and requests, and a generative AI model is created based on them. The user accesses the system and uses a device to register this generative AI model. The device then sends the user's input information to the server, which then stores this data in a database.

[0383] When multiple users want to use a service together, for example when a group of friends decides on a restaurant, each user's generative AI model is requested from the server. The server collects these models and preprocesses the data, specifically standardizing the format and imputing missing values. A fusion algorithm is then applied to create a shared generative AI model that balances each user's preferences and needs.

[0384] The server uses this shared generative AI model to generate optimal recommendations for the user. For example, if multiple friends like Japanese and Italian food, the server might list fusion restaurants as candidates. These recommendations are then sent to the device and presented to the user.

[0385] Furthermore, the server generates the optimal interface for the use case based on the user's request and the current situation. For example, in the case of a ticket vending machine, a menu optimized for the user is automatically displayed. This interface is also displayed on the terminal, making it easy for the user to operate.

[0386] Examples:

[0387] User A registers his / her preference as "Japanese food," his / her budget as "under 3,000 yen," and his / her meal time as "evening." User B registers his / her preference as "Italian food," his / her budget as "under 4,000 yen," and his / her meal time as "evening." If both users want to choose a restaurant in the same group, each user's generative AI model is sent to the server. The server collects this data and performs preprocessing. A fusion algorithm then creates a shared generative AI model that matches each user's preference, such as a restaurant that serves Japanese and Italian fusion cuisine.

[0388] The server uses a shared generative AI model to generate optimal recommendations and create a candidate list. This list is sent to the device and displayed to User A and User B. User A and User B can then choose a restaurant that satisfies both of them from this list. In addition, the device displays an interface that is tailored to the situation, making operation intuitive and easy.

[0389] This system not only makes it possible to provide optimal information that takes into account the preferences and needs of multiple users in a balanced manner, but also provides a flexible interface that suits the situation.

[0390] The processing flow will be explained below.

[0391] Step 1:

[0392] Users set their own preferences and requests by entering information such as their preferred genre, budget, and desired time slot through the device interface.

[0393] Step 2:

[0394] The device receives the input information and creates a generative AI model, which learns the user's preferences and requests and converts them into the required data structure.

[0395] Step 3:

[0396] The device sends the generated AI model data to the server, which receives the data and stores it in a database.

[0397] Step 4:

[0398] When multiple users use the service together, each user's generated AI model is requested from the server, such as when a group of friends want to decide on a restaurant together.

[0399] Step 5:

[0400] The server collects the data for the requested generative AI model and performs preprocessing, which includes standardizing the data format and filling in missing values.

[0401] Step 6:

[0402] The server then runs a fusion algorithm based on the pre-processed data, creating a shared generative AI model that balances each user's preferences and needs.

[0403] Step 7:

[0404] The server uses a shared generative AI model to generate optimal recommendations, including options that balance the preferences of multiple users, such as "restaurants that serve Japanese and Italian fusion cuisine."

[0405] Step 8:

[0406] The server sends the generated recommendations to the terminal, which receives the recommendations and presents them to the user. The user can then select from the displayed list.

[0407] Step 9:

[0408] The server generates the optimal interface based on the user's request and the current situation, and sends it to the device, providing a display tailored to a specific use case, such as the interface of a ticket vending machine.

[0409] Step 10:

[0410] The terminal displays the optimal interface sent from the server, allowing the user to use the system in an easy-to-use environment.

[0411] In this way, by performing specific processing at each step, a system is realized that provides optimal recommendations that reflect the preferences and requests of multiple users in a balanced manner.

[0412] Example 1

[0413] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0414] In modern information provision systems, it is extremely difficult to generate recommendations that reflect the preferences and needs of multiple users in a balanced manner and provide an appropriate interface. In particular, when user preferences vary greatly or when an interface that adapts to real-time situations is required, it is difficult for general systems to respond flexibly and appropriately. This leads to issues such as a poor user experience and difficulty in achieving satisfaction.

[0415] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0416] In this invention, the server includes means for registering a generation program that allows a user to set their own preferences and requirements, means for collecting generation program data from multiple users, means for preprocessing the collected data, standardizing formats, and completing missing values, means for creating a shared generation program that uses a fusion algorithm to reflect the preferences and requirements of each user in a balanced manner, means for generating optimal recommendations using the generated shared generation program, means for presenting the generated optimal recommendations to the user, means for generating an optimal interface based on the user's request content and situation and displaying it to the user, and means for error handling. This makes it possible to provide flexible and appropriate recommendations and interfaces in real time, taking into account the preferences and requirements of multiple users in a balanced manner.

[0417] A "generator" is a program that is created based on data entered by a user to reflect their preferences and requirements.

[0418] "Means for collecting" refers to a method or device for collecting generator data from multiple users.

[0419] "Preprocessing" refers to the process of analyzing collected data, standardizing the format, and filling in missing values.

[0420] A "fusion algorithm" is a calculation method for unifying multiple generation programs and creating a shared generation program that reflects each user's preferences and requirements in a balanced manner.

[0421] A "shared generation program" is an integrated generation program created to reflect each user's preferences and requests in a balanced manner.

[0422] "Means for generating optimal recommendations" refers to a method or apparatus for generating the best suggestions or candidates for each user using a shared generator.

[0423] "Means for generating an optimal interface" refers to a method or device for creating a screen display or operating means that allows the user to operate intuitively and easily based on the user's request and situation.

[0424] "Error handling" refers to a method or device for appropriately handling unexpected errors or abnormalities that occur in a system, thereby maintaining normal system operation.

[0425] The system according to the present invention is composed of a user, a server, and a terminal, and each component operates in cooperation with each other. Specific embodiments will be described below.

[0426] overview:

[0427] In this system, users set their own preferences and requirements and register a generator program based on those preferences. The server collects generator data from multiple users, preprocesses it, and then applies a fusion algorithm to create a shared generator program that reflects each user's preferences and requirements in a balanced manner. Finally, the shared generator program is used to generate optimal recommendations and provide them to the user.

[0428] Equipment and software used:

[0429] Server: Database management system (e.g., MySQL, PostgreSQL), generative AI model (e.g., OpenAI GPT-4)

[0430] Devices: Smartphones, PCs

[0431] Software: Data preprocessing programs, fusion algorithms

[0432] System Action:

[0433] First, the user inputs their preferences and requests into the terminal. For example, the user may register information such as a preference for Japanese food, a budget of 3,000 yen or less, and a preference for dinner time. Based on this, a generation program is created and sent to the server via the terminal.

[0434] The server stores the received generation programs in a database. Next, the server collects data from the generation programs collected by multiple users and preprocesses the data. Specifically, it standardizes the data format and performs processes such as filling in missing values.

[0435] After the preprocessing is complete, the server applies a fusion algorithm to create a shared generator that balances each user's preferences and needs, enabling it to recommend recommendations that will satisfy both a group of friends who like Japanese and Italian food, for example.

[0436] Using the created shared generation program, the server generates optimal recommendations. These recommendations are sent to the device and presented to the user. The user can review the recommendations and take action as necessary. The server also generates an optimal interface based on the user's request and the current situation, and displays it on the device.

[0437] Examples:

[0438] Example prompt sentence:

[0439] "I like Japanese food, my budget is under 3,000 yen, and I'd like to eat in the evening. Can you recommend a suitable restaurant?"

[0440] User A registers his / her preference as "Japanese food," his / her budget as "under 3,000 yen," and his / her meal time as "evening." User B registers his / her preference as "Italian food," his / her budget as "under 4,000 yen," and his / her meal time as "evening." If both users want to choose a restaurant in the same group, each user's generator program is sent to the server. The server collects this data and performs preprocessing. The fusion algorithm then creates a shared generator program that matches each user's preference, such as a restaurant that serves Japanese-Italian fusion cuisine.

[0441] The server uses a shared generation program to generate optimal recommendations and create a candidate list. This list is sent to the device and displayed to User A and User B. User A and User B can then choose a restaurant that satisfies both of them from this list. In addition, the device displays an interface that is tailored to the situation, making operation intuitive and easy.

[0442] This system not only makes it possible to provide optimal information that takes into account the preferences and needs of multiple users in a balanced manner, but also provides a flexible interface that suits the situation.

[0443] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0444] Step 1:

[0445] The user inputs their preferences and requests into the terminal. For example, the user may input information such as a preference for Japanese food, a budget of 3,000 yen or less, and a preference for dinner time. The input information becomes data for creating a generation program within the terminal.

[0446] Input: User preference (Japanese food), budget (under 3000 yen), meal time (evening)

[0447] Output: Data format for generator

[0448] Specific behavior:

[0449] The user uses a smartphone or PC interface to input information such as preferences, budget, and time. This input is then appropriately converted within the device and a data format is created for the generation program.

[0450] Step 2:

[0451] The terminal sends the generation program and the user's input data to the server, which stores the received data in a database.

[0452] Input: Data format for generator

[0453] Output: Data stored in the server database

[0454] Specific behavior:

[0455] The terminal generates a data packet containing the generation program and transmits it over the network to the server, which then analyzes the received data and stores it in a database.

[0456] Step 3:

[0457] The server collects multiple user generated programs from the database and preprocesses them, including standardizing data formats and filling in missing values.

[0458] Input: Generator for multiple users in a database

[0459] Output: Preprocessed generator data

[0460] Specific behavior:

[0461] The server retrieves the generation program from the database, unifies the data formats, performs imputation processing if there are missing values, and generates preprocessed data.

[0462] Step 4:

[0463] The server uses the preprocessed data to apply a fusion algorithm to create a shared generator that balances each user's preferences and requirements.

[0464] Input: Preprocessed generator data

[0465] Output: Shared generator

[0466] Specific behavior:

[0467] The server applies a fusion algorithm (e.g., weighted average method, clustering) to the preprocessed data to generate a shared generator that reflects the preferences and requirements of multiple users.

[0468] Step 5:

[0469] The server uses a shared generator to generate optimal recommendations for each user, which are then sent to the terminal and presented to the user.

[0470] Input: Shared Generator

[0471] Output: Best recommendation and recommendation list

[0472] Specific behavior:

[0473] The server creates the optimal recommendations for each user from the generated shared generation program and generates a recommendation list. This list is sent to the terminal and displayed on the user's smartphone or PC.

[0474] Step 6:

[0475] The server generates an optimal interface based on the user's request and the current situation, and the terminal displays this interface to the user.

[0476] Input: User request details and status

[0477] Output: Optimized interface

[0478] Specific behavior:

[0479] The server selects an appropriate interface template based on the user's context (such as location and time of use), renders the selected template, and sends it to the device. The device displays the received template, allowing the user to operate it intuitively.

[0480] In this way, the systems work together to balance user preferences and requirements and provide optimal recommendations and interfaces.

[0481] (Application example 1)

[0482] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0483] While conventional content distribution services can reflect the preferences of individual users, it is difficult to recommend content that reflects the preferences and needs of all users in a balanced manner when multiple users are watching together. In addition, there are limited ways for multiple users to share their viewing experience in real time, which can lead to a decrease in satisfaction with the shared viewing experience.

[0484] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0485] In this invention, the server includes means for registering a generative artificial intelligence model for a user to set his or her preferences and requirements, means for collecting data on the generative artificial intelligence model from multiple users, means for preprocessing the collected data and using a fusion algorithm to create a shared generative artificial intelligence model that reflects the preferences and requirements of each user in a balanced manner, means for recommending optimal content using the generated shared generative artificial intelligence model, means for presenting the recommended optimal content to the user, and means for providing a real-time function for multiple users to share their viewing experience, thereby enabling the recommendation of optimal content that takes into account the preferences and requirements of multiple users in a balanced manner and joint viewing in real time.

[0486] "User" refers to an individual or group of people who access the System and use it to set their own preferences and requirements.

[0487] A "generative artificial intelligence model" refers to an algorithm or dataset that is generated based on user preferences and requests.

[0488] "Means of registration" refers to the method by which a user inputs their preferences and requests and stores a generative artificial intelligence model based on them in the system.

[0489] "Means of collection" refers to the method of aggregating data for the generative AI model provided by multiple users.

[0490] "Preprocessing" refers to the process of standardizing the format of collected data and filling in missing values.

[0491] A "fusion algorithm" refers to a technical method for balancing the preferences and needs of multiple users to create a new shared generative model.

[0492] A "shared generative AI model" is a model created to reflect the preferences and requirements of multiple users, meeting everyone's needs in a balanced manner.

[0493] "Recommendation means" refers to a method of selecting and presenting optimal content using a shared generative artificial intelligence model.

[0494] "Means of presentation" refers to the method of displaying the generated recommendation content to the user.

[0495] "Real-time functionality" refers to functionality that allows multiple users to share a viewing experience and communicate simultaneously.

[0496] System configuration

[0497] The system according to the present invention includes a server, a terminal, and a user. The user sets his or her preferences and requests and registers a generative AI model. The server collects generative AI model data from multiple users and combines it to create a shared generative AI model. The terminal presents the generated recommendations and optimized interface to the user.

[0498] System action

[0499] 1. User preference data input:

[0500] Users use a terminal to input their preferences and requests into the system, such as movie genres, favorite actors, and viewing times.

[0501] 2. Registering a generative AI model:

[0502] Based on the information entered by the user, the terminal generates a generative artificial intelligence model and sends it to the server, which stores the received data in a database (e.g., MySQL).

[0503] 3. Data collection and preprocessing:

[0504] Generative AI models collected from multiple users are preprocessed by the server, which includes standardizing formats and filling in missing values. This process uses machine learning frameworks such as TensorFlow and PyTorch that run on the server.

[0505] 4. Applying the fusion algorithm:

[0506] After the preprocessing is complete, the server applies a fusion algorithm to the data to create a shared generative AI model that reflects the preferences and needs of multiple users in a balanced manner. This shared generative model takes each user's preferences into full consideration.

[0507] 5. Content Recommendations:

[0508] The server uses a shared live AI model to recommend optimal content, for example, movies containing elements of comedy and suspense for multiple users to watch.

[0509] 6. Recommendations:

[0510] The generated list of optimal content is sent to the terminal and presented to the user, who can then select the content they prefer.

[0511] 7. Real-time function:

[0512] The server uses Firebase Realtime Database to provide real-time functionality for multiple users to share the viewing experience, allowing users to watch content together while chatting in real time.

[0513] Specific examples

[0514] User A: "Action movies", "Comedy dramas"

[0515] User B: "Romance movies", "Suspense dramas"

[0516] Based on this information, the server recommends content that reflects the preferences of both parties in a balanced manner.

[0517] Example prompt for a generative AI model:

[0518] User A: Action movies, comedy dramas

[0519] User B: Romance movies, suspense dramas

[0520] Requirements for a shared generative AI model: Recommending optimal video content to satisfy multiple preferences in the evening

[0521] This system enables optimal content recommendations that balance the preferences and needs of multiple users, and enables real-time collaborative viewing.

[0522] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0523] Step 1: Enter user preference data

[0524] Users use a device to input their preferences and requests (e.g., movie genres, favorite actors, viewing times). The device receives these inputs and formats the data for later transmission to the server. The input data includes genres and keywords selected by the user, and is used by the device as basic information for generating a generative artificial intelligence model.

[0525] Input: User preferences and request data

[0526] Output: Formatted user preference data

[0527] Step 2: Registering a Generative AI Model

[0528] The device generates a generative AI model based on the preference data acquired from the user. The generated model is sent to the server via API and stored in a server-side database (e.g., MySQL).

[0529] Input: Formatted user preference data

[0530] Output: Generative AI model

[0531] Step 3: Data collection and preprocessing

[0532] The server collects generative AI models sent by multiple users. The collected data undergoes preprocessing, such as standardizing the format and filling in missing values. During this process, the data is cleaned using frameworks such as TensorFlow.

[0533] Input: Generative AI model

[0534] Output: A preprocessed generative AI model

[0535] Step 4: Applying the fusion algorithm

[0536] The server applies a fusion algorithm to the preprocessed generative AI model, creating a shared generative AI model that balances the preferences and needs of multiple users. This fusion operation is performed using a machine learning framework such as PyTorch.

[0537] Input: A preprocessed generative AI model

[0538] Output: Shared generative AI model

[0539] Step 5: Content Recommendations

[0540] The server uses a shared generative artificial intelligence model to recommend the best content, taking into account the user's preferences and past viewing history to generate a list of the best content. This recommendation is made in real time.

[0541] Input: Shared generative artificial intelligence model

[0542] Output: Recommended content list

[0543] Step 6: Present recommended content

[0544] The server sends a list of recommended content to the device, which receives it and displays it in an interface optimized for the user. The user can then select and watch the content of their interest from the list.

[0545] Input: Recommended content list

[0546] Output: Optimized content list displayed on user device

[0547] Step 7: Providing real-time functionality

[0548] The server uses Firebase Realtime Database to provide functionality for multiple users to share their viewing experiences in real time, specifically allowing users to chat with each other in real time and share their viewing status.

[0549] Input: Viewer information and chat messages

[0550] Output: Real-time synchronized viewing information and messages

[0551] The above processing steps enable the preferences and needs of multiple users to be considered in a balanced manner, and optimal content recommendations and real-time joint viewing can be realized.

[0552] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0553] System configuration

[0554] The system according to the present invention includes a server, a terminal, a user, and an emotion engine. The user sets his or her preferences and requests and registers a generative AI model. The server collects generative AI model data from multiple users and combines it to create a shared generative AI model. Furthermore, the emotion engine recognizes the user's emotions and reflects them in the system. The terminal presents the generated recommendations and optimized interface to the user.

[0555] System action

[0556] First, the user inputs their preferences and requests, and a generative AI model is created based on them. The user accesses the system and uses a device to register this generative AI model. The device then sends the user's input information to the server, which then stores this data in a database.

[0557] When multiple users want to use a service together, for example when deciding on a restaurant for a group of friends, each user's generative AI model is requested from the server. The server collects these models and preprocesses the data, specifically standardizing the format and imputing missing values. A fusion algorithm is then applied to create a shared generative AI model that balances each user's preferences and needs.

[0558] The server uses the results of this shared generative AI model and the emotion engine to generate optimal recommendations for the user. For example, if multiple friends like Japanese and Italian food, the server might list fusion restaurants as candidates. These recommendations are then sent to the device and presented to the user.

[0559] Furthermore, the server generates the optimal interface for the use case based on the user's request and the current situation. For example, in the case of a ticket vending machine, a menu optimized for the user is automatically displayed. This interface is also displayed on the terminal, making it easy for the user to operate.

[0560] Emotion engine processing

[0561] The emotion engine uses sensors and cameras to recognize the user's emotions in real time. While the user is using the device, it analyzes their facial expressions, voice, and text input to determine their emotional state. The server takes in the data from this emotion engine and uses it to generate generative AI models and provide recommendations. This makes it possible to make suggestions tailored to the situation, such as recommending places where a user who is feeling stressed can relax.

[0562] Specific examples

[0563] User A registers his / her preference as "Japanese food," his / her budget as "under 3,000 yen," and his / her meal time as "evening." User B registers his / her preference as "Italian food," his / her budget as "under 4,000 yen," and his / her meal time as "evening." If both users want to choose a restaurant in the same group, each user's generative AI model is sent to the server. The server collects this data and performs preprocessing. A fusion algorithm then creates a shared generative AI model that matches each user's preference, such as a restaurant that serves Japanese and Italian fusion cuisine.

[0564] Furthermore, if User A is feeling stressed, the emotion engine recognizes this state and notifies the server. Based on this, the server adjusts the recommendation content to suggest relaxing environments and services. The server uses the results of the shared generative AI model and data from the emotion engine to generate optimal recommendations and create a candidate list. This list is sent to the device and displayed to User A and User B.

[0565] The server then generates the optimal interface for each use case and displays it on the device. For example, if User A is feeling stressed, it displays an interface with a menu that helps them relax.

[0566] This system not only makes it possible to provide optimal information that reflects the preferences and needs of multiple users in a balanced manner, but also provides a flexible interface that responds to the user's emotional state and usage situation.

[0567] The processing flow will be explained below.

[0568] Step 1:

[0569] Users set their own preferences and requests. Specifically, they input information such as their preferred genre, budget, and desired time slot through the interface on their device.

[0570] Step 2:

[0571] The device receives the input information and creates a generative AI model, which learns the user's preferences and requests and structures the data based on that information.

[0572] Step 3:

[0573] The device sends the generated AI model data to the server, which receives the data and stores it in a database.

[0574] Step 4:

[0575] When multiple users use the service together, for example when a group of friends want to choose a restaurant, each user's generated AI model is requested from the server.

[0576] Step 5:

[0577] The server collects the requested data for the generative AI model and performs preprocessing, which includes standardizing the data format and filling in missing values.

[0578] Step 6:

[0579] The server runs a fusion algorithm based on the preprocessed data to create a shared generative AI model that reflects each user's preferences and requirements in a balanced manner.

[0580] Step 7:

[0581] The server receives emotion data generated by the emotion engine from the user's device. Specifically, sensors and cameras installed on the device analyze the user's facial expressions, voice, and text input to determine their emotional state.

[0582] Step 8:

[0583] The server integrates the generated shared generative AI model with the emotion data to generate optimal recommendations, taking into account data from the emotion engine and adjusting the recommendations accordingly.

[0584] Step 9:

[0585] The server sends the generated optimal recommendations to the device, which receives the recommendations and presents them to the user, who can then select from the displayed list.

[0586] Step 10:

[0587] The server generates the most appropriate interface based on the user's request, emotional state, and the situation at hand. For example, a user feeling stressed will be shown a menu that helps them relax.

[0588] Step 11:

[0589] The terminal displays the optimal interface sent from the server, allowing the user to use the system in an easy-to-use environment.

[0590] Examples:

[0591] User A sets his preference as "Japanese food," his budget as "under 3,000 yen," and his meal time as "evening," and creates a generative AI model. User B sets his preference as "Italian food," his budget as "under 4,000 yen," and his meal time as "evening," and creates a generative AI model. If both users want to choose a restaurant in the same group, each user's generative AI model is sent to the server. The server collects this data and performs preprocessing. A fusion algorithm then creates a shared generative AI model that reflects each user's preferences, such as a restaurant that serves Japanese and Italian fusion cuisine.

[0592] Next, if User A is feeling stressed, the emotion engine recognizes this state and sends the data to the server. The server then adjusts the recommendation content taking this emotion data into account, suggesting, for example, a relaxing restaurant that serves Japanese and Italian fusion cuisine. The optimal recommendation is generated, and a candidate list is sent to the device and displayed to the user.

[0593] Finally, the server generates an optimal interface and adjusts the display content according to the user's state based on the emotional data. A relaxing menu interface is displayed on the device, providing an environment that is easy for the user to operate. In this way, the system provides optimal information by taking into account the preferences and emotional states of multiple users in a balanced manner.

[0594] Example 2

[0595] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0596] Conventional recommendation systems have limitations in reflecting user preferences and requests, making it difficult to incorporate the intentions of multiple users in a balanced manner. Furthermore, they lack the ability to provide recommendations that take into account the user's emotional state and optimal operation screens that are tailored to the situation.

[0597] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for registering a generative machine learning model for a user to set his or her preferences and requirements; means for collecting data of the generative machine learning model from multiple users; means for preprocessing the collected data and creating a shared generative machine learning model that reflects the preferences and requirements of each user in a balanced manner using a fusion algorithm; means for generating optimal recommendations using the generated shared generative machine learning model; emotion recognition means for recognizing the emotional state of the user in real time; means for providing the generated optimal recommendation to the user using the result of combining the data obtained from the emotion recognition means; and means for presenting the generated optimal recommendation to the user. This makes it possible to not only precisely reflect the preferences and requirements of users and incorporate the intentions of multiple users in a balanced manner, but also to provide optimal recommendations and operation screens according to the emotional state and situation.

[0598] A "generative machine learning model" is a data model created using machine learning algorithms based on user preferences and requirements.

[0599] "Means of registration" refers to the function for storing and managing user input information and generated machine learning models in the system's database or cloud service.

[0600] The "means of collection" is a function for centrally collecting and managing generative machine learning models and related data provided by multiple users.

[0601] "Preprocessing" refers to processes for improving data quality, such as standardizing the format of collected data and filling in missing values.

[0602] A "fusion algorithm" is a computational method for creating a new model that reflects each user's preferences and requirements in a balanced manner, based on data from a generative machine learning model collected from multiple users.

[0603] A "shared generative machine learning model" is a generative machine learning model that reflects the preferences and requirements of multiple users and combines them in a balanced manner.

[0604] The "means for generating recommendations" is a function for providing appropriate suggestions and options to the user using the generated shared generative machine learning model.

[0605] The "emotion recognition means" is a function that analyzes the user's facial expressions, voice, text input, etc. in real time to determine the user's emotional state.

[0606] The "means for providing" is a function for displaying the generated recommendations to the user and making them actually usable.

[0607] The "presentation means" is a function for visually or audibly presenting the recommendation content to the user.

[0608] Describe in detail the implementation of the invention

[0609] The present invention relates to a system that utilizes a generative machine learning model that reflects user preferences and requirements, creates a shared generative machine learning model that incorporates the requirements of multiple users in a balanced manner, and provides optimal recommendations. The system is also characterized by recognizing the user's emotional state in real time and providing recommendations and operation screens that correspond to that state.

[0610] First, a user inputs their preferences and requests using a device. The device formats this information and creates a generative machine learning model. This generative machine learning model is sent from the device to a server and stored in a database on the server. Next, the server collects the generative machine learning model data sent by multiple users and performs preprocessing. This preprocessing includes standardizing the format and filling in missing values. The collected data is then subjected to a fusion algorithm to create a shared generative machine learning model that reflects each user's preferences and requests in a balanced manner.

[0611] Using this shared generative machine learning model, the server generates optimal recommendations for users. For example, for a user group that likes "Japanese" and "Italian," it might list fusion restaurants as candidates. This optimized recommendation is sent to the user's device and presented visually or audibly.

[0612] Furthermore, the system uses emotion recognition to recognize the user's emotional state in real time. While the user is operating the device, the emotion engine analyzes the user's facial expressions, voice, and text through the camera, microphone, and text input to determine their emotional state. The data obtained from the emotion recognition is sent to the server and reflected in the generated recommendations. For example, if the user is feeling stressed, recommendations will be generated that provide a relaxing environment or service.

[0613] This system also dynamically generates the optimal operation screen based on the user's request and situation. For example, a user feeling stressed will be shown an easy-to-operate relaxation menu. This interface is displayed on the user's device and is designed to be intuitive for the user to operate.

[0614] Specific examples

[0615] User A likes "Japanese food," has a budget of "less than 3,000 yen," and registers his / her meal time as "evening." User B likes "Italian food," has a budget of "less than 4,000 yen," and also registers his / her meal time as "evening." If both users want to choose a restaurant in the same group, the generative machine learning models of each user are sent to the server. The server collects this data and performs preprocessing. After that, a fusion algorithm recommends restaurants that serve Japanese and Italian fusion cuisine.

[0616] Examples of prompts include:

[0617] "I like Japanese food, my budget is under 3,000 yen, and I eat in the evening. My friend likes Italian food, my budget is under 4,000 yen, and we also eat in the evening. What kind of restaurant would you recommend?"

[0618] This not only allows the system to precisely reflect users' preferences and requests and incorporate the intentions of multiple users in a balanced manner, but also makes it possible to provide optimal recommendations and operation screens that are tailored to the user's emotional state and situation.

[0619] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0620] Step 1:

[0621] The user inputs their preferences and requests. Using a terminal, the user inputs detailed information such as preferences, budget, and meal times. Input information may include "Japanese food," "budget 3,000 yen," and "dinner." This information is formatted by the terminal and converted into a unified format such as JSON.

[0622] Step 2:

[0623] A generative machine learning model is created and registered. The device creates a generative machine learning model based on the input data. The generated model is sent to the server, which registers it in a database. As an output, the generative machine learning model is saved on the server.

[0624] Step 3:

[0625] Collecting data for generative machine learning models. The server collects data for generative machine learning models sent by multiple users. The collected data is compiled for preprocessing. Multiple generative machine learning models are included as input.

[0626] Step 4:

[0627] The server preprocesses the collected data. The server performs preprocessing such as standardizing the format of the collected data and completing missing values. For example, it standardizes the data format for each user and estimates and completes missing data. The output is a preprocessed dataset.

[0628] Step 5:

[0629] A fusion algorithm is applied to create a shared generative machine learning model. The server uses the preprocessed data to create a shared generative machine learning model that balances the preferences and needs of multiple users. For example, data from users who like "Japanese food" and "Italian food" is integrated to generate a recommendation model for fusion cuisine. A shared generative machine learning model is generated as the output.

[0630] Step 6:

[0631] Emotion recognition is performed. While the user is using the device, the emotion engine analyzes the user's facial expressions and voice using sensors and cameras to determine their emotional state. The server collects this data and understands the user's emotional state. Real-time emotional data is included as input.

[0632] Step 7:

[0633] Generate optimal recommendations. The server integrates the shared generative machine learning model with emotion recognition data to generate optimal recommendations. For example, if the user is feeling stressed, it will recommend restaurants where they can relax. The output is an optimized recommendation list.

[0634] Step 8:

[0635] Present the recommendations to the user. The server sends the generated recommendation list to the terminal and presents it to the user. For example, a list of fusion restaurants is displayed on the user's terminal. The input contains the optimal recommendation list, and the output is a screen that the user can visually confirm.

[0636] Step 9:

[0637] The optimal interface is generated and displayed. The server generates the optimal operation screen based on the user's request and emotional state. For example, a user who is feeling stressed will be shown a simple and easy-to-understand relaxation menu. The optimal interface is then displayed on the terminal as output.

[0638] (Application example 2)

[0639] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0640] While there are existing recommendation systems based on user preferences and requests, it is difficult to provide flexible recommendations that take into account each individual's emotional state. It is also difficult to integrate the preferences of multiple users and make recommendations that satisfy everyone. Furthermore, there are few systems in physical stores that can provide an optimal interface based on the user's emotions.

[0641] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0642] In this invention, the server includes means for registering a generative artificial intelligence model for a user to set their own preferences and requirements, means for collecting data on the generative artificial intelligence model from multiple users, means for preprocessing the collected data and creating a shared generative artificial intelligence model that reflects each user's preferences and requirements in a balanced manner using a fusion algorithm, means for generating optimal recommendations using the generated shared generative artificial intelligence model, means for presenting the generated optimal recommendations to the user, means for recognizing the user's emotions in real time using an emotion recognition engine, and means for adjusting the recommendation content based on the recognized emotions and automatically generating an optimal interface. This makes it possible to provide more optimal recommendations and interfaces that take both the user's preferences and emotions into consideration.

[0643] A "user" is someone who uses the system to register their preferences and needs and receive recommendations.

[0644] "Preferences" is a concept that refers to the things and conditions that a user prefers.

[0645] "Requirements" refer to the specific requests and conditions that users have for the system.

[0646] A "generative artificial intelligence model" is an artificial intelligence-based data model that is generated based on user preferences and requirements.

[0647] "Means of registration" refers to the functions and methods by which users can input and save their preferences and requests in the system.

[0648] "Means of collection" refers to the functions and methods for collecting data for the generative artificial intelligence model sent by multiple users.

[0649] "Preprocessing" is the process of processing collected data, such as filling in missing values ​​and standardizing the format.

[0650] A "fusion algorithm" is an algorithm that balances the preferences and needs of multiple users to create a single shared generative artificial intelligence model.

[0651] A "shared generative artificial intelligence model" is a common generative artificial intelligence model created by combining the preferences and requests of multiple users.

[0652] "Means for generating" refers to methods and functions for generating optimal recommendations using a shared generative artificial intelligence model.

[0653] "Recommendation" refers to recommended information and services provided by a system based on a user's preferences and requests.

[0654] The "presentation means" refers to a method or function for showing the generated recommended information or services to the user.

[0655] An "emotion recognition engine" is a system or device that analyzes a user's facial expressions, voice, text, etc. to recognize their emotional state in real time.

[0656] "Means of recognition" refers to functions and methods for capturing the user's emotions in real time, analyzing them, and making judgments.

[0657] "Tuning" means methods or functions for modifying and optimizing recommendations based on perceived sentiment.

[0658] "Interface" refers to the display screen and operation method for exchanging information between the user and the system.

[0659] "Automatic generation means" refers to methods or functions that allow the system to operate on its own based on specific conditions and create an appropriate interface.

[0660] The system for implementing this invention includes a server, a terminal, a user, and an emotion engine. The user sets his or her preferences and requests, and generates a generative artificial intelligence model based on them. The generated model is sent from the user's terminal to the server and stored in the server's database.

[0661] System configuration and program processing

[0662] server

[0663] The server collects data for generative AI models sent by multiple users and preprocesses it. This preprocessing involves standardizing the format and filling in missing values. Next, a fusion algorithm is applied to create a shared generative AI model that reflects each user's preferences and requests in a balanced manner. Optimal recommendations are generated based on this created model. Furthermore, emotional data from an emotion recognition engine is incorporated, the recommendation content is adjusted, and an optimal interface is generated according to the situation.

[0664] Specific software used includes scikit-learn for data preprocessing and fusion algorithms and the face_recognition library for emotion recognition.

[0665] Terminal

[0666] The device provides a means for users to input their preferences and requests and register them on the server. It also displays recommended information and optimized interfaces received from the server to the user. The device is equipped with sensors and cameras to transmit the user's facial expressions and voice to an emotion recognition engine in real time.

[0667] As a specific example, smartphones and tablets can be used as devices, as they have built-in cameras and microphones and can be used to collect data for emotion recognition.

[0668] User

[0669] Users input their preferences and requests through their devices, and a generative AI model is generated based on this information and registered on the server. Furthermore, the user's emotional data is also collected in real time and sent to the server. This allows the system to provide optimal recommendations tailored to the user's situation.

[0670] Specific examples

[0671] For example, if User A specifies his / her preference and requirement that he / she prefers Japanese food and is only available in the evening, this information is sent to the server as a generative AI model. Similarly, if User B specifies his / her preference and requirement that he / she prefers Italian food and has a budget of 4,000 yen or less, the same applies. By combining this information, the server can recommend restaurants that offer Japanese-Italian fusion cuisine.

[0672] If User A is feeling stressed, the emotion recognition engine will recognize that emotion in real time and notify the server. Based on this, the server will adjust its recommendations to suggest relaxing environments and services.

[0673] Prompt Sentence Examples

[0674] "Please recommend restaurants based on User A's preference for Japanese food and User B's preference for Italian food. Also, if User A is feeling stressed, please generate appropriate recommendations that will help them relax."

[0675] In this way, the system can simultaneously consider user preferences and emotions to provide optimal recommendations and interfaces for physical stores.

[0676] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0677] Step 1:

[0678] Users input their preferences and requests to create a generative AI model.

[0679] The user uses a device to input their preferences and requests, such as their preferred cuisine genre, budget, and time of day. The input data is converted into a generative AI model on the device. Inputs include preferences and requests such as "Japanese food," "under 3,000 yen," and "evening." The output is a generative AI model that reflects the user's preferences and requests.

[0680] Step 2:

[0681] Send the generative AI model from the device to the server

[0682] The generative AI model entered by the user is sent from the device to the server. The device sends the input data in packets. The generative AI model is used as input, and the model is saved on the server side as output.

[0683] Step 3:

[0684] The server collects and preprocesses the generated AI model data from multiple users.

[0685] The server receives generative AI model data sent by multiple users. The received data undergoes preprocessing, such as format standardization and missing value completion. The input is the generative AI models from multiple users, and the output is the preprocessed data.

[0686] Step 4:

[0687] The pre-processed data is processed through a fusion algorithm to create a shared generative AI model.

[0688] The server inputs the preprocessed data into a fusion algorithm to create a shared generative AI model that reflects each user's preferences and requirements in a balanced manner. The input is the preprocessed data, and the output is the shared generative AI model.

[0689] Step 5:

[0690] The server uses an emotion recognition engine to recognize the user's emotions in real time and captures the data.

[0691] The device sends the user's facial image and voice to the emotion recognition engine in real time. The server captures the recognized emotion data and stores it in a database. The input is the user's facial image and voice, and the output is the recognized emotional state data.

[0692] Step 6:

[0693] The server adjusts the recommendation content based on the emotional data and generates the optimal interface.

[0694] The server analyzes the acquired emotional data and adjusts the recommendation content based on that data. It also automatically generates an optimal interface for the user. The input is emotional state data, and the output is the adjusted recommendation content and the optimal interface.

[0695] Step 7:

[0696] The server sends the recommendations and the optimal interface to the user's device.

[0697] The generated recommendation content and interface are sent from the server to the user's device, which receives them and displays them to the user. The input is the recommendation content and interface data, and the output is the information presented to the user.

[0698] Step 8:

[0699] The user interacts with the recommendations and interface and sends feedback to the server.

[0700] The user operates the recommended content and interface provided through the device, and sends feedback from the device to the server as needed. The input is the user's operations and feedback data, and the output is the feedback information accumulated on the server.

[0701] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0702] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0703] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0704] [Third embodiment]

[0705] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0706] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0707] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0708] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0709] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0710] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0711] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0712] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0713] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0714] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0715] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0716] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0717] System configuration:

[0718] The system of the present invention includes a server, a terminal, and a user. The user sets their preferences and requirements and registers a generative AI model. The server collects generative AI model data from multiple users and combines it to create a shared generative AI model. The terminal presents the generated recommendations and optimized interface to the user.

[0719] System Action:

[0720] First, the user inputs their preferences and requests, and a generative AI model is created based on them. The user accesses the system and uses a device to register this generative AI model. The device then sends the user's input information to the server, which then stores this data in a database.

[0721] When multiple users want to use a service together, for example when a group of friends decides on a restaurant, each user's generative AI model is requested from the server. The server collects these models and preprocesses the data, specifically standardizing the format and imputing missing values. A fusion algorithm is then applied to create a shared generative AI model that balances each user's preferences and needs.

[0722] The server uses this shared generative AI model to generate optimal recommendations for the user. For example, if multiple friends like Japanese and Italian food, the server might list fusion restaurants as candidates. These recommendations are then sent to the device and presented to the user.

[0723] Furthermore, the server generates the optimal interface for the use case based on the user's request and the current situation. For example, in the case of a ticket vending machine, a menu optimized for the user is automatically displayed. This interface is also displayed on the terminal, making it easy for the user to operate.

[0724] Examples:

[0725] User A registers his / her preference as "Japanese food," his / her budget as "under 3,000 yen," and his / her meal time as "evening." User B registers his / her preference as "Italian food," his / her budget as "under 4,000 yen," and his / her meal time as "evening." If both users want to choose a restaurant in the same group, each user's generative AI model is sent to the server. The server collects this data and performs preprocessing. A fusion algorithm then creates a shared generative AI model that matches each user's preference, such as a restaurant that serves Japanese and Italian fusion cuisine.

[0726] The server uses a shared generative AI model to generate optimal recommendations and create a candidate list. This list is sent to the device and displayed to User A and User B. User A and User B can then choose a restaurant that satisfies both of them from this list. In addition, the device displays an interface that is tailored to the situation, making operation intuitive and easy.

[0727] This system not only makes it possible to provide optimal information that takes into account the preferences and needs of multiple users in a balanced manner, but also provides a flexible interface that suits the situation.

[0728] The processing flow will be explained below.

[0729] Step 1:

[0730] Users set their own preferences and requests by entering information such as their preferred genre, budget, and desired time slot through the device interface.

[0731] Step 2:

[0732] The device receives the input information and creates a generative AI model, which learns the user's preferences and requests and converts them into the required data structure.

[0733] Step 3:

[0734] The device sends the generated AI model data to the server, which receives the data and stores it in a database.

[0735] Step 4:

[0736] When multiple users use the service together, each user's generated AI model is requested from the server, such as when a group of friends want to decide on a restaurant together.

[0737] Step 5:

[0738] The server collects the data for the requested generative AI model and performs preprocessing, which includes standardizing the data format and filling in missing values.

[0739] Step 6:

[0740] The server then runs a fusion algorithm based on the pre-processed data, creating a shared generative AI model that balances each user's preferences and needs.

[0741] Step 7:

[0742] The server uses a shared generative AI model to generate optimal recommendations, including options that balance the preferences of multiple users, such as "restaurants that serve Japanese and Italian fusion cuisine."

[0743] Step 8:

[0744] The server sends the generated recommendations to the terminal, which receives the recommendations and presents them to the user. The user can then select from the displayed list.

[0745] Step 9:

[0746] The server generates the optimal interface based on the user's request and the current situation, and sends it to the device, providing a display tailored to a specific use case, such as the interface of a ticket vending machine.

[0747] Step 10:

[0748] The terminal displays the optimal interface sent from the server, allowing the user to use the system in an easy-to-use environment.

[0749] In this way, by performing specific processing at each step, a system is realized that provides optimal recommendations that reflect the preferences and requests of multiple users in a balanced manner.

[0750] Example 1

[0751] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0752] In modern information provision systems, it is extremely difficult to generate recommendations that reflect the preferences and needs of multiple users in a balanced manner and provide an appropriate interface. In particular, when user preferences vary greatly or when an interface that adapts to real-time situations is required, it is difficult for general systems to respond flexibly and appropriately. This leads to issues such as a poor user experience and difficulty in achieving satisfaction.

[0753] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0754] In this invention, the server includes means for registering a generation program that allows a user to set their own preferences and requirements, means for collecting generation program data from multiple users, means for preprocessing the collected data, standardizing formats, and completing missing values, means for creating a shared generation program that uses a fusion algorithm to reflect the preferences and requirements of each user in a balanced manner, means for generating optimal recommendations using the generated shared generation program, means for presenting the generated optimal recommendations to the user, means for generating an optimal interface based on the user's request content and situation and displaying it to the user, and means for error handling. This makes it possible to provide flexible and appropriate recommendations and interfaces in real time, taking into account the preferences and requirements of multiple users in a balanced manner.

[0755] A "generator" is a program that is created based on data entered by a user to reflect their preferences and requirements.

[0756] "Means for collecting" refers to a method or device for collecting generator data from multiple users.

[0757] "Preprocessing" refers to the process of analyzing collected data, standardizing the format, and filling in missing values.

[0758] A "fusion algorithm" is a calculation method for unifying multiple generation programs and creating a shared generation program that reflects each user's preferences and requirements in a balanced manner.

[0759] A "shared generation program" is an integrated generation program created to reflect each user's preferences and requests in a balanced manner.

[0760] "Means for generating optimal recommendations" refers to a method or apparatus for generating the best suggestions or candidates for each user using a shared generator.

[0761] "Means for generating an optimal interface" refers to a method or device for creating a screen display or operating means that allows the user to operate intuitively and easily based on the user's request and situation.

[0762] "Error handling" refers to a method or device for appropriately handling unexpected errors or abnormalities that occur in a system, thereby maintaining normal system operation.

[0763] The system according to the present invention is composed of a user, a server, and a terminal, and each component operates in cooperation with each other. Specific embodiments will be described below.

[0764] overview:

[0765] In this system, users set their own preferences and requirements and register a generator program based on those preferences. The server collects generator data from multiple users, preprocesses it, and then applies a fusion algorithm to create a shared generator program that reflects each user's preferences and requirements in a balanced manner. Finally, the shared generator program is used to generate optimal recommendations and provide them to the user.

[0766] Equipment and software used:

[0767] Server: Database management system (e.g., MySQL, PostgreSQL), generative AI model (e.g., OpenAI GPT-4)

[0768] Devices: Smartphones, PCs

[0769] Software: Data preprocessing programs, fusion algorithms

[0770] System Action:

[0771] First, the user inputs their preferences and requests into the terminal. For example, the user may register information such as a preference for Japanese food, a budget of 3,000 yen or less, and a preference for dinner time. Based on this, a generation program is created and sent to the server via the terminal.

[0772] The server stores the received generation programs in a database. Next, the server collects data from the generation programs collected by multiple users and preprocesses the data. Specifically, it standardizes the data format and performs processes such as filling in missing values.

[0773] After the preprocessing is complete, the server applies a fusion algorithm to create a shared generator that balances each user's preferences and needs, enabling it to recommend recommendations that will satisfy both a group of friends who like Japanese and Italian food, for example.

[0774] Using the created shared generation program, the server generates optimal recommendations. These recommendations are sent to the device and presented to the user. The user can review the recommendations and take action as necessary. The server also generates an optimal interface based on the user's request and the current situation, and displays it on the device.

[0775] Examples:

[0776] Example prompt sentence:

[0777] "I like Japanese food, my budget is under 3,000 yen, and I'd like to eat in the evening. Can you recommend a suitable restaurant?"

[0778] User A registers his / her preference as "Japanese food," his / her budget as "under 3,000 yen," and his / her meal time as "evening." User B registers his / her preference as "Italian food," his / her budget as "under 4,000 yen," and his / her meal time as "evening." If both users want to choose a restaurant in the same group, each user's generator program is sent to the server. The server collects this data and performs preprocessing. The fusion algorithm then creates a shared generator program that matches each user's preference, such as a restaurant that serves Japanese-Italian fusion cuisine.

[0779] The server uses a shared generation program to generate optimal recommendations and create a candidate list. This list is sent to the device and displayed to User A and User B. User A and User B can then choose a restaurant that satisfies both of them from this list. In addition, the device displays an interface that is tailored to the situation, making operation intuitive and easy.

[0780] This system not only makes it possible to provide optimal information that takes into account the preferences and needs of multiple users in a balanced manner, but also provides a flexible interface that suits the situation.

[0781] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0782] Step 1:

[0783] The user inputs their preferences and requests into the terminal. For example, the user may input information such as a preference for Japanese food, a budget of 3,000 yen or less, and a preference for dinner time. The input information becomes data for creating a generation program within the terminal.

[0784] Input: User preference (Japanese food), budget (under 3000 yen), meal time (evening)

[0785] Output: Data format for generator

[0786] Specific behavior:

[0787] The user uses a smartphone or PC interface to input information such as preferences, budget, and time. This input is then appropriately converted within the device and a data format is created for the generation program.

[0788] Step 2:

[0789] The terminal sends the generation program and the user's input data to the server, which stores the received data in a database.

[0790] Input: Data format for generator

[0791] Output: Data stored in the server database

[0792] Specific behavior:

[0793] The terminal generates a data packet containing the generation program and transmits it over the network to the server, which then analyzes the received data and stores it in a database.

[0794] Step 3:

[0795] The server collects multiple user generated programs from the database and preprocesses them, including standardizing data formats and filling in missing values.

[0796] Input: Generator for multiple users in a database

[0797] Output: Preprocessed generator data

[0798] Specific behavior:

[0799] The server retrieves the generation program from the database, unifies the data formats, performs imputation processing if there are missing values, and generates preprocessed data.

[0800] Step 4:

[0801] The server uses the preprocessed data to apply a fusion algorithm to create a shared generator that balances each user's preferences and requirements.

[0802] Input: Preprocessed generator data

[0803] Output: Shared generator

[0804] Specific behavior:

[0805] The server applies a fusion algorithm (e.g., weighted average method, clustering) to the preprocessed data to generate a shared generator that reflects the preferences and requirements of multiple users.

[0806] Step 5:

[0807] The server uses a shared generator to generate optimal recommendations for each user, which are then sent to the terminal and presented to the user.

[0808] Input: Shared Generator

[0809] Output: Best recommendation and recommendation list

[0810] Specific behavior:

[0811] The server creates the optimal recommendations for each user from the generated shared generation program and generates a recommendation list. This list is sent to the terminal and displayed on the user's smartphone or PC.

[0812] Step 6:

[0813] The server generates an optimal interface based on the user's request and the current situation, and the terminal displays this interface to the user.

[0814] Input: User request details and status

[0815] Output: Optimized interface

[0816] Specific behavior:

[0817] The server selects an appropriate interface template based on the user's context (such as location and time of use), renders the selected template, and sends it to the device. The device displays the received template, allowing the user to operate it intuitively.

[0818] In this way, the systems work together to balance user preferences and requirements and provide optimal recommendations and interfaces.

[0819] (Application example 1)

[0820] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0821] While conventional content distribution services can reflect the preferences of individual users, it is difficult to recommend content that reflects the preferences and needs of all users in a balanced manner when multiple users are watching together. In addition, there are limited ways for multiple users to share their viewing experience in real time, which can lead to a decrease in satisfaction with the shared viewing experience.

[0822] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0823] In this invention, the server includes means for registering a generative artificial intelligence model for a user to set his or her preferences and requirements, means for collecting data on the generative artificial intelligence model from multiple users, means for preprocessing the collected data and using a fusion algorithm to create a shared generative artificial intelligence model that reflects the preferences and requirements of each user in a balanced manner, means for recommending optimal content using the generated shared generative artificial intelligence model, means for presenting the recommended optimal content to the user, and means for providing a real-time function for multiple users to share their viewing experience, thereby enabling the recommendation of optimal content that takes into account the preferences and requirements of multiple users in a balanced manner and joint viewing in real time.

[0824] "User" refers to an individual or group of people who access the System and use it to set their own preferences and requirements.

[0825] A "generative artificial intelligence model" refers to an algorithm or dataset that is generated based on user preferences and requests.

[0826] "Means of registration" refers to the method by which a user inputs their preferences and requests and stores a generative artificial intelligence model based on them in the system.

[0827] "Means of collection" refers to the method of aggregating data for the generative AI model provided by multiple users.

[0828] "Preprocessing" refers to the process of standardizing the format of collected data and filling in missing values.

[0829] A "fusion algorithm" refers to a technical method for balancing the preferences and needs of multiple users to create a new shared generative model.

[0830] A "shared generative AI model" is a model created to reflect the preferences and requirements of multiple users, meeting everyone's needs in a balanced manner.

[0831] "Recommendation means" refers to a method of selecting and presenting optimal content using a shared generative artificial intelligence model.

[0832] "Means of presentation" refers to the method of displaying the generated recommendation content to the user.

[0833] "Real-time functionality" refers to functionality that allows multiple users to share a viewing experience and communicate simultaneously.

[0834] System configuration

[0835] The system according to the present invention includes a server, a terminal, and a user. The user sets his or her preferences and requests and registers a generative AI model. The server collects generative AI model data from multiple users and combines it to create a shared generative AI model. The terminal presents the generated recommendations and optimized interface to the user.

[0836] System action

[0837] 1. User preference data input:

[0838] Users use a terminal to input their preferences and requests into the system, such as movie genres, favorite actors, and viewing times.

[0839] 2. Registering a generative AI model:

[0840] Based on the information entered by the user, the terminal generates a generative artificial intelligence model and sends it to the server, which stores the received data in a database (e.g., MySQL).

[0841] 3. Data collection and preprocessing:

[0842] Generative AI models collected from multiple users are preprocessed by the server, which includes standardizing formats and filling in missing values. This process uses machine learning frameworks such as TensorFlow and PyTorch that run on the server.

[0843] 4. Applying the fusion algorithm:

[0844] After the preprocessing is complete, the server applies a fusion algorithm to the data to create a shared generative AI model that reflects the preferences and needs of multiple users in a balanced manner. This shared generative model takes each user's preferences into full consideration.

[0845] 5. Content Recommendations:

[0846] The server uses a shared live AI model to recommend optimal content, for example, movies containing elements of comedy and suspense for multiple users to watch.

[0847] 6. Recommendations:

[0848] The generated list of optimal content is sent to the terminal and presented to the user, who can then select the content they prefer.

[0849] 7. Real-time function:

[0850] The server uses Firebase Realtime Database to provide real-time functionality for multiple users to share the viewing experience, allowing users to watch content together while chatting in real time.

[0851] Specific examples

[0852] User A: "Action movies", "Comedy dramas"

[0853] User B: "Romance movies", "Suspense dramas"

[0854] Based on this information, the server recommends content that reflects the preferences of both parties in a balanced manner.

[0855] Example prompt for a generative AI model:

[0856] User A: Action movies, comedy dramas

[0857] User B: Romance movies, suspense dramas

[0858] Requirements for a shared generative AI model: Recommending optimal video content to satisfy multiple preferences in the evening

[0859] This system enables optimal content recommendations that balance the preferences and needs of multiple users, and enables real-time collaborative viewing.

[0860] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0861] Step 1: Enter user preference data

[0862] Users use a device to input their preferences and requests (e.g., movie genres, favorite actors, viewing times). The device receives these inputs and formats the data for later transmission to the server. The input data includes genres and keywords selected by the user, and is used by the device as basic information for generating a generative artificial intelligence model.

[0863] Input: User preferences and request data

[0864] Output: Formatted user preference data

[0865] Step 2: Registering a Generative AI Model

[0866] The device generates a generative AI model based on the preference data acquired from the user. The generated model is sent to the server via API and stored in a server-side database (e.g., MySQL).

[0867] Input: Formatted user preference data

[0868] Output: Generative AI model

[0869] Step 3: Data collection and preprocessing

[0870] The server collects generative AI models sent by multiple users. The collected data undergoes preprocessing, such as standardizing the format and filling in missing values. During this process, the data is cleaned using frameworks such as TensorFlow.

[0871] Input: Generative AI model

[0872] Output: A preprocessed generative AI model

[0873] Step 4: Applying the fusion algorithm

[0874] The server applies a fusion algorithm to the preprocessed generative AI model, creating a shared generative AI model that balances the preferences and needs of multiple users. This fusion operation is performed using a machine learning framework such as PyTorch.

[0875] Input: A preprocessed generative AI model

[0876] Output: Shared generative AI model

[0877] Step 5: Content Recommendations

[0878] The server uses a shared generative artificial intelligence model to recommend the best content, taking into account the user's preferences and past viewing history to generate a list of the best content. This recommendation is made in real time.

[0879] Input: Shared generative artificial intelligence model

[0880] Output: Recommended content list

[0881] Step 6: Present recommended content

[0882] The server sends a list of recommended content to the device, which receives it and displays it in an interface optimized for the user. The user can then select and watch the content of their interest from the list.

[0883] Input: Recommended content list

[0884] Output: Optimized content list displayed on user device

[0885] Step 7: Providing real-time functionality

[0886] The server uses Firebase Realtime Database to provide functionality for multiple users to share their viewing experiences in real time, specifically allowing users to chat with each other in real time and share their viewing status.

[0887] Input: Viewer information and chat messages

[0888] Output: Real-time synchronized viewing information and messages

[0889] The above processing steps enable the preferences and needs of multiple users to be considered in a balanced manner, and optimal content recommendations and real-time joint viewing can be realized.

[0890] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0891] System configuration

[0892] The system according to the present invention includes a server, a terminal, a user, and an emotion engine. The user sets his or her preferences and requests and registers a generative AI model. The server collects generative AI model data from multiple users and combines it to create a shared generative AI model. Furthermore, the emotion engine recognizes the user's emotions and reflects them in the system. The terminal presents the generated recommendations and optimized interface to the user.

[0893] System action

[0894] First, the user inputs their preferences and requests, and a generative AI model is created based on them. The user accesses the system and uses a device to register this generative AI model. The device then sends the user's input information to the server, which then stores this data in a database.

[0895] When multiple users want to use a service together, for example when deciding on a restaurant for a group of friends, each user's generative AI model is requested from the server. The server collects these models and preprocesses the data, specifically standardizing the format and imputing missing values. A fusion algorithm is then applied to create a shared generative AI model that balances each user's preferences and needs.

[0896] The server uses the results of this shared generative AI model and the emotion engine to generate optimal recommendations for the user. For example, if multiple friends like Japanese and Italian food, the server might list fusion restaurants as candidates. These recommendations are then sent to the device and presented to the user.

[0897] Furthermore, the server generates the optimal interface for the use case based on the user's request and the current situation. For example, in the case of a ticket vending machine, a menu optimized for the user is automatically displayed. This interface is also displayed on the terminal, making it easy for the user to operate.

[0898] Emotion engine processing

[0899] The emotion engine uses sensors and cameras to recognize the user's emotions in real time. While the user is using the device, it analyzes their facial expressions, voice, and text input to determine their emotional state. The server takes in the data from this emotion engine and uses it to generate generative AI models and provide recommendations. This makes it possible to make suggestions tailored to the situation, such as recommending places where a user who is feeling stressed can relax.

[0900] Specific examples

[0901] User A registers his / her preference as "Japanese food," his / her budget as "under 3,000 yen," and his / her meal time as "evening." User B registers his / her preference as "Italian food," his / her budget as "under 4,000 yen," and his / her meal time as "evening." If both users want to choose a restaurant in the same group, each user's generative AI model is sent to the server. The server collects this data and performs preprocessing. A fusion algorithm then creates a shared generative AI model that matches each user's preference, such as a restaurant that serves Japanese and Italian fusion cuisine.

[0902] Furthermore, if User A is feeling stressed, the emotion engine recognizes this state and notifies the server. Based on this, the server adjusts the recommendation content to suggest relaxing environments and services. The server uses the results of the shared generative AI model and data from the emotion engine to generate optimal recommendations and create a candidate list. This list is sent to the device and displayed to User A and User B.

[0903] The server then generates the optimal interface for each use case and displays it on the device. For example, if User A is feeling stressed, it displays an interface with a menu that helps them relax.

[0904] This system not only makes it possible to provide optimal information that reflects the preferences and needs of multiple users in a balanced manner, but also provides a flexible interface that responds to the user's emotional state and usage situation.

[0905] The processing flow will be explained below.

[0906] Step 1:

[0907] Users set their own preferences and requests. Specifically, they input information such as their preferred genre, budget, and desired time slot through the interface on their device.

[0908] Step 2:

[0909] The device receives the input information and creates a generative AI model, which learns the user's preferences and requests and structures the data based on that information.

[0910] Step 3:

[0911] The device sends the generated AI model data to the server, which receives the data and stores it in a database.

[0912] Step 4:

[0913] When multiple users use the service together, for example when a group of friends want to choose a restaurant, each user's generated AI model is requested from the server.

[0914] Step 5:

[0915] The server collects the requested data for the generative AI model and performs preprocessing, which includes standardizing the data format and filling in missing values.

[0916] Step 6:

[0917] The server runs a fusion algorithm based on the preprocessed data to create a shared generative AI model that reflects each user's preferences and requirements in a balanced manner.

[0918] Step 7:

[0919] The server receives emotion data generated by the emotion engine from the user's device. Specifically, sensors and cameras installed on the device analyze the user's facial expressions, voice, and text input to determine their emotional state.

[0920] Step 8:

[0921] The server integrates the generated shared generative AI model with the emotion data to generate optimal recommendations, taking into account data from the emotion engine and adjusting the recommendations accordingly.

[0922] Step 9:

[0923] The server sends the generated optimal recommendations to the device, which receives the recommendations and presents them to the user, who can then select from the displayed list.

[0924] Step 10:

[0925] The server generates the most appropriate interface based on the user's request, emotional state, and the situation at hand. For example, a user feeling stressed will be shown a menu that helps them relax.

[0926] Step 11:

[0927] The terminal displays the optimal interface sent from the server, allowing the user to use the system in an easy-to-use environment.

[0928] Examples:

[0929] User A sets his preference as "Japanese food," his budget as "under 3,000 yen," and his meal time as "evening," and creates a generative AI model. User B sets his preference as "Italian food," his budget as "under 4,000 yen," and his meal time as "evening," and creates a generative AI model. If both users want to choose a restaurant in the same group, each user's generative AI model is sent to the server. The server collects this data and performs preprocessing. A fusion algorithm then creates a shared generative AI model that reflects each user's preferences, such as a restaurant that serves Japanese and Italian fusion cuisine.

[0930] Next, if User A is feeling stressed, the emotion engine recognizes this state and sends the data to the server. The server then adjusts the recommendation content taking this emotion data into account, suggesting, for example, a relaxing restaurant that serves Japanese and Italian fusion cuisine. The optimal recommendation is generated, and a candidate list is sent to the device and displayed to the user.

[0931] Finally, the server generates an optimal interface and adjusts the display content according to the user's state based on the emotional data. A relaxing menu interface is displayed on the device, providing an environment that is easy for the user to operate. In this way, the system provides optimal information by taking into account the preferences and emotional states of multiple users in a balanced manner.

[0932] Example 2

[0933] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0934] Conventional recommendation systems have limitations in reflecting user preferences and requests, making it difficult to incorporate the intentions of multiple users in a balanced manner. Furthermore, they lack the ability to provide recommendations that take into account the user's emotional state and optimal operation screens that are tailored to the situation.

[0935] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for registering a generative machine learning model for a user to set his or her preferences and requirements; means for collecting data of the generative machine learning model from multiple users; means for preprocessing the collected data and creating a shared generative machine learning model that reflects the preferences and requirements of each user in a balanced manner using a fusion algorithm; means for generating optimal recommendations using the generated shared generative machine learning model; emotion recognition means for recognizing the emotional state of the user in real time; means for providing the generated optimal recommendation to the user using the result of combining the data obtained from the emotion recognition means; and means for presenting the generated optimal recommendation to the user. This makes it possible to not only precisely reflect the preferences and requirements of users and incorporate the intentions of multiple users in a balanced manner, but also to provide optimal recommendations and operation screens according to the emotional state and situation.

[0936] A "generative machine learning model" is a data model created using machine learning algorithms based on user preferences and requirements.

[0937] "Means of registration" refers to the function for storing and managing user input information and generated machine learning models in the system's database or cloud service.

[0938] The "means of collection" is a function for centrally collecting and managing generative machine learning models and related data provided by multiple users.

[0939] "Preprocessing" refers to processes for improving data quality, such as standardizing the format of collected data and filling in missing values.

[0940] A "fusion algorithm" is a computational method for creating a new model that reflects each user's preferences and requirements in a balanced manner, based on data from a generative machine learning model collected from multiple users.

[0941] A "shared generative machine learning model" is a generative machine learning model that reflects the preferences and requirements of multiple users and combines them in a balanced manner.

[0942] The "means for generating recommendations" is a function for providing appropriate suggestions and options to the user using the generated shared generative machine learning model.

[0943] The "emotion recognition means" is a function that analyzes the user's facial expressions, voice, text input, etc. in real time to determine the user's emotional state.

[0944] The "means for providing" is a function for displaying the generated recommendations to the user and making them actually usable.

[0945] The "presentation means" is a function for visually or audibly presenting the recommendation content to the user.

[0946] Describe in detail the implementation of the invention

[0947] The present invention relates to a system that utilizes a generative machine learning model that reflects user preferences and requirements, creates a shared generative machine learning model that incorporates the requirements of multiple users in a balanced manner, and provides optimal recommendations. The system is also characterized by recognizing the user's emotional state in real time and providing recommendations and operation screens that correspond to that state.

[0948] First, a user inputs their preferences and requests using a device. The device formats this information and creates a generative machine learning model. This generative machine learning model is sent from the device to a server and stored in a database on the server. Next, the server collects the generative machine learning model data sent by multiple users and performs preprocessing. This preprocessing includes standardizing the format and filling in missing values. The collected data is then subjected to a fusion algorithm to create a shared generative machine learning model that reflects each user's preferences and requests in a balanced manner.

[0949] Using this shared generative machine learning model, the server generates optimal recommendations for users. For example, for a user group that likes "Japanese" and "Italian," it might list fusion restaurants as candidates. This optimized recommendation is sent to the user's device and presented visually or audibly.

[0950] Furthermore, the system uses emotion recognition to recognize the user's emotional state in real time. While the user is operating the device, the emotion engine analyzes the user's facial expressions, voice, and text through the camera, microphone, and text input to determine their emotional state. The data obtained from the emotion recognition is sent to the server and reflected in the generated recommendations. For example, if the user is feeling stressed, recommendations will be generated that provide a relaxing environment or service.

[0951] This system also dynamically generates the optimal operation screen based on the user's request and situation. For example, a user feeling stressed will be shown an easy-to-operate relaxation menu. This interface is displayed on the user's device and is designed to be intuitive for the user to operate.

[0952] Specific examples

[0953] User A likes "Japanese food," has a budget of "less than 3,000 yen," and registers his / her meal time as "evening." User B likes "Italian food," has a budget of "less than 4,000 yen," and also registers his / her meal time as "evening." If both users want to choose a restaurant in the same group, the generative machine learning models of each user are sent to the server. The server collects this data and performs preprocessing. After that, a fusion algorithm recommends restaurants that serve Japanese and Italian fusion cuisine.

[0954] Examples of prompts include:

[0955] "I like Japanese food, my budget is under 3,000 yen, and I eat in the evening. My friend likes Italian food, my budget is under 4,000 yen, and we also eat in the evening. What kind of restaurant would you recommend?"

[0956] This not only allows the system to precisely reflect users' preferences and requests and incorporate the intentions of multiple users in a balanced manner, but also makes it possible to provide optimal recommendations and operation screens that are tailored to the user's emotional state and situation.

[0957] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0958] Step 1:

[0959] The user inputs their preferences and requests. Using a terminal, the user inputs detailed information such as preferences, budget, and meal times. Input information may include "Japanese food," "budget 3,000 yen," and "dinner." This information is formatted by the terminal and converted into a unified format such as JSON.

[0960] Step 2:

[0961] A generative machine learning model is created and registered. The device creates a generative machine learning model based on the input data. The generated model is sent to the server, which registers it in a database. As an output, the generative machine learning model is saved on the server.

[0962] Step 3:

[0963] Collecting data for generative machine learning models. The server collects data for generative machine learning models sent by multiple users. The collected data is compiled for preprocessing. Multiple generative machine learning models are included as input.

[0964] Step 4:

[0965] The server preprocesses the collected data. The server performs preprocessing such as standardizing the format of the collected data and completing missing values. For example, it standardizes the data format for each user and estimates and completes missing data. The output is a preprocessed dataset.

[0966] Step 5:

[0967] A fusion algorithm is applied to create a shared generative machine learning model. The server uses the preprocessed data to create a shared generative machine learning model that balances the preferences and needs of multiple users. For example, data from users who like "Japanese food" and "Italian food" is integrated to generate a recommendation model for fusion cuisine. A shared generative machine learning model is generated as the output.

[0968] Step 6:

[0969] Emotion recognition is performed. While the user is using the device, the emotion engine analyzes the user's facial expressions and voice using sensors and cameras to determine their emotional state. The server collects this data and understands the user's emotional state. Real-time emotional data is included as input.

[0970] Step 7:

[0971] Generate optimal recommendations. The server integrates the shared generative machine learning model with emotion recognition data to generate optimal recommendations. For example, if the user is feeling stressed, it will recommend restaurants where they can relax. The output is an optimized recommendation list.

[0972] Step 8:

[0973] Present the recommendations to the user. The server sends the generated recommendation list to the terminal and presents it to the user. For example, a list of fusion restaurants is displayed on the user's terminal. The input contains the optimal recommendation list, and the output is a screen that the user can visually confirm.

[0974] Step 9:

[0975] The optimal interface is generated and displayed. The server generates the optimal operation screen based on the user's request and emotional state. For example, a user who is feeling stressed will be shown a simple and easy-to-understand relaxation menu. The optimal interface is then displayed on the terminal as output.

[0976] (Application example 2)

[0977] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0978] While there are existing recommendation systems based on user preferences and requests, it is difficult to provide flexible recommendations that take into account each individual's emotional state. It is also difficult to integrate the preferences of multiple users and make recommendations that satisfy everyone. Furthermore, there are few systems in physical stores that can provide an optimal interface based on the user's emotions.

[0979] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0980] In this invention, the server includes means for registering a generative artificial intelligence model for a user to set their own preferences and requirements, means for collecting data on the generative artificial intelligence model from multiple users, means for preprocessing the collected data and creating a shared generative artificial intelligence model that reflects each user's preferences and requirements in a balanced manner using a fusion algorithm, means for generating optimal recommendations using the generated shared generative artificial intelligence model, means for presenting the generated optimal recommendations to the user, means for recognizing the user's emotions in real time using an emotion recognition engine, and means for adjusting the recommendation content based on the recognized emotions and automatically generating an optimal interface. This makes it possible to provide more optimal recommendations and interfaces that take both the user's preferences and emotions into consideration.

[0981] A "user" is someone who uses the system to register their preferences and needs and receive recommendations.

[0982] "Preferences" is a concept that refers to the things and conditions that a user prefers.

[0983] "Requirements" refer to the specific requests and conditions that users have for the system.

[0984] A "generative artificial intelligence model" is an artificial intelligence-based data model that is generated based on user preferences and requirements.

[0985] "Means of registration" refers to the functions and methods by which users can input and save their preferences and requests in the system.

[0986] "Means of collection" refers to the functions and methods for collecting data for the generative artificial intelligence model sent by multiple users.

[0987] "Preprocessing" is the process of processing collected data, such as filling in missing values ​​and standardizing the format.

[0988] A "fusion algorithm" is an algorithm that balances the preferences and needs of multiple users to create a single shared generative artificial intelligence model.

[0989] A "shared generative artificial intelligence model" is a common generative artificial intelligence model created by combining the preferences and requests of multiple users.

[0990] "Means for generating" refers to methods and functions for generating optimal recommendations using a shared generative artificial intelligence model.

[0991] "Recommendation" refers to recommended information and services provided by a system based on a user's preferences and requests.

[0992] The "presentation means" refers to a method or function for showing the generated recommended information or services to the user.

[0993] An "emotion recognition engine" is a system or device that analyzes a user's facial expressions, voice, text, etc. to recognize their emotional state in real time.

[0994] "Means of recognition" refers to functions and methods for capturing the user's emotions in real time, analyzing them, and making judgments.

[0995] "Tuning" means methods or functions for modifying and optimizing recommendations based on perceived sentiment.

[0996] "Interface" refers to the display screen and operation method for exchanging information between the user and the system.

[0997] "Automatic generation means" refers to methods or functions that allow the system to operate on its own based on specific conditions and create an appropriate interface.

[0998] The system for implementing this invention includes a server, a terminal, a user, and an emotion engine. The user sets his or her preferences and requests, and generates a generative artificial intelligence model based on them. The generated model is sent from the user's terminal to the server and stored in the server's database.

[0999] System configuration and program processing

[1000] server

[1001] The server collects data for generative AI models sent by multiple users and preprocesses it. This preprocessing involves standardizing the format and filling in missing values. Next, a fusion algorithm is applied to create a shared generative AI model that reflects each user's preferences and requests in a balanced manner. Optimal recommendations are generated based on this created model. Furthermore, emotional data from an emotion recognition engine is incorporated, the recommendation content is adjusted, and an optimal interface is generated according to the situation.

[1002] Specific software used includes scikit-learn for data preprocessing and fusion algorithms and the face_recognition library for emotion recognition.

[1003] Terminal

[1004] The device provides a means for users to input their preferences and requests and register them on the server. It also displays recommended information and optimized interfaces received from the server to the user. The device is equipped with sensors and cameras to transmit the user's facial expressions and voice to an emotion recognition engine in real time.

[1005] As a specific example, smartphones and tablets can be used as devices, as they have built-in cameras and microphones and can be used to collect data for emotion recognition.

[1006] User

[1007] Users input their preferences and requests through their devices, and a generative AI model is generated based on this information and registered on the server. Furthermore, the user's emotional data is also collected in real time and sent to the server. This allows the system to provide optimal recommendations tailored to the user's situation.

[1008] Specific examples

[1009] For example, if User A specifies his / her preference and requirement that he / she prefers Japanese food and is only available in the evening, this information is sent to the server as a generative AI model. Similarly, if User B specifies his / her preference and requirement that he / she prefers Italian food and has a budget of 4,000 yen or less, the same applies. By combining this information, the server can recommend restaurants that offer Japanese-Italian fusion cuisine.

[1010] If User A is feeling stressed, the emotion recognition engine will recognize that emotion in real time and notify the server. Based on this, the server will adjust its recommendations to suggest relaxing environments and services.

[1011] Prompt Sentence Examples

[1012] "Please recommend restaurants based on User A's preference for Japanese food and User B's preference for Italian food. Also, if User A is feeling stressed, please generate appropriate recommendations that will help them relax."

[1013] In this way, the system can simultaneously consider user preferences and emotions to provide optimal recommendations and interfaces for physical stores.

[1014] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1015] Step 1:

[1016] Users input their preferences and requests to create a generative AI model.

[1017] The user uses a device to input their preferences and requests, such as their preferred cuisine genre, budget, and time of day. The input data is converted into a generative AI model on the device. Inputs include preferences and requests such as "Japanese food," "under 3,000 yen," and "evening." The output is a generative AI model that reflects the user's preferences and requests.

[1018] Step 2:

[1019] Send the generative AI model from the device to the server

[1020] The generative AI model entered by the user is sent from the device to the server. The device sends the input data in packets. The generative AI model is used as input, and the model is saved on the server side as output.

[1021] Step 3:

[1022] The server collects and preprocesses the generated AI model data from multiple users.

[1023] The server receives generative AI model data sent by multiple users. The received data undergoes preprocessing, such as format standardization and missing value completion. The input is the generative AI models from multiple users, and the output is the preprocessed data.

[1024] Step 4:

[1025] The pre-processed data is processed through a fusion algorithm to create a shared generative AI model.

[1026] The server inputs the preprocessed data into a fusion algorithm to create a shared generative AI model that reflects each user's preferences and requirements in a balanced manner. The input is the preprocessed data, and the output is the shared generative AI model.

[1027] Step 5:

[1028] The server uses an emotion recognition engine to recognize the user's emotions in real time and captures the data.

[1029] The device sends the user's facial image and voice to the emotion recognition engine in real time. The server captures the recognized emotion data and stores it in a database. The input is the user's facial image and voice, and the output is the recognized emotional state data.

[1030] Step 6:

[1031] The server adjusts the recommendation content based on the emotional data and generates the optimal interface.

[1032] The server analyzes the acquired emotional data and adjusts the recommendation content based on that data. It also automatically generates an optimal interface for the user. The input is emotional state data, and the output is the adjusted recommendation content and the optimal interface.

[1033] Step 7:

[1034] The server sends the recommendations and the optimal interface to the user's device.

[1035] The generated recommendation content and interface are sent from the server to the user's device, which receives them and displays them to the user. The input is the recommendation content and interface data, and the output is the information presented to the user.

[1036] Step 8:

[1037] The user interacts with the recommendations and interface and sends feedback to the server.

[1038] The user operates the recommended content and interface provided through the device, and sends feedback from the device to the server as needed. The input is the user's operations and feedback data, and the output is the feedback information accumulated on the server.

[1039] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1040] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1041] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1042] [Fourth embodiment]

[1043] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1044] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1045] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1046] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1047] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1048] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1049] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1050] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1051] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1052] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1053] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1054] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1055] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1056] System configuration:

[1057] The system of the present invention includes a server, a terminal, and a user. The user sets their preferences and requirements and registers a generative AI model. The server collects generative AI model data from multiple users and combines it to create a shared generative AI model. The terminal presents the generated recommendations and optimized interface to the user.

[1058] System Action:

[1059] First, the user inputs their preferences and requests, and a generative AI model is created based on them. The user accesses the system and uses a device to register this generative AI model. The device then sends the user's input information to the server, which then stores this data in a database.

[1060] When multiple users want to use a service together, for example when a group of friends decides on a restaurant, each user's generative AI model is requested from the server. The server collects these models and preprocesses the data, specifically standardizing the format and imputing missing values. A fusion algorithm is then applied to create a shared generative AI model that balances each user's preferences and needs.

[1061] The server uses this shared generative AI model to generate optimal recommendations for the user. For example, if multiple friends like Japanese and Italian food, the server might list fusion restaurants as candidates. These recommendations are then sent to the device and presented to the user.

[1062] Furthermore, the server generates the optimal interface for the use case based on the user's request and the current situation. For example, in the case of a ticket vending machine, a menu optimized for the user is automatically displayed. This interface is also displayed on the terminal, making it easy for the user to operate.

[1063] Examples:

[1064] User A registers his / her preference as "Japanese food," his / her budget as "under 3,000 yen," and his / her meal time as "evening." User B registers his / her preference as "Italian food," his / her budget as "under 4,000 yen," and his / her meal time as "evening." If both users want to choose a restaurant in the same group, each user's generative AI model is sent to the server. The server collects this data and performs preprocessing. A fusion algorithm then creates a shared generative AI model that matches each user's preference, such as a restaurant that serves Japanese and Italian fusion cuisine.

[1065] The server uses a shared generative AI model to generate optimal recommendations and create a candidate list. This list is sent to the device and displayed to User A and User B. User A and User B can then choose a restaurant that satisfies both of them from this list. In addition, the device displays an interface that is tailored to the situation, making operation intuitive and easy.

[1066] This system not only makes it possible to provide optimal information that takes into account the preferences and needs of multiple users in a balanced manner, but also provides a flexible interface that suits the situation.

[1067] The processing flow will be explained below.

[1068] Step 1:

[1069] Users set their own preferences and requests by entering information such as their preferred genre, budget, and desired time slot through the device interface.

[1070] Step 2:

[1071] The device receives the input information and creates a generative AI model, which learns the user's preferences and requests and converts them into the required data structure.

[1072] Step 3:

[1073] The device sends the generated AI model data to the server, which receives the data and stores it in a database.

[1074] Step 4:

[1075] When multiple users use the service together, each user's generated AI model is requested from the server, such as when a group of friends want to decide on a restaurant together.

[1076] Step 5:

[1077] The server collects the data for the requested generative AI model and performs preprocessing, which includes standardizing the data format and filling in missing values.

[1078] Step 6:

[1079] The server then runs a fusion algorithm based on the pre-processed data, creating a shared generative AI model that balances each user's preferences and needs.

[1080] Step 7:

[1081] The server uses a shared generative AI model to generate optimal recommendations, including options that balance the preferences of multiple users, such as "restaurants that serve Japanese and Italian fusion cuisine."

[1082] Step 8:

[1083] The server sends the generated recommendations to the terminal, which receives the recommendations and presents them to the user. The user can then select from the displayed list.

[1084] Step 9:

[1085] The server generates the optimal interface based on the user's request and the current situation, and sends it to the device, providing a display tailored to a specific use case, such as the interface of a ticket vending machine.

[1086] Step 10:

[1087] The terminal displays the optimal interface sent from the server, allowing the user to use the system in an easy-to-use environment.

[1088] In this way, by performing specific processing at each step, a system is realized that provides optimal recommendations that reflect the preferences and requests of multiple users in a balanced manner.

[1089] Example 1

[1090] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1091] In modern information provision systems, it is extremely difficult to generate recommendations that reflect the preferences and needs of multiple users in a balanced manner and provide an appropriate interface. In particular, when user preferences vary greatly or when an interface that adapts to real-time situations is required, it is difficult for general systems to respond flexibly and appropriately. This leads to issues such as a poor user experience and difficulty in achieving satisfaction.

[1092] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1093] In this invention, the server includes means for registering a generation program that allows a user to set their own preferences and requirements, means for collecting generation program data from multiple users, means for preprocessing the collected data, standardizing formats, and completing missing values, means for creating a shared generation program that uses a fusion algorithm to reflect the preferences and requirements of each user in a balanced manner, means for generating optimal recommendations using the generated shared generation program, means for presenting the generated optimal recommendations to the user, means for generating an optimal interface based on the user's request content and situation and displaying it to the user, and means for error handling. This makes it possible to provide flexible and appropriate recommendations and interfaces in real time, taking into account the preferences and requirements of multiple users in a balanced manner.

[1094] A "generator" is a program that is created based on data entered by a user to reflect their preferences and requirements.

[1095] "Means for collecting" refers to a method or device for collecting generator data from multiple users.

[1096] "Preprocessing" refers to the process of analyzing collected data, standardizing the format, and filling in missing values.

[1097] A "fusion algorithm" is a calculation method for unifying multiple generation programs and creating a shared generation program that reflects each user's preferences and requirements in a balanced manner.

[1098] A "shared generation program" is an integrated generation program created to reflect each user's preferences and requests in a balanced manner.

[1099] "Means for generating optimal recommendations" refers to a method or apparatus for generating the best suggestions or candidates for each user using a shared generator.

[1100] "Means for generating an optimal interface" refers to a method or device for creating a screen display or operating means that allows the user to operate intuitively and easily based on the user's request and situation.

[1101] "Error handling" refers to a method or device for appropriately handling unexpected errors or abnormalities that occur in a system, thereby maintaining normal system operation.

[1102] The system according to the present invention is composed of a user, a server, and a terminal, and each component operates in cooperation with each other. Specific embodiments will be described below.

[1103] overview:

[1104] In this system, users set their own preferences and requirements and register a generator program based on those preferences. The server collects generator data from multiple users, preprocesses it, and then applies a fusion algorithm to create a shared generator program that reflects each user's preferences and requirements in a balanced manner. Finally, the shared generator program is used to generate optimal recommendations and provide them to the user.

[1105] Equipment and software used:

[1106] Server: Database management system (e.g., MySQL, PostgreSQL), generative AI model (e.g., OpenAI GPT-4)

[1107] Devices: Smartphones, PCs

[1108] Software: Data preprocessing programs, fusion algorithms

[1109] System Action:

[1110] First, the user inputs their preferences and requests into the terminal. For example, the user may register information such as a preference for Japanese food, a budget of 3,000 yen or less, and a preference for dinner time. Based on this, a generation program is created and sent to the server via the terminal.

[1111] The server stores the received generation programs in a database. Next, the server collects data from the generation programs collected by multiple users and preprocesses the data. Specifically, it standardizes the data format and performs processes such as filling in missing values.

[1112] After the preprocessing is complete, the server applies a fusion algorithm to create a shared generator that balances each user's preferences and needs, enabling it to recommend recommendations that will satisfy both a group of friends who like Japanese and Italian food, for example.

[1113] Using the created shared generation program, the server generates optimal recommendations. These recommendations are sent to the device and presented to the user. The user can review the recommendations and take action as necessary. The server also generates an optimal interface based on the user's request and the current situation, and displays it on the device.

[1114] Examples:

[1115] Example prompt sentence:

[1116] "I like Japanese food, my budget is under 3,000 yen, and I'd like to eat in the evening. Can you recommend a suitable restaurant?"

[1117] User A registers his / her preference as "Japanese food," his / her budget as "under 3,000 yen," and his / her meal time as "evening." User B registers his / her preference as "Italian food," his / her budget as "under 4,000 yen," and his / her meal time as "evening." If both users want to choose a restaurant in the same group, each user's generator program is sent to the server. The server collects this data and performs preprocessing. The fusion algorithm then creates a shared generator program that matches each user's preference, such as a restaurant that serves Japanese-Italian fusion cuisine.

[1118] The server uses a shared generation program to generate optimal recommendations and create a candidate list. This list is sent to the device and displayed to User A and User B. User A and User B can then choose a restaurant that satisfies both of them from this list. In addition, the device displays an interface that is tailored to the situation, making operation intuitive and easy.

[1119] This system not only makes it possible to provide optimal information that takes into account the preferences and needs of multiple users in a balanced manner, but also provides a flexible interface that suits the situation.

[1120] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1121] Step 1:

[1122] The user inputs their preferences and requests into the terminal. For example, the user may input information such as a preference for Japanese food, a budget of 3,000 yen or less, and a preference for dinner time. The input information becomes data for creating a generation program within the terminal.

[1123] Input: User preference (Japanese food), budget (under 3000 yen), meal time (evening)

[1124] Output: Data format for generator

[1125] Specific behavior:

[1126] The user uses a smartphone or PC interface to input information such as preferences, budget, and time. This input is then appropriately converted within the device and a data format is created for the generation program.

[1127] Step 2:

[1128] The terminal sends the generation program and the user's input data to the server, which stores the received data in a database.

[1129] Input: Data format for generator

[1130] Output: Data stored in the server database

[1131] Specific behavior:

[1132] The terminal generates a data packet containing the generation program and transmits it over the network to the server, which then analyzes the received data and stores it in a database.

[1133] Step 3:

[1134] The server collects multiple user generated programs from the database and preprocesses them, including standardizing data formats and filling in missing values.

[1135] Input: Generator for multiple users in a database

[1136] Output: Preprocessed generator data

[1137] Specific behavior:

[1138] The server retrieves the generation program from the database, unifies the data formats, performs imputation processing if there are missing values, and generates preprocessed data.

[1139] Step 4:

[1140] The server uses the preprocessed data to apply a fusion algorithm to create a shared generator that balances each user's preferences and requirements.

[1141] Input: Preprocessed generator data

[1142] Output: Shared generator

[1143] Specific behavior:

[1144] The server applies a fusion algorithm (e.g., weighted average method, clustering) to the preprocessed data to generate a shared generator that reflects the preferences and requirements of multiple users.

[1145] Step 5:

[1146] The server uses a shared generator to generate optimal recommendations for each user, which are then sent to the terminal and presented to the user.

[1147] Input: Shared Generator

[1148] Output: Best recommendation and recommendation list

[1149] Specific behavior:

[1150] The server creates the optimal recommendations for each user from the generated shared generation program and generates a recommendation list. This list is sent to the terminal and displayed on the user's smartphone or PC.

[1151] Step 6:

[1152] The server generates an optimal interface based on the user's request and the current situation, and the terminal displays this interface to the user.

[1153] Input: User request details and status

[1154] Output: Optimized interface

[1155] Specific behavior:

[1156] The server selects an appropriate interface template based on the user's context (such as location and time of use), renders the selected template, and sends it to the device. The device displays the received template, allowing the user to operate it intuitively.

[1157] In this way, the systems work together to balance user preferences and requirements and provide optimal recommendations and interfaces.

[1158] (Application example 1)

[1159] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1160] While conventional content distribution services can reflect the preferences of individual users, it is difficult to recommend content that reflects the preferences and needs of all users in a balanced manner when multiple users are watching together. In addition, there are limited ways for multiple users to share their viewing experience in real time, which can lead to a decrease in satisfaction with the shared viewing experience.

[1161] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1162] In this invention, the server includes means for registering a generative artificial intelligence model for a user to set his or her preferences and requirements, means for collecting data on the generative artificial intelligence model from multiple users, means for preprocessing the collected data and using a fusion algorithm to create a shared generative artificial intelligence model that reflects the preferences and requirements of each user in a balanced manner, means for recommending optimal content using the generated shared generative artificial intelligence model, means for presenting the recommended optimal content to the user, and means for providing a real-time function for multiple users to share their viewing experience, thereby enabling the recommendation of optimal content that takes into account the preferences and requirements of multiple users in a balanced manner and joint viewing in real time.

[1163] "User" refers to an individual or group of people who access the System and use it to set their own preferences and requirements.

[1164] A "generative artificial intelligence model" refers to an algorithm or dataset that is generated based on user preferences and requests.

[1165] "Means of registration" refers to the method by which a user inputs their preferences and requests and stores a generative artificial intelligence model based on them in the system.

[1166] "Means of collection" refers to the method of aggregating data for the generative AI model provided by multiple users.

[1167] "Preprocessing" refers to the process of standardizing the format of collected data and filling in missing values.

[1168] A "fusion algorithm" refers to a technical method for balancing the preferences and needs of multiple users to create a new shared generative model.

[1169] A "shared generative AI model" is a model created to reflect the preferences and requirements of multiple users, meeting everyone's needs in a balanced manner.

[1170] "Recommendation means" refers to a method of selecting and presenting optimal content using a shared generative artificial intelligence model.

[1171] "Means of presentation" refers to the method of displaying the generated recommendation content to the user.

[1172] "Real-time functionality" refers to functionality that allows multiple users to share a viewing experience and communicate simultaneously.

[1173] System configuration

[1174] The system according to the present invention includes a server, a terminal, and a user. The user sets his or her preferences and requests and registers a generative AI model. The server collects generative AI model data from multiple users and combines it to create a shared generative AI model. The terminal presents the generated recommendations and optimized interface to the user.

[1175] System action

[1176] 1. User preference data input:

[1177] Users use a terminal to input their preferences and requests into the system, such as movie genres, favorite actors, and viewing times.

[1178] 2. Registering a generative AI model:

[1179] Based on the information entered by the user, the terminal generates a generative artificial intelligence model and sends it to the server, which stores the received data in a database (e.g., MySQL).

[1180] 3. Data collection and preprocessing:

[1181] Generative AI models collected from multiple users are preprocessed by the server, which includes standardizing formats and filling in missing values. This process uses machine learning frameworks such as TensorFlow and PyTorch that run on the server.

[1182] 4. Applying the fusion algorithm:

[1183] After the preprocessing is complete, the server applies a fusion algorithm to the data to create a shared generative AI model that reflects the preferences and needs of multiple users in a balanced manner. This shared generative model takes each user's preferences into full consideration.

[1184] 5. Content Recommendations:

[1185] The server uses a shared live AI model to recommend optimal content, for example, movies containing elements of comedy and suspense for multiple users to watch.

[1186] 6. Recommendations:

[1187] The generated list of optimal content is sent to the terminal and presented to the user, who can then select the content they prefer.

[1188] 7. Real-time function:

[1189] The server uses Firebase Realtime Database to provide real-time functionality for multiple users to share the viewing experience, allowing users to watch content together while chatting in real time.

[1190] Specific examples

[1191] User A: "Action movies", "Comedy dramas"

[1192] User B: "Romance movies", "Suspense dramas"

[1193] Based on this information, the server recommends content that reflects the preferences of both parties in a balanced manner.

[1194] Example prompt for a generative AI model:

[1195] User A: Action movies, comedy dramas

[1196] User B: Romance movies, suspense dramas

[1197] Requirements for a shared generative AI model: Recommending optimal video content to satisfy multiple preferences in the evening

[1198] This system enables optimal content recommendations that balance the preferences and needs of multiple users, and enables real-time collaborative viewing.

[1199] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1200] Step 1: Enter user preference data

[1201] Users use a device to input their preferences and requests (e.g., movie genres, favorite actors, viewing times). The device receives these inputs and formats the data for later transmission to the server. The input data includes genres and keywords selected by the user, and is used by the device as basic information for generating a generative artificial intelligence model.

[1202] Input: User preferences and request data

[1203] Output: Formatted user preference data

[1204] Step 2: Registering a Generative AI Model

[1205] The device generates a generative AI model based on the preference data acquired from the user. The generated model is sent to the server via API and stored in a server-side database (e.g., MySQL).

[1206] Input: Formatted user preference data

[1207] Output: Generative AI model

[1208] Step 3: Data collection and preprocessing

[1209] The server collects generative AI models sent by multiple users. The collected data undergoes preprocessing, such as standardizing the format and filling in missing values. During this process, the data is cleaned using frameworks such as TensorFlow.

[1210] Input: Generative AI model

[1211] Output: A preprocessed generative AI model

[1212] Step 4: Applying the fusion algorithm

[1213] The server applies a fusion algorithm to the preprocessed generative AI model, creating a shared generative AI model that balances the preferences and needs of multiple users. This fusion operation is performed using a machine learning framework such as PyTorch.

[1214] Input: A preprocessed generative AI model

[1215] Output: Shared generative AI model

[1216] Step 5: Content Recommendations

[1217] The server uses a shared generative artificial intelligence model to recommend the best content, taking into account the user's preferences and past viewing history to generate a list of the best content. This recommendation is made in real time.

[1218] Input: Shared generative artificial intelligence model

[1219] Output: Recommended content list

[1220] Step 6: Present recommended content

[1221] The server sends a list of recommended content to the device, which receives it and displays it in an interface optimized for the user. The user can then select and watch the content of their interest from the list.

[1222] Input: Recommended content list

[1223] Output: Optimized content list displayed on user device

[1224] Step 7: Providing real-time functionality

[1225] The server uses Firebase Realtime Database to provide functionality for multiple users to share their viewing experiences in real time, specifically allowing users to chat with each other in real time and share their viewing status.

[1226] Input: Viewer information and chat messages

[1227] Output: Real-time synchronized viewing information and messages

[1228] The above processing steps enable the preferences and needs of multiple users to be considered in a balanced manner, and optimal content recommendations and real-time joint viewing can be realized.

[1229] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1230] System configuration

[1231] The system according to the present invention includes a server, a terminal, a user, and an emotion engine. The user sets his or her preferences and requests and registers a generative AI model. The server collects generative AI model data from multiple users and combines it to create a shared generative AI model. Furthermore, the emotion engine recognizes the user's emotions and reflects them in the system. The terminal presents the generated recommendations and optimized interface to the user.

[1232] System action

[1233] First, the user inputs their preferences and requests, and a generative AI model is created based on them. The user accesses the system and uses a device to register this generative AI model. The device then sends the user's input information to the server, which then stores this data in a database.

[1234] When multiple users want to use a service together, for example when deciding on a restaurant for a group of friends, each user's generative AI model is requested from the server. The server collects these models and preprocesses the data, specifically standardizing the format and imputing missing values. A fusion algorithm is then applied to create a shared generative AI model that balances each user's preferences and needs.

[1235] The server uses the results of this shared generative AI model and the emotion engine to generate optimal recommendations for the user. For example, if multiple friends like Japanese and Italian food, the server might list fusion restaurants as candidates. These recommendations are then sent to the device and presented to the user.

[1236] Furthermore, the server generates the optimal interface for the use case based on the user's request and the current situation. For example, in the case of a ticket vending machine, a menu optimized for the user is automatically displayed. This interface is also displayed on the terminal, making it easy for the user to operate.

[1237] Emotion engine processing

[1238] The emotion engine uses sensors and cameras to recognize the user's emotions in real time. While the user is using the device, it analyzes their facial expressions, voice, and text input to determine their emotional state. The server takes in the data from this emotion engine and uses it to generate generative AI models and provide recommendations. This makes it possible to make suggestions tailored to the situation, such as recommending places where a user who is feeling stressed can relax.

[1239] Specific examples

[1240] User A registers his / her preference as "Japanese food," his / her budget as "under 3,000 yen," and his / her meal time as "evening." User B registers his / her preference as "Italian food," his / her budget as "under 4,000 yen," and his / her meal time as "evening." If both users want to choose a restaurant in the same group, each user's generative AI model is sent to the server. The server collects this data and performs preprocessing. A fusion algorithm then creates a shared generative AI model that matches each user's preference, such as a restaurant that serves Japanese and Italian fusion cuisine.

[1241] Furthermore, if User A is feeling stressed, the emotion engine recognizes this state and notifies the server. Based on this, the server adjusts the recommendation content to suggest relaxing environments and services. The server uses the results of the shared generative AI model and data from the emotion engine to generate optimal recommendations and create a candidate list. This list is sent to the device and displayed to User A and User B.

[1242] The server then generates the optimal interface for each use case and displays it on the device. For example, if User A is feeling stressed, it displays an interface with a menu that helps them relax.

[1243] This system not only makes it possible to provide optimal information that reflects the preferences and needs of multiple users in a balanced manner, but also provides a flexible interface that responds to the user's emotional state and usage situation.

[1244] The processing flow will be explained below.

[1245] Step 1:

[1246] Users set their own preferences and requests. Specifically, they input information such as their preferred genre, budget, and desired time slot through the interface on their device.

[1247] Step 2:

[1248] The device receives the input information and creates a generative AI model, which learns the user's preferences and requests and structures the data based on that information.

[1249] Step 3:

[1250] The device sends the generated AI model data to the server, which receives the data and stores it in a database.

[1251] Step 4:

[1252] When multiple users use the service together, for example when a group of friends want to choose a restaurant, each user's generated AI model is requested from the server.

[1253] Step 5:

[1254] The server collects the requested data for the generative AI model and performs preprocessing, which includes standardizing the data format and filling in missing values.

[1255] Step 6:

[1256] The server runs a fusion algorithm based on the preprocessed data to create a shared generative AI model that reflects each user's preferences and requirements in a balanced manner.

[1257] Step 7:

[1258] The server receives emotion data generated by the emotion engine from the user's device. Specifically, sensors and cameras installed on the device analyze the user's facial expressions, voice, and text input to determine their emotional state.

[1259] Step 8:

[1260] The server integrates the generated shared generative AI model with the emotion data to generate optimal recommendations, taking into account data from the emotion engine and adjusting the recommendations accordingly.

[1261] Step 9:

[1262] The server sends the generated optimal recommendations to the device, which receives the recommendations and presents them to the user, who can then select from the displayed list.

[1263] Step 10:

[1264] The server generates the most appropriate interface based on the user's request, emotional state, and the situation at hand. For example, a user feeling stressed will be shown a menu that helps them relax.

[1265] Step 11:

[1266] The terminal displays the optimal interface sent from the server, allowing the user to use the system in an easy-to-use environment.

[1267] Examples:

[1268] User A sets his preference as "Japanese food," his budget as "under 3,000 yen," and his meal time as "evening," and creates a generative AI model. User B sets his preference as "Italian food," his budget as "under 4,000 yen," and his meal time as "evening," and creates a generative AI model. If both users want to choose a restaurant in the same group, each user's generative AI model is sent to the server. The server collects this data and performs preprocessing. A fusion algorithm then creates a shared generative AI model that reflects each user's preferences, such as a restaurant that serves Japanese and Italian fusion cuisine.

[1269] Next, if User A is feeling stressed, the emotion engine recognizes this state and sends the data to the server. The server then adjusts the recommendation content taking this emotion data into account, suggesting, for example, a relaxing restaurant that serves Japanese and Italian fusion cuisine. The optimal recommendation is generated, and a candidate list is sent to the device and displayed to the user.

[1270] Finally, the server generates an optimal interface and adjusts the display content according to the user's state based on the emotional data. A relaxing menu interface is displayed on the device, providing an environment that is easy for the user to operate. In this way, the system provides optimal information by taking into account the preferences and emotional states of multiple users in a balanced manner.

[1271] Example 2

[1272] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1273] Conventional recommendation systems have limitations in reflecting user preferences and requests, making it difficult to incorporate the intentions of multiple users in a balanced manner. Furthermore, they lack the ability to provide recommendations that take into account the user's emotional state and optimal operation screens that are tailored to the situation.

[1274] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for registering a generative machine learning model for a user to set his or her preferences and requirements; means for collecting data of the generative machine learning model from multiple users; means for preprocessing the collected data and creating a shared generative machine learning model that reflects the preferences and requirements of each user in a balanced manner using a fusion algorithm; means for generating optimal recommendations using the generated shared generative machine learning model; emotion recognition means for recognizing the emotional state of the user in real time; means for providing the generated optimal recommendation to the user using the result of combining the data obtained from the emotion recognition means; and means for presenting the generated optimal recommendation to the user. This makes it possible to not only precisely reflect the preferences and requirements of users and incorporate the intentions of multiple users in a balanced manner, but also to provide optimal recommendations and operation screens according to the emotional state and situation.

[1275] A "generative machine learning model" is a data model created using machine learning algorithms based on user preferences and requirements.

[1276] "Means of registration" refers to the function for storing and managing user input information and generated machine learning models in the system's database or cloud service.

[1277] The "means of collection" is a function for centrally collecting and managing generative machine learning models and related data provided by multiple users.

[1278] "Preprocessing" refers to processes for improving data quality, such as standardizing the format of collected data and filling in missing values.

[1279] A "fusion algorithm" is a computational method for creating a new model that reflects each user's preferences and requirements in a balanced manner, based on data from a generative machine learning model collected from multiple users.

[1280] A "shared generative machine learning model" is a generative machine learning model that reflects the preferences and requirements of multiple users and combines them in a balanced manner.

[1281] The "means for generating recommendations" is a function for providing appropriate suggestions and options to the user using the generated shared generative machine learning model.

[1282] The "emotion recognition means" is a function that analyzes the user's facial expressions, voice, text input, etc. in real time to determine the user's emotional state.

[1283] The "means for providing" is a function for displaying the generated recommendations to the user and making them actually usable.

[1284] The "presentation means" is a function for visually or audibly presenting the recommendation content to the user.

[1285] Describe in detail the implementation of the invention

[1286] The present invention relates to a system that utilizes a generative machine learning model that reflects user preferences and requirements, creates a shared generative machine learning model that incorporates the requirements of multiple users in a balanced manner, and provides optimal recommendations. The system is also characterized by recognizing the user's emotional state in real time and providing recommendations and operation screens that correspond to that state.

[1287] First, a user inputs their preferences and requests using a device. The device formats this information and creates a generative machine learning model. This generative machine learning model is sent from the device to a server and stored in a database on the server. Next, the server collects the generative machine learning model data sent by multiple users and performs preprocessing. This preprocessing includes standardizing the format and filling in missing values. The collected data is then subjected to a fusion algorithm to create a shared generative machine learning model that reflects each user's preferences and requests in a balanced manner.

[1288] Using this shared generative machine learning model, the server generates optimal recommendations for users. For example, for a user group that likes "Japanese" and "Italian," it might list fusion restaurants as candidates. This optimized recommendation is sent to the user's device and presented visually or audibly.

[1289] Furthermore, the system uses emotion recognition to recognize the user's emotional state in real time. While the user is operating the device, the emotion engine analyzes the user's facial expressions, voice, and text through the camera, microphone, and text input to determine their emotional state. The data obtained from the emotion recognition is sent to the server and reflected in the generated recommendations. For example, if the user is feeling stressed, recommendations will be generated that provide a relaxing environment or service.

[1290] This system also dynamically generates the optimal operation screen based on the user's request and situation. For example, a user feeling stressed will be shown an easy-to-operate relaxation menu. This interface is displayed on the user's device and is designed to be intuitive for the user to operate.

[1291] Specific examples

[1292] User A likes "Japanese food," has a budget of "less than 3,000 yen," and registers his / her meal time as "evening." User B likes "Italian food," has a budget of "less than 4,000 yen," and also registers his / her meal time as "evening." If both users want to choose a restaurant in the same group, the generative machine learning models of each user are sent to the server. The server collects this data and performs preprocessing. After that, a fusion algorithm recommends restaurants that serve Japanese and Italian fusion cuisine.

[1293] Examples of prompts include:

[1294] "I like Japanese food, my budget is under 3,000 yen, and I eat in the evening. My friend likes Italian food, my budget is under 4,000 yen, and we also eat in the evening. What kind of restaurant would you recommend?"

[1295] This not only allows the system to precisely reflect users' preferences and requests and incorporate the intentions of multiple users in a balanced manner, but also makes it possible to provide optimal recommendations and operation screens that are tailored to the user's emotional state and situation.

[1296] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1297] Step 1:

[1298] The user inputs their preferences and requests. Using a terminal, the user inputs detailed information such as preferences, budget, and meal times. Input information may include "Japanese food," "budget 3,000 yen," and "dinner." This information is formatted by the terminal and converted into a unified format such as JSON.

[1299] Step 2:

[1300] A generative machine learning model is created and registered. The device creates a generative machine learning model based on the input data. The generated model is sent to the server, which registers it in a database. As an output, the generative machine learning model is saved on the server.

[1301] Step 3:

[1302] Collecting data for generative machine learning models. The server collects data for generative machine learning models sent by multiple users. The collected data is compiled for preprocessing. Multiple generative machine learning models are included as input.

[1303] Step 4:

[1304] The server preprocesses the collected data. The server performs preprocessing such as standardizing the format of the collected data and completing missing values. For example, it standardizes the data format for each user and estimates and completes missing data. The output is a preprocessed dataset.

[1305] Step 5:

[1306] A fusion algorithm is applied to create a shared generative machine learning model. The server uses the preprocessed data to create a shared generative machine learning model that balances the preferences and needs of multiple users. For example, data from users who like "Japanese food" and "Italian food" is integrated to generate a recommendation model for fusion cuisine. A shared generative machine learning model is generated as the output.

[1307] Step 6:

[1308] Emotion recognition is performed. While the user is using the device, the emotion engine analyzes the user's facial expressions and voice using sensors and cameras to determine their emotional state. The server collects this data and understands the user's emotional state. Real-time emotional data is included as input.

[1309] Step 7:

[1310] Generate optimal recommendations. The server integrates the shared generative machine learning model with emotion recognition data to generate optimal recommendations. For example, if the user is feeling stressed, it will recommend restaurants where they can relax. The output is an optimized recommendation list.

[1311] Step 8:

[1312] Present the recommendations to the user. The server sends the generated recommendation list to the terminal and presents it to the user. For example, a list of fusion restaurants is displayed on the user's terminal. The input contains the optimal recommendation list, and the output is a screen that the user can visually confirm.

[1313] Step 9:

[1314] The optimal interface is generated and displayed. The server generates the optimal operation screen based on the user's request and emotional state. For example, a user who is feeling stressed will be shown a simple and easy-to-understand relaxation menu. The optimal interface is then displayed on the terminal as output.

[1315] (Application example 2)

[1316] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1317] While there are existing recommendation systems based on user preferences and requests, it is difficult to provide flexible recommendations that take into account each individual's emotional state. It is also difficult to integrate the preferences of multiple users and make recommendations that satisfy everyone. Furthermore, there are few systems in physical stores that can provide an optimal interface based on the user's emotions.

[1318] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1319] In this invention, the server includes means for registering a generative artificial intelligence model for a user to set their own preferences and requirements, means for collecting data on the generative artificial intelligence model from multiple users, means for preprocessing the collected data and creating a shared generative artificial intelligence model that reflects each user's preferences and requirements in a balanced manner using a fusion algorithm, means for generating optimal recommendations using the generated shared generative artificial intelligence model, means for presenting the generated optimal recommendations to the user, means for recognizing the user's emotions in real time using an emotion recognition engine, and means for adjusting the recommendation content based on the recognized emotions and automatically generating an optimal interface. This makes it possible to provide more optimal recommendations and interfaces that take both the user's preferences and emotions into consideration.

[1320] A "user" is someone who uses the system to register their preferences and needs and receive recommendations.

[1321] "Preferences" is a concept that refers to the things and conditions that a user prefers.

[1322] "Requirements" refer to the specific requests and conditions that users have for the system.

[1323] A "generative artificial intelligence model" is an artificial intelligence-based data model that is generated based on user preferences and requirements.

[1324] "Means of registration" refers to the functions and methods by which users can input and save their preferences and requests in the system.

[1325] "Means of collection" refers to the functions and methods for collecting data for the generative artificial intelligence model sent by multiple users.

[1326] "Preprocessing" is the process of processing collected data, such as filling in missing values ​​and standardizing the format.

[1327] A "fusion algorithm" is an algorithm that balances the preferences and needs of multiple users to create a single shared generative artificial intelligence model.

[1328] A "shared generative artificial intelligence model" is a common generative artificial intelligence model created by combining the preferences and requests of multiple users.

[1329] "Means for generating" refers to methods and functions for generating optimal recommendations using a shared generative artificial intelligence model.

[1330] "Recommendation" refers to recommended information and services provided by a system based on a user's preferences and requests.

[1331] The "presentation means" refers to a method or function for showing the generated recommended information or services to the user.

[1332] An "emotion recognition engine" is a system or device that analyzes a user's facial expressions, voice, text, etc. to recognize their emotional state in real time.

[1333] "Means of recognition" refers to functions and methods for capturing the user's emotions in real time, analyzing them, and making judgments.

[1334] "Tuning" means methods or functions for modifying and optimizing recommendations based on perceived sentiment.

[1335] "Interface" refers to the display screen and operation method for exchanging information between the user and the system.

[1336] "Automatic generation means" refers to methods or functions that allow the system to operate on its own based on specific conditions and create an appropriate interface.

[1337] The system for implementing this invention includes a server, a terminal, a user, and an emotion engine. The user sets his or her preferences and requests, and generates a generative artificial intelligence model based on them. The generated model is sent from the user's terminal to the server and stored in the server's database.

[1338] System configuration and program processing

[1339] server

[1340] The server collects data for generative AI models sent by multiple users and preprocesses it. This preprocessing involves standardizing the format and filling in missing values. Next, a fusion algorithm is applied to create a shared generative AI model that reflects each user's preferences and requests in a balanced manner. Optimal recommendations are generated based on this created model. Furthermore, emotional data from an emotion recognition engine is incorporated, the recommendation content is adjusted, and an optimal interface is generated according to the situation.

[1341] Specific software used includes scikit-learn for data preprocessing and fusion algorithms and the face_recognition library for emotion recognition.

[1342] Terminal

[1343] The device provides a means for users to input their preferences and requests and register them on the server. It also displays recommended information and optimized interfaces received from the server to the user. The device is equipped with sensors and cameras to transmit the user's facial expressions and voice to an emotion recognition engine in real time.

[1344] As a specific example, smartphones and tablets can be used as devices, as they have built-in cameras and microphones and can be used to collect data for emotion recognition.

[1345] User

[1346] Users input their preferences and requests through their devices, and a generative AI model is generated based on this information and registered on the server. Furthermore, the user's emotional data is also collected in real time and sent to the server. This allows the system to provide optimal recommendations tailored to the user's situation.

[1347] Specific examples

[1348] For example, if User A specifies his / her preference and requirement that he / she prefers Japanese food and is only available in the evening, this information is sent to the server as a generative AI model. Similarly, if User B specifies his / her preference and requirement that he / she prefers Italian food and has a budget of 4,000 yen or less, the same applies. By combining this information, the server can recommend restaurants that offer Japanese-Italian fusion cuisine.

[1349] If User A is feeling stressed, the emotion recognition engine will recognize that emotion in real time and notify the server. Based on this, the server will adjust its recommendations to suggest relaxing environments and services.

[1350] Prompt Sentence Examples

[1351] "Please recommend restaurants based on User A's preference for Japanese food and User B's preference for Italian food. Also, if User A is feeling stressed, please generate appropriate recommendations that will help them relax."

[1352] In this way, the system can simultaneously consider user preferences and emotions to provide optimal recommendations and interfaces for physical stores.

[1353] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1354] Step 1:

[1355] Users input their preferences and requests to create a generative AI model.

[1356] The user uses a device to input their preferences and requests, such as their preferred cuisine genre, budget, and time of day. The input data is converted into a generative AI model on the device. Inputs include preferences and requests such as "Japanese food," "under 3,000 yen," and "evening." The output is a generative AI model that reflects the user's preferences and requests.

[1357] Step 2:

[1358] Send the generative AI model from the device to the server

[1359] The generative AI model entered by the user is sent from the device to the server. The device sends the input data in packets. The generative AI model is used as input, and the model is saved on the server side as output.

[1360] Step 3:

[1361] The server collects and preprocesses the generated AI model data from multiple users.

[1362] The server receives generative AI model data sent by multiple users. The received data undergoes preprocessing, such as format standardization and missing value completion. The input is the generative AI models from multiple users, and the output is the preprocessed data.

[1363] Step 4:

[1364] The pre-processed data is processed through a fusion algorithm to create a shared generative AI model.

[1365] The server inputs the preprocessed data into a fusion algorithm to create a shared generative AI model that reflects each user's preferences and requirements in a balanced manner. The input is the preprocessed data, and the output is the shared generative AI model.

[1366] Step 5:

[1367] The server uses an emotion recognition engine to recognize the user's emotions in real time and captures the data.

[1368] The device sends the user's facial image and voice to the emotion recognition engine in real time. The server captures the recognized emotion data and stores it in a database. The input is the user's facial image and voice, and the output is the recognized emotional state data.

[1369] Step 6:

[1370] The server adjusts the recommendation content based on the emotional data and generates the optimal interface.

[1371] The server analyzes the acquired emotional data and adjusts the recommendation content based on that data. It also automatically generates an optimal interface for the user. The input is emotional state data, and the output is the adjusted recommendation content and the optimal interface.

[1372] Step 7:

[1373] The server sends the recommendations and the optimal interface to the user's device.

[1374] The generated recommendation content and interface are sent from the server to the user's device, which receives them and displays them to the user. The input is the recommendation content and interface data, and the output is the information presented to the user.

[1375] Step 8:

[1376] The user interacts with the recommendations and interface and sends feedback to the server.

[1377] The user operates the recommended content and interface provided through the device, and sends feedback from the device to the server as needed. The input is the user's operations and feedback data, and the output is the feedback information accumulated on the server.

[1378] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1379] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1380] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1381] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1382] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1383] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1384] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1385] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1386] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1387] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1388] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1389] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1390] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1391] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1392] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1393] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1394] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1395] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1396] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1397] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1398] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1399] The following is further disclosed regarding the above embodiment.

[1400] (Claim 1)

[1401] A means for the user to register a generative artificial intelligence model to set his / her preferences and requirements;

[1402] a means for collecting data for the generative artificial intelligence model from a plurality of users;

[1403] A means for preprocessing the collected data and creating a shared generative artificial intelligence model that reflects each user's preferences and requirements in a balanced manner using a fusion algorithm;

[1404] A means for generating optimal recommendations using the generated shared generative artificial intelligence model;

[1405] A means for presenting the generated optimal recommendations to the user; and

[1406] A system including:

[1407] (Claim 2)

[1408] 2. The system according to claim 1, further comprising means for generating an optimal interface based on the content of a user's request and a situation, and displaying the optimal interface to the user.

[1409] (Claim 3)

[1410] 10. The system of claim 1, further comprising means for performing error handling.

[1411] "Example 1"

[1412] (Claim 1)

[1413] a means for a user to register a generator to set his / her preferences and requirements;

[1414] a means for collecting generator data from a plurality of users;

[1415] A means for preprocessing the collected data and creating a sharing generation program that reflects each user's preferences and requirements in a balanced manner using a fusion algorithm;

[1416] A means for generating optimal recommendations using the generated shared generation program;

[1417] A means for presenting the generated optimal recommendations to the user; and

[1418] A means for generating an optimal interface based on the user's request and situation and displaying it to the user;

[1419] A system including:

[1420] (Claim 2)

[1421] 10. The system of claim 1, further comprising means for performing preprocessing of the collected data, such as standardizing the format and imputing missing values.

[1422] (Claim 3)

[1423] 10. The system of claim 1, further comprising means for performing error handling.

[1424] "Application Example 1"

[1425] (Claim 1)

[1426] A means for the user to register a generative artificial intelligence model to set his / her preferences and requirements;

[1427] a means for collecting data for the generative artificial intelligence model from a plurality of users;

[1428] A means for preprocessing the collected data and creating a shared generative artificial intelligence model that reflects each user's preferences and requirements in a balanced manner using a fusion algorithm;

[1429] A means for recommending optimal content using the generated shared generative artificial intelligence model;

[1430] A means of presenting the best recommended content to users,

[1431] A means to provide real-time capabilities for multiple users to share a viewing experience; and

[1432] A system including:

[1433] (Claim 2)

[1434] 2. The system according to claim 1, further comprising means for generating an optimal interface based on the content of a user's request and a situation, and displaying the optimal interface to the user.

[1435] (Claim 3)

[1436] 10. The system of claim 1, further comprising means for performing error handling.

[1437] "Example 2: Combining Emotion Engines"

[1438] (Claim 1)

[1439] A means for users to register generative machine learning models to set their preferences and requirements;

[1440] a means for collecting data for the generative machine learning model from a plurality of users;

[1441] A means for preprocessing the collected data and creating a shared generative machine learning model that reflects each user's preferences and requirements in a balanced manner using a fusion algorithm;

[1442] A means for generating optimal recommendations using the generated shared generative machine learning model;

[1443] emotion recognition means for recognizing the emotional state of a user in real time;

[1444] a means for providing a user with an optimal recommendation generated using a result of combining data obtained from the emotion recognition means;

[1445] means for presenting the generated optimal recommendations to the user;

[1446] A system including:

[1447] (Claim 2)

[1448] 2. The system according to claim 1, further comprising means for generating an optimum operation screen based on the contents of a user's request and a situation, and displaying the optimum operation screen to the user.

[1449] (Claim 3)

[1450] 10. The system of claim 1, further comprising means for performing error handling.

[1451] "Application example 2 when combining emotion engines"

[1452] (Claim 1)

[1453] A means for the user to register a generative artificial intelligence model to set his / her preferences and requirements;

[1454] a means for collecting data for the generative artificial intelligence model from a plurality of users;

[1455] A means for preprocessing the collected data and creating a shared generative artificial intelligence model that reflects each user's preferences and requirements in a balanced manner using a fusion algorithm;

[1456] A means for generating optimal recommendations using the generated shared generative artificial intelligence model;

[1457] A means for presenting the generated optimal recommendations to the user; and

[1458] A means for recognizing a user's emotions in real time using an emotion recognition engine;

[1459] A means for automatically generating an optimal interface by adjusting the recommendation content based on the recognized emotions;

[1460] A system including:

[1461] (Claim 2)

[1462] 2. The system according to claim 1, further comprising means for generating an optimal interface based on the content of a user's request and a situation, and displaying the optimal interface to the user.

[1463] (Claim 3)

[1464] 10. The system of claim 1, further comprising means for performing error handling. [Explanation of symbols]

[1465] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for the user to register a generative artificial intelligence model to set his / her preferences and requirements; a means for collecting data for the generative artificial intelligence model from a plurality of users; A means for preprocessing the collected data and creating a shared generative artificial intelligence model that reflects each user's preferences and requirements in a balanced manner using a fusion algorithm; A means for generating optimal recommendations using the generated shared generative artificial intelligence model; A means for presenting the generated optimal recommendations to the user; and A system including:

2. 2. The system according to claim 1, further comprising means for generating an optimal interface based on the content of a user's request and a situation, and displaying the optimal interface to the user.

3. 10. The system of claim 1, further comprising means for performing error handling.

Images