System

Abstract

A system is provided.SOLUTION: A system, comprising: means for registering and managing generative models in different fields; means for matching collaboration and competition between the generative models; means for evaluating products output by the generative models; and means for providing an interface for the generative models and users to collaboratively conduct creative activities.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional generative models often train based on the same dataset, resulting in monotonous outputs. Furthermore, there is little interaction between generative models, and sharing of training data and generated results is limited, preventing the diversity and creativity of generative models from being fully utilized. The present invention aims to enable generative models from different fields to stimulate each other, thereby generating diverse ideas and expression methods, and improving the learning efficiency of generative models and the creativity of their results. [Means for solving the problem]

[0005] To solve this problem, the following means are provided.

[0006] We provide a system consisting of a means for registering and managing generative models from different fields, a means for matching collaboration and competition between said generative models, a means for evaluating the products output by said generative models, and a means for providing an interface for said generative models and users to engage in collaborative creative activities.This creates an environment in which generative models can stimulate each other, promoting the creation of new ideas and methods of expression, and thereby improving the diversity and creativity of generative models.

[0007] A "generative model" is a type of artificial intelligence technology that generates new data and representations based on a training dataset.

[0008] "Collaboration" refers to the process in which different generative models work together to create a product.

[0009] "Competition" refers to the process in which generative models compete with each other over the quality and diversity of their products.

[0010] "Matching" is a process that uses an algorithm to form optimal pairs or groups based on the characteristics of the generative model.

[0011] "Evaluation" refers to the process of having experts and general users judge the quality and diversity of the products created and then compiling the evaluation results.

[0012] "Interface" refers to the means of user interaction that a user uses to interact with a generative model and perform creative activities.

[0013] "Progress monitoring" is the process of tracking the collaboration and competition of generative models in real time and providing feedback as needed.

[0014] "Storing generated results" refers to the process of recording the data and products generated by a generative model in a database.

[0015] "General User" refers to a user who does not have specialized knowledge and who uses the Platform.

[0016] "Evaluation and Awards" refers to the process of recognizing and awarding excellent generative models and their developers based on the evaluation results of their products. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a system that improves the diversity and creativity of generative models through collaboration and competition between them in different fields. This system has functions for registering and managing generative models, matching between generative models, evaluating the results, and realizing collaborative creation with users.

[0039] 1. Registering and Managing Generative Models

[0040] User:

[0041] Users register generative models on the platform. For example, an interface is provided for entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used). After registration is complete, users can share their generative models with other users.

[0042] server:

[0043] The server receives information about the generative model sent by the user and stores it in a database, allowing management of the characteristics of the generative model (performance, genre, training data, etc.).

[0044] 2. Matching and project launch

[0045] server:

[0046] The server automatically matches models from different fields based on the registered generative models. For example, it pairs a music generation model with a video generation model. The matching results are notified to the user, who can then start a collaboration project.

[0047] User:

[0048] Upon receiving the notification, the user can start the project by checking the project details and making the necessary settings (such as the format of the generated output and output nodes).

[0049] 3. Implementing the generation process

[0050] Device:

[0051] The generative model runs on the user's device and executes the generation process based on the specified parameters. For example, a music generation model generates a unique piece of music, and a video generation model creates a video to go along with it.

[0052] server:

[0053] The server monitors the progress of the generation process in real time, stores the generated results, and provides feedback to the user if necessary.

[0054] 4. Product Evaluation and Awards

[0055] server:

[0056] We will hold regular generative AI contests, where submitted products will be evaluated by experts and general users. The evaluation results will be compiled, and outstanding generative models and their developers will be awarded prizes.

[0057] User:

[0058] Users who participate in the contest can submit their creations and rate and comment on other creations, allowing for more diverse feedback.

[0059] 5. User-participatory creative activities

[0060] User:

[0061] Ordinary users can also access the platform and use generative models to create their own works. For example, a user can create a song using a music generation model and then create a video to accompany that song using a video generation model.

[0062] server:

[0063] The server provides an interface for users to store their generated work and share it with other users.

[0064] As a concrete example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches these models and proposes a project to jointly generate artworks (images + music). Users A and B generate artworks using their respective models, and the generated artworks are published on the platform and evaluated by other users.

[0065] In this way, the present invention promotes interaction between generative AI models, enabling the exploration of diverse expression techniques and improving the quality of the products.

[0066] The processing flow will be explained below.

[0067] Step 1:

[0068] Registering a Generative Model

[0069] User:

[0070] Enter detailed information about your generative model (model name, genre, purpose, dataset used, etc.) into the registration form and submit it.

[0071] server:

[0072] The submitted generative model data is received and saved in the database. Once saved, a notification is sent to the user to notify them of the completion of registration.

[0073] Step 2:

[0074] Managing Generative Models

[0075] server:

[0076] It updates the Generative Model information in the database and provides a user-accessible management screen that displays a list of registered Generative Models and their respective characteristics.

[0077] Step 3:

[0078] Matching implementation

[0079] server:

[0080] It analyzes the generative model information in the database, runs an algorithm to pair generative models from different fields for collaboration or competition, and generates matching results and notifies the user.

[0081] Step 4:

[0082] Starting a project

[0083] User:

[0084] You will receive a notification of the matching result, check the details, and if you agree to start the project, make the necessary settings (product format, output node, etc.).

[0085] server:

[0086] Receives configuration information from the user and prepares the project execution environment.

[0087] Step 5:

[0088] Implementing the generation process

[0089] Device:

[0090] On the device where the user's generative model runs, the generative process is started based on the specified parameters, and an artifact (e.g., music or video) is created.

[0091] server:

[0092] Monitor the progress of the generation process in real time, store the generated results in a database, and check the feedback and notify the user if necessary.

[0093] Step 6:

[0094] Product evaluation

[0095] server:

[0096] We provide an interface for experts and general users to evaluate the generated models, and we aggregate the evaluation results and award outstanding generative models.

[0097] User:

[0098] You can participate in product evaluations and leave comments. You can also view other users' evaluations.

[0099] Step 7:

[0100] User-participation creative activities

[0101] User:

[0102] Ordinary users can access the platform and use generative models to create their own works. For example, they can select a music generation model to create a song, and then use a video generation model to create a video that matches the song.

[0103] server:

[0104] It provides an interface for users to save and share their creations with other users, and also manages other users' comments on shared creations.

[0105] This allows for greater diversity and creativity in generative models.

[0106] Example 1

[0107] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0108] In conventional generative AI model systems, it was difficult for generative models from different fields to efficiently collaborate and produce diverse creative works. Furthermore, there was a lack of mechanisms for evaluating the generated results, and insufficient feedback on improving the quality of the generative models was also an issue. Furthermore, the hurdles for general users to use generative AI models for creative activities were high, making it difficult to produce diverse creative works. A system that can solve these issues and bring out more creativity and diversity is needed.

[0109] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0110] In this invention, the server includes means for registering and managing generative AI models, means for matching collaboration and competition between generative AI models, means for evaluating the products output by the generative AI models, means for providing an interface for generative AI models and users to collaborate on creative activities, means for receiving detailed information about the generative AI models and saving it in a database, means for automatically matching generative AI models of different genres and notifying users, and means for monitoring the progress of the generation process in real time and saving the generation results. This allows generative models from different fields to collaborate efficiently, improve the quality of generative models through evaluation and feedback, and enable general users to easily engage in creative activities.

[0111] A "generative AI model" is an algorithm that is trained to perform a specific problem or task using artificial intelligence techniques.

[0112] "Registration and management" refers to the process of registering the details of a generative AI model on the platform, storing that information in the database, and updating or deleting it as needed.

[0113] "Collaboration and competition" refers to a situation in which different generative AI models work together to collaborate or compete to create products.

[0114] "Matching" is the process of automatically selecting and linking generative AI models with different genres and characteristics.

[0115] "Products" are the results output by generative AI models, and can take various forms such as music, video, images, and text.

[0116] "Evaluation" refers to the act of experts and general users scoring and commenting on the quality and performance of the products output by a generative AI model.

[0117] "Interface" means the user interface and functionality that allows a user to interact with a generative AI model and engage in collaborative creative activities.

[0118] A "database" is a software system for efficiently storing and managing data such as detailed information about generative AI models and generated results.

[0119] "Notifications" are messages that convey information to users, such as the matching results of the generative AI model and the progress of the generation project.

[0120] "Progress monitoring" means checking in real time whether the generation process is proceeding as planned or whether any abnormalities have occurred.

[0121] "Storage" refers to the act of recording the output product of the generation process and the data generated during that process in a database or storage.

[0122] This invention is a system that improves the diversity and creativity of generative AI models by allowing them to collaborate and compete with each other in different fields. This system has functions for registering and managing generative AI models, matching between generative AI models, evaluating the results, and realizing collaborative creation with users.

[0123] Specific actions

[0124] Users register generative AI models on the platform. For example, an interface is provided for entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used). When registering a "music generative model," users enter information such as "music generative model A," "music," "music production," and "music dataset 1."

[0125] The server receives information about the generative AI model sent by the user and stores it in a database. This allows the server to manage the characteristics of the generative AI model (performance, genre, training data, etc.). For example, a record such as "Music Generative Model A, Music, Music Production, Music Dataset 1" is registered in the database.

[0126] The server automatically matches models from different fields based on information about generative AI models registered in the database. For example, it might select a pair of "music generation model A" and "video generation model B." It then proposes a new project for the selected pair and notifies the user.

[0127] After receiving the notification, the user checks the details of the proposed project and makes the necessary settings (such as the format of the generated output and output nodes) on the user interface. For example, they can set the length of the music to be generated and the video resolution. Once the settings are complete, they press the "Start Project" button.

[0128] The generative AI model runs on the device and executes the generation process based on the specified parameters. For example, a music generation model generates a unique song, and a video generation model creates a video in conjunction with it.

[0129] The server monitors the progress of the generation process in real time and saves the generated results. After the generation is complete, it saves the results in a database and notifies the user of the results. Specifically, it saves the generated music and video files and sends a notification to the user.

[0130] The server will hold a generative AI contest and evaluate the products submitted by users. It will compile evaluations from experts and general users and award outstanding generative models and their developers. For example, if "Music Generation Model A" receives a high evaluation based on the evaluation criteria, the results will be made public.

[0131] Ordinary users can access the platform and use generative AI models to create their own works. For example, a user can create a song using a music generation model and then create a video to go with that song using a video generation model.

[0132] The server stores user-generated works and provides an interface for sharing them with other users, e.g., user-generated music and videos are shared so that other users can view and rate them.

[0133] Examples and prompts

[0134] For example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches these models and proposes a project to jointly generate an artwork (image + music). Users A and B create artwork using their respective models, and the resulting artwork is published on the platform and evaluated by other users.

[0135] The following can be used as a specific prompt:

[0136] "Please run a program that generates a landscape image and generates background music that matches that image. We will use 'Landscape Image Generation Model A' as the image generation model and 'Music Generation Model A' as the background music generation model."

[0137] In this way, the present invention promotes interaction between generative AI models, enabling the exploration of diverse expression methods and improving the quality of the products.

[0138] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0139] Step 1:

[0140] Registering a generative AI model

[0141] Users register generative AI models on the platform by accessing a dedicated interface and entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used).

[0142] Input: Detailed information about the generative model (model name, genre, purpose, dataset used, etc.)

[0143] Output: Send registration application data

[0144] Specific operation: The user enters information such as "Music Generation Model A," "Music," "Music Production," and "Music Dataset 1" into the interface and presses the registration button.

[0145] Step 2:

[0146] Receiving and saving generated AI model registration information

[0147] The server receives the registration information for the generative AI model sent by the user and stores it in a database.

[0148] Input: Registration information for the generative model from the user

[0149] Output: Record added to database

[0150] Specific operation: The server receives the information "Music generation model A, music, music production, music dataset 1" from the user and stores it in the database.

[0151] Step 3:

[0152] Matching between generative models

[0153] The server automatically matches models from different fields based on information about generative AI models registered in the database.

[0154] Input: Information about the generative AI model stored in the database

[0155] Output: Matching result notification

[0156] Specific operation: The server selects a pair of "music generation model A" and "video generation model B" and sends a "project proposal" notification to the user.

[0157] Step 4:

[0158] Project details

[0159] The user receives a notification, reviews the details of the proposed project, and makes any necessary configurations (such as artifact format and output nodes).

[0160] Input: Project proposal notification, configuration information (format, output node, etc.)

[0161] Output: Sending a notification that the setup is complete

[0162] Specific operations: The user sets the length of the music to be generated, the resolution of the video, etc., and presses the "Start Project" button.

[0163] Step 5:

[0164] Implementing the generation process

[0165] The generative AI model runs on the device and executes the generation process based on the specified parameters.

[0166] Input: Setting parameters (song length, video resolution, etc.)

[0167] Output: Generated products (music files, video files)

[0168] Specific operation: "Music generation model A" generates a five-minute piece of music on User A's device, and at the same time, "video generation model B" generates a video linked to the music on User B's device.

[0169] Step 6:

[0170] Product monitoring and preservation

[0171] The server monitors the progress of the generation process in real time and stores the generation results.

[0172] Input: Progress data of the generation process

[0173] Output: Save generated results to database, progress notification

[0174] Specific operation: After generation is complete, the generated music and video files are saved in a database and the user is notified of the results.

[0175] Step 7:

[0176] Product evaluation

[0177] The server hosts a generative AI contest and performs a process to evaluate the products submitted by users.

[0178] Input: Submitted product

[0179] Output: Evaluation results summary, award notification

[0180] Specific operation: We will aggregate evaluations from experts and general users and award outstanding generative models and their developers.

[0181] Step 8:

[0182] User-participation creative activities

[0183] Ordinary users can also access the platform and use generative AI models to create their own unique works.

[0184] Input: Creation request using generative model

[0185] Output: Generated creative works (music, video, etc.)

[0186] Specific operation: The user creates a song using the music generation model and a video using the video generation model.

[0187] Step 9:

[0188] Sharing the results

[0189] The server provides an interface for users to store their generated work and share it with other users.

[0190] Input: Generated artwork

[0191] Output: Provides an interface for sharing

[0192] Specific operation: The generated music and video will be published on the platform and can be rated by other users.

[0193] (Application example 1)

[0194] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0195] Conventional generative AI models have focused on outputting products specialized in a single field, making it difficult for generative models from different fields to collaborate with each other or for users to participate in the generation process in real time. Furthermore, there is a lack of a mechanism for effectively utilizing evaluations and feedback of products to update the generated content, which limits the quality and diversity of the products.

[0196] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0197] In this invention, the server includes a means for registering and managing generative models from different fields, a means for matching collaboration and competition between generative models, a means for evaluating the products output by the generative models, a means for providing an interface for the generative models and users to engage in collaborative creative activities, and a means for linking different generative models to generate and view interactive content. This enables the linking of generative models from different fields to generate diverse, high-quality interactive content and user participation in real time.

[0198] A "generative model" is an algorithm that generates new data or content based on a specific dataset or algorithm.

[0199] “Registration and Management” refers to the process of adding a Generative Model to the Platform, storing its information, and updating or modifying it as needed.

[0200] "Collaboration" refers to the activity of different generative models working together to jointly produce a single product.

[0201] "Competition" refers to the process in which different generative models compete against each other in terms of their performance and output.

[0202] "Matching" refers to the process of selecting the optimal combination based on the characteristics and parameters of different generative models.

[0203] "Interactive content" refers to content whose generated content changes dynamically as users operate and provide feedback in real time.

[0204] "Interface" refers to the screen or tool that allows the user to input and operate the generative model.

[0205] "Evaluation" refers to the activity of diagnosing the product output by a generative model based on its quality, performance, etc., and providing the results as feedback.

[0206] "Real-time monitoring" refers to the process of instantly checking the progress of the generation process and maintaining a state in which necessary responses can be taken immediately.

[0207] "Preservation" refers to the process of securely storing generated data and content for later use and evaluation.

[0208] "Feedback" refers to the process of returning evaluations of the product and suggestions for improvement to the model.

[0209] This invention provides a system that links generative models from different fields to enable the generation and viewing of interactive content. An embodiment of this system is described in detail below.

[0210] 1. System Configuration

[0211] Server: The server contains the following main means:

[0212] Generative Model Management

[0213] Generative model matching method

[0214] Product evaluation method

[0215] Interface provision means

[0216] Real-time monitoring and feedback measures

[0217] To achieve these goals, the server uses high-performance computing hardware and a web framework such as Flask.

[0218] User device: Users access the system using a smartphone or head-mounted display (HMD) to generate and view content. A dedicated app for operating and linking different generative models is installed on the smartphone.

[0219] Generative Models: Generative models are algorithms that are specialized for a particular domain, such as music generation models or video generation models. They are hosted on a server and accessed via API endpoints.

[0220] 2. Operation overview

[0221] Users use a smartphone app to register generative models on the platform. After entering model information (model name, genre, purpose, dataset used, etc.), the model is saved on the server. The registered generative model is managed by the server and updated as needed.

[0222] Next, the server automatically matches models from different fields based on the characteristics of the registered generative models. For example, a music generation model and a video generation model are paired and notified to the user. The user can then start the project by checking the project details and configuring the necessary settings.

[0223] 3. Content Creation and Viewing

[0224] The user inputs a prompt to generate interactive content, for example:

[0225] Example prompt:

[0226] Input to the music generation model: "Generate a fast-paced, electronic-feeling soundtrack."

[0227] Input to the video generation model: "Generate a video with a fast camera movement against a futuristic cityscape."

[0228] Based on this, the server calls the music generation model and video generation model, respectively, and performs generation. The generated music and video are sent to the user's device in real time and can be viewed on a smartphone or HMD. Based on user feedback, the server updates the generated content as appropriate and advances the generation process.

[0229] The generated content is stored on the server and can be shared and rated by other users. The evaluation results are aggregated and outstanding generative models and their developers are awarded.

[0230] This allows generative models from different fields to work together, enabling the generation of diverse, high-quality interactive content and real-time user participation.

[0231] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0232] Step 1: Registering the Generative Model

[0233] Users use a smartphone app to register a generative model on the platform. Input includes detailed information about the generative model (model name, genre, purpose, dataset used, etc.). These details are sent to the server and stored in a database. As an output, a message is sent to the user informing them that the generative model has been registered.

[0234] Step 2: Managing the Generative Model

[0235] The server manages information about registered generative models. It references the characteristics of the generative models stored in the database (performance, genre, training data, etc.) and updates and corrects them as needed. This management process periodically checks the information about the generative models to keep them up to date. As an output, a report containing the latest information about the generative models is generated.

[0236] Step 3: Matching the generative model

[0237] The server automatically matches models from different fields based on the characteristics of the registered generative models. For example, it analyzes features to find the optimal combination to pair a music generation model with a video generation model. The characteristic information of each generative model is used as input, and the matching results are reported to the user as output.

[0238] Step 4: Starting the project

[0239] Upon receiving the notification, the user checks the project details and makes the necessary settings (such as the product format and output nodes). The input includes the matched generative model and project details. As an output, a project start confirmation message is sent to the server, and the project is launched.

[0240] Step 5: Implementing the generation process

[0241] When a project starts, the user inputs a prompt, for example, "Please generate a soundtrack with a fast tempo and an electronic feel." The server receives this prompt and sends generation requests to the music generation model and the video generation model. The generative models perform the generation work based on the specified parameters and return the output results (music and video) to the server.

[0242] Step 6: Integrating the products

[0243] The server receives the generated results (music and video) sent from the generative model. To integrate these generated results, the server adjusts the timing and coding of the music and video. The input includes the music generation results and the video generation results, and interactive content is generated as the output.

[0244] Step 7: Interactive viewing

[0245] The generated interactive content is delivered to the user device in real time. The user views the content using a smartphone or HMD and sends feedback to the server. The input includes the user's viewing data and feedback, and the output is an instruction to update the generated content.

[0246] Step 8: Update the generation process

[0247] The server updates the generated content based on user feedback. It analyzes the generated content for improvement and sends new parameters to the generative model for regeneration. This process aims to improve the quality of the product and increase user satisfaction. The input includes user feedback data, and the output is the improved product that is redistributed.

[0248] Step 9: Evaluate and share your creations

[0249] The final generated content is stored on a server and published on the platform for general users to evaluate. Users can rate and comment on other creations and provide their own feedback. The input includes evaluation data, and the output is the evaluation results that are fed back to the developer of the generative model.

[0250] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0251] The present invention is a system that aims to improve the diversity and creativity of generative models from different fields by allowing them to collaborate and compete with each other. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions, allowing the output of the generative model to be customized according to the user's emotions. An embodiment of this system is described in detail below.

[0252] System Configuration

[0253] The system mainly consists of the following components:

[0254] A server that registers and manages generative models

[0255] A server that performs matching between generative models

[0256] User device equipped with an emotion engine

[0257] A server that provides evaluation and award functions for products

[0258] An interface for collaborative creative activities between users and generative models

[0259] 1. Registering and Managing Generative Models

[0260] User:

[0261] Users register generative models on the platform. When registering, they are provided with an interface to input detailed information about the generative model (such as the model name, genre, purpose, and dataset used).

[0262] server:

[0263] The server receives the information about the generative model sent by the user and stores it in a database, allowing for centralized management of the characteristic information of the generative model.

[0264] 2. Matching and project launch

[0265] server:

[0266] The server pairs generative models from different fields based on the registered generative model information, and conducts matching for collaboration and competition. The matching results are notified to the user.

[0267] User:

[0268] The user who receives the notification checks the details of the project and makes the necessary settings (format of the generated output, output node, etc.), which starts the project.

[0269] 3. Implementing the generation process

[0270] Device:

[0271] The generation process is initiated based on the specified parameters on the device where the user's generative model is running. The emotion engine has the ability to recognize emotions by analyzing the user's facial expressions and voice data.

[0272] server:

[0273] The server monitors the progress of the generation process in real time, stores the generated results in a database, and receives feedback from the emotion engine to adjust the output of the generative model.

[0274] 4. Product Evaluation and Awards

[0275] server:

[0276] The server provides an interface for experts and general users to evaluate the generated models, and aggregates the evaluation results to recognize and reward outstanding generative models and their developers.

[0277] User:

[0278] Users can participate in rating the products and leave comments, which provides feedback for improvements to the generative model and new ideas.

[0279] 5. User-participatory creative activities

[0280] User:

[0281] Ordinary users can access the platform and create their own works using generative models. For example, a user can select a music generation model, create a song, and then create a video to go with the song using a video generation model.

[0282] server:

[0283] The server provides an interface for users to store and share their generated works with other users, and can further customize the creations based on data from the emotion engine.

[0284] Specific examples

[0285] For example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches the two, and users A and B start a project to jointly create an artwork (image + music). When user B creates music, the emotion engine analyzes user B's facial expressions in real time and adjusts the tempo and tone of the music in real time according to the emotion. The final artwork created is published on the platform and receives ratings and comments from other users.

[0286] In this way, the present invention increases the diversity and creativity of generative models, while at the same time providing a more personalized experience by recognizing the user's emotions and customizing the product.

[0287] The processing flow will be explained below.

[0288] Step 1:

[0289] Registering a Generative Model

[0290] User:

[0291] The user accesses the registration form for the generative model and enters detailed information about the generative model (model name, genre, purpose, dataset used, etc.). By clicking the submit button, the information is sent to the server.

[0292] server:

[0293] The submitted generative model data is received and saved in the database. Once saved, a notification of registration completion is sent to the user.

[0294] Step 2:

[0295] Managing Generative Models

[0296] server:

[0297] The system periodically updates the information about generative models in the database and provides a user-accessible management screen that displays a list of registered generative models and their respective characteristics.

[0298] User:

[0299] You can view and edit information about registered generative models through the management screen.

[0300] Step 3:

[0301] Matching implementation

[0302] server:

[0303] It analyzes the generative model information in the database, runs an algorithm to pair generative models from different fields, generates a list of paired models, and notifies the user of the matching results.

[0304] Step 4:

[0305] Starting a project

[0306] User:

[0307] Receive a notification of the matching result, confirm the project details, and if you agree to start the project, configure the required settings such as the output format and output node.

[0308] server:

[0309] Receives configuration information from the user and prepares the project execution environment. Once preparations are complete, notifies the user that the project has started.

[0310] Step 5:

[0311] Implementing the generation process

[0312] Device:

[0313] The generation process is initiated based on the specified parameters on the device where the user's generative model is running. During this process, the emotion engine analyzes the user's facial expressions and voice in real time to collect emotion data.

[0314] server:

[0315] The progress of the generation process is monitored in real time, the generation results are stored in a database, and the output of the generative model is adjusted based on the emotion data obtained from the emotion engine.

[0316] Step 6:

[0317] Product evaluation

[0318] server:

[0319] We provide an interface for experts and general users to evaluate the generated models. We aggregate the evaluation results and recognize and reward excellent generative models and their developers.

[0320] User:

[0321] Through the rating interface, you can rate the product, leave comments, and view other users' ratings.

[0322] Step 7:

[0323] User-participation creative activities

[0324] User:

[0325] Ordinary users can access the platform and create their own works using the generative models provided. For example, they can select a music generation model to create a song, and then use a video generation model to create a video to go along with that song.

[0326] server:

[0327] It stores user-generated works in a database, provides an interface for sharing them with other users, and manages other users' comments on the shared works.

[0328] Examples:

[0329] User A registers an image generation model on the platform, and User B registers a music generation model. The server matches these models and proposes a project to jointly create an artwork. When User B creates music, the emotion engine analyzes User B's facial expressions in real time and adjusts the tempo and tone of the music according to their emotions. The final artwork is published on the platform and can be rated and commented on by other users.

[0330] In this way, the present invention not only increases the diversity and creativity of generative models, but also provides a personalized experience that responds to the user's emotions.

[0331] Example 2

[0332] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0333] In conventional generative modeling systems, collaboration and competition between generative models from different fields was difficult, limiting the diversity and creativity of the products they produced. Furthermore, they lacked methods for providing personalized products that respond to the user's emotions. This resulted in a uniform user experience that lacked individuality, reducing the appeal and value of the products.

[0334] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for registering and managing generative models from different fields, means for matching collaboration and competition between generative models, means for customizing products output by generative models based on user emotions, means for evaluating products output by generative models, and means for providing an interface for generative models and users to collaborate on creative activities. This facilitates collaboration and competition between generative models and makes it possible to provide personalized products that correspond to user emotions.

[0335] A "generative model" is a machine learning algorithm that generates new data or products for a specific purpose or use.

[0336] A "means for registration and management" is a system or interface that allows users to input detailed information about generative models and store and organize that information.

[0337] "Matching means" is a function that analyzes the features and characteristics of different generative models and performs pairings for optimal collaboration or competition.

[0338] The "means for customization" is a system that analyzes the user's emotions and adjusts the characteristics and parameters of the product based on those emotions.

[0339] The "means of evaluation" is a mechanism for evaluating the quality and suitability of the products output by the generative model based on feedback from users and experts.

[0340] The "interface for creative activities" is a user interface used when a generative model and a user collaborate on creative activities, and allows for adjustment of input and output.

[0341] This system aims to improve the diversity and creativity of generative models by enabling collaboration and competition between them in different fields. Characteristic features of the system include the registration and management of generative models, matching between generative models, evaluation of products, customization of products based on user emotions, and provision of an interface for collaborative creation.

[0342] System Configuration

[0343] The system consists of the following elements:

[0344] A server that registers and manages generative models

[0345] A server that performs matching between generative models

[0346] User device equipped with an emotion engine

[0347] A server that provides evaluation and award functions for products

[0348] An interface for collaborative creative activities between users and generative models

[0349] 1. Registering and Managing Generative Models

[0350] User:

[0351] Users register their generative models on the platform. When registering, an interface is provided for entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used). For example, users can enter data into the form using a web browser and then press the "Register" button to send the generative model information to the server.

[0352] server:

[0353] The server receives the generative model information sent by the user and stores it in a database. This storage enables centralized management of generative models. The received data is parsed into JSON format and stored in an SQL database such as MySQL.

[0354] 2. Matching the Generative Model and Starting the Project

[0355] server:

[0356] The server pairs generative models from different fields based on the registered generative model information, and performs matching for collaboration and competition. For example, k-means clustering is used as a matching algorithm. The matching results are notified to the user.

[0357] User:

[0358] After receiving the notification, the user checks the details of the project and makes the necessary settings (such as the format of the generated output, output nodes, etc.). For example, the user selects the project format and presses the "Start Project" button to start the project.

[0359] 3. Implementing the generation process

[0360] Device:

[0361] The device where the user's generative model runs starts the generation process based on the specified parameters. The device's emotion engine analyzes the user's facial and voice data to recognize the user's emotions. Examples of use cases include libraries such as OpenCV and TensorFlow.

[0362] server:

[0363] The server monitors the progress of the generation process in real time, stores the generation results in a database, and receives feedback from the emotion engine to adjust the output of the generative model. The generation results are stored in cloud storage, and the metadata is stored in a database.

[0364] 4. Product Evaluation and Awards

[0365] server:

[0366] The server provides an interface for evaluating the generated models and accepts evaluations from general users and experts. Evaluation scores and comments are stored in a database and periodically compiled. Based on this, an algorithm is run to reward excellent generative models and their developers, and the results are notified.

[0367] User:

[0368] Users can participate in the evaluation of the generated results and leave comments, which leads to improvements in the generative model and new ideas.

[0369] 5. User-participatory creative activities

[0370] User:

[0371] Ordinary users can access the platform and create their own works using the provided generative models. For example, they can create a song using the music generation model and then create a video to accompany that song using the video generation model.

[0372] server:

[0373] The server provides an interface for users to store their generated works and share them with other users, and also allows for further customization of the works based on data from the emotion engine.

[0374] Specific examples

[0375] For example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches the two, and users A and B start a project to jointly create an artwork (images and music). When user B creates music, the emotion engine analyzes user B's facial expressions in real time and adjusts the tempo and tone of the music according to their emotions. The final artwork created is published on the platform and receives ratings and comments from other users.

[0376] Example prompt

[0377] The following examples could be used as prompts for a generative AI model:

[0378] "Using image generation models and music generation models, we create artwork based on the user's emotions. First, user A generates a landscape painting using the image generation model. Based on that landscape painting, user B creates background music using the music generation model. The tempo and tone of the background music can be adjusted according to user B's emotions (joy, sadness, surprise, etc.)."

[0379] In this way, the system of the present invention increases the diversity and creativity of generative models, while at the same time providing a more personalized experience by recognizing the user's emotions and customizing the product.

[0380] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0381] Step 1: Registering the Generative Model

[0382] User:

[0383] The user enters detailed information about the generative model (such as the model name, genre, purpose, and dataset used) into the interface and presses the "Register" button. The input can be in text or file format.

[0384] Specific behavior:

[0385] Users access the registration page via a web browser or smartphone app and enter the required information. When the user presses the "Register" button, the entered information is sent to the server as an HTTP POST request.

[0386] input:

[0387] Detailed information about the generative model (model name, genre, purpose, dataset used, etc.).

[0388] output:

[0389] An HTTP POST request containing detailed information about the generated model.

[0390] Step 2: Managing the Generative Model

[0391] server:

[0392] The server receives the generative model information sent by the user, stores it in a database, and notifies the user when the storage is complete.

[0393] Specific behavior:

[0394] The server analyzes the data of the received HTTP POST request and parses it into JSON format. The analyzed data is saved in a database such as MySQL or PostgreSQL using an INSERT statement. Once the saving is complete, the user is notified via HTTP response or email.

[0395] input:

[0396] Detailed information about the generated model sent as an HTTP POST request.

[0397] output:

[0398] Information about the generated model saved in the database and notification of completion of saving.

[0399] Step 3: Matching the generative model

[0400] server:

[0401] The server matches generative models from different fields based on the generative model information stored in the database, using a matching algorithm such as k-means clustering.

[0402] Specific behavior:

[0403] The server runs a regularly scheduled job to retrieve generative model information from the database, and then runs the k-means clustering algorithm based on the retrieved dataset to determine matching pairs. The matching results are then sent to users via a notification system.

[0404] input:

[0405] A set of information about the generative model stored in a database.

[0406] output:

[0407] Informing generative model matching pairs across disciplines.

[0408] Step 4: Starting the project

[0409] User:

[0410] The user who receives the notification checks the project details, sets up the necessary settings (product format, output node, etc.), and clicks the start button for the project.

[0411] Specific behavior:

[0412] The user clicks on the notification to access the project details screen and make the necessary settings. Once the settings are complete, they press the "Start Project" button, which sends the project settings information to the server.

[0413] input:

[0414] Project configuration information (product format, output nodes, etc.).

[0415] output:

[0416] Project start request.

[0417] Step 5: Implementing the generation process

[0418] Device:

[0419] The user's generative model starts the generation process based on the specified parameters, and the emotion engine installed on the device analyzes the user's facial expressions and voice data to recognize emotions.

[0420] Specific behavior:

[0421] The device collects the user's facial and voice data using a camera and microphone, and analyzes it using an emotion engine (e.g., OpenCV or TensorFlow). Based on the analysis results, the parameters of the generative model are dynamically adjusted and the generation process is initiated. The generated results are stored locally or in the cloud.

[0422] input:

[0423] User emotion data (facial expressions, voice).

[0424] output:

[0425] The product based on the adjusted parameters.

[0426] server:

[0427] It monitors the progress of the generation process in real time, stores the generated results in a database, and receives feedback from the emotion engine to adjust the output of the generative model.

[0428] Specific behavior:

[0429] The server receives progress data from the device and updates the monitoring panel. Once generated, it uploads the generated product to cloud storage and stores the metadata in a database. It may also send regeneration commands to the generative model based on feedback from the emotion engine.

[0430] input:

[0431] Progress data for the generation process, and feedback for the emotion engine.

[0432] output:

[0433] Generation results and regeneration instructions stored in a database.

[0434] Step 6: Evaluate the product

[0435] server:

[0436] It provides an interface for evaluating the generated results, accepts evaluations from users and experts, and aggregates the evaluation results.

[0437] Specific behavior:

[0438] The server dynamically generates evaluation pages for products and provides an interface where users can enter evaluation scores and comments. The evaluation data from users and experts is stored in a database and periodically aggregated.

[0439] input:

[0440] Evaluation data (scores, comments) from users and experts.

[0441] output:

[0442] Aggregated evaluation results.

[0443] Step 7: Awarding the product

[0444] server:

[0445] Based on the evaluation results, an algorithm will be run to award outstanding generative models and their developers. The award results will be notified.

[0446] Specific behavior:

[0447] The server runs an algorithm to determine the recipients of the awards based on the aggregated results, and then notifies the selected generative models and developers of the awards via in-app messages and emails.

[0448] input:

[0449] Aggregated evaluation results.

[0450] output:

[0451] Notification of award recipients.

[0452] Step 8: User-participation creative activity

[0453] User:

[0454] Ordinary users access the platform and create their own works using the provided generative models. They then publish the works on the platform and receive ratings and comments from other users.

[0455] Specific behavior:

[0456] Users select a generative model and create a landscape painting using the image generation model. They then upload the landscape painting to the platform and set it to be public. The published work can then receive ratings and comments from other users.

[0457] input:

[0458] Artwork data based on generative models.

[0459] output:

[0460] Published generated works and their ratings and comments.

[0461] This clarifies the processing flow of the entire system, allowing for a detailed understanding of what inputs and outputs exist at each step and what specific operations are performed.

[0462] (Application example 2)

[0463] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0464] Modern generative AI models lack collaboration and competition to improve their diversity and creativity, and the generated content is rarely properly tailored to the user's emotions, resulting in insufficient personalization of the user experience.

[0465] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0466] In this invention, the server includes a means for registering and managing generative models from different fields, a means for matching collaboration and competition between generative models, a means for evaluating the products output by the generative models, a means for providing an interface for collaborative creative activities between the generative models and users, and a means for recognizing the user's emotions and adjusting the products based on those emotions. This increases the diversity and creativity of generative AI models and makes it possible to provide personalized products according to the user's emotions.

[0467] "Generative models from different fields" is a general term for AI models that operate in different genres or applications, such as image generation models and music generation models.

[0468] "Means for registration and management" refers to an interface or program that provides the functionality to collect, classify, and store information about generative models on a server.

[0469] "Means for matching collaboration and competition" refers to algorithms or programs that pair different generative models together to enable them to collaborate or compete.

[0470] "Means for evaluating the product" refers to an interface or system for analyzing and evaluating the results output by the generative model.

[0471] An "interface for users to engage in collaborative creative activities" is a system that provides a user interface and tools that allow users to create content in collaboration with generative models.

[0472] "Means for recognizing emotions and adjusting the generated content based on those emotions" refers to algorithms or programs that analyze the user's emotions in real time using a camera or microphone, and change or adjust the generated content based on the results.

[0473] This invention is a system that allows generative AI models from different fields to collaborate and compete with each other, improving their diversity and creativity. This system incorporates an emotion engine that recognizes the user's emotions, allowing the output of the generative model to be customized according to the user's emotions.

[0474] System Configuration

[0475] 1. Register and manage generative models:

[0476] Users register their generative models on the platform. When registering, an interface is provided for entering detailed information about the generative model (model name, genre, purpose, dataset used, etc.). The server receives this information and stores it in a database, enabling centralized management.

[0477] 2. Matching different generative models:

[0478] The server pairs generative models from different fields based on the registered generative model information, and performs matching for collaboration and competition. The matching results are notified to the user, who can then start the project by checking the project details and making the necessary settings.

[0479] 3. Implementing the generation process:

[0480] The device running the generative model starts the generation process based on specified parameters. The emotion engine analyzes the user's facial expressions and voice data to recognize emotions. The server monitors the progress of the generation process in real time and stores the generation results in a database. It also receives feedback from the emotion engine and adjusts the output of the generative model as appropriate.

[0481] 4. Product Evaluation and Awards:

[0482] The server provides an interface for experts and general users to evaluate the generated models, and has a mechanism to aggregate the evaluation results and recognize and award excellent generative models and their developers.

[0483] 5. User-participatory creative activities:

[0484] Ordinary users can access the platform and create their own works using generative models. For example, a user can choose a music generation model, create a song, and then create a video using a video generation model to accompany the song. The server stores the generated works and provides an interface for sharing them with other users.

[0485] Hardware and software used

[0486] Camera: Used to capture the user's facial expressions.

[0487] Microphone: Used to capture the user's voice.

[0488] OpenCV: An image processing library used to acquire and process camera images.

[0489] Librosa: An audio processing library for reading and analyzing audio files.

[0490] Transformers: A library that implements natural language processing (NLP) models for emotion recognition.

[0491] Specific examples

[0492] For example, suppose a user uses the "Collabo Creator" app to register a landscape image generation model and a classical music generation model. Once the generative models are matched, the app performs real-time emotion analysis using the user's camera and microphone. If the user is moved, the app can slow down the tempo of the generated music and add a sunset element to the generated scenery. This content can be shared with other users and rated.

[0493] Prompt Sentence Examples

[0494] "We combine user-input image generation models with music generation models, recognize emotions in real time using a camera and microphone, and customize generated content based on emotions."

[0495] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0496] Step 1:

[0497] The user registers a generative model on the platform. The user inputs detailed information about the generative model (model name, genre, purpose, dataset used, etc.) and sends it to the server through the interface. The input is the detailed information about the generative model, and the output is information about the generative model stored in the server's database.

[0498] Step 2:

[0499] The server pairs generative models from different fields based on the registered generative model information, and performs matching for collaboration and competition. The input is the information of the generative models stored in the database, and the output is the combination information of the paired generative models.

[0500] Step 3:

[0501] The server notifies the user of the matching results. The user receives the notification, checks the project details, and makes the necessary settings (such as the format of the generated output, output nodes, etc.). The input at this time is the matching results and the user's settings information, and the output is an instruction to start the project.

[0502] Step 4:

[0503] The project generation process begins on the device. The emotion engine captures the user's facial expressions and voice data in real time via the camera and microphone, and recognizes emotions. The input is the user's facial expression data and voice data, and the output is the recognized emotion information. In the processing step, OpenCV analyzes facial expressions, and Librosa processes the voice to extract emotions.

[0504] Step 5:

[0505] The server receives feedback from the emotion engine and adjusts the output of the generative model. The input is the recognized emotion information, and the output is an emotion-adjusted artifact (e.g., music tempo or image color tone). The processing step involves retraining or adjusting the generative model in real time with parameters corresponding to the emotion information.

[0506] Step 6:

[0507] The generated artifacts are sent to a server and stored in a database. The input is the emotion-adjusted artifact, and the output is the artifact stored in the database. The processing step involves storing the artifacts in appropriate storage and managing them in an easily accessible format.

[0508] Step 7:

[0509] The server provides an interface that allows experts and general users to evaluate the products. The input is the saved products, and the output is the evaluation data and the evaluation results. In the processing step, the evaluation interface is provided and the user evaluations are aggregated.

[0510] Step 8:

[0511] Users can participate in the evaluation of the product and leave comments. The input is the product and the evaluation interface, and the output is evaluation points and comments. In the processing step, user feedback is collected and the generative model is improved based on the evaluation results.

[0512] Step 9:

[0513] The server recognizes the developers of excellent products and generative models and notifies the user. The input is the evaluation result, and the output is the recognition notice and recognition list. In the processing step, the server analyzes the evaluation result, selects excellent products, and issues a recognition notice.

[0514] Step 10:

[0515] Users can share their work with other users. The input is the generated work, and the output is a sharing link or a public page. In the processing step, a link is generated or a public page is created to share the work with other users.

[0516] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0517] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0518] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0519] [Second embodiment]

[0520] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0521] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0522] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0523] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0524] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0525] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0526] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0527] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0528] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0529] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0530] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0531] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0532] This invention is a system that improves the diversity and creativity of generative models through collaboration and competition between them in different fields. This system has functions for registering and managing generative models, matching between generative models, evaluating the results, and realizing collaborative creation with users.

[0533] 1. Registering and Managing Generative Models

[0534] User:

[0535] Users register generative models on the platform. For example, an interface is provided for entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used). After registration is complete, users can share their generative models with other users.

[0536] server:

[0537] The server receives information about the generative model sent by the user and stores it in a database, allowing management of the characteristics of the generative model (performance, genre, training data, etc.).

[0538] 2. Matching and project launch

[0539] server:

[0540] The server automatically matches models from different fields based on the registered generative models. For example, it pairs a music generation model with a video generation model. The matching results are notified to the user, who can then start a collaboration project.

[0541] User:

[0542] Upon receiving the notification, the user can start the project by checking the project details and making the necessary settings (such as the format of the generated output and output nodes).

[0543] 3. Implementing the generation process

[0544] Device:

[0545] The generative model runs on the user's device and executes the generation process based on the specified parameters. For example, a music generation model generates a unique piece of music, and a video generation model creates a video to go along with it.

[0546] server:

[0547] The server monitors the progress of the generation process in real time, stores the generated results, and provides feedback to the user if necessary.

[0548] 4. Product Evaluation and Awards

[0549] server:

[0550] We will hold regular generative AI contests, where submitted products will be evaluated by experts and general users. The evaluation results will be compiled, and outstanding generative models and their developers will be awarded prizes.

[0551] User:

[0552] Users who participate in the contest can submit their creations and rate and comment on other creations, allowing for more diverse feedback.

[0553] 5. User-participatory creative activities

[0554] User:

[0555] Ordinary users can also access the platform and use generative models to create their own works. For example, a user can create a song using a music generation model and then create a video to accompany that song using a video generation model.

[0556] server:

[0557] The server provides an interface for users to store their generated work and share it with other users.

[0558] As a concrete example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches these models and proposes a project to jointly generate artworks (images + music). Users A and B generate artworks using their respective models, and the generated artworks are published on the platform and evaluated by other users.

[0559] In this way, the present invention promotes interaction between generative AI models, enabling the exploration of diverse expression techniques and improving the quality of the products.

[0560] The processing flow will be explained below.

[0561] Step 1:

[0562] Registering a Generative Model

[0563] User:

[0564] Enter detailed information about your generative model (model name, genre, purpose, dataset used, etc.) into the registration form and submit it.

[0565] server:

[0566] The submitted generative model data is received and saved in the database. Once saved, a notification is sent to the user to notify them of the completion of registration.

[0567] Step 2:

[0568] Managing Generative Models

[0569] server:

[0570] It updates the Generative Model information in the database and provides a user-accessible management screen that displays a list of registered Generative Models and their respective characteristics.

[0571] Step 3:

[0572] Matching implementation

[0573] server:

[0574] Analyzes the generative model information in the database and runs an algorithm to pair generative models from different fields for collaboration or competition. Generates matching results and notifies the user.

[0575] Step 4:

[0576] Starting a project

[0577] User:

[0578] You will receive a notification of the matching result, check the details, and if you agree to start the project, make the necessary settings (product format, output node, etc.).

[0579] server:

[0580] Receives configuration information from the user and prepares the project execution environment.

[0581] Step 5:

[0582] Implementing the generation process

[0583] Device:

[0584] On the device where the user's generative model runs, the generative process is started based on the specified parameters, and an artifact (e.g., music or video) is created.

[0585] server:

[0586] Monitor the progress of the generation process in real time, store the generated results in a database, review the feedback and notify the user if necessary.

[0587] Step 6:

[0588] Product evaluation

[0589] server:

[0590] We provide an interface for experts and general users to evaluate the generated models, and we aggregate the evaluation results and award outstanding generative models.

[0591] User:

[0592] You can participate in product evaluations and leave comments. You can also view other users' evaluations.

[0593] Step 7:

[0594] User-participation creative activities

[0595] User:

[0596] Ordinary users can access the platform and use generative models to create their own works. For example, they can select a music generation model to create a song, and then use a video generation model to create a video that matches the song.

[0597] server:

[0598] It provides an interface for users to save and share their creations with other users, and also manages other users' comments on shared creations.

[0599] This allows for greater diversity and creativity in generative models.

[0600] Example 1

[0601] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0602] In conventional generative AI model systems, it was difficult for generative models from different fields to efficiently collaborate and produce diverse creative works. Furthermore, there was a lack of mechanisms for evaluating the generated results, and insufficient feedback on improving the quality of the generative models was also an issue. Furthermore, the hurdles for general users to use generative AI models for creative activities were high, making it difficult to produce diverse creative works. A system that can solve these issues and bring out more creativity and diversity is needed.

[0603] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0604] In this invention, the server includes means for registering and managing generative AI models, means for matching collaboration and competition between generative AI models, means for evaluating the products output by the generative AI models, means for providing an interface for generative AI models and users to collaborate on creative activities, means for receiving detailed information about the generative AI models and saving it in a database, means for automatically matching generative AI models of different genres and notifying users, and means for monitoring the progress of the generation process in real time and saving the generation results. This allows generative models from different fields to collaborate efficiently, improve the quality of generative models through evaluation and feedback, and enable general users to easily engage in creative activities.

[0605] A "generative AI model" is an algorithm that is trained to perform a specific problem or task using artificial intelligence techniques.

[0606] "Registration and management" refers to the process of registering the details of a generative AI model on the platform, storing that information in the database, and updating or deleting it as needed.

[0607] "Collaboration and competition" refers to a situation in which different generative AI models work together to collaborate or compete to create products.

[0608] "Matching" is the process of automatically selecting and linking generative AI models with different genres and characteristics.

[0609] "Products" are the results output by generative AI models, and can take various forms such as music, video, images, and text.

[0610] "Evaluation" refers to the act of experts and general users scoring and commenting on the quality and performance of the products output by a generative AI model.

[0611] "Interface" means the user interface and functionality that allows a user to interact with a generative AI model and engage in collaborative creative activities.

[0612] A "database" is a software system for efficiently storing and managing data such as detailed information about generative AI models and generated results.

[0613] "Notifications" are messages that convey information to users, such as the matching results of the generative AI model and the progress of the generation project.

[0614] "Progress monitoring" means checking in real time whether the generation process is proceeding as planned or whether any abnormalities have occurred.

[0615] "Storage" refers to the act of recording the output product of the generation process and the data generated during that process in a database or storage.

[0616] This invention is a system that improves the diversity and creativity of generative AI models by allowing them to collaborate and compete with each other in different fields. This system has functions for registering and managing generative AI models, matching between generative AI models, evaluating the results, and realizing collaborative creation with users.

[0617] Specific actions

[0618] Users register generative AI models on the platform. For example, an interface is provided for entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used). When registering a "music generative model," users enter information such as "music generative model A," "music," "music production," and "music dataset 1."

[0619] The server receives information about the generative AI model sent by the user and stores it in a database. This allows the server to manage the characteristics of the generative AI model (performance, genre, training data, etc.). For example, a record such as "Music Generative Model A, Music, Music Production, Music Dataset 1" is registered in the database.

[0620] The server automatically matches models from different fields based on information about generative AI models registered in the database. For example, it might select a pair of "music generation model A" and "video generation model B." It then proposes a new project for the selected pair and notifies the user.

[0621] After receiving the notification, the user checks the details of the proposed project and makes the necessary settings (such as the format of the generated output and output nodes) on the user interface. For example, they can set the length of the music to be generated and the video resolution. Once the settings are complete, they press the "Start Project" button.

[0622] The generative AI model runs on the device and executes the generation process based on the specified parameters. For example, a music generation model generates a unique song, and a video generation model creates a video in conjunction with it.

[0623] The server monitors the progress of the generation process in real time and saves the generated results. After the generation is complete, it saves the results in a database and notifies the user of the results. Specifically, it saves the generated music and video files and sends a notification to the user.

[0624] The server will hold a generative AI contest and evaluate the products submitted by users. It will compile evaluations from experts and general users and award outstanding generative models and their developers. For example, if "Music Generation Model A" receives a high evaluation based on the evaluation criteria, the results will be made public.

[0625] Ordinary users can access the platform and use generative AI models to create their own works. For example, a user can create a song using a music generation model and then create a video to go with that song using a video generation model.

[0626] The server stores user-generated works and provides an interface for sharing them with other users, e.g., user-generated music and videos are shared so that other users can view and rate them.

[0627] Examples and prompts

[0628] For example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches these models and proposes a project to jointly generate an artwork (image + music). Users A and B create artwork using their respective models, and the resulting artwork is published on the platform and evaluated by other users.

[0629] The following can be used as a specific prompt:

[0630] "Please run a program that generates a landscape image and generates background music that matches that image. We will use 'Landscape Image Generation Model A' as the image generation model and 'Music Generation Model A' as the background music generation model."

[0631] In this way, the present invention promotes interaction between generative AI models, enabling the exploration of diverse expression methods and improving the quality of the products.

[0632] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0633] Step 1:

[0634] Registering a generative AI model

[0635] Users register generative AI models on the platform by accessing a dedicated interface and entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used).

[0636] Input: Detailed information about the generative model (model name, genre, purpose, dataset used, etc.)

[0637] Output: Send registration application data

[0638] Specific operation: The user enters information such as "Music Generation Model A," "Music," "Music Production," and "Music Dataset 1" into the interface and presses the registration button.

[0639] Step 2:

[0640] Receiving and saving generated AI model registration information

[0641] The server receives the registration information for the generative AI model sent by the user and stores it in a database.

[0642] Input: Registration information for the generative model from the user

[0643] Output: Record added to database

[0644] Specific operation: The server receives the information "Music generation model A, music, music production, music dataset 1" from the user and stores it in the database.

[0645] Step 3:

[0646] Matching between generative models

[0647] The server automatically matches models from different fields based on information about generative AI models registered in the database.

[0648] Input: Information about the generative AI model stored in the database

[0649] Output: Matching result notification

[0650] Specific operation: The server selects a pair of "music generation model A" and "video generation model B" and sends a "project proposal" notification to the user.

[0651] Step 4:

[0652] Project details

[0653] The user receives a notification, reviews the details of the proposed project, and makes any necessary configurations (such as artifact format and output nodes).

[0654] Input: Project proposal notification, configuration information (format, output node, etc.)

[0655] Output: Sending a notification that the setup is complete

[0656] Specific operations: The user sets the length of the music to be generated, the resolution of the video, etc., and presses the "Start Project" button.

[0657] Step 5:

[0658] Implementing the generation process

[0659] The generative AI model runs on the device and executes the generation process based on the specified parameters.

[0660] Input: Setting parameters (song length, video resolution, etc.)

[0661] Output: Generated products (music files, video files)

[0662] Specific operation: "Music generation model A" generates a five-minute piece of music on User A's device, and at the same time, "video generation model B" generates a video linked to the music on User B's device.

[0663] Step 6:

[0664] Product monitoring and preservation

[0665] The server monitors the progress of the generation process in real time and stores the generation results.

[0666] Input: Progress data of the generation process

[0667] Output: Save generated results to database, progress notification

[0668] Specific operation: After generation is complete, the generated music and video files are saved in a database and the user is notified of the results.

[0669] Step 7:

[0670] Product evaluation

[0671] The server hosts a generative AI contest and performs a process to evaluate the products submitted by users.

[0672] Input: Submitted product

[0673] Output: Evaluation results summary, award notification

[0674] Specific operation: We will aggregate evaluations from experts and general users and award outstanding generative models and their developers.

[0675] Step 8:

[0676] User-participation creative activities

[0677] Ordinary users can also access the platform and use generative AI models to create their own unique works.

[0678] Input: Creation request using generative model

[0679] Output: Generated creative works (music, video, etc.)

[0680] Specific operation: The user creates a song using the music generation model and a video using the video generation model.

[0681] Step 9:

[0682] Sharing the results

[0683] The server provides an interface for users to store their generated work and share it with other users.

[0684] Input: Generated artwork

[0685] Output: Provides an interface for sharing

[0686] Specific operation: The generated music and video will be published on the platform and can be rated by other users.

[0687] (Application example 1)

[0688] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0689] Conventional generative AI models have focused on outputting products specialized in a single field, making it difficult for generative models from different fields to collaborate with each other or for users to participate in the generation process in real time. Furthermore, there is a lack of a mechanism for effectively utilizing evaluations and feedback of products to update the generated content, which limits the quality and diversity of the products.

[0690] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0691] In this invention, the server includes a means for registering and managing generative models from different fields, a means for matching collaboration and competition between generative models, a means for evaluating the products output by the generative models, a means for providing an interface for the generative models and users to engage in collaborative creative activities, and a means for linking different generative models to generate and view interactive content. This enables the linking of generative models from different fields to generate diverse, high-quality interactive content and user participation in real time.

[0692] A "generative model" is an algorithm that generates new data or content based on a specific dataset or algorithm.

[0693] “Registration and Management” refers to the process of adding a Generative Model to the Platform, storing its information, and updating or modifying it as needed.

[0694] "Collaboration" refers to the activity of different generative models working together to jointly produce a single product.

[0695] "Competition" refers to the process in which different generative models compete against each other in terms of their performance and output.

[0696] "Matching" refers to the process of selecting the optimal combination based on the characteristics and parameters of different generative models.

[0697] "Interactive content" refers to content whose generated content changes dynamically as users operate and provide feedback in real time.

[0698] "Interface" refers to the screen or tool that allows the user to input and operate the generative model.

[0699] "Evaluation" refers to the activity of diagnosing the product output by a generative model based on its quality, performance, etc., and providing the results as feedback.

[0700] "Real-time monitoring" refers to the process of instantly checking the progress of the generation process and maintaining a state in which necessary responses can be taken immediately.

[0701] "Preservation" refers to the process of securely storing generated data and content for later use and evaluation.

[0702] "Feedback" refers to the process of returning evaluations of the product and suggestions for improvement to the model.

[0703] This invention provides a system that links generative models from different fields to enable the generation and viewing of interactive content. An embodiment of this system is described in detail below.

[0704] 1. System Configuration

[0705] Server: The server contains the following main means:

[0706] Generative model management

[0707] Generative model matching method

[0708] Product evaluation method

[0709] Interface provision means

[0710] Real-time monitoring and feedback measures

[0711] To achieve these goals, the server uses high-performance computing hardware and a web framework such as Flask.

[0712] User device: Users access the system using a smartphone or head-mounted display (HMD) to generate and view content. A dedicated app for operating and linking different generative models is installed on the smartphone.

[0713] Generative Models: Generative models are algorithms specialized for a specific domain, such as music generation models or video generation models. They are hosted on a server and accessed via API endpoints.

[0714] 2. Operation overview

[0715] Users use a smartphone app to register generative models on the platform. After entering model information (model name, genre, purpose, dataset used, etc.), the model is saved on the server. The registered generative model is managed by the server and updated as needed.

[0716] Next, the server automatically matches models from different fields based on the characteristics of the registered generative models. For example, a music generation model and a video generation model are paired and notified to the user. The user can then start the project by checking the project details and configuring the necessary settings.

[0717] 3. Content Creation and Viewing

[0718] The user inputs a prompt to generate interactive content, for example:

[0719] Example prompt:

[0720] Input to the music generation model: "Generate a fast-paced, electronic-feeling soundtrack."

[0721] Input to the video generation model: "Generate a video with a fast camera movement against a futuristic cityscape."

[0722] Based on this, the server calls the music generation model and video generation model, respectively, and performs generation. The generated music and video are sent to the user's device in real time and can be viewed on a smartphone or HMD. Based on user feedback, the server updates the generated content as appropriate and advances the generation process.

[0723] The generated content is stored on the server and can be shared and rated by other users. The evaluation results are aggregated and outstanding generative models and their developers are awarded.

[0724] This allows generative models from different fields to work together, enabling the generation of diverse, high-quality interactive content and real-time user participation.

[0725] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0726] Step 1: Registering the Generative Model

[0727] Users use a smartphone app to register a generative model on the platform. Input includes detailed information about the generative model (model name, genre, purpose, dataset used, etc.). These details are sent to the server and stored in a database. As an output, a message is sent to the user informing them that the generative model has been registered.

[0728] Step 2: Managing the Generative Model

[0729] The server manages information about registered generative models. It references the characteristics of the generative models stored in the database (performance, genre, training data, etc.) and updates and corrects them as needed. This management process periodically checks the information about the generative models to keep them up to date. As an output, a report containing the latest information about the generative models is generated.

[0730] Step 3: Matching the generative model

[0731] The server automatically matches models from different fields based on the characteristics of the registered generative models. For example, it analyzes features to find the optimal combination to pair a music generation model with a video generation model. The characteristic information of each generative model is used as input, and the matching results are reported to the user as output.

[0732] Step 4: Starting the project

[0733] Upon receiving the notification, the user checks the project details and makes the necessary settings (such as the product format and output nodes). The input includes the matched generative model and project details. As an output, a project start confirmation message is sent to the server, and the project is launched.

[0734] Step 5: Implementing the generation process

[0735] When a project starts, the user inputs a prompt, for example, "Please generate a soundtrack with a fast tempo and an electronic feel." The server receives this prompt and sends generation requests to the music generation model and the video generation model. The generative models perform the generation work based on the specified parameters and return the output results (music and video) to the server.

[0736] Step 6: Integrating the products

[0737] The server receives the generated results (music and video) sent from the generative model. To integrate these generated results, the server adjusts the timing and coding of the music and video. The input includes the music generation results and the video generation results, and interactive content is generated as the output.

[0738] Step 7: Interactive viewing

[0739] The generated interactive content is delivered to the user device in real time. The user views the content using a smartphone or HMD and sends feedback to the server. The input includes the user's viewing data and feedback, and the output is an instruction to update the generated content.

[0740] Step 8: Update the generation process

[0741] The server updates the generated content based on user feedback. It analyzes the generated content for improvement and sends new parameters to the generative model for regeneration. This process aims to improve the quality of the product and increase user satisfaction. The input includes user feedback data, and the output is the improved product that is redistributed.

[0742] Step 9: Rate and share your creation

[0743] The final generated content is stored on a server and published on the platform for general users to evaluate. Users can rate and comment on other creations and provide their own feedback. The input includes evaluation data, and the output is the evaluation results that are fed back to the developer of the generative model.

[0744] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0745] The present invention is a system that aims to improve the diversity and creativity of generative models from different fields by allowing them to collaborate and compete with each other. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions, allowing the output of the generative model to be customized according to the user's emotions. An embodiment of this system is described in detail below.

[0746] System Configuration

[0747] The system mainly consists of the following components:

[0748] A server that registers and manages generative models

[0749] A server that performs matching between generative models

[0750] User device equipped with an emotion engine

[0751] A server that provides evaluation and award functions for products

[0752] An interface for collaborative creative activities between users and generative models

[0753] 1. Registering and Managing Generative Models

[0754] User:

[0755] Users register generative models on the platform. When registering, they are provided with an interface to input detailed information about the generative model (such as the model name, genre, purpose, and dataset used).

[0756] server:

[0757] The server receives the information about the generative model sent by the user and stores it in a database, allowing for centralized management of the characteristic information of the generative model.

[0758] 2. Matching and project launch

[0759] server:

[0760] The server pairs generative models from different fields based on the registered generative model information, and conducts matching for collaboration and competition. The matching results are notified to the user.

[0761] User:

[0762] The user who receives the notification checks the details of the project and makes the necessary settings (format of the generated output, output node, etc.), which starts the project.

[0763] 3. Implementing the generation process

[0764] Device:

[0765] The generation process is initiated based on the specified parameters on the device where the user's generative model is running. The emotion engine has the ability to recognize emotions by analyzing the user's facial expressions and voice data.

[0766] server:

[0767] The server monitors the progress of the generation process in real time, stores the generated results in a database, and receives feedback from the emotion engine to adjust the output of the generative model.

[0768] 4. Product Evaluation and Awards

[0769] server:

[0770] The server provides an interface for experts and general users to evaluate the generated models, and aggregates the evaluation results to recognize and reward outstanding generative models and their developers.

[0771] User:

[0772] Users can participate in rating the products and leave comments, which provides feedback for improvements to the generative model and new ideas.

[0773] 5. User-participatory creative activities

[0774] User:

[0775] Ordinary users can access the platform and create their own works using generative models. For example, a user can select a music generation model, create a song, and then create a video to go with the song using a video generation model.

[0776] server:

[0777] The server provides an interface for users to store and share their generated works with other users, and can further customize the creations based on data from the emotion engine.

[0778] Specific examples

[0779] For example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches the two, and users A and B start a project to jointly create an artwork (image + music). When user B creates music, the emotion engine analyzes user B's facial expressions in real time and adjusts the tempo and tone of the music in real time according to the emotion. The final artwork created is published on the platform and receives ratings and comments from other users.

[0780] In this way, the present invention increases the diversity and creativity of generative models, while at the same time providing a more personalized experience by recognizing the user's emotions and customizing the product.

[0781] The processing flow will be explained below.

[0782] Step 1:

[0783] Registering a Generative Model

[0784] User:

[0785] The user accesses the registration form for the generative model and enters detailed information about the generative model (model name, genre, purpose, dataset used, etc.). By clicking the submit button, the information is sent to the server.

[0786] server:

[0787] The submitted generative model data is received and saved in the database. Once saved, a notification of registration completion is sent to the user.

[0788] Step 2:

[0789] Managing Generative Models

[0790] server:

[0791] The system periodically updates the information about generative models in the database and provides a user-accessible management screen that displays a list of registered generative models and their respective characteristics.

[0792] User:

[0793] You can view and edit information about registered generative models through the management screen.

[0794] Step 3:

[0795] Matching implementation

[0796] server:

[0797] It analyzes the generative model information in the database, runs an algorithm to pair generative models from different fields, generates a list of paired models, and notifies the user of the matching results.

[0798] Step 4:

[0799] Starting a project

[0800] User:

[0801] Receive a notification of the matching result, confirm the project details, and if you agree to start the project, configure the required settings such as the output format and output node.

[0802] server:

[0803] Receives configuration information from the user and prepares the project execution environment. Once preparations are complete, notifies the user that the project has started.

[0804] Step 5:

[0805] Implementing the generation process

[0806] Device:

[0807] The generation process is initiated based on the specified parameters on the device where the user's generative model is running. During this process, the emotion engine analyzes the user's facial expressions and voice in real time to collect emotion data.

[0808] server:

[0809] The progress of the generation process is monitored in real time, the generation results are stored in a database, and the output of the generative model is adjusted based on the emotion data obtained from the emotion engine.

[0810] Step 6:

[0811] Product evaluation

[0812] server:

[0813] We provide an interface for experts and general users to evaluate the generated models. We aggregate the evaluation results and recognize and reward excellent generative models and their developers.

[0814] User:

[0815] Through the rating interface, you can rate the product, leave comments, and view other users' ratings.

[0816] Step 7:

[0817] User-participation creative activities

[0818] User:

[0819] Ordinary users can access the platform and create their own works using the generative models provided. For example, they can select a music generation model to create a song, and then use a video generation model to create a video to go along with that song.

[0820] server:

[0821] It stores user-generated works in a database, provides an interface for sharing them with other users, and manages other users' comments on the shared works.

[0822] Examples:

[0823] User A registers an image generation model on the platform, and User B registers a music generation model. The server matches these models and proposes a project to jointly create an artwork. When User B creates music, the emotion engine analyzes User B's facial expressions in real time and adjusts the tempo and tone of the music according to their emotions. The final artwork is published on the platform and can be rated and commented on by other users.

[0824] In this way, the present invention not only increases the diversity and creativity of generative models, but also provides a personalized experience that responds to the user's emotions.

[0825] Example 2

[0826] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0827] In conventional generative modeling systems, collaboration and competition between generative models from different fields was difficult, limiting the diversity and creativity of the products they produced. Furthermore, they lacked methods for providing personalized products that respond to the user's emotions. This resulted in a uniform user experience that lacked individuality, reducing the appeal and value of the products.

[0828] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for registering and managing generative models from different fields, means for matching collaboration and competition between generative models, means for customizing products output by generative models based on user emotions, means for evaluating products output by generative models, and means for providing an interface for generative models and users to collaborate on creative activities. This facilitates collaboration and competition between generative models and makes it possible to provide personalized products that correspond to user emotions.

[0829] A "generative model" is a machine learning algorithm that generates new data or products for a specific purpose or use.

[0830] A "means for registration and management" is a system or interface that allows users to input detailed information about generative models and store and organize that information.

[0831] "Matching means" is a function that analyzes the features and characteristics of different generative models and performs pairings for optimal collaboration or competition.

[0832] The "means for customization" is a system that analyzes the user's emotions and adjusts the characteristics and parameters of the product based on those emotions.

[0833] The "means of evaluation" is a mechanism for evaluating the quality and suitability of the products output by the generative model based on feedback from users and experts.

[0834] The "interface for creative activities" is a user interface used when a generative model and a user collaborate on creative activities, and allows for adjustment of input and output.

[0835] This system aims to improve the diversity and creativity of generative models by enabling collaboration and competition between them in different fields. Characteristic features of the system include the registration and management of generative models, matching between generative models, evaluation of products, customization of products based on user emotions, and provision of an interface for collaborative creation.

[0836] System Configuration

[0837] The system consists of the following elements:

[0838] A server that registers and manages generative models

[0839] A server that performs matching between generative models

[0840] User device equipped with an emotion engine

[0841] A server that provides evaluation and award functions for products

[0842] An interface for collaborative creative activities between users and generative models

[0843] 1. Registering and Managing Generative Models

[0844] User:

[0845] Users register their generative models on the platform. When registering, an interface is provided for entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used). For example, users can enter data into the form using a web browser and then press the "Register" button to send the generative model information to the server.

[0846] server:

[0847] The server receives the generative model information sent by the user and stores it in a database. This storage enables centralized management of generative models. The received data is parsed into JSON format and stored in an SQL database such as MySQL.

[0848] 2. Matching the Generative Model and Starting the Project

[0849] server:

[0850] The server pairs generative models from different fields based on the registered generative model information, and performs matching for collaboration and competition. For example, k-means clustering is used as a matching algorithm. The matching results are notified to the user.

[0851] User:

[0852] After receiving the notification, the user checks the details of the project and makes the necessary settings (such as the format of the generated output, output nodes, etc.). For example, the user selects the project format and presses the "Start Project" button to start the project.

[0853] 3. Implementing the generation process

[0854] Device:

[0855] The device where the user's generative model runs starts the generation process based on the specified parameters. The device's emotion engine analyzes the user's facial and voice data to recognize the user's emotions. Examples of use cases include libraries such as OpenCV and TensorFlow.

[0856] server:

[0857] The server monitors the progress of the generation process in real time, stores the generation results in a database, and receives feedback from the emotion engine to adjust the output of the generative model. The generation results are stored in cloud storage, and the metadata is stored in a database.

[0858] 4. Product Evaluation and Awards

[0859] server:

[0860] The server provides an interface for evaluating the generated models and accepts evaluations from general users and experts. Evaluation scores and comments are stored in a database and periodically compiled. Based on this, an algorithm is run to reward excellent generative models and their developers, and the results are notified.

[0861] User:

[0862] Users can participate in the evaluation of the generated results and leave comments, which leads to improvements in the generative model and new ideas.

[0863] 5. User-participatory creative activities

[0864] User:

[0865] Ordinary users can access the platform and create their own works using the provided generative models. For example, they can create a song using the music generation model and then create a video to accompany that song using the video generation model.

[0866] server:

[0867] The server provides an interface for users to store their generated works and share them with other users, and also allows for further customization of the works based on data from the emotion engine.

[0868] Specific examples

[0869] For example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches the two, and users A and B start a project to jointly create an artwork (images and music). When user B creates music, the emotion engine analyzes user B's facial expressions in real time and adjusts the tempo and tone of the music according to their emotions. The final artwork created is published on the platform and receives ratings and comments from other users.

[0870] Example prompt

[0871] The following examples could be used as prompts for a generative AI model:

[0872] "Using image generation models and music generation models, we create artwork based on the user's emotions. First, user A generates a landscape painting using the image generation model. Based on that landscape painting, user B creates background music using the music generation model. The tempo and tone of the background music can be adjusted according to user B's emotions (joy, sadness, surprise, etc.)."

[0873] In this way, the system of the present invention increases the diversity and creativity of generative models, while at the same time providing a more personalized experience by recognizing the user's emotions and customizing the product.

[0874] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0875] Step 1: Registering the Generative Model

[0876] User:

[0877] The user enters detailed information about the generative model (such as the model name, genre, purpose, and dataset used) into the interface and presses the "Register" button. The input can be in text or file format.

[0878] Specific behavior:

[0879] Users access the registration page via a web browser or smartphone app and enter the required information. When the user presses the "Register" button, the entered information is sent to the server as an HTTP POST request.

[0880] input:

[0881] Detailed information about the generative model (model name, genre, purpose, dataset used, etc.).

[0882] output:

[0883] An HTTP POST request containing detailed information about the generated model.

[0884] Step 2: Managing the Generative Model

[0885] server:

[0886] The server receives the generative model information sent by the user, stores it in a database, and notifies the user when the storage is complete.

[0887] Specific behavior:

[0888] The server analyzes the data of the received HTTP POST request and parses it into JSON format. The analyzed data is saved in a database such as MySQL or PostgreSQL using an INSERT statement. Once the saving is complete, the user is notified via HTTP response or email.

[0889] input:

[0890] Detailed information about the generated model sent as an HTTP POST request.

[0891] output:

[0892] Information about the generated model saved in the database and notification of completion of saving.

[0893] Step 3: Matching the generative model

[0894] server:

[0895] The server matches generative models from different fields based on the generative model information stored in the database, using a matching algorithm such as k-means clustering.

[0896] Specific behavior:

[0897] The server runs a regularly scheduled job to retrieve generative model information from the database, and then runs the k-means clustering algorithm based on the retrieved dataset to determine matching pairs. The matching results are then sent to users via a notification system.

[0898] input:

[0899] A set of information about the generative model stored in a database.

[0900] output:

[0901] Informing generative model matching pairs across disciplines.

[0902] Step 4: Starting the project

[0903] User:

[0904] The user who receives the notification checks the project details, sets up the necessary settings (product format, output node, etc.), and clicks the start button for the project.

[0905] Specific behavior:

[0906] The user clicks on the notification to access the project details screen and make the necessary settings. Once the settings are complete, they press the "Start Project" button, which sends the project settings information to the server.

[0907] input:

[0908] Project configuration information (product format, output nodes, etc.).

[0909] output:

[0910] Project start request.

[0911] Step 5: Implementing the generation process

[0912] Device:

[0913] The user's generative model starts the generation process based on the specified parameters, and the emotion engine installed on the device analyzes the user's facial expressions and voice data to recognize emotions.

[0914] Specific behavior:

[0915] The device collects the user's facial and voice data using a camera and microphone, and analyzes it using an emotion engine (e.g., OpenCV or TensorFlow). Based on the analysis results, the parameters of the generative model are dynamically adjusted and the generation process is initiated. The generated results are stored locally or in the cloud.

[0916] input:

[0917] User emotion data (facial expressions, voice).

[0918] output:

[0919] The product based on the adjusted parameters.

[0920] server:

[0921] It monitors the progress of the generation process in real time, stores the generated results in a database, and receives feedback from the emotion engine to adjust the output of the generative model.

[0922] Specific behavior:

[0923] The server receives progress data from the device and updates the monitoring panel. Once generated, it uploads the generated product to cloud storage and stores the metadata in a database. It may also send regeneration commands to the generative model based on feedback from the emotion engine.

[0924] input:

[0925] Progress data for the generation process, and feedback for the emotion engine.

[0926] output:

[0927] Generation results and regeneration instructions stored in a database.

[0928] Step 6: Evaluate the product

[0929] server:

[0930] It provides an interface for evaluating the generated results, accepts evaluations from users and experts, and aggregates the evaluation results.

[0931] Specific behavior:

[0932] The server dynamically generates evaluation pages for products and provides an interface where users can enter evaluation scores and comments. The evaluation data from users and experts is stored in a database and periodically aggregated.

[0933] input:

[0934] Evaluation data (scores, comments) from users and experts.

[0935] output:

[0936] Aggregated evaluation results.

[0937] Step 7: Awarding the product

[0938] server:

[0939] Based on the evaluation results, an algorithm will be run to award outstanding generative models and their developers. The award results will be notified.

[0940] Specific behavior:

[0941] The server runs an algorithm to determine the recipients of the awards based on the aggregated results, and then notifies the selected generative models and developers of the awards via in-app messages and emails.

[0942] input:

[0943] Aggregated evaluation results.

[0944] output:

[0945] Notification of award recipients.

[0946] Step 8: User-participation creative activity

[0947] User:

[0948] Ordinary users access the platform and create their own works using the provided generative models. They then publish the works on the platform and receive ratings and comments from other users.

[0949] Specific behavior:

[0950] Users select a generative model and create a landscape painting using the image generation model. They then upload the landscape painting to the platform and set it to be public. The published work can then receive ratings and comments from other users.

[0951] input:

[0952] Artwork data based on generative models.

[0953] output:

[0954] Published generated works and their ratings and comments.

[0955] This clarifies the processing flow of the entire system, allowing for a detailed understanding of what inputs and outputs exist at each step and what specific operations are performed.

[0956] (Application example 2)

[0957] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0958] Modern generative AI models lack collaboration and competition to improve their diversity and creativity, and the generated content is rarely properly tailored to the user's emotions, resulting in insufficient personalization of the user experience.

[0959] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0960] In this invention, the server includes a means for registering and managing generative models from different fields, a means for matching collaboration and competition between generative models, a means for evaluating the products output by the generative models, a means for providing an interface for collaborative creative activities between the generative models and users, and a means for recognizing the user's emotions and adjusting the products based on those emotions. This increases the diversity and creativity of generative AI models and makes it possible to provide personalized products according to the user's emotions.

[0961] "Generative models from different fields" is a general term for AI models that operate in different genres or applications, such as image generation models and music generation models.

[0962] "Means for registration and management" refers to an interface or program that provides the functionality to collect, classify, and store information about generative models on a server.

[0963] "Means for matching collaboration and competition" refers to algorithms or programs that pair different generative models together to enable them to collaborate or compete.

[0964] "Means for evaluating the product" refers to an interface or system for analyzing and evaluating the results output by the generative model.

[0965] An "interface for users to engage in collaborative creative activities" is a system that provides a user interface and tools that allow users to create content in collaboration with generative models.

[0966] "Means for recognizing emotions and adjusting the generated content based on those emotions" refers to algorithms or programs that analyze the user's emotions in real time using a camera or microphone, and change or adjust the generated content based on the results.

[0967] This invention is a system that allows generative AI models from different fields to collaborate and compete with each other, improving their diversity and creativity. This system incorporates an emotion engine that recognizes the user's emotions, allowing the output of the generative model to be customized according to the user's emotions.

[0968] System Configuration

[0969] 1. Register and manage generative models:

[0970] Users register their generative models on the platform. When registering, an interface is provided for entering detailed information about the generative model (model name, genre, purpose, dataset used, etc.). The server receives this information and stores it in a database, enabling centralized management.

[0971] 2. Matching different generative models:

[0972] The server pairs generative models from different fields based on the registered generative model information, and performs matching for collaboration and competition. The matching results are notified to the user, who can then start the project by checking the project details and making the necessary settings.

[0973] 3. Implementing the generation process:

[0974] The device running the generative model starts the generation process based on specified parameters. The emotion engine analyzes the user's facial expressions and voice data to recognize emotions. The server monitors the progress of the generation process in real time and stores the generation results in a database. It also receives feedback from the emotion engine and adjusts the output of the generative model as appropriate.

[0975] 4. Product Evaluation and Awards:

[0976] The server provides an interface for experts and general users to evaluate the generated models, and has a mechanism to aggregate the evaluation results and recognize and award excellent generative models and their developers.

[0977] 5. User-participatory creative activities:

[0978] Ordinary users can access the platform and create their own works using generative models. For example, a user can choose a music generation model, create a song, and then create a video using a video generation model to accompany the song. The server stores the generated works and provides an interface for sharing them with other users.

[0979] Hardware and software used

[0980] Camera: Used to capture the user's facial expressions.

[0981] Microphone: Used to capture the user's voice.

[0982] OpenCV: An image processing library used to acquire and process camera images.

[0983] Librosa: An audio processing library for reading and analyzing audio files.

[0984] Transformers: A library that implements natural language processing (NLP) models for emotion recognition.

[0985] Specific examples

[0986] For example, suppose a user uses the "Collabo Creator" app to register a landscape image generation model and a classical music generation model. Once the generative models are matched, the app performs real-time emotion analysis using the user's camera and microphone. If the user is moved, the app can slow down the tempo of the generated music and add a sunset element to the generated scenery. This content can be shared with other users and rated.

[0987] Prompt Sentence Examples

[0988] "We combine user-input image generation models with music generation models, recognize emotions in real time using a camera and microphone, and customize generated content based on emotions."

[0989] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0990] Step 1:

[0991] The user registers a generative model on the platform. The user inputs detailed information about the generative model (model name, genre, purpose, dataset used, etc.) and sends it to the server through the interface. The input is the detailed information about the generative model, and the output is information about the generative model stored in the server's database.

[0992] Step 2:

[0993] The server pairs generative models from different fields based on the registered generative model information, and performs matching for collaboration and competition. The input is the information of the generative models stored in the database, and the output is the combination information of the paired generative models.

[0994] Step 3:

[0995] The server notifies the user of the matching results. The user receives the notification, checks the project details, and makes the necessary settings (such as the format of the generated output, output nodes, etc.). The input at this time is the matching results and the user's settings information, and the output is an instruction to start the project.

[0996] Step 4:

[0997] The project generation process begins on the device. The emotion engine captures the user's facial expressions and voice data in real time via the camera and microphone, and recognizes emotions. The input is the user's facial expression data and voice data, and the output is the recognized emotion information. In the processing step, OpenCV analyzes facial expressions, and Librosa processes the voice to extract emotions.

[0998] Step 5:

[0999] The server receives feedback from the emotion engine and adjusts the output of the generative model. The input is the recognized emotion information, and the output is an emotion-adjusted artifact (e.g., music tempo or image color tone). The processing step involves retraining or adjusting the generative model in real time with parameters corresponding to the emotion information.

[1000] Step 6:

[1001] The generated artifacts are sent to a server and stored in a database. The input is the emotion-adjusted artifact, and the output is the artifact stored in the database. The processing step involves storing the artifacts in appropriate storage and managing them in an easily accessible format.

[1002] Step 7:

[1003] The server provides an interface that allows experts and general users to evaluate the products. The input is the saved products, and the output is the evaluation data and the evaluation results. In the processing step, the evaluation interface is provided and the user evaluations are aggregated.

[1004] Step 8:

[1005] Users can participate in the evaluation of the product and leave comments. The input is the product and the evaluation interface, and the output is evaluation points and comments. In the processing step, user feedback is collected and the generative model is improved based on the evaluation results.

[1006] Step 9:

[1007] The server recognizes the developers of excellent products and generative models and notifies the user. The input is the evaluation result, and the output is the recognition notice and recognition list. In the processing step, the server analyzes the evaluation result, selects excellent products, and issues a recognition notice.

[1008] Step 10:

[1009] Users can share their work with other users. The input is the generated work, and the output is a sharing link or a public page. In the processing step, a link is generated or a public page is created to share the work with other users.

[1010] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1011] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1012] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1013] [Third embodiment]

[1014] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1015] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1016] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1017] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1018] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1019] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1020] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1021] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1022] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1023] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1024] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1025] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1026] This invention is a system that improves the diversity and creativity of generative models through collaboration and competition between them in different fields. This system has functions for registering and managing generative models, matching between generative models, evaluating the results, and realizing collaborative creation with users.

[1027] 1. Registering and Managing Generative Models

[1028] User:

[1029] Users register generative models on the platform. For example, an interface is provided for entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used). After registration is complete, users can share their generative models with other users.

[1030] server:

[1031] The server receives information about the generative model sent by the user and stores it in a database, allowing management of the characteristics of the generative model (performance, genre, training data, etc.).

[1032] 2. Matching and project launch

[1033] server:

[1034] The server automatically matches models from different fields based on the registered generative models. For example, it pairs a music generation model with a video generation model. The matching results are notified to the user, who can then start a collaboration project.

[1035] User:

[1036] Upon receiving the notification, the user can start the project by checking the project details and making the necessary settings (such as the format of the generated output and output nodes).

[1037] 3. Implementing the generation process

[1038] Device:

[1039] The generative model runs on the user's device and executes the generation process based on the specified parameters. For example, a music generation model generates a unique piece of music, and a video generation model creates a video to go along with it.

[1040] server:

[1041] The server monitors the progress of the generation process in real time, stores the generated results, and provides feedback to the user if necessary.

[1042] 4. Product Evaluation and Awards

[1043] server:

[1044] We will hold regular generative AI contests, where submitted products will be evaluated by experts and general users. The evaluation results will be compiled, and outstanding generative models and their developers will be awarded prizes.

[1045] User:

[1046] Users who participate in the contest can submit their creations and rate and comment on other creations, allowing for more diverse feedback.

[1047] 5. User-participatory creative activities

[1048] User:

[1049] Ordinary users can also access the platform and use generative models to create their own works. For example, a user can create a song using a music generation model and then create a video to accompany that song using a video generation model.

[1050] server:

[1051] The server provides an interface for users to store their generated work and share it with other users.

[1052] As a concrete example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches these models and proposes a project to jointly generate artworks (images + music). Users A and B generate artworks using their respective models, and the generated artworks are published on the platform and evaluated by other users.

[1053] In this way, the present invention promotes interaction between generative AI models, enabling the exploration of diverse expression techniques and improving the quality of the products.

[1054] The processing flow will be explained below.

[1055] Step 1:

[1056] Registering a Generative Model

[1057] User:

[1058] Enter detailed information about your generative model (model name, genre, purpose, dataset used, etc.) into the registration form and submit it.

[1059] server:

[1060] The submitted generative model data is received and saved in the database. Once saved, a notification is sent to the user to notify them of the completion of registration.

[1061] Step 2:

[1062] Managing Generative Models

[1063] server:

[1064] It updates the Generative Model information in the database and provides a user-accessible management screen that displays a list of registered Generative Models and their respective characteristics.

[1065] Step 3:

[1066] Matching implementation

[1067] server:

[1068] Analyzes the generative model information in the database and runs an algorithm to pair generative models from different fields for collaboration or competition. Generates matching results and notifies the user.

[1069] Step 4:

[1070] Starting a project

[1071] User:

[1072] You will receive a notification of the matching result, check the details, and if you agree to start the project, make the necessary settings (product format, output node, etc.).

[1073] server:

[1074] Receives configuration information from the user and prepares the project execution environment.

[1075] Step 5:

[1076] Implementing the generation process

[1077] Device:

[1078] On the device where the user's generative model runs, the generative process is started based on the specified parameters, and an artifact (e.g., music or video) is created.

[1079] server:

[1080] Monitor the progress of the generation process in real time, store the generated results in a database, review the feedback and notify the user if necessary.

[1081] Step 6:

[1082] Product evaluation

[1083] server:

[1084] We provide an interface for experts and general users to evaluate the generated models, and we aggregate the evaluation results and award outstanding generative models.

[1085] User:

[1086] You can participate in product evaluations and leave comments. You can also view other users' evaluations.

[1087] Step 7:

[1088] User-participation creative activities

[1089] User:

[1090] Ordinary users can access the platform and use generative models to create their own works. For example, they can select a music generation model to create a song, and then use a video generation model to create a video that matches the song.

[1091] server:

[1092] It provides an interface for users to save and share their creations with other users, and also manages other users' comments on shared creations.

[1093] This allows for greater diversity and creativity in generative models.

[1094] Example 1

[1095] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1096] In conventional generative AI model systems, it was difficult for generative models from different fields to efficiently collaborate and produce diverse creative works. Furthermore, there was a lack of mechanisms for evaluating the generated results, and insufficient feedback on improving the quality of the generative models was also an issue. Furthermore, the hurdles for general users to use generative AI models for creative activities were high, making it difficult to produce diverse creative works. A system that can solve these issues and bring out more creativity and diversity is needed.

[1097] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1098] In this invention, the server includes means for registering and managing generative AI models, means for matching collaboration and competition between generative AI models, means for evaluating the products output by the generative AI models, means for providing an interface for generative AI models and users to collaborate on creative activities, means for receiving detailed information about the generative AI models and saving it in a database, means for automatically matching generative AI models of different genres and notifying users, and means for monitoring the progress of the generation process in real time and saving the generation results. This allows generative models from different fields to collaborate efficiently, improve the quality of generative models through evaluation and feedback, and enable general users to easily engage in creative activities.

[1099] A "generative AI model" is an algorithm that is trained to perform a specific problem or task using artificial intelligence techniques.

[1100] "Registration and management" refers to the process of registering the details of a generative AI model on the platform, storing that information in the database, and updating or deleting it as needed.

[1101] "Collaboration and competition" refers to a situation in which different generative AI models work together to collaborate or compete to create products.

[1102] "Matching" is the process of automatically selecting and linking generative AI models with different genres and characteristics.

[1103] "Products" are the results output by generative AI models, and can take various forms such as music, video, images, and text.

[1104] "Evaluation" refers to the act of experts and general users scoring and commenting on the quality and performance of the products output by a generative AI model.

[1105] "Interface" means the user interface and functionality that allows a user to interact with a generative AI model and engage in collaborative creative activities.

[1106] A "database" is a software system for efficiently storing and managing data such as detailed information about generative AI models and generated results.

[1107] "Notifications" are messages that convey information to users, such as the matching results of the generative AI model and the progress of the generation project.

[1108] "Progress monitoring" means checking in real time whether the generation process is proceeding as planned or whether any abnormalities have occurred.

[1109] "Storage" refers to the act of recording the output product of the generation process and the data generated during that process in a database or storage.

[1110] This invention is a system that improves the diversity and creativity of generative AI models by allowing them to collaborate and compete with each other in different fields. This system has functions for registering and managing generative AI models, matching between generative AI models, evaluating the results, and realizing collaborative creation with users.

[1111] Specific actions

[1112] Users register generative AI models on the platform. For example, an interface is provided for entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used). When registering a "music generative model," users enter information such as "music generative model A," "music," "music production," and "music dataset 1."

[1113] The server receives information about the generative AI model sent by the user and stores it in a database. This allows the server to manage the characteristics of the generative AI model (performance, genre, training data, etc.). For example, a record such as "Music Generative Model A, Music, Music Production, Music Dataset 1" is registered in the database.

[1114] The server automatically matches models from different fields based on information about generative AI models registered in the database. For example, it might select a pair of "music generation model A" and "video generation model B." It then proposes a new project for the selected pair and notifies the user.

[1115] After receiving the notification, the user checks the details of the proposed project and makes the necessary settings (such as the format of the generated output and output nodes) on the user interface. For example, they can set the length of the music to be generated and the video resolution. Once the settings are complete, they press the "Start Project" button.

[1116] The generative AI model runs on the device and executes the generation process based on the specified parameters. For example, a music generation model generates a unique song, and a video generation model creates a video in conjunction with it.

[1117] The server monitors the progress of the generation process in real time and saves the generated results. After the generation is complete, it saves the results in a database and notifies the user of the results. Specifically, it saves the generated music and video files and sends a notification to the user.

[1118] The server will hold a generative AI contest and evaluate the products submitted by users. It will compile evaluations from experts and general users and award outstanding generative models and their developers. For example, if "Music Generation Model A" receives a high evaluation based on the evaluation criteria, the results will be made public.

[1119] Ordinary users can access the platform and use generative AI models to create their own works. For example, a user can create a song using a music generation model and then create a video to go with that song using a video generation model.

[1120] The server stores user-generated works and provides an interface for sharing them with other users, e.g., user-generated music and videos are shared so that other users can view and rate them.

[1121] Examples and prompts

[1122] For example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches these models and proposes a project to jointly generate an artwork (image + music). Users A and B create artwork using their respective models, and the resulting artwork is published on the platform and evaluated by other users.

[1123] The following can be used as a specific prompt:

[1124] "Please run a program that generates a landscape image and generates background music that matches that image. We will use 'Landscape Image Generation Model A' as the image generation model and 'Music Generation Model A' as the background music generation model."

[1125] In this way, the present invention promotes interaction between generative AI models, enabling the exploration of diverse expression methods and improving the quality of the products.

[1126] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1127] Step 1:

[1128] Registering a generative AI model

[1129] Users register generative AI models on the platform by accessing a dedicated interface and entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used).

[1130] Input: Detailed information about the generative model (model name, genre, purpose, dataset used, etc.)

[1131] Output: Send registration application data

[1132] Specific operation: The user enters information such as "Music Generation Model A," "Music," "Music Production," and "Music Dataset 1" into the interface and presses the registration button.

[1133] Step 2:

[1134] Receiving and saving generated AI model registration information

[1135] The server receives the registration information for the generative AI model sent by the user and stores it in a database.

[1136] Input: Registration information for the generative model from the user

[1137] Output: Record added to database

[1138] Specific operation: The server receives the information "Music generation model A, music, music production, music dataset 1" from the user and stores it in the database.

[1139] Step 3:

[1140] Matching between generative models

[1141] The server automatically matches models from different fields based on information about generative AI models registered in the database.

[1142] Input: Information about the generative AI model stored in the database

[1143] Output: Matching result notification

[1144] Specific operation: The server selects a pair of "music generation model A" and "video generation model B" and sends a "project proposal" notification to the user.

[1145] Step 4:

[1146] Project details

[1147] The user receives a notification, reviews the details of the proposed project, and makes any necessary configurations (such as artifact format and output nodes).

[1148] Input: Project proposal notification, configuration information (format, output node, etc.)

[1149] Output: Sending a notification that the setup is complete

[1150] Specific operations: The user sets the length of the music to be generated, the resolution of the video, etc., and presses the "Start Project" button.

[1151] Step 5:

[1152] Implementing the generation process

[1153] The generative AI model runs on the device and executes the generation process based on the specified parameters.

[1154] Input: Setting parameters (song length, video resolution, etc.)

[1155] Output: Generated products (music files, video files)

[1156] Specific operation: "Music generation model A" generates a five-minute piece of music on User A's device, and at the same time, "video generation model B" generates a video linked to the music on User B's device.

[1157] Step 6:

[1158] Product monitoring and preservation

[1159] The server monitors the progress of the generation process in real time and stores the generation results.

[1160] Input: Progress data of the generation process

[1161] Output: Save generated results to database, progress notification

[1162] Specific operation: After generation is complete, the generated music and video files are saved in a database and the user is notified of the results.

[1163] Step 7:

[1164] Product evaluation

[1165] The server hosts a generative AI contest and performs a process to evaluate the products submitted by users.

[1166] Input: Submitted product

[1167] Output: Evaluation results summary, award notification

[1168] Specific operation: We will aggregate evaluations from experts and general users and award outstanding generative models and their developers.

[1169] Step 8:

[1170] User-participation creative activities

[1171] Ordinary users can also access the platform and use generative AI models to create their own unique works.

[1172] Input: Creation request using generative model

[1173] Output: Generated creative works (music, video, etc.)

[1174] Specific operation: The user creates a song using the music generation model and a video using the video generation model.

[1175] Step 9:

[1176] Sharing the results

[1177] The server provides an interface for users to store their generated work and share it with other users.

[1178] Input: Generated artwork

[1179] Output: Provides an interface for sharing

[1180] Specific operation: The generated music and video will be published on the platform and can be rated by other users.

[1181] (Application example 1)

[1182] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1183] Conventional generative AI models have focused on outputting products specialized in a single field, making it difficult for generative models from different fields to collaborate with each other or for users to participate in the generation process in real time. Furthermore, there is a lack of a mechanism for effectively utilizing evaluations and feedback of products to update the generated content, which limits the quality and diversity of the products.

[1184] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1185] In this invention, the server includes a means for registering and managing generative models from different fields, a means for matching collaboration and competition between generative models, a means for evaluating the products output by the generative models, a means for providing an interface for the generative models and users to engage in collaborative creative activities, and a means for linking different generative models to generate and view interactive content. This enables the linking of generative models from different fields to generate diverse, high-quality interactive content and user participation in real time.

[1186] A "generative model" is an algorithm that generates new data or content based on a specific dataset or algorithm.

[1187] “Registration and Management” refers to the process of adding a Generative Model to the Platform, storing its information, and updating or modifying it as needed.

[1188] "Collaboration" refers to the activity of different generative models working together to jointly produce a single product.

[1189] "Competition" refers to the process in which different generative models compete against each other in terms of their performance and output.

[1190] "Matching" refers to the process of selecting the optimal combination based on the characteristics and parameters of different generative models.

[1191] "Interactive content" refers to content whose generated content changes dynamically as users operate and provide feedback in real time.

[1192] "Interface" refers to the screen or tool that allows the user to input and operate the generative model.

[1193] "Evaluation" refers to the activity of diagnosing the product output by a generative model based on its quality, performance, etc., and providing the results as feedback.

[1194] "Real-time monitoring" refers to the process of instantly checking the progress of the generation process and maintaining a state in which necessary responses can be taken immediately.

[1195] "Preservation" refers to the process of securely storing generated data and content for later use and evaluation.

[1196] "Feedback" refers to the process of returning evaluations of the product and suggestions for improvement to the model.

[1197] This invention provides a system that links generative models from different fields to enable the generation and viewing of interactive content. An embodiment of this system is described in detail below.

[1198] 1. System Configuration

[1199] Server: The server contains the following main means:

[1200] Generative model management

[1201] Generative model matching method

[1202] Product evaluation method

[1203] Interface provision means

[1204] Real-time monitoring and feedback measures

[1205] To achieve these goals, the server uses high-performance computing hardware and a web framework such as Flask.

[1206] User device: Users access the system using a smartphone or head-mounted display (HMD) to generate and view content. A dedicated app for operating and linking different generative models is installed on the smartphone.

[1207] Generative Models: Generative models are algorithms specialized for a specific domain, such as music generation models or video generation models. They are hosted on a server and accessed via API endpoints.

[1208] 2. Operation overview

[1209] Users use a smartphone app to register generative models on the platform. After entering model information (model name, genre, purpose, dataset used, etc.), the model is saved on the server. The registered generative model is managed by the server and updated as needed.

[1210] Next, the server automatically matches models from different fields based on the characteristics of the registered generative models. For example, a music generation model and a video generation model are paired and notified to the user. The user can then start the project by checking the project details and configuring the necessary settings.

[1211] 3. Content Creation and Viewing

[1212] The user inputs a prompt to generate interactive content, for example:

[1213] Example prompt:

[1214] Input to the music generation model: "Generate a fast-paced, electronic-feeling soundtrack."

[1215] Input to the video generation model: "Generate a video with a fast camera movement against a futuristic cityscape."

[1216] Based on this, the server calls the music generation model and video generation model, respectively, and performs generation. The generated music and video are sent to the user's device in real time and can be viewed on a smartphone or HMD. Based on user feedback, the server updates the generated content as appropriate and advances the generation process.

[1217] The generated content is stored on the server and can be shared and rated by other users. The evaluation results are aggregated and outstanding generative models and their developers are awarded.

[1218] This allows generative models from different fields to work together, enabling the generation of diverse, high-quality interactive content and real-time user participation.

[1219] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1220] Step 1: Registering the Generative Model

[1221] Users use a smartphone app to register a generative model on the platform. Input includes detailed information about the generative model (model name, genre, purpose, dataset used, etc.). These details are sent to the server and stored in a database. As an output, a message is sent to the user informing them that the generative model has been registered.

[1222] Step 2: Managing the Generative Model

[1223] The server manages information about registered generative models. It references the characteristics of the generative models stored in the database (performance, genre, training data, etc.) and updates and corrects them as needed. This management process periodically checks the information about the generative models to keep them up to date. As an output, a report containing the latest information about the generative models is generated.

[1224] Step 3: Matching the generative model

[1225] The server automatically matches models from different fields based on the characteristics of the registered generative models. For example, it analyzes features to find the optimal combination to pair a music generation model with a video generation model. The characteristic information of each generative model is used as input, and the matching results are reported to the user as output.

[1226] Step 4: Starting the project

[1227] Upon receiving the notification, the user checks the project details and makes the necessary settings (such as the product format and output nodes). The input includes the matched generative model and project details. As an output, a project start confirmation message is sent to the server, and the project is launched.

[1228] Step 5: Implementing the generation process

[1229] When a project starts, the user inputs a prompt, for example, "Please generate a soundtrack with a fast tempo and an electronic feel." The server receives this prompt and sends generation requests to the music generation model and the video generation model. The generative models perform the generation work based on the specified parameters and return the output results (music and video) to the server.

[1230] Step 6: Integrating the products

[1231] The server receives the generated results (music and video) sent from the generative model. To integrate these generated results, the server adjusts the timing and coding of the music and video. The input includes the music generation results and the video generation results, and interactive content is generated as the output.

[1232] Step 7: Interactive viewing

[1233] The generated interactive content is delivered to the user device in real time. The user views the content using a smartphone or HMD and sends feedback to the server. The input includes the user's viewing data and feedback, and the output is an instruction to update the generated content.

[1234] Step 8: Update the generation process

[1235] The server updates the generated content based on user feedback. It analyzes the generated content for improvement and sends new parameters to the generative model for regeneration. This process aims to improve the quality of the product and increase user satisfaction. The input includes user feedback data, and the output is the improved product that is redistributed.

[1236] Step 9: Rate and share your creation

[1237] The final generated content is stored on a server and published on the platform for general users to evaluate. Users can rate and comment on other creations and provide their own feedback. The input includes evaluation data, and the output is the evaluation results that are fed back to the developer of the generative model.

[1238] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1239] The present invention is a system that aims to improve the diversity and creativity of generative models from different fields by allowing them to collaborate and compete with each other. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions, allowing the output of the generative model to be customized according to the user's emotions. An embodiment of this system is described in detail below.

[1240] System Configuration

[1241] The system mainly consists of the following components:

[1242] A server that registers and manages generative models

[1243] A server that performs matching between generative models

[1244] User device equipped with an emotion engine

[1245] A server that provides evaluation and award functions for products

[1246] An interface for collaborative creative activities between users and generative models

[1247] 1. Registering and Managing Generative Models

[1248] User:

[1249] Users register generative models on the platform. When registering, they are provided with an interface to input detailed information about the generative model (such as the model name, genre, purpose, and dataset used).

[1250] server:

[1251] The server receives the information about the generative model sent by the user and stores it in a database, allowing for centralized management of the characteristic information of the generative model.

[1252] 2. Matching and project launch

[1253] server:

[1254] The server pairs generative models from different fields based on the registered generative model information, and conducts matching for collaboration and competition. The matching results are notified to the user.

[1255] User:

[1256] The user who receives the notification checks the details of the project and makes the necessary settings (format of the generated output, output node, etc.), which starts the project.

[1257] 3. Implementing the generation process

[1258] Device:

[1259] The generation process is initiated based on the specified parameters on the device where the user's generative model is running. The emotion engine has the ability to recognize emotions by analyzing the user's facial expressions and voice data.

[1260] server:

[1261] The server monitors the progress of the generation process in real time, stores the generated results in a database, and receives feedback from the emotion engine to adjust the output of the generative model.

[1262] 4. Product Evaluation and Awards

[1263] server:

[1264] The server provides an interface for experts and general users to evaluate the generated models, and aggregates the evaluation results to recognize and reward outstanding generative models and their developers.

[1265] User:

[1266] Users can participate in rating the products and leave comments, which provides feedback for improvements to the generative model and new ideas.

[1267] 5. User-participatory creative activities

[1268] User:

[1269] Ordinary users can access the platform and create their own works using generative models. For example, a user can select a music generation model, create a song, and then create a video to go with the song using a video generation model.

[1270] server:

[1271] The server provides an interface for users to store and share their generated works with other users, and can further customize the creations based on data from the emotion engine.

[1272] Specific examples

[1273] For example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches the two, and users A and B start a project to jointly create an artwork (image + music). When user B creates music, the emotion engine analyzes user B's facial expressions in real time and adjusts the tempo and tone of the music in real time according to the emotion. The final artwork created is published on the platform and receives ratings and comments from other users.

[1274] In this way, the present invention increases the diversity and creativity of generative models, while at the same time providing a more personalized experience by recognizing the user's emotions and customizing the product.

[1275] The processing flow will be explained below.

[1276] Step 1:

[1277] Registering a Generative Model

[1278] User:

[1279] The user accesses the registration form for the generative model and enters detailed information about the generative model (model name, genre, purpose, dataset used, etc.). By clicking the submit button, the information is sent to the server.

[1280] server:

[1281] The submitted generative model data is received and saved in the database. Once saved, a notification of registration completion is sent to the user.

[1282] Step 2:

[1283] Managing Generative Models

[1284] server:

[1285] The system periodically updates the information about generative models in the database and provides a user-accessible management screen that displays a list of registered generative models and their respective characteristics.

[1286] User:

[1287] You can view and edit information about registered generative models through the management screen.

[1288] Step 3:

[1289] Matching implementation

[1290] server:

[1291] It analyzes the generative model information in the database, runs an algorithm to pair generative models from different fields, generates a list of paired models, and notifies the user of the matching results.

[1292] Step 4:

[1293] Starting a project

[1294] User:

[1295] Receive a notification of the matching result, confirm the project details, and if you agree to start the project, configure the required settings such as the output format and output node.

[1296] server:

[1297] Receives configuration information from the user and prepares the project execution environment. Once preparations are complete, notifies the user that the project has started.

[1298] Step 5:

[1299] Implementing the generation process

[1300] Device:

[1301] The generation process is initiated based on the specified parameters on the device where the user's generative model is running. During this process, the emotion engine analyzes the user's facial expressions and voice in real time to collect emotion data.

[1302] server:

[1303] The progress of the generation process is monitored in real time, the generation results are stored in a database, and the output of the generative model is adjusted based on the emotion data obtained from the emotion engine.

[1304] Step 6:

[1305] Product evaluation

[1306] server:

[1307] We provide an interface for experts and general users to evaluate the generated models, and we aggregate the evaluation results to recognize and reward excellent generative models and their developers.

[1308] User:

[1309] Through the rating interface, you can rate the product, leave comments, and view other users' ratings.

[1310] Step 7:

[1311] User-participation creative activities

[1312] User:

[1313] Ordinary users can access the platform and create their own works using the generative models provided. For example, they can select a music generation model to create a song, and then use a video generation model to create a video to go along with that song.

[1314] server:

[1315] It stores user-generated works in a database, provides an interface for sharing them with other users, and manages other users' comments on the shared works.

[1316] Examples:

[1317] User A registers an image generation model on the platform, and User B registers a music generation model. The server matches these models and proposes a project to jointly create an artwork. When User B creates music, the emotion engine analyzes User B's facial expressions in real time and adjusts the tempo and tone of the music according to their emotions. The final artwork is published on the platform and can be rated and commented on by other users.

[1318] In this way, the present invention not only increases the diversity and creativity of generative models, but also provides a personalized experience that responds to the user's emotions.

[1319] Example 2

[1320] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1321] In conventional generative modeling systems, collaboration and competition between generative models from different fields was difficult, limiting the diversity and creativity of the products they produced. Furthermore, they lacked methods for providing personalized products that respond to the user's emotions. This resulted in a uniform user experience that lacked individuality, reducing the appeal and value of the products.

[1322] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for registering and managing generative models from different fields, means for matching collaboration and competition between generative models, means for customizing products output by generative models based on user emotions, means for evaluating products output by generative models, and means for providing an interface for generative models and users to collaborate on creative activities. This facilitates collaboration and competition between generative models and makes it possible to provide personalized products that correspond to user emotions.

[1323] A "generative model" is a machine learning algorithm that generates new data or products for a specific purpose or use.

[1324] A "means for registration and management" is a system or interface that allows users to input detailed information about generative models and store and organize that information.

[1325] "Matching means" is a function that analyzes the features and characteristics of different generative models and performs pairings for optimal collaboration or competition.

[1326] The "means for customization" is a system that analyzes the user's emotions and adjusts the characteristics and parameters of the product based on those emotions.

[1327] The "means of evaluation" is a mechanism for evaluating the quality and suitability of the products output by the generative model based on feedback from users and experts.

[1328] The "interface for creative activities" is a user interface used when a generative model and a user collaborate on creative activities, and allows for adjustment of input and output.

[1329] This system aims to improve the diversity and creativity of generative models by enabling collaboration and competition between them in different fields. Characteristic features of the system include the registration and management of generative models, matching between generative models, evaluation of products, customization of products based on user emotions, and provision of an interface for collaborative creation.

[1330] System Configuration

[1331] The system consists of the following elements:

[1332] A server that registers and manages generative models

[1333] A server that performs matching between generative models

[1334] User device equipped with an emotion engine

[1335] A server that provides evaluation and award functions for products

[1336] An interface for collaborative creative activities between users and generative models

[1337] 1. Registering and Managing Generative Models

[1338] User:

[1339] Users register their generative models on the platform. When registering, an interface is provided for entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used). For example, users can enter data into the form using a web browser and then press the "Register" button to send the generative model information to the server.

[1340] server:

[1341] The server receives the generative model information sent by the user and stores it in a database. This storage enables centralized management of generative models. The received data is parsed into JSON format and stored in an SQL database such as MySQL.

[1342] 2. Matching the Generative Model and Starting the Project

[1343] server:

[1344] The server pairs generative models from different fields based on the registered generative model information, and performs matching for collaboration and competition. For example, k-means clustering is used as a matching algorithm. The matching results are notified to the user.

[1345] User:

[1346] After receiving the notification, the user checks the details of the project and makes the necessary settings (such as the format of the generated output, output nodes, etc.). For example, the user selects the project format and presses the "Start Project" button to start the project.

[1347] 3. Implementing the generation process

[1348] Device:

[1349] The device where the user's generative model runs starts the generation process based on the specified parameters. The device's emotion engine analyzes the user's facial and voice data to recognize the user's emotions. Examples of use cases include libraries such as OpenCV and TensorFlow.

[1350] server:

[1351] The server monitors the progress of the generation process in real time, stores the generation results in a database, and receives feedback from the emotion engine to adjust the output of the generative model. The generation results are stored in cloud storage, and the metadata is stored in a database.

[1352] 4. Product Evaluation and Awards

[1353] server:

[1354] The server provides an interface for evaluating the generated models and accepts evaluations from general users and experts. Evaluation scores and comments are stored in a database and periodically compiled. Based on this, an algorithm is run to reward excellent generative models and their developers, and the results are notified.

[1355] User:

[1356] Users can participate in the evaluation of the generated results and leave comments, which leads to improvements in the generative model and new ideas.

[1357] 5. User-participatory creative activities

[1358] User:

[1359] Ordinary users can access the platform and create their own works using the provided generative models. For example, they can create a song using the music generation model and then create a video to accompany that song using the video generation model.

[1360] server:

[1361] The server provides an interface for users to store and share their generated works with other users, and allows further customization of the works based on data from the emotion engine.

[1362] Specific examples

[1363] For example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches the two, and users A and B start a project to jointly create an artwork (images and music). When user B creates music, the emotion engine analyzes user B's facial expressions in real time and adjusts the tempo and tone of the music according to their emotions. The final artwork created is published on the platform and receives ratings and comments from other users.

[1364] Example prompt

[1365] The following examples could be used as prompts for a generative AI model:

[1366] "Using image generation models and music generation models, we create artwork based on the user's emotions. First, user A generates a landscape painting using the image generation model. Based on that landscape painting, user B creates background music using the music generation model. The tempo and tone of the background music can be adjusted according to user B's emotions (joy, sadness, surprise, etc.)."

[1367] In this way, the system of the present invention increases the diversity and creativity of generative models, while at the same time providing a more personalized experience by recognizing the user's emotions and customizing the product.

[1368] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1369] Step 1: Registering the Generative Model

[1370] User:

[1371] The user enters detailed information about the generative model (such as the model name, genre, purpose, and dataset used) into the interface and presses the "Register" button. The input can be in text or file format.

[1372] Specific behavior:

[1373] Users access the registration page via a web browser or smartphone app and enter the required information. When the user presses the "Register" button, the entered information is sent to the server as an HTTP POST request.

[1374] input:

[1375] Detailed information about the generative model (model name, genre, purpose, dataset used, etc.).

[1376] output:

[1377] An HTTP POST request containing detailed information about the generated model.

[1378] Step 2: Managing the Generative Model

[1379] server:

[1380] The server receives the information about the generative model sent by the user, stores it in a database, and notifies the user when the storage is complete.

[1381] Specific behavior:

[1382] The server analyzes the data of the received HTTP POST request and parses it into JSON format. The analyzed data is saved in a database such as MySQL or PostgreSQL using an INSERT statement. Once the saving is complete, the user is notified via HTTP response or email.

[1383] input:

[1384] Detailed information about the generated model sent as an HTTP POST request.

[1385] output:

[1386] Information about the generated model saved in the database and notification of completion of saving.

[1387] Step 3: Matching the generative model

[1388] server:

[1389] The server matches generative models from different fields based on the generative model information stored in the database, using a matching algorithm such as k-means clustering.

[1390] Specific behavior:

[1391] The server runs a regularly scheduled job to retrieve generative model information from the database, and then runs the k-means clustering algorithm based on the retrieved dataset to determine matching pairs. The matching results are then sent to users via a notification system.

[1392] input:

[1393] A set of information about the generative model stored in a database.

[1394] output:

[1395] Informing generative model matching pairs across disciplines.

[1396] Step 4: Starting the project

[1397] User:

[1398] The user who receives the notification checks the project details, sets up the necessary settings (product format, output node, etc.), and clicks the start button for the project.

[1399] Specific behavior:

[1400] The user clicks on the notification to access the project details screen and make the necessary settings. Once the settings are complete, they press the "Start Project" button, which sends the project settings information to the server.

[1401] input:

[1402] Project configuration information (product format, output nodes, etc.).

[1403] output:

[1404] Project start request.

[1405] Step 5: Implementing the generation process

[1406] Device:

[1407] The user's generative model starts the generation process based on the specified parameters, and the emotion engine installed on the device analyzes the user's facial expressions and voice data to recognize emotions.

[1408] Specific behavior:

[1409] The device collects the user's facial and voice data using a camera and microphone, and analyzes it using an emotion engine (e.g., OpenCV or TensorFlow). Based on the analysis results, the parameters of the generative model are dynamically adjusted and the generation process is initiated. The generated results are stored locally or in the cloud.

[1410] input:

[1411] User emotion data (facial expressions, voice).

[1412] output:

[1413] The product based on the adjusted parameters.

[1414] server:

[1415] It monitors the progress of the generation process in real time, stores the generated results in a database, and receives feedback from the emotion engine to adjust the output of the generative model.

[1416] Specific behavior:

[1417] The server receives progress data from the device and updates the monitoring panel. Once generated, it uploads the generated product to cloud storage and stores the metadata in a database. It may also send regeneration commands to the generative model based on feedback from the emotion engine.

[1418] input:

[1419] Progress data for the generation process, and feedback for the emotion engine.

[1420] output:

[1421] Generation results and regeneration instructions stored in a database.

[1422] Step 6: Evaluate the product

[1423] server:

[1424] It provides an interface for evaluating the generated results, accepts evaluations from users and experts, and aggregates the evaluation results.

[1425] Specific behavior:

[1426] The server dynamically generates evaluation pages for products and provides an interface where users can enter evaluation scores and comments. The evaluation data from users and experts is stored in a database and periodically aggregated.

[1427] input:

[1428] Evaluation data (scores, comments) from users and experts.

[1429] output:

[1430] Aggregated evaluation results.

[1431] Step 7: Awarding the product

[1432] server:

[1433] Based on the evaluation results, an algorithm will be run to award outstanding generative models and their developers. The award results will be notified.

[1434] Specific behavior:

[1435] The server runs an algorithm to determine the recipients of the awards based on the aggregated results, and notifies the selected generative models and developers of the awards via in-app messages and emails.

[1436] input:

[1437] Aggregated evaluation results.

[1438] output:

[1439] Notification of award recipients.

[1440] Step 8: User-participation creative activity

[1441] User:

[1442] Ordinary users access the platform and create their own works using the provided generative models. They then publish the works on the platform and receive ratings and comments from other users.

[1443] Specific behavior:

[1444] Users select a generative model and create a landscape painting using the image generation model. They then upload the landscape painting to the platform and set it to be public. The published work can then receive ratings and comments from other users.

[1445] input:

[1446] Artwork data based on generative models.

[1447] output:

[1448] Published generated works and their ratings and comments.

[1449] This clarifies the processing flow of the entire system, allowing for a detailed understanding of what inputs and outputs exist at each step and what specific operations are performed.

[1450] (Application example 2)

[1451] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1452] Modern generative AI models lack collaboration and competition to improve their diversity and creativity, and the generated content is rarely properly tailored to the user's emotions, resulting in insufficient personalization of the user experience.

[1453] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1454] In this invention, the server includes a means for registering and managing generative models from different fields, a means for matching collaboration and competition between generative models, a means for evaluating the products output by the generative models, a means for providing an interface for collaborative creative activities between the generative models and users, and a means for recognizing the user's emotions and adjusting the products based on those emotions. This increases the diversity and creativity of generative AI models and makes it possible to provide personalized products according to the user's emotions.

[1455] "Generative models from different fields" is a general term for AI models that operate in different genres or applications, such as image generation models and music generation models.

[1456] "Means for registration and management" refers to an interface or program that provides the functionality to collect, classify, and store information about generative models on a server.

[1457] "Means for matching collaboration and competition" refers to algorithms or programs that pair different generative models together to enable them to collaborate or compete.

[1458] "Means for evaluating the product" refers to an interface or system for analyzing and evaluating the results output by the generative model.

[1459] An "interface for users to engage in collaborative creative activities" is a system that provides a user interface and tools that allow users to create content in collaboration with generative models.

[1460] "Means for recognizing emotions and adjusting the generated content based on those emotions" refers to algorithms or programs that analyze the user's emotions in real time using a camera or microphone, and change or adjust the generated content based on the results.

[1461] This invention is a system that allows generative AI models from different fields to collaborate and compete with each other, improving their diversity and creativity. This system incorporates an emotion engine that recognizes the user's emotions, allowing the output of the generative model to be customized according to the user's emotions.

[1462] System Configuration

[1463] 1. Register and manage generative models:

[1464] Users register their generative models on the platform. When registering, an interface is provided for entering detailed information about the generative model (model name, genre, purpose, dataset used, etc.). The server receives this information and stores it in a database, enabling centralized management.

[1465] 2. Matching different generative models:

[1466] The server pairs generative models from different fields based on the registered generative model information, and performs matching for collaboration and competition. The matching results are notified to the user, who can then start the project by checking the project details and making the necessary settings.

[1467] 3. Implementing the generation process:

[1468] The device running the generative model starts the generation process based on specified parameters. The emotion engine analyzes the user's facial expressions and voice data to recognize emotions. The server monitors the progress of the generation process in real time and stores the generation results in a database. It also receives feedback from the emotion engine and adjusts the output of the generative model as appropriate.

[1469] 4. Product Evaluation and Awards:

[1470] The server provides an interface for experts and general users to evaluate the generated models, and has a mechanism to aggregate the evaluation results and recognize and award excellent generative models and their developers.

[1471] 5. User-participatory creative activities:

[1472] Ordinary users can access the platform and create their own works using generative models. For example, a user can choose a music generation model, create a song, and then create a video using a video generation model to accompany the song. The server stores the generated works and provides an interface for sharing them with other users.

[1473] Hardware and software used

[1474] Camera: Used to capture the user's facial expressions.

[1475] Microphone: Used to capture the user's voice.

[1476] OpenCV: An image processing library used to acquire and process camera images.

[1477] Librosa: An audio processing library for reading and analyzing audio files.

[1478] Transformers: A library that implements natural language processing (NLP) models for emotion recognition.

[1479] Specific examples

[1480] For example, suppose a user uses the "Collabo Creator" app to register a landscape image generation model and a classical music generation model. Once the generative models are matched, the app performs real-time emotion analysis using the user's camera and microphone. If the user is moved, the app can slow down the tempo of the generated music and add a sunset element to the generated scenery. This content can be shared with other users and rated.

[1481] Prompt Sentence Examples

[1482] "We combine user-input image generation models with music generation models, recognize emotions in real time using a camera and microphone, and customize generated content based on emotions."

[1483] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1484] Step 1:

[1485] The user registers a generative model on the platform. The user inputs detailed information about the generative model (model name, genre, purpose, dataset used, etc.) and sends it to the server through the interface. The input is the detailed information about the generative model, and the output is information about the generative model stored in the server's database.

[1486] Step 2:

[1487] The server pairs generative models from different fields based on the registered generative model information, and performs matching for collaboration and competition. The input is the information of the generative models stored in the database, and the output is the combination information of the paired generative models.

[1488] Step 3:

[1489] The server notifies the user of the matching results. The user receives the notification, checks the project details, and makes the necessary settings (such as the format of the generated output, output nodes, etc.). The input at this time is the matching results and the user's settings information, and the output is an instruction to start the project.

[1490] Step 4:

[1491] The project generation process begins on the device. The emotion engine captures the user's facial expressions and voice data in real time via the camera and microphone, and recognizes emotions. The input is the user's facial expression data and voice data, and the output is the recognized emotion information. In the processing step, OpenCV analyzes facial expressions, and Librosa processes the voice to extract emotions.

[1492] Step 5:

[1493] The server receives feedback from the emotion engine and adjusts the output of the generative model. The input is the recognized emotion information, and the output is an emotion-adjusted artifact (e.g., music tempo or image color tone). The processing step involves retraining or adjusting the generative model in real time with parameters corresponding to the emotion information.

[1494] Step 6:

[1495] The generated artifacts are sent to a server and stored in a database. The input is the emotion-adjusted artifact, and the output is the artifact stored in the database. The processing step involves storing the artifacts in appropriate storage and managing them in an easily accessible format.

[1496] Step 7:

[1497] The server provides an interface that allows experts and general users to evaluate the products. The input is the saved products, and the output is the evaluation data and the evaluation results. In the processing step, the evaluation interface is provided and the user evaluations are aggregated.

[1498] Step 8:

[1499] Users can participate in the evaluation of the product and leave comments. The input is the product and the evaluation interface, and the output is evaluation points and comments. In the processing step, user feedback is collected and the generative model is improved based on the evaluation results.

[1500] Step 9:

[1501] The server recognizes the developers of excellent products and generative models and notifies the user. The input is the evaluation result, and the output is the recognition notice and recognition list. In the processing step, the evaluation result is analyzed, excellent products are selected, and a recognition notice is issued.

[1502] Step 10:

[1503] Users can share their work with other users. The input is the generated work, and the output is a sharing link or a public page. In the processing step, a link is generated or a public page is created to share the work with other users.

[1504] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1505] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1506] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1507] [Fourth embodiment]

[1508] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1509] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1510] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1511] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1512] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1513] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1514] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1515] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1516] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1517] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1518] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1519] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1520] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1521] This invention is a system that improves the diversity and creativity of generative models through collaboration and competition between them in different fields. This system has functions for registering and managing generative models, matching between generative models, evaluating the results, and realizing collaborative creation with users.

[1522] 1. Registering and Managing Generative Models

[1523] User:

[1524] Users register generative models on the platform. For example, an interface is provided for entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used). After registration is complete, users can share their generative models with other users.

[1525] server:

[1526] The server receives information about the generative model sent by the user and stores it in a database, allowing management of the characteristics of the generative model (performance, genre, training data, etc.).

[1527] 2. Matching and project launch

[1528] server:

[1529] The server automatically matches models from different fields based on the registered generative models. For example, it pairs a music generation model with a video generation model. The matching results are notified to the user, who can then start a collaboration project.

[1530] User:

[1531] Upon receiving the notification, the user can start the project by checking the project details and making the necessary settings (such as the format of the generated output and output nodes).

[1532] 3. Implementing the generation process

[1533] Device:

[1534] The generative model runs on the user's device and executes the generation process based on the specified parameters. For example, a music generation model generates a unique piece of music, and a video generation model creates a video to go along with it.

[1535] server:

[1536] The server monitors the progress of the generation process in real time, stores the generated results, and provides feedback to the user if necessary.

[1537] 4. Product Evaluation and Awards

[1538] server:

[1539] We will hold regular generative AI contests, where submitted products will be evaluated by experts and general users. The evaluation results will be compiled, and outstanding generative models and their developers will be awarded prizes.

[1540] User:

[1541] Users who participate in the contest can submit their creations and rate and comment on other creations, allowing for more diverse feedback.

[1542] 5. User-participatory creative activities

[1543] User:

[1544] Ordinary users can also access the platform and use generative models to create their own works. For example, a user can create a song using a music generation model and then create a video to accompany that song using a video generation model.

[1545] server:

[1546] The server provides an interface for users to store their generated work and share it with other users.

[1547] As a concrete example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches these models and proposes a project to jointly generate artworks (images + music). Users A and B generate artworks using their respective models, and the generated artworks are published on the platform and evaluated by other users.

[1548] In this way, the present invention promotes interaction between generative AI models, enabling the exploration of diverse expression techniques and improving the quality of the products.

[1549] The processing flow will be explained below.

[1550] Step 1:

[1551] Registering a Generative Model

[1552] User:

[1553] Enter detailed information about your generative model (model name, genre, purpose, dataset used, etc.) into the registration form and submit it.

[1554] server:

[1555] The submitted generative model data is received and saved in the database. Once saved, a notification is sent to the user to notify them of the completion of registration.

[1556] Step 2:

[1557] Managing Generative Models

[1558] server:

[1559] It updates the Generative Model information in the database and provides a user-accessible management screen that displays a list of registered Generative Models and their respective characteristics.

[1560] Step 3:

[1561] Matching implementation

[1562] server:

[1563] Analyzes the generative model information in the database and runs an algorithm to pair generative models from different fields for collaboration or competition. Generates matching results and notifies the user.

[1564] Step 4:

[1565] Starting a project

[1566] User:

[1567] You will receive a notification of the matching result, check the details, and if you agree to start the project, make the necessary settings (product format, output node, etc.).

[1568] server:

[1569] Receives configuration information from the user and prepares the project execution environment.

[1570] Step 5:

[1571] Implementing the generation process

[1572] Device:

[1573] On the device where the user's generative model runs, the generative process is started based on the specified parameters, and an artifact (e.g., music or video) is created.

[1574] server:

[1575] Monitor the progress of the generation process in real time, store the generated results in a database, review the feedback and notify the user if necessary.

[1576] Step 6:

[1577] Product evaluation

[1578] server:

[1579] We provide an interface for experts and general users to evaluate the generated models, and we aggregate the evaluation results and award outstanding generative models.

[1580] User:

[1581] You can participate in product evaluations and leave comments. You can also view other users' evaluations.

[1582] Step 7:

[1583] User-participation creative activities

[1584] User:

[1585] Ordinary users can access the platform and use generative models to create their own works. For example, they can select a music generation model to create a song, and then use a video generation model to create a video that matches the song.

[1586] server:

[1587] It provides an interface for users to save and share their creations with other users, and also manages other users' comments on shared creations.

[1588] This allows for greater diversity and creativity in generative models.

[1589] Example 1

[1590] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1591] In conventional generative AI model systems, it was difficult for generative models from different fields to efficiently collaborate and produce diverse creative works. Furthermore, there was a lack of mechanisms for evaluating the generated results, and insufficient feedback on improving the quality of the generative models was also an issue. Furthermore, the hurdles for general users to use generative AI models for creative activities were high, making it difficult to produce diverse creative works. A system that can solve these issues and bring out more creativity and diversity is needed.

[1592] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1593] In this invention, the server includes means for registering and managing generative AI models, means for matching collaboration and competition between generative AI models, means for evaluating the products output by the generative AI models, means for providing an interface for generative AI models and users to collaborate on creative activities, means for receiving detailed information about the generative AI models and saving it in a database, means for automatically matching generative AI models of different genres and notifying users, and means for monitoring the progress of the generation process in real time and saving the generation results. This allows generative models from different fields to collaborate efficiently, improve the quality of generative models through evaluation and feedback, and enable general users to easily engage in creative activities.

[1594] A "generative AI model" is an algorithm that is trained to perform a specific problem or task using artificial intelligence techniques.

[1595] "Registration and management" refers to the process of registering the details of a generative AI model on the platform, storing that information in the database, and updating or deleting it as needed.

[1596] "Collaboration and competition" refers to a situation in which different generative AI models work together to collaborate or compete to create products.

[1597] "Matching" is the process of automatically selecting and linking generative AI models with different genres and characteristics.

[1598] "Products" are the results output by generative AI models, and can take various forms such as music, video, images, and text.

[1599] "Evaluation" refers to the act of experts and general users scoring and commenting on the quality and performance of the products output by a generative AI model.

[1600] "Interface" means the user interface and functionality that allows a user to interact with a generative AI model and engage in collaborative creative activities.

[1601] A "database" is a software system for efficiently storing and managing data such as detailed information about generative AI models and generated results.

[1602] "Notifications" are messages that convey information to users, such as the matching results of the generative AI model and the progress of the generation project.

[1603] "Progress monitoring" means checking in real time whether the generation process is proceeding as planned or whether any abnormalities have occurred.

[1604] "Storage" refers to the act of recording the output product of the generation process and the data generated during that process in a database or storage.

[1605] This invention is a system that improves the diversity and creativity of generative AI models by allowing them to collaborate and compete with each other in different fields. This system has functions for registering and managing generative AI models, matching between generative AI models, evaluating the results, and realizing collaborative creation with users.

[1606] Specific actions

[1607] Users register generative AI models on the platform. For example, an interface is provided for entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used). When registering a "music generative model," users enter information such as "music generative model A," "music," "music production," and "music dataset 1."

[1608] The server receives information about the generative AI model sent by the user and stores it in a database. This allows the server to manage the characteristics of the generative AI model (performance, genre, training data, etc.). For example, a record such as "Music Generative Model A, Music, Music Production, Music Dataset 1" is registered in the database.

[1609] The server automatically matches models from different fields based on information about generative AI models registered in the database. For example, it might select a pair of "music generation model A" and "video generation model B." It then proposes a new project for the selected pair and notifies the user.

[1610] After receiving the notification, the user checks the details of the proposed project and makes the necessary settings (such as the format of the generated output and output nodes) on the user interface. For example, they can set the length of the music to be generated and the video resolution. Once the settings are complete, they press the "Start Project" button.

[1611] The generative AI model runs on the device and executes the generation process based on the specified parameters. For example, a music generation model generates a unique song, and a video generation model creates a video in conjunction with it.

[1612] The server monitors the progress of the generation process in real time and saves the generated results. After the generation is complete, it saves the results in a database and notifies the user of the results. Specifically, it saves the generated music and video files and sends a notification to the user.

[1613] The server will hold a generative AI contest and evaluate the products submitted by users. It will compile evaluations from experts and general users and award outstanding generative models and their developers. For example, if "Music Generation Model A" receives a high evaluation based on the evaluation criteria, the results will be made public.

[1614] Ordinary users can access the platform and use generative AI models to create their own works. For example, a user can create a song using a music generation model and then create a video to go with that song using a video generation model.

[1615] The server stores user-generated works and provides an interface for sharing them with other users, e.g., user-generated music and videos are shared so that other users can view and rate them.

[1616] Examples and prompts

[1617] For example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches these models and proposes a project to jointly generate an artwork (image + music). Users A and B create artwork using their respective models, and the resulting artwork is published on the platform and evaluated by other users.

[1618] The following can be used as a specific prompt:

[1619] "Please run a program that generates a landscape image and generates background music that matches that image. We will use 'Landscape Image Generation Model A' as the image generation model and 'Music Generation Model A' as the background music generation model."

[1620] In this way, the present invention promotes interaction between generative AI models, enabling the exploration of diverse expression methods and improving the quality of the products.

[1621] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1622] Step 1:

[1623] Registering a generative AI model

[1624] Users register generative AI models on the platform by accessing a dedicated interface and entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used).

[1625] Input: Detailed information about the generative model (model name, genre, purpose, dataset used, etc.)

[1626] Output: Send registration application data

[1627] Specific operation: The user enters information such as "Music Generation Model A," "Music," "Music Production," and "Music Dataset 1" into the interface and presses the registration button.

[1628] Step 2:

[1629] Receiving and saving generated AI model registration information

[1630] The server receives the registration information for the generative AI model sent by the user and stores it in a database.

[1631] Input: Registration information for the generative model from the user

[1632] Output: Record added to database

[1633] Specific operation: The server receives the information "Music generation model A, music, music production, music dataset 1" from the user and stores it in the database.

[1634] Step 3:

[1635] Matching between generative models

[1636] The server automatically matches models from different fields based on information about generative AI models registered in the database.

[1637] Input: Information about the generative AI model stored in the database

[1638] Output: Matching result notification

[1639] Specific operation: The server selects a pair of "music generation model A" and "video generation model B" and sends a "project proposal" notification to the user.

[1640] Step 4:

[1641] Project details

[1642] The user receives a notification, reviews the details of the proposed project, and makes any necessary configurations (such as artifact format and output nodes).

[1643] Input: Project proposal notification, configuration information (format, output node, etc.)

[1644] Output: Sending a notification that the setup is complete

[1645] Specific operations: The user sets the length of the music to be generated, the resolution of the video, etc., and presses the "Start Project" button.

[1646] Step 5:

[1647] Implementing the generation process

[1648] The generative AI model runs on the device and executes the generation process based on the specified parameters.

[1649] Input: Setting parameters (song length, video resolution, etc.)

[1650] Output: Generated products (music files, video files)

[1651] Specific operation: "Music generation model A" generates a five-minute piece of music on User A's device, and at the same time, "video generation model B" generates a video linked to the music on User B's device.

[1652] Step 6:

[1653] Product monitoring and preservation

[1654] The server monitors the progress of the generation process in real time and stores the generation results.

[1655] Input: Progress data of the generation process

[1656] Output: Save generated results to database, progress notification

[1657] Specific operation: After generation is complete, the generated music and video files are saved in a database and the user is notified of the results.

[1658] Step 7:

[1659] Product evaluation

[1660] The server hosts a generative AI contest and performs a process to evaluate the products submitted by users.

[1661] Input: Submitted product

[1662] Output: Evaluation results summary, award notification

[1663] Specific operation: We will aggregate evaluations from experts and general users and award outstanding generative models and their developers.

[1664] Step 8:

[1665] User-participation creative activities

[1666] Ordinary users can also access the platform and use generative AI models to create their own unique works.

[1667] Input: Creation request using generative model

[1668] Output: Generated creative works (music, video, etc.)

[1669] Specific operation: The user creates a song using the music generation model and a video using the video generation model.

[1670] Step 9:

[1671] Sharing the results

[1672] The server provides an interface for users to store their generated work and share it with other users.

[1673] Input: Generated artwork

[1674] Output: Provides an interface for sharing

[1675] Specific operation: The generated music and video will be published on the platform and can be rated by other users.

[1676] (Application example 1)

[1677] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1678] Conventional generative AI models have focused on outputting products specialized in a single field, making it difficult for generative models from different fields to collaborate with each other or for users to participate in the generation process in real time. Furthermore, there is a lack of a mechanism for effectively utilizing evaluations and feedback of products to update the generated content, which limits the quality and diversity of the products.

[1679] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1680] In this invention, the server includes a means for registering and managing generative models from different fields, a means for matching collaboration and competition between generative models, a means for evaluating the products output by the generative models, a means for providing an interface for the generative models and users to engage in collaborative creative activities, and a means for linking different generative models to generate and view interactive content. This enables the linking of generative models from different fields to generate diverse, high-quality interactive content and user participation in real time.

[1681] A "generative model" is an algorithm that generates new data or content based on a specific dataset or algorithm.

[1682] “Registration and Management” refers to the process of adding a Generative Model to the Platform, storing its information, and updating or modifying it as needed.

[1683] "Collaboration" refers to the activity of different generative models working together to jointly produce a single product.

[1684] "Competition" refers to the process in which different generative models compete against each other in terms of their performance and output.

[1685] "Matching" refers to the process of selecting the optimal combination based on the characteristics and parameters of different generative models.

[1686] "Interactive content" refers to content whose generated content changes dynamically as users operate and provide feedback in real time.

[1687] "Interface" refers to the screen or tool that allows the user to input and operate the generative model.

[1688] "Evaluation" refers to the activity of diagnosing the product output by a generative model based on its quality, performance, etc., and providing the results as feedback.

[1689] "Real-time monitoring" refers to the process of instantly checking the progress of the generation process and maintaining a state in which necessary responses can be taken immediately.

[1690] "Preservation" refers to the process of securely storing generated data and content for later use and evaluation.

[1691] "Feedback" refers to the process of returning evaluations of the product and suggestions for improvement to the model.

[1692] This invention provides a system that links generative models from different fields to enable the generation and viewing of interactive content. An embodiment of this system is described in detail below.

[1693] 1. System Configuration

[1694] Server: The server contains the following main means:

[1695] Generative model management

[1696] Generative model matching method

[1697] Product evaluation method

[1698] Interface provision means

[1699] Real-time monitoring and feedback measures

[1700] To achieve these goals, the server uses high-performance computing hardware and a web framework such as Flask.

[1701] User device: Users access the system using a smartphone or head-mounted display (HMD) to generate and view content. A dedicated app for operating and linking different generative models is installed on the smartphone.

[1702] Generative Models: Generative models are algorithms specialized for a specific domain, such as music generation models or video generation models. They are hosted on a server and accessed via API endpoints.

[1703] 2. Operation overview

[1704] Users use a smartphone app to register generative models on the platform. After entering model information (model name, genre, purpose, dataset used, etc.), the model is saved on the server. The registered generative model is managed by the server and updated as needed.

[1705] Next, the server automatically matches models from different fields based on the characteristics of the registered generative models. For example, a music generation model and a video generation model are paired and notified to the user. The user can then start the project by checking the project details and configuring the necessary settings.

[1706] 3. Content Creation and Viewing

[1707] The user inputs a prompt to generate interactive content, for example:

[1708] Example prompt:

[1709] Input to the music generation model: "Generate a fast-paced, electronic-feeling soundtrack."

[1710] Input to the video generation model: "Generate a video with a fast camera movement against a futuristic cityscape."

[1711] Based on this, the server calls the music generation model and video generation model, respectively, and performs generation. The generated music and video are sent to the user's device in real time and can be viewed on a smartphone or HMD. Based on user feedback, the server updates the generated content as appropriate and advances the generation process.

[1712] The generated content is stored on the server and can be shared and rated by other users. The evaluation results are aggregated and outstanding generative models and their developers are awarded.

[1713] This allows generative models from different fields to work together, enabling the generation of diverse, high-quality interactive content and real-time user participation.

[1714] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1715] Step 1: Registering the Generative Model

[1716] Users use a smartphone app to register a generative model on the platform. Input includes detailed information about the generative model (model name, genre, purpose, dataset used, etc.). These details are sent to the server and stored in a database. As an output, a message is sent to the user informing them that the generative model has been registered.

[1717] Step 2: Managing the Generative Model

[1718] The server manages information about registered generative models. It references the characteristics of the generative models stored in the database (performance, genre, training data, etc.) and updates and corrects them as needed. This management process periodically checks the information about the generative models to keep them up to date. As an output, a report containing the latest information about the generative models is generated.

[1719] Step 3: Matching the generative model

[1720] The server automatically matches models from different fields based on the characteristics of the registered generative models. For example, it analyzes features to find the optimal combination to pair a music generation model with a video generation model. The characteristic information of each generative model is used as input, and the matching results are reported to the user as output.

[1721] Step 4: Starting the project

[1722] Upon receiving the notification, the user checks the project details and makes the necessary settings (such as the product format and output nodes). The input includes the matched generative model and project details. As an output, a project start confirmation message is sent to the server, and the project is launched.

[1723] Step 5: Implementing the generation process

[1724] When a project starts, the user inputs a prompt, for example, "Please generate a soundtrack with a fast tempo and an electronic feel." The server receives this prompt and sends generation requests to the music generation model and the video generation model. The generative models perform the generation work based on the specified parameters and return the output results (music and video) to the server.

[1725] Step 6: Integrating the products

[1726] The server receives the generated results (music and video) sent from the generative model. To integrate these generated results, the server adjusts the timing and coding of the music and video. The input includes the music generation results and the video generation results, and interactive content is generated as the output.

[1727] Step 7: Interactive viewing

[1728] The generated interactive content is delivered to the user device in real time. The user views the content using a smartphone or HMD and sends feedback to the server. The input includes the user's viewing data and feedback, and the output is an instruction to update the generated content.

[1729] Step 8: Update the generation process

[1730] The server updates the generated content based on user feedback. It analyzes the generated content for improvement and sends new parameters to the generative model for regeneration. This process aims to improve the quality of the product and increase user satisfaction. The input includes user feedback data, and the output is the improved product that is redistributed.

[1731] Step 9: Rate and share your creation

[1732] The final generated content is stored on a server and published on the platform for general users to evaluate. Users can rate and comment on other creations and provide their own feedback. The input includes evaluation data, and the output is the evaluation results that are fed back to the developer of the generative model.

[1733] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1734] The present invention is a system that aims to improve the diversity and creativity of generative models from different fields by allowing them to collaborate and compete with each other. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions, allowing the output of the generative model to be customized according to the user's emotions. An embodiment of this system is described in detail below.

[1735] System Configuration

[1736] The system mainly consists of the following components:

[1737] A server that registers and manages generative models

[1738] A server that performs matching between generative models

[1739] User device equipped with an emotion engine

[1740] A server that provides evaluation and award functions for products

[1741] An interface for collaborative creative activities between users and generative models

[1742] 1. Registering and Managing Generative Models

[1743] User:

[1744] Users register generative models on the platform. When registering, they are provided with an interface to input detailed information about the generative model (such as the model name, genre, purpose, and dataset used).

[1745] server:

[1746] The server receives the information about the generative model sent by the user and stores it in a database, allowing for centralized management of the characteristic information of the generative model.

[1747] 2. Matching and project launch

[1748] server:

[1749] The server pairs generative models from different fields based on the registered generative model information, and conducts matching for collaboration and competition. The matching results are notified to the user.

[1750] User:

[1751] The user who receives the notification checks the details of the project and makes the necessary settings (format of the generated output, output node, etc.), which starts the project.

[1752] 3. Implementing the generation process

[1753] Device:

[1754] The generation process is initiated based on the specified parameters on the device where the user's generative model is running. The emotion engine has the ability to recognize emotions by analyzing the user's facial expressions and voice data.

[1755] server:

[1756] The server monitors the progress of the generation process in real time, stores the generated results in a database, and receives feedback from the emotion engine to adjust the output of the generative model.

[1757] 4. Product Evaluation and Awards

[1758] server:

[1759] The server provides an interface for experts and general users to evaluate the generated models, and aggregates the evaluation results to recognize and reward outstanding generative models and their developers.

[1760] User:

[1761] Users can participate in rating the products and leave comments, which provides feedback for improvements to the generative model and new ideas.

[1762] 5. User-participatory creative activities

[1763] User:

[1764] Ordinary users can access the platform and create their own works using generative models. For example, a user can select a music generation model, create a song, and then create a video to go with the song using a video generation model.

[1765] server:

[1766] The server provides an interface for users to store and share their generated works with other users, and can further customize the creations based on data from the emotion engine.

[1767] Specific examples

[1768] For example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches the two, and users A and B start a project to jointly create an artwork (image + music). When user B creates music, the emotion engine analyzes user B's facial expressions in real time and adjusts the tempo and tone of the music in real time according to the emotion. The final artwork created is published on the platform and receives ratings and comments from other users.

[1769] In this way, the present invention increases the diversity and creativity of generative models, while at the same time providing a more personalized experience by recognizing the user's emotions and customizing the product.

[1770] The processing flow will be explained below.

[1771] Step 1:

[1772] Registering a Generative Model

[1773] User:

[1774] The user accesses the registration form for the generative model and enters detailed information about the generative model (model name, genre, purpose, dataset used, etc.). By clicking the submit button, the information is sent to the server.

[1775] server:

[1776] The submitted generative model data is received and saved in the database. Once saved, a notification of registration completion is sent to the user.

[1777] Step 2:

[1778] Managing Generative Models

[1779] server:

[1780] The system periodically updates the information about generative models in the database and provides a user-accessible management screen that displays a list of registered generative models and their respective characteristics.

[1781] User:

[1782] You can view and edit information about registered generative models through the management screen.

[1783] Step 3:

[1784] Matching implementation

[1785] server:

[1786] It analyzes the generative model information in the database, runs an algorithm to pair generative models from different fields, generates a list of paired models, and notifies the user of the matching results.

[1787] Step 4:

[1788] Starting a project

[1789] User:

[1790] Receive a notification of the matching result, confirm the project details, and if you agree to start the project, configure the required settings such as the output format and output node.

[1791] server:

[1792] Receives configuration information from the user and prepares the project execution environment. Once preparations are complete, notifies the user that the project has started.

[1793] Step 5:

[1794] Implementing the generation process

[1795] Device:

[1796] The generation process is initiated based on the specified parameters on the device where the user's generative model is running. During this process, the emotion engine analyzes the user's facial expressions and voice in real time to collect emotion data.

[1797] server:

[1798] The progress of the generation process is monitored in real time, the generation results are stored in a database, and the output of the generative model is adjusted based on the emotion data obtained from the emotion engine.

[1799] Step 6:

[1800] Product evaluation

[1801] server:

[1802] We provide an interface for experts and general users to evaluate the generated models, and we aggregate the evaluation results to recognize and reward excellent generative models and their developers.

[1803] User:

[1804] Through the rating interface, you can rate the product, leave comments, and view other users' ratings.

[1805] Step 7:

[1806] User-participation creative activities

[1807] User:

[1808] Ordinary users can access the platform and create their own works using the generative models provided. For example, they can select a music generation model to create a song, and then use a video generation model to create a video to go along with that song.

[1809] server:

[1810] It stores user-generated works in a database, provides an interface for sharing them with other users, and manages other users' comments on the shared works.

[1811] Examples:

[1812] User A registers an image generation model on the platform, and User B registers a music generation model. The server matches these models and proposes a project to jointly create an artwork. When User B creates music, the emotion engine analyzes User B's facial expressions in real time and adjusts the tempo and tone of the music according to their emotions. The final artwork is published on the platform and can be rated and commented on by other users.

[1813] In this way, the present invention not only increases the diversity and creativity of generative models, but also provides a personalized experience that responds to the user's emotions.

[1814] Example 2

[1815] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1816] In conventional generative modeling systems, collaboration and competition between generative models from different fields was difficult, limiting the diversity and creativity of the products they produced. Furthermore, they lacked methods for providing personalized products that respond to the user's emotions. This resulted in a uniform user experience that lacked individuality, reducing the appeal and value of the products.

[1817] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for registering and managing generative models from different fields, means for matching collaboration and competition between generative models, means for customizing products output by generative models based on user emotions, means for evaluating products output by generative models, and means for providing an interface for generative models and users to collaborate on creative activities. This facilitates collaboration and competition between generative models and makes it possible to provide personalized products that correspond to user emotions.

[1818] A "generative model" is a machine learning algorithm that generates new data or products for a specific purpose or use.

[1819] A "means for registration and management" is a system or interface that allows users to input detailed information about generative models and store and organize that information.

[1820] "Matching means" is a function that analyzes the features and characteristics of different generative models and performs pairings for optimal collaboration or competition.

[1821] The "means for customization" is a system that analyzes the user's emotions and adjusts the characteristics and parameters of the product based on those emotions.

[1822] The "means of evaluation" is a mechanism for evaluating the quality and suitability of the products output by the generative model based on feedback from users and experts.

[1823] The "interface for creative activities" is a user interface used when a generative model and a user collaborate on creative activities, and allows for adjustment of input and output.

[1824] This system aims to improve the diversity and creativity of generative models by enabling collaboration and competition between them in different fields. Characteristic features of the system include the registration and management of generative models, matching between generative models, evaluation of products, customization of products based on user emotions, and provision of an interface for collaborative creation.

[1825] System Configuration

[1826] The system consists of the following elements:

[1827] A server that registers and manages generative models

[1828] A server that performs matching between generative models

[1829] User device equipped with an emotion engine

[1830] A server that provides evaluation and award functions for products

[1831] An interface for collaborative creative activities between users and generative models

[1832] 1. Registering and Managing Generative Models

[1833] User:

[1834] Users register their generative models on the platform. When registering, an interface is provided for entering detailed information about the generative model (such as the model name, genre, purpose, and dataset used). For example, users can enter data into the form using a web browser and then press the "Register" button to send the generative model information to the server.

[1835] server:

[1836] The server receives the generative model information sent by the user and stores it in a database. This storage enables centralized management of generative models. The received data is parsed into JSON format and stored in an SQL database such as MySQL.

[1837] 2. Matching the Generative Model and Starting the Project

[1838] server:

[1839] The server pairs generative models from different fields based on the registered generative model information, and performs matching for collaboration and competition. For example, k-means clustering is used as a matching algorithm. The matching results are notified to the user.

[1840] User:

[1841] After receiving the notification, the user checks the details of the project and makes the necessary settings (such as the format of the generated output, output nodes, etc.). For example, the user selects the project format and presses the "Start Project" button to start the project.

[1842] 3. Implementing the generation process

[1843] Device:

[1844] The device where the user's generative model runs starts the generation process based on the specified parameters. The device's emotion engine analyzes the user's facial and voice data to recognize the user's emotions. Examples of use cases include libraries such as OpenCV and TensorFlow.

[1845] server:

[1846] The server monitors the progress of the generation process in real time, stores the generation results in a database, and receives feedback from the emotion engine to adjust the output of the generative model. The generation results are stored in cloud storage, and the metadata is stored in a database.

[1847] 4. Product Evaluation and Awards

[1848] server:

[1849] The server provides an interface for evaluating the generated models and accepts evaluations from general users and experts. Evaluation scores and comments are stored in a database and periodically compiled. Based on this, an algorithm is run to reward excellent generative models and their developers, and the results are notified.

[1850] User:

[1851] Users can participate in the evaluation of the generated results and leave comments, which leads to improvements in the generative model and new ideas.

[1852] 5. User-participatory creative activities

[1853] User:

[1854] Ordinary users can access the platform and create their own works using the provided generative models. For example, they can create a song using the music generation model and then create a video to accompany that song using the video generation model.

[1855] server:

[1856] The server provides an interface for users to store and share their generated works with other users, and allows further customization of the works based on data from the emotion engine.

[1857] Specific examples

[1858] For example, user A registers an image generation model on the platform, and user B registers a music generation model. The server matches the two, and users A and B start a project to jointly create an artwork (images and music). When user B creates music, the emotion engine analyzes user B's facial expressions in real time and adjusts the tempo and tone of the music according to their emotions. The final artwork created is published on the platform and receives ratings and comments from other users.

[1859] Example prompt

[1860] The following examples could be used as prompts for a generative AI model:

[1861] "Using image generation models and music generation models, we create artwork based on the user's emotions. First, user A generates a landscape painting using the image generation model. Based on that landscape painting, user B creates background music using the music generation model. The tempo and tone of the background music can be adjusted according to user B's emotions (joy, sadness, surprise, etc.)."

[1862] In this way, the system of the present invention increases the diversity and creativity of generative models, while at the same time providing a more personalized experience by recognizing the user's emotions and customizing the product.

[1863] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1864] Step 1: Registering the Generative Model

[1865] User:

[1866] The user enters detailed information about the generative model (such as the model name, genre, purpose, and dataset used) into the interface and presses the "Register" button. The input can be in text or file format.

[1867] Specific behavior:

[1868] Users access the registration page via a web browser or smartphone app and enter the required information. When the user presses the "Register" button, the entered information is sent to the server as an HTTP POST request.

[1869] input:

[1870] Detailed information about the generative model (model name, genre, purpose, dataset used, etc.).

[1871] output:

[1872] An HTTP POST request containing detailed information about the generated model.

[1873] Step 2: Managing the Generative Model

[1874] server:

[1875] The server receives the information about the generative model sent by the user, stores it in a database, and notifies the user when the storage is complete.

[1876] Specific behavior:

[1877] The server analyzes the data of the received HTTP POST request and parses it into JSON format. The analyzed data is saved in a database such as MySQL or PostgreSQL using an INSERT statement. Once the saving is complete, the user is notified via HTTP response or email.

[1878] input:

[1879] Detailed information about the generated model sent as an HTTP POST request.

[1880] output:

[1881] Information about the generated model saved in the database and notification of completion of saving.

[1882] Step 3: Matching the generative model

[1883] server:

[1884] The server matches generative models from different fields based on the generative model information stored in the database, using a matching algorithm such as k-means clustering.

[1885] Specific behavior:

[1886] The server runs a regularly scheduled job to retrieve generative model information from the database, and then runs the k-means clustering algorithm based on the retrieved dataset to determine matching pairs. The matching results are then sent to users via a notification system.

[1887] input:

[1888] A set of information about the generative model stored in a database.

[1889] output:

[1890] Informing generative model matching pairs across disciplines.

[1891] Step 4: Starting the project

[1892] User:

[1893] The user who receives the notification checks the project details, sets up the necessary settings (product format, output node, etc.), and clicks the start button for the project.

[1894] Specific behavior:

[1895] The user clicks on the notification to access the project details screen and make the necessary settings. Once the settings are complete, they press the "Start Project" button, which sends the project settings information to the server.

[1896] input:

[1897] Project configuration information (product format, output nodes, etc.).

[1898] output:

[1899] Project start request.

[1900] Step 5: Implementing the generation process

[1901] Device:

[1902] The user's generative model starts the generation process based on the specified parameters, and the emotion engine installed on the device analyzes the user's facial expressions and voice data to recognize emotions.

[1903] Specific behavior:

[1904] The device collects the user's facial and voice data using a camera and microphone, and analyzes it using an emotion engine (e.g., OpenCV or TensorFlow). Based on the analysis results, the parameters of the generative model are dynamically adjusted and the generation process is initiated. The generated results are stored locally or in the cloud.

[1905] input:

[1906] User emotion data (facial expressions, voice).

[1907] output:

[1908] The product based on the adjusted parameters.

[1909] server:

[1910] It monitors the progress of the generation process in real time, stores the generated results in a database, and receives feedback from the emotion engine to adjust the output of the generative model.

[1911] Specific behavior:

[1912] The server receives progress data from the device and updates the monitoring panel. Once generated, it uploads the generated product to cloud storage and stores the metadata in a database. It may also send regeneration commands to the generative model based on feedback from the emotion engine.

[1913] input:

[1914] Progress data for the generation process, and feedback for the emotion engine.

[1915] output:

[1916] Generation results and regeneration instructions stored in a database.

[1917] Step 6: Evaluate the product

[1918] server:

[1919] It provides an interface for evaluating the generated results, accepts evaluations from users and experts, and aggregates the evaluation results.

[1920] Specific behavior:

[1921] The server dynamically generates evaluation pages for products and provides an interface where users can enter evaluation scores and comments. The evaluation data from users and experts is stored in a database and periodically aggregated.

[1922] input:

[1923] Evaluation data (scores, comments) from users and experts.

[1924] output:

[1925] Aggregated evaluation results.

[1926] Step 7: Awarding the product

[1927] server:

[1928] Based on the evaluation results, an algorithm will be run to award outstanding generative models and their developers. The award results will be notified.

[1929] Specific behavior:

[1930] The server runs an algorithm to determine the recipients of the awards based on the aggregated results, and notifies the selected generative models and developers of the awards via in-app messages and emails.

[1931] input:

[1932] Aggregated evaluation results.

[1933] output:

[1934] Notification of award recipients.

[1935] Step 8: User-participation creative activity

[1936] User:

[1937] Ordinary users access the platform and create their own works using the provided generative models. They then publish the works on the platform and receive ratings and comments from other users.

[1938] Specific behavior:

[1939] Users select a generative model and create a landscape painting using the image generation model. They then upload the landscape painting to the platform and set it to be public. The published work can then receive ratings and comments from other users.

[1940] input:

[1941] Artwork data based on generative models.

[1942] output:

[1943] Published generated works and their ratings and comments.

[1944] This clarifies the processing flow of the entire system, allowing for a detailed understanding of what inputs and outputs exist at each step and what specific operations are performed.

[1945] (Application example 2)

[1946] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1947] Modern generative AI models lack collaboration and competition to improve their diversity and creativity, and the generated content is rarely properly tailored to the user's emotions, resulting in insufficient personalization of the user experience.

[1948] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1949] In this invention, the server includes a means for registering and managing generative models from different fields, a means for matching collaboration and competition between generative models, a means for evaluating the products output by the generative models, a means for providing an interface for collaborative creative activities between the generative models and users, and a means for recognizing the user's emotions and adjusting the products based on those emotions. This increases the diversity and creativity of generative AI models and makes it possible to provide personalized products according to the user's emotions.

[1950] "Generative models from different fields" is a general term for AI models that operate in different genres or applications, such as image generation models and music generation models.

[1951] "Means for registration and management" refers to an interface or program that provides the functionality to collect, classify, and store information about generative models on a server.

[1952] "Means for matching collaboration and competition" refers to algorithms or programs that pair different generative models together to enable them to collaborate or compete.

[1953] "Means for evaluating the product" refers to an interface or system for analyzing and evaluating the results output by the generative model.

[1954] An "interface for users to engage in collaborative creative activities" is a system that provides a user interface and tools that allow users to create content in collaboration with generative models.

[1955] "Means for recognizing emotions and adjusting the generated content based on those emotions" refers to algorithms or programs that analyze the user's emotions in real time using a camera or microphone, and change or adjust the generated content based on the results.

[1956] This invention is a system that allows generative AI models from different fields to collaborate and compete with each other, improving their diversity and creativity. This system incorporates an emotion engine that recognizes the user's emotions, allowing the output of the generative model to be customized according to the user's emotions.

[1957] System Configuration

[1958] 1. Register and manage generative models:

[1959] Users register their generative models on the platform. When registering, an interface is provided for entering detailed information about the generative model (model name, genre, purpose, dataset used, etc.). The server receives this information and stores it in a database, enabling centralized management.

[1960] 2. Matching different generative models:

[1961] The server pairs generative models from different fields based on the registered generative model information, and performs matching for collaboration and competition. The matching results are notified to the user, who can then start the project by checking the project details and making the necessary settings.

[1962] 3. Implementing the generation process:

[1963] The device running the generative model starts the generation process based on specified parameters. The emotion engine analyzes the user's facial expressions and voice data to recognize emotions. The server monitors the progress of the generation process in real time and stores the generation results in a database. It also receives feedback from the emotion engine and adjusts the output of the generative model as appropriate.

[1964] 4. Product Evaluation and Awards:

[1965] The server provides an interface for experts and general users to evaluate the generated models, and has a mechanism to aggregate the evaluation results and recognize and award excellent generative models and their developers.

[1966] 5. User-participatory creative activities:

[1967] Ordinary users can access the platform and create their own works using generative models. For example, a user can choose a music generation model, create a song, and then create a video using a video generation model to accompany the song. The server stores the generated works and provides an interface for sharing them with other users.

[1968] Hardware and software used

[1969] Camera: Used to capture the user's facial expressions.

[1970] Microphone: Used to capture the user's voice.

[1971] OpenCV: An image processing library used to acquire and process camera images.

[1972] Librosa: An audio processing library for reading and analyzing audio files.

[1973] Transformers: A library that implements natural language processing (NLP) models for emotion recognition.

[1974] Specific examples

[1975] For example, suppose a user uses the "Collabo Creator" app to register a landscape image generation model and a classical music generation model. Once the generative models are matched, the app performs real-time emotion analysis using the user's camera and microphone. If the user is moved, the app can slow down the tempo of the generated music and add a sunset element to the generated scenery. This content can be shared with other users and rated.

[1976] Prompt Sentence Examples

[1977] "We combine user-input image generation models with music generation models, recognize emotions in real time using a camera and microphone, and customize generated content based on emotions."

[1978] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1979] Step 1:

[1980] The user registers a generative model on the platform. The user inputs detailed information about the generative model (model name, genre, purpose, dataset used, etc.) and sends it to the server through the interface. The input is the detailed information about the generative model, and the output is information about the generative model stored in the server's database.

[1981] Step 2:

[1982] The server pairs generative models from different fields based on the registered generative model information, and performs matching for collaboration and competition. The input is the information of the generative models stored in the database, and the output is the combination information of the paired generative models.

[1983] Step 3:

[1984] The server notifies the user of the matching results. The user receives the notification, checks the project details, and makes the necessary settings (such as the format of the generated output, output nodes, etc.). The input at this time is the matching results and the user's settings information, and the output is an instruction to start the project.

[1985] Step 4:

[1986] The project generation process begins on the device. The emotion engine captures the user's facial expressions and voice data in real time via the camera and microphone, and recognizes emotions. The input is the user's facial expression data and voice data, and the output is the recognized emotion information. In the processing step, OpenCV analyzes facial expressions, and Librosa processes the voice to extract emotions.

[1987] Step 5:

[1988] The server receives feedback from the emotion engine and adjusts the output of the generative model. The input is the recognized emotion information, and the output is an emotion-adjusted artifact (e.g., music tempo or image color tone). The processing step involves retraining or adjusting the generative model in real time with parameters corresponding to the emotion information.

[1989] Step 6:

[1990] The generated artifacts are sent to a server and stored in a database. The input is the emotion-adjusted artifact, and the output is the artifact stored in the database. The processing step involves storing the artifacts in appropriate storage and managing them in an easily accessible format.

[1991] Step 7:

[1992] The server provides an interface that allows experts and general users to evaluate the products. The input is the saved products, and the output is the evaluation data and the evaluation results. In the processing step, the evaluation interface is provided and the user evaluations are aggregated.

[1993] Step 8:

[1994] Users can participate in the evaluation of the product and leave comments. The input is the product and the evaluation interface, and the output is evaluation points and comments. In the processing step, user feedback is collected and the generative model is improved based on the evaluation results.

[1995] Step 9:

[1996] The server recognizes the developers of excellent products and generative models and notifies the user. The input is the evaluation result, and the output is the recognition notice and recognition list. In the processing step, the evaluation result is analyzed, excellent products are selected, and a recognition notice is issued.

[1997] Step 10:

[1998] Users can share their work with other users. The input is the generated work, and the output is a sharing link or a public page. In the processing step, a link is generated or a public page is created to share the work with other users.

[1999] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2000] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2001] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2002] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2003] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2004] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2005] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2006] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2007] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2008] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2009] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2010] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2011] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2012] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2013] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2014] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2015] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2016] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2017] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2018] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2019] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2020] The following is further disclosed regarding the above embodiment.

[2021] (Claim 1)

[2022] A means to register and manage generative models from different fields,

[2023] means for matching collaboration and competition between said generative models;

[2024] A means for evaluating a product output by the generative model;

[2025] A means for providing an interface for a user to collaborate with the generative model in creative activities;

[2026] A system including:

[2027] (Claim 2)

[2028] and means for monitoring the progress of said collaboration and competition in real time and storing the results of said collaboration and competition.

[2029] 10. The system of claim 1.

[2030] (Claim 3)

[2031] The system includes a means for allowing general users to evaluate the generated product, and a means for evaluating a...

Claims

1. A means for registering and managing generative models from different domains; means for matching collaboration and competition between said generative models; A means for evaluating a product output by the generative model; A means for providing an interface for a user to collaborate with the generative model in creative activities; A system including:

2. and means for monitoring the progress of said collaboration and competition in real time and storing the results of said collaboration and competition. The system of claim 1 .

3. The system includes a means for allowing general users to evaluate the generated product, and a means for evaluating and awarding the generative model based on the evaluation results. The system of claim 1 .

Images