System

Abstract

A system is provided.SOLUTION: A system comprising: means for providing a user interface for selecting a plurality of generator artificial intelligence models; means for collecting documents and sample data of the selected generator artificial intelligence models; means for analyzing the collected documents and sample data to identify functionality of each generator artificial intelligence model; means for generating an integrated prompt based on the identified functionality; means for building a new generator artificial intelligence model with the generated prompt; and means for providing the built generator artificial intelligence model to a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, multiple generative artificial intelligence (AI) models specialized for specific purposes have emerged. However, these typically function independently, requiring users to share context between each of the generative AI models, which is a time-consuming task. Furthermore, when combining different generative AI models, it is cumbersome to individually understand and appropriately utilize the characteristics and functions of each model, which reduces efficiency. Therefore, there is a need for a system that can integrate the functions of multiple generative AI models and enable users to use them consistently. [Means for solving the problem]

[0005] The present invention provides a user interface that allows a user to select multiple generative AI models, and collects documents and sample data for the selected models. The collected documents and sample data are analyzed to identify the capabilities of each generative AI model. Then, integrated prompts are generated based on the identified capabilities, and a new generative AI model is constructed using the generated prompts. Finally, by providing this new generative AI model to the user, the user can seamlessly utilize the capabilities of multiple generative AI models in a single system, reducing the need for shared context and effort.

[0006] Specifically, the system includes a means for analyzing the document of the generative AI model using natural language processing technology to understand its characteristics, a means for analyzing the sample data to identify input and output formats, a means for generating prompts and automatically constructing a new generative AI model based on the prompts, and a means for providing the constructed generative AI model to a user's device. This system allows users to use multiple generative AI models in an integrated manner, improving convenience.

[0007] A "generative artificial intelligence model" is an artificial intelligence system that is trained to generate specific outputs in response to specific inputs.

[0008] A "user interface" is an interface through which a user interacts with a system, and is a means by which the user makes selections and inputs.

[0009] "Documentation" refers to a document that describes the functions, usage, characteristics, etc. of a generative artificial intelligence model.

[0010] "Sample data" refers to example inputs to a generative artificial intelligence model and their corresponding output examples, and is data used to concretely demonstrate the model's functions.

[0011] "Natural language processing technology" is a technology for analyzing and understanding text data, and is used to analyze the characteristics of language.

[0012] A "prompt" is an instruction given to a generative artificial intelligence model to obtain a specific output.

[0013] "Integration" refers to combining multiple different things into one, and in this system it means combining the functions of multiple generative artificial intelligence models into one integrated model.

[0014] "Building" involves creating a new generative artificial intelligence model based on predefined prompts.

[0015] To "analyze" means to examine and understand the structure and properties of given data or documents.

[0016] "Providing" means handing over the newly generated model and information to the user in a usable state.

[0017] A "terminal" is an electronic device operated by a user, such as a computer or smartphone, that interfaces with the system. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] Program processing overview

[0040] The present invention provides a system that allows users to integrate and use multiple generative artificial intelligence models (generative AI models). This system allows users to select models through a user interface, analyze their documentation and sample data, understand the functions of each model, generate new prompts, and finally construct a new integrated generative AI model to provide to the user.

[0041] Program Operation

[0042] 1. User model selection

[0043] Through the system's user interface, users can select from multiple generative AI models they wish to use, based on their specific tasks and objectives.

[0044] 2. Collecting model information

[0045] The device sends documents and sample data (example inputs and outputs) for each generative AI model selected by the user to the server.

[0046] 3. Document Analysis

[0047] The server analyzes the received documents using natural language processing (NLP) technology to identify the characteristics and functions of each generative AI model. For example, a document about an AI model that explains a paper will be analyzed to identify its function of explaining the paper.

[0048] 4. Analysis of sample data

[0049] The server analyzes sample data and understands the specific input and output formats of each model. For example, based on the input example of a code auto-generation AI model, it understands that when program specifications are input, code will be output.

[0050] 5. Prompt Generation

[0051] The server generates new prompts based on the capabilities of each identified model, which are designed to reflect the user's intent and integrate the capabilities of multiple generative AI models.

[0052] 6. Building a new generative AI model

[0053] The server then builds a new generative AI model based on the generated prompts, which provides different functions such as paper explanation, code generation, and illustration generation in a single, unified interface.

[0054] 7. Providing a Generative Model

[0055] The server then provides this newly constructed generative AI model to the user's device, where the user can use this integrated model to seamlessly execute a series of tasks.

[0056] Specific examples

[0057] For example, the following scenario is possible:

[0058] A user wants to read a paper, understand its content, generate related code based on the paper, and create an illustration to visually represent it. First, the user logs into the Mix Box platform and selects one of three models: a paper explanation AI model, an automatic code generation AI model, or an illustration generation AI model.

[0059] The device sends the documents and sample data of these models to the server, which analyzes them to understand the characteristics of each model. The server then creates an integrated prompt, such as "generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code." Based on this prompt, the server then builds a new generative AI model and provides it to the user.

[0060] Users input a paper using the new generative AI model, which then generates a summary of the paper, along with code and related illustrations based on the summary, seamlessly creating a consistent process that allows users to efficiently and conveniently leverage the capabilities of multiple generative AI models.

[0061] In this way, the present invention provides a system that allows users to use the functions of multiple generative AI models in an integrated manner, and is realized in a concrete form for sharing context and reducing effort.

[0062] The processing flow will be explained below.

[0063] Step 1:

[0064] Users log in to the Mix Box platform and select the generative AI model they want to use. Specific examples include a paper explanation AI model, an automatic code generation AI model, and an illustration generation AI model.

[0065] Step 2:

[0066] The device collects documentation and sample data (example inputs and outputs) for each generative AI model selected by the user, and sends this data to the server.

[0067] Step 3:

[0068] The server analyzes the received documents using natural language processing (NLP) technology. For example, from a document about an AI model explaining a paper, it understands that the model has the function of explaining a paper. It also analyzes AI models for automatic code generation and illustration generation in the same way.

[0069] Step 4:

[0070] The server analyzes the received sample data. Specifically, it identifies the input and output formats of each generative AI model. For example, it understands that the input example of a code auto-generation AI model is a program specification, and that the output example is code based on that specification.

[0071] Step 5:

[0072] The server generates an integrated prompt based on the analysis results. This prompt combines the functions of multiple generative AI models. For example, it creates a prompt that says, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[0073] Step 6:

[0074] The server uses these prompts to build a new generative AI model that combines the capabilities of paper commentary, code generation, and illustration generation.

[0075] Step 7:

[0076] The server then provides the newly constructed generative AI model to the user's device, allowing the user to seamlessly execute a series of tasks using this integrated model.

[0077] Step 8:

[0078] Users can then use the new generative AI models provided to them to perform specific tasks. For example, by inputting a paper, a summary of the paper is generated, code based on the summary is generated, and then illustrations related to the code are generated. In this way, users can efficiently use the functions of multiple generative AI models in a single system.

[0079] Through these steps, the system enables users to integrate the functions of multiple generative AI models, reducing the need for context sharing and effort.

[0080] Example 1

[0081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0082] Currently, there is no system for effectively integrating multiple generative AI models. Users must select models individually, input data for each model separately, and obtain results. This process is time-consuming, labor-intensive, and inefficient. Furthermore, it is difficult to integrate data and functions between different generative AI models, making it difficult to obtain consistent results.

[0083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0084] In this invention, the server includes means for providing a user interface for selecting multiple generative AI models, means for collecting information and sample data on the selected generative AI models, means for analyzing the collected documents and sample data and identifying functions of each generative AI model, means for generating integrated prompts based on the identified functions, means for providing the constructed generative AI model to the user, means for generating prompts that reflect the user's intentions and integrate the functions of the multiple generative AI models, and means for constructing a new generative AI model based on the generated prompts and seamlessly executing a series of tasks. This enables the user to efficiently integrate and use multiple generative AI models and obtain consistent results.

[0085] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates output such as text, images, or program code based on user input.

[0086] A "user interface" refers to a screen or input device that allows a user to interact with a system, and is the means by which a user performs the necessary operations and selections.

[0087] "Sample data" refers to a dataset that contains specific examples to demonstrate the format and content of inputs and outputs to a generative artificial intelligence model.

[0088] "Documentation" refers to documents that describe the functions, specifications, and usage of a generative artificial intelligence model, and includes information essential for understanding and using the model.

[0089] "Natural language processing technology" is a general term for technologies that enable computers to understand and manipulate text data, and includes technologies for analyzing documents and extracting meaning.

[0090] A "prompt" is text given to a generative artificial intelligence model in the form of an instruction or question, and is an input sentence that causes the model to perform a desired action.

[0091] This invention relates to a system that allows users to use multiple generative artificial intelligence models (generative AI models) in an integrated manner. This system mainly consists of a server, a terminal, and a user. The processes and roles of each are explained in detail below.

[0092] System Overview

[0093] The user selects a generative AI model through the system's user interface, after which the server and device work together to collect and analyze the necessary data, ultimately providing the user with an integrated generative AI model.

[0094] Hardware and Software

[0095] server

[0096] A server equipped with high-performance computers and natural language processing technology (e.g., using spaCy or NLTK as an NLP toolkit).

[0097] Terminal

[0098] A computer, tablet, or smartphone with an internet connection.

[0099] User Interface

[0100] The interface is provided using a web browser or a dedicated application.

[0101] Data processing and calculation

[0102] 1. User model selection

[0103] Through the interface, users can select the generative AI model they want to use, such as an AI model for paper commentary, automatic code generation, or illustration generation.

[0104] 2. Collecting model information

[0105] The device collects documentation and sample data about the model selected by the user and sends it to the server, including the model's features, specifications, and example input and output data.

[0106] 3. Analyzing Documents and Sample Data

[0107] The server analyzes the collected documents using natural language processing technology and extracts the characteristics and functions of each generative AI model. For example, from documents of an AI model that explains academic papers, it identifies the literature explanation function.

[0108] The server also analyzes sample data to understand the input and output formats of each model. For example, from an example input for a code auto-generation AI model, it identifies the program specification input and the corresponding code output.

[0109] 4. Prompt generation and building a new generative AI model

[0110] The server generates an integrated prompt based on the analysis results. This prompt reflects the user's intent and integrates the functions of multiple generative AI models. For example, it creates a prompt that asks, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[0111] Based on these prompts, we build a new generative AI model that integrates the functions of paper explanation, code generation, and illustration generation.

[0112] 5. Providing a Generative Model

[0113] The server provides the newly constructed generative AI model to the user, who can then use the model to seamlessly execute different tasks.

[0114] Specific examples

[0115] For example, consider a scenario where a user logs in to the Mix Box platform and performs the following steps: First, the user selects an AI model for explaining papers, then an AI model for automatically generating code, and finally an AI model for generating illustrations. The device then sends these documents and sample data to the server, which then analyzes each data and generates an integrated prompt.

[0116] An example prompt might be, "Generate a summary of a paper, generate Python code based on that summary, and generate illustrations related to that code." A new generative AI model is built based on this prompt, allowing users to efficiently and seamlessly utilize multiple AI functions.

[0117] In this way, the present invention provides a system that allows users to integrate the functions of multiple generative AI models and achieve their goals through a consistent process.

[0118] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0119] Step 1:

[0120] User model selection

[0121] Specific actions

[0122] Users access the system's user interface and select from multiple generative artificial intelligence models according to their goals and challenges.

[0123] input

[0124] A list of generative AI models based on user selections (clicks, etc.)

[0125] output

[0126] A list of selected generative AI models

[0127] Step 2:

[0128] Collecting Model Information

[0129] Specific actions

[0130] The device collects documentation and sample data for the generative AI model selected by the user, including API calls and retrieval from a database, and sends the collected data to a server.

[0131] input

[0132] A list of generative AI models selected by the user

[0133] output

[0134] Documentation and sample data for each model

[0135] Step 3:

[0136] Document Parsing

[0137] Specific actions

[0138] The server uses natural language processing (NLP) technology to analyze the documents of each generative AI model received from the device, thereby extracting the characteristics and functions of each AI model.

[0139] input

[0140] Documentation for each generative AI model (PDF, HTML, etc.)

[0141] output

[0142] A list of the characteristics and capabilities of each AI model

[0143] Step 4:

[0144] Analysis of sample data

[0145] Specific actions

[0146] The server analyzes the sample data received from each generative AI model to understand the specific input and output formats, thereby understanding how each model operates.

[0147] input

[0148] Sample data for each generative AI model (JSON, CSV, etc.)

[0149] output

[0150] Details of the input and output formats for each model

[0151] Step 5:

[0152] Prompt Generation

[0153] Specific actions

[0154] The server generates new prompts based on the characteristics and capabilities of each analyzed model, which are designed to reflect the user's intent and integrate the capabilities of multiple AI models.

[0155] input

[0156] Details of the characteristics and functions of each model, as well as input and output formats

[0157] output

[0158] Integrated prompts

[0159] Step 6:

[0160] Building a new generative AI model

[0161] Specific actions

[0162] The server then uses the generated prompts to build a new generative AI model, which is designed to integrate multiple functions and guide users through tasks in a consistent process.

[0163] input

[0164] Integrated prompts

[0165] output

[0166] New generative AI models

[0167] Step 7:

[0168] Providing generative AI models

[0169] Specific actions

[0170] The server provides the constructed generative AI model to the user, who can then use this model to seamlessly execute a series of tasks.

[0171] input

[0172] New generative AI models

[0173] output

[0174] Generative AI models provided by users

[0175] (Application example 1)

[0176] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0177] Conventional electronic payment services have difficulty proposing optimal payment methods, offering promotions, and providing coupons to users. Furthermore, functions such as budget management and fraud detection are provided separately, resulting in the inconvenience of users having to use multiple applications and services. Therefore, there is a need for a system that can provide these multiple functions in a unified manner, improving the user experience.

[0178] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0179] In this invention, the server includes means for providing a user interface for selecting multiple generative artificial intelligence models, means for collecting documents and sample data for the selected generative artificial intelligence models, means for analyzing the collected documents and sample data to identify functions of each generative artificial intelligence model, means for generating integrated prompts based on the identified functions, means for constructing a new generative artificial intelligence model using the generated prompts, means for suggesting an appropriate electronic payment method to the user, means for analyzing past purchase history and market trends to generate and suggest promotions and coupons, means for analyzing input income and expenditure data to provide a budget management plan, means for analyzing transaction data in real time to detect fraudulent transactions, and means for providing the constructed generative artificial intelligence model and various suggestions and notifications to the user. This allows users to receive comprehensive electronic payment support in a single system, improving convenience and safety.

[0180] A "user interface" is an interface that has an operation screen and input means for a user to select from multiple generative artificial intelligence models.

[0181] "Documentation" refers to the specifications, instructions, and technical documentation for the generative artificial intelligence model.

[0182] "Sample data" refers to a dataset that includes example inputs and outputs of a generative artificial intelligence model.

[0183] "Natural language processing technology" is a technology for analyzing text data and understanding human language, including morphological analysis and contextual understanding.

[0184] An "integrated prompt" is a new input instruction that integrates the functions of multiple generative artificial intelligence models and reflects the user's intentions.

[0185] An "electronic payment method" is a means by which a user pays for products online or in a brick-and-mortar store, and includes, for example, credit cards, QR code payments, digital wallets, etc.

[0186] "Promotion" refers to a discount or campaign to promote a particular product or service to users.

[0187] A "Coupon" is electronic or physical evidence of a discount under certain conditions.

[0188] A "budget management plan" is a plan for proper financial management that is created based on the user's income and expenditure data.

[0189] "Unauthorized Transactions" are transactions conducted in an unauthorized manner, including, for example, theft of credit cards and other fraudulent activities.

[0190] "Real-time" refers to a time frame in which data is processed immediately and results are reflected immediately.

[0191] "Generative AI model" is a general term for AI models that have the ability to generate an output corresponding to a specific input.

[0192] MODE FOR CARRYING OUT THE INVENTION

[0193] System Overview

[0194] The present invention is a system that allows users to integrate and use multiple generative AI models, particularly in electronic payment services. This system allows users to select models through a user interface (UI), analyze their documentation and sample data, understand the functions of each model, generate new prompts, and finally construct a new integrated generative AI model to provide to the user.

[0195] Hardware and software used

[0196] Frontend: React Native (Mobile App Development)

[0197] Backend: Node.js (server side), MongoDB (database)

[0198] Using AI models: Python, TensorFlow, GPT models

[0199] Processing flow

[0200] 1. User model selection and document collection:

[0201] The user selects multiple generative AI models (e.g., a payment method suggestion model, a coupon generation model, a budget management model, and a fraudulent transaction detection model) through the UI of a smartphone app.

[0202] Documentation and sample data for the selected model are sent to the server.

[0203] 2. Analyzing the documentation and sample data:

[0204] The server analyzes the received documents using natural language processing (NLP) techniques to identify the characteristics and functions of each generative AI model.

[0205] The server parses the sample data and understands the specific input and output formats for each model.

[0206] 3. Generating new prompts and building an integrated model:

[0207] The server generates new prompts based on the capabilities of each identified model, which are designed to reflect the user's intent and integrate the capabilities of multiple generative AI models.

[0208] Based on the generated prompts, a new generative AI model is built that can suggest appropriate electronic payment methods to users, generate and suggest promotions and coupons, provide budgeting plans, and detect fraudulent transactions.

[0209] 4. User Provision and Notification:

[0210] The server provides this newly constructed generative AI model and various suggestions and notifications to the user's smartphone app, allowing the user to seamlessly perform a series of tasks using this integrated model.

[0211] Specific examples

[0212] Example 1:

[0213] When users try to buy a drink, the app will recommend QR code payment and provide the latest promotional coupons. In addition, the app will suggest appropriate budget management plans and help users manage their overall spending based on their past shopping history.

[0214] Example 2:

[0215] When a user pays for a subscription service, the fraud detection model detects anomalous behavior and immediately alerts the user, allowing them to take immediate action and protect themselves from fraudulent transactions.

[0216] Prompt Sentence Examples

[0217] "Generate recommended coupons based on the user's last five purchases and current market promotions."

[0218] "Based on the income and expense data entered, please suggest the best monthly budget management plan for this user."

[0219] "Detect fraudulent transactions based on the latest transaction data and generate alerts."

[0220] As described above, the present invention enables users to receive comprehensive electronic payment support in one system, improving convenience and security.

[0221] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0222] Step 1:

[0223] The user selects multiple generative AI models through the UI of a smartphone app. Specifically, the user can select a payment method suggestion model, a coupon generation model, a budget management model, or a fraudulent transaction detection model. The input is the ID and name of the AI ​​model selected by the user, and the output is a list of the selected AI models.

[0224] Step 2:

[0225] The terminal collects the documents and sample data of the selected generative AI model and sends them to the server. The specific operation performed by the terminal is to call the API corresponding to the selected AI model to obtain the documents and sample data. The input here is the list of AI models selected in step 1, and the output is a set of documents and sample data for each model.

[0226] Step 3:

[0227] The server analyzes the received documents using natural language processing (NLP) techniques to identify the capabilities of each generative AI model. Specifically, the server performs morphological analysis and contextual understanding to extract important keywords and phrases from the documents. The input is the document obtained in step 2, and the output is a list of the characteristics and capabilities of each AI model.

[0228] Step 4:

[0229] The server analyzes the sample data and understands the specific input and output formats of each model. Specifically, the server analyzes the format of the sample data and identifies typical input data patterns and the corresponding output data formats. The input here is the sample data obtained in step 2, and the output is a list of input and output formats for each model.

[0230] Step 5:

[0231] The server generates a new prompt based on the capabilities of each identified model. This prompt is designed to reflect the user's intent and integrate the capabilities of multiple generative AI models. Specifically, the server optimizes the service by combining appropriate prompts to suit the user's purpose. The input is the list of capabilities and data formats obtained in Step 3 and Step 4, and the output is a new integrated prompt.

[0232] Step 6:

[0233] The server builds a new generative AI model based on the generated prompt. Specifically, the server calls the API of each AI model to send an integrated prompt, and then integrates the output to create a new model. The input is the prompt generated in step 5, and the output is a new integrated generative AI model.

[0234] Step 7:

[0235] The server provides this newly constructed generative AI model and various suggestions and notifications to the user's smartphone app. Specifically, the server provides the generated model as an endpoint and allows the user to access it. The user can use this integrated model to receive electronic payment method suggestions, obtain promotions and coupons, create budget management plans, and monitor fraudulent transactions. The input is the integrated generative AI model constructed in step 6, and the output is various services provided through the user's application interface.

[0236] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0237] Program processing overview

[0238] This invention provides a system that allows users to integrate multiple generative artificial intelligence models (generative AI models), and further combines it with an emotion engine that recognizes the user's emotions and adjusts output accordingly. This system allows users to select models through a user interface, analyzes their documents and sample data, understands the functions of each model, generates new prompts, and finally constructs a new integrated generative AI model to provide to the user.

[0239] Program Operation

[0240] 1. User model selection

[0241] Through the system's user interface, users can select from multiple generative AI models based on their specific tasks and goals. For example, they can choose from an AI model for explaining papers, an AI model for automatically generating code, or an AI model for generating illustrations.

[0242] 2. Collecting model information

[0243] The device collects documentation and sample data (example inputs and outputs) related to the generative AI model selected by the user and sends this to the server.

[0244] 3. Document Analysis

[0245] The server analyzes the received documents using natural language processing (NLP) technology. For example, from a document about an AI model explaining a paper, it understands that the model has the function of explaining a paper. It also analyzes AI models for automatic code generation and illustration generation in the same way.

[0246] 4. Analysis of sample data

[0247] The server analyzes the sample data and understands the specific input and output formats of each model. For example, it understands that the input example of a code auto-generation AI model is a program specification, and the output example is code based on that specification.

[0248] 5. Prompt Generation

[0249] The server generates an integrated prompt based on the analysis results. This prompt combines the functions of multiple generative AI models. For example, it creates a prompt that says, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[0250] 6. Use of Emotion Engines

[0251] The server uses an emotion engine to recognize the user's emotions. The recognized emotions are reflected in the output of the generative AI model. For example, if the user is feeling stressed, the server generates a gentle explanation in accordance with the user's emotions and an illustration to soothe the user's mood.

[0252] 7. Building a new generative AI model

[0253] The server then builds a new generative AI model based on the generated prompts and the user's emotional information. This new model integrates the functions of paper explanation, code generation, and illustration generation, and provides optimal output according to the user's emotions.

[0254] 8. Providing a Generative Model

[0255] The server then provides this newly constructed generative AI model to the user's device, where the user can use this integrated model to seamlessly execute a series of tasks.

[0256] Specific examples

[0257] For example, the following scenario is possible:

[0258] A user wants to read a paper, understand its content, generate related code based on the paper, and create an illustration to visually represent it. First, the user logs into the Mix Box platform and selects one of three models: a paper explanation AI model, an automatic code generation AI model, or an illustration generation AI model.

[0259] The device sends the documents and sample data of these models to the server, which analyzes them to understand the characteristics of each model. The server then creates an integrated prompt that asks, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code." The server then uses an emotion engine to analyze the user's emotions and makes adjustments based on those emotions to build a new generative AI model.

[0260] Users input a paper using the new generative AI model, which then generates a summary of the paper, along with code and related illustrations based on the summary, seamlessly. This consistent process is delivered in a way that takes into account the user's emotional state, allowing users to efficiently and comfortably leverage the capabilities of multiple generative AI models.

[0261] In this way, the present invention provides a system that allows users to use the functions of multiple generative AI models in an integrated manner, and by combining it with an emotion engine, the system reflects the user's emotional state in the output content, thereby realizing a concrete form that improves the user experience while reducing the need for context sharing and effort.

[0262] The processing flow will be explained below.

[0263] Step 1:

[0264] Users log in to the Mix Box platform and select multiple generative AI models they want to use. For example, they can choose an AI model for explaining papers, an AI model for automatically generating code, or an AI model for generating illustrations.

[0265] Step 2:

[0266] The device collects documentation and sample data (example inputs and outputs) for each generative AI model selected by the user and sends this to the server.

[0267] Step 3:

[0268] The server analyzes the received documents using natural language processing (NLP) technology. For example, from a document about an AI model explaining a paper, it understands that this model has the function of explaining a paper. Similarly, it analyzes AI models for automatic code generation and illustration generation.

[0269] Step 4:

[0270] The server analyzes the received sample data and understands the specific input and output formats of each model. For example, it understands that the input example of a code auto-generation AI model is a program specification, and that the output example is code based on that specification.

[0271] Step 5:

[0272] The server generates an integrated prompt based on the analysis results. This prompt combines the functions of multiple generative AI models. For example, it creates a prompt that says, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[0273] Step 6:

[0274] The server uses an emotion engine to recognize the user's emotions, for example, by analyzing the tone and speed of the user's keyboard and voice input to identify their current emotional state (e.g., stress, happiness, fatigue).

[0275] Step 7:

[0276] The server adjusts the generated prompts based on the user's emotional information obtained from the emotion engine. For example, if the user is feeling stressed, the server changes the explanatory text to a gentler tone and adjusts the illustrations to have a more relaxing effect.

[0277] Step 8:

[0278] Based on the tailored prompts, the server builds a new generative AI model that integrates the functions of paper explanation, code generation, and illustration generation, while providing output that responds to the user's emotions.

[0279] Step 9:

[0280] The server then provides the newly constructed generative AI model to the user's device, allowing the user to seamlessly execute a series of tasks using this integrated model.

[0281] Step 10:

[0282] Users use the provided new generative AI model to perform specific tasks. For example, by inputting a paper, the system generates a summary of the paper, generates code based on the summary, and then generates an illustration related to the code. The output content is provided according to the user's emotional state, allowing users to work efficiently and comfortably.

[0283] In this way, this system allows users to integrate the functions of multiple generative AI models, and by combining it with an emotion engine, it reflects the user's emotional state in the output content, improving the user experience.

[0284] Example 2

[0285] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0286] Conventional generative AI models require users to use each individual model, and integrating the outputs of each model requires a lot of time and effort. Furthermore, the results output without considering the user's emotions often do not necessarily respond to the user's needs. This detracts from the overall user experience.

[0287] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0288] In this invention, the server includes means for providing a user interface for selecting multiple generative AI models, means for collecting documents and sample data for the selected generative AI models, means for analyzing the collected documents and sample data and identifying functions of each generative AI model, means for generating an integrated prompt based on the identified functions, means for constructing a new generative AI model based on the analysis results and user emotion information, means for recognizing the user's emotions using an emotion engine and reflecting them in the output of the generative AI model, and means for providing the constructed generative AI model to the user. This makes it possible to efficiently integrate multiple generative AI models and provide optimal output according to the user's emotions.

[0289] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates new information or data based on input data.

[0290] A "user interface" is an interface through which a user interacts with software or a system.

[0291] A "document" is a text or file that contains an explanation or specification of a generative artificial intelligence model.

[0292] "Sample data" refers to specific input and output examples used to demonstrate the functionality of a generative artificial intelligence model.

[0293] "Natural language processing technology" is a technology that enables computers to understand and generate human language.

[0294] A "prompt" is an input instruction or question to a generative artificial intelligence model.

[0295] An "emotion engine" is a technology that recognizes a user's emotions and adjusts responses and output based on the results.

[0296] An "integrated prompt" is a consistent instruction that combines the capabilities of multiple generative artificial intelligence models.

[0297] The "new generative artificial intelligence model" is a new AI model that integrates the functions of multiple existing generative artificial intelligence models to provide optimal output according to the user's emotions.

[0298] The present invention is a system that allows users to use multiple generative AI models in an integrated manner, and also combines an emotion engine that recognizes the user's emotions and adjusts the output accordingly. This system is implemented by combining the following processes.

[0299] First, the user selects the generative AI model they want to use through a user interface, such as a web browser or application on a computer or smart device. The user can choose from a paper explanation AI model, an automatic code generation AI model, an illustration generation AI model, and so on.

[0300] The device then collects documentation and sample data for the selected generative AI model, often in the form of text or JSON files, and transmits the collected data to a server via the HTTPS protocol.

[0301] The server analyzes the received documents using natural language processing (NLP) techniques. Specifically, it uses Python's NLTK library and SpaCy to extract the functions and features of each generative AI model. Sample data is also analyzed using data analysis libraries such as Pandas and NumPy to identify the input and output formats for each model.

[0302] Based on the analysis results, the server generates integrated prompts, which are generated using large-scale language models such as GPT-3. Specifically, the prompts include "generate a summary of the paper, generate code based on that summary, and generate illustrations related to that code."

[0303] Furthermore, the server uses an emotion engine to recognize the user's emotions. The emotion engine uses, for example, IBM Watson's Emotion Analysis. It determines the user's emotional state based on the user's voice and text input obtained from the user interface. The recognized emotion is reflected in the output of the generative AI model. For example, if the user is feeling stressed, a gentle tone of voice corresponding to that emotion is generated.

[0304] The server then builds a new generative AI model based on the generated prompts and the user's emotional information. This new generative AI model is built using machine learning frameworks such as TensorFlow and PyTorch. This makes it possible to integrate functions for paper explanations, code generation, and illustration generation, while providing optimal output based on the user's emotional state.

[0305] Finally, the server provides the constructed generative AI model to the user's device. This process also uses HTTPS to ensure secure communication. The user can then use this integrated model to seamlessly execute a series of tasks.

[0306] As a concrete example, the following scenario can be considered.

[0307] When a user types, "Generate a summary of this paper," the server analyzes the user's input and creates a prompt to generate the summary. Furthermore, if the user is feeling stressed, the server generates a prompt saying, "Generate a summary of this paper in a gentle tone." Then, based on the summarized content, it generates a prompt saying, "Generate appropriate code based on the summarized paper content." Finally, it generates a prompt saying, "Create an illustration related to the generated code." The system seamlessly executes this series of processes.

[0308] Through the above steps, the present invention enables users to use multiple generative artificial intelligence models in an integrated manner, and furthermore, by reflecting the user's emotions in the output content, provides an efficient and appropriate user experience.

[0309] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0310] Step 1: User model selection

[0311] Specific operation: The user selects the generative AI model they want to use through a user interface (e.g., a web browser or application). Available models include an AI model for explaining papers, an AI model for automatically generating code, and an AI model for generating illustrations.

[0312] Input: The model type selected by the user.

[0313] Output: The information of the selected model is saved in the internal database.

[0314] Step 2: Gathering model information

[0315] Specific operations: The terminal collects documents and sample data for the generative artificial intelligence model selected through the user interface, including document files containing descriptions and specifications of the model, as well as example inputs and outputs.

[0316] Input: The model type selected by the user.

[0317] Output: Collected documents and sample data are sent to the server.

[0318] Step 3: Parsing the document

[0319] How it works: The server analyzes the received document files using natural language processing (NLP) techniques to identify the capabilities of each generative AI model. Specifically, it analyzes the content of the documents using Python's NLTK library and SpaCy.

[0320] Input: Collected document files.

[0321] Data processing: Tokenize the text information in the document and extract important keywords and context.

[0322] Output: Data about the capabilities of each model is stored in an internal database.

[0323] Step 4: Analyzing the sample data

[0324] Specific operation: The server analyzes the received sample data and identifies the specific input and output formats for each model. This analysis is performed using data analysis libraries such as Pandas and NumPy.

[0325] Input: Collected sample data.

[0326] Data processing: Analyze sample data to extract input and output formats.

[0327] Output: Data about the input and output formats for each model is stored in an internal database.

[0328] Step 5: Generate prompts

[0329] What it does: The server generates a unified prompt based on the analysis results, using a large-scale language model such as GPT-3.

[0330] Input: Data about the capabilities, input formats, and output formats of each model.

[0331] Data calculation: Combine the functions of each model to generate the optimal prompt.

[0332] Output: The integrated prompt is generated.

[0333] Step 6: Use the Emotion Engine

[0334] Specific operation: The server recognizes the user's emotions using an emotion engine. The emotion engine uses IBM Watson's Emotion Analysis. It analyzes the user's voice and text input obtained from the user interface to identify the user's emotional state.

[0335] Input: User voice and text input.

[0336] Data calculation: The emotion engine analyzes the emotional state.

[0337] Output: Data about the user's emotional state.

[0338] Step 7: Building a new generative AI model

[0339] How it works: The server builds a new generative AI model based on the generated prompts and the user's emotional information, using machine learning frameworks such as TensorFlow and PyTorch.

[0340] Input: Synthesized prompt sentence and user emotional state data.

[0341] Data calculation: Build an optimal generative AI model from prompt sentences and emotional information.

[0342] Output: A new generative artificial intelligence model.

[0343] Step 8: Serving the generative model

[0344] Specific operation: The server provides the constructed generative AI model to the user's device using the HTTPS protocol to ensure secure communication.

[0345] Input: A new generative artificial intelligence model.

[0346] Output: The generative AI model provided to the user's device.

[0347] (Application example 2)

[0348] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0349] In systems that use conventional generative AI models, users have to spend a lot of time and effort integrating multiple models. Furthermore, outputs generated without considering the user's emotional state can degrade the quality of the user experience. This makes it difficult to achieve the desired results for specific applications.

[0350] The specification processing by specification processing unit 290 of data processing device 12 in application example 2 is realized by the following means. In this invention, the server includes means for providing a user interface for selecting multiple generative AI models, means for collecting documents and sample data for the selected generative AI models, means for analyzing the collected documents and sample data and identifying the functions of each generative AI model, means for generating an integrated prompt based on the identified functions, means for constructing a new generative AI model using the generated prompt, means for recognizing a user's emotion and adjusting the output content of the generative AI model based on the recognized emotion, and means for providing the constructed generative AI model to the user. This allows the user to use multiple generative AI models in an integrated manner and further enables output adjustment based on emotion, providing a higher quality user experience.

[0351] A "generative artificial intelligence model" is an artificial intelligence algorithm that has the ability to receive input for a specific task and generate a corresponding output.

[0352] "User interface" is a general term for the screens and operating means that allow users to access and operate systems and software.

[0353] "Documentation" refers to instructions and specifications regarding the functions and usage of generative artificial intelligence models.

[0354] "Sample data" is a set of example inputs and outputs that demonstrate how a generative artificial intelligence model works.

[0355] "Analysis" is the act of examining data or documents in detail to identify meanings and patterns.

[0356] An "integrated prompt" is a command that combines the functions of multiple generative artificial intelligence models.

[0357] "Emotion recognition" refers to analyzing a user's input and behavior to identify their emotional state.

[0358] "Adjusting output" refers to adjusting the information or results generated based on the perceived emotion.

[0359] A "new generative artificial intelligence model" is a new, improved model that integrates the functions and characteristics of multiple existing generative artificial intelligence models.

[0360] "Provision" refers to the act of making a particular service or feature available to users.

[0361] A specific embodiment of the present invention will be described. The present invention is a system that can use multiple generative artificial intelligence models in an integrated manner, and further recognizes the user's emotions and adjusts the output accordingly.

[0362] First, a user interface is provided that allows users to select from multiple generative AI models. Through this interface, users can select the model they need based on their purpose, such as a product recommendation model or a review generation model.

[0363] Next, documents and sample data related to the selected generative AI models are collected and sent to a server, which analyzes the collected documents and sample data using natural language processing techniques (e.g., TensorFlow, PyTorch) to identify the capabilities of each generative AI model.

[0364] The server then generates a unified prompt based on the capabilities of each identified model. This prompt seamlessly combines the capabilities of multiple generative AI models, such as the following prompt:

[0365] "User is feeling stressed. Please recommend some relaxation items."

[0366] Furthermore, the server uses an emotion analysis model (e.g., EmotionAnalyzer) to recognize the user's emotions. Based on the recognized emotions, the output of the generative AI model can be adjusted. For example, if the user is feeling stressed, a gentle tone of voice or relaxation-related products will be recommended to match that emotion.

[0367] Finally, these methods are combined to build a new generative AI model, which is then provided to users. Users can access this integrated model through devices such as smartphones or head-mounted displays (HMDs) to seamlessly perform a series of tasks. This system allows users to receive high-quality services that respond to their emotional state.

[0368] As a concrete example, consider the case where a user is searching for "relaxing products" in a virtual store and enters "I've been very busy and stressed lately." Based on this input, the server generates the following prompt:

[0369] "User is feeling stressed. Please recommend some relaxation items."

[0370] The product recommendation model then uses these prompts to suggest appropriate relaxation items to the user, enabling the user to efficiently and comfortably utilize the capabilities of multiple generative AI models.

[0371] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0372] Step 1:

[0373] User model selection

[0374] The user selects from multiple generative AI models they wish to use through the system's user interface. This interface allows the user to select a specific generative AI model for a purpose, such as automatic product recommendation or review generation. The input is the user's model selection, and the output is a list of the selected models.

[0375] Step 2:

[0376] Collecting Model Information

[0377] The terminal collects documents and sample data related to the generative artificial intelligence models selected by the user, where the input is a list of selected models and the output is the documents and sample data corresponding to those models, and this data is sent to the server.

[0378] Step 3:

[0379] Document Parsing

[0380] The server analyzes the collected documents using natural language processing techniques, such as TensorFlow or PyTorch, to understand the content of the documents and identify the functions of each generative artificial intelligence model. The input here is the collected documents, and the output is a list of the model's functions.

[0381] Step 4:

[0382] Analysis of sample data

[0383] The server analyzes the collected sample data and understands the specific input and output formats of each model, which allows it to determine what data is required for a specific task. The input is the sample data, and the output is the specific information for the input and output formats of each model.

[0384] Step 5:

[0385] Prompt Generation

[0386] The server generates an integrated prompt based on the analysis results. For example, this prompt might be in the form of "The user is feeling stressed. Please recommend some relaxation items." Here, the functions of each model are combined to create a prompt that meets the user's needs. The input is the analysis results (model functions and input / output format), and the output is the integrated prompt.

[0387] Step 6:

[0388] Use of emotion engine

[0389] The server uses an emotion engine to recognize the user's emotions. The recognized emotions are reflected in the output of the generative AI model. For example, EmotionAnalyzer analyzes the user's input, "I've been very busy and stressed lately," and identifies the user's emotional state as stress. The input is the user's text input, and the output is the identified emotional state.

[0390] Step 7:

[0391] Building a new generative artificial intelligence model

[0392] The server constructs a new generative AI model based on the generated prompt and the user's emotional information. This new model integrates the necessary functions to respond to the prompt and can provide optimal output based on the user's emotional state. The input is the prompt and emotional information, and the output is the newly constructed generative AI model.

[0393] Step 8:

[0394] Providing a generative model

[0395] The server provides this newly constructed generative AI model to the user's device. The user can use this integrated model through a device such as a smartphone or head-mounted display (HMD) to seamlessly perform a series of tasks. The input is the new generative AI model, and the output is the services and functions that the user receives.

[0396] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0397] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0398] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0399] [Second embodiment]

[0400] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0401] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0402] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0403] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0404] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0405] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0406] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0407] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0408] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0409] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0410] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0411] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0412] Program processing overview

[0413] The present invention provides a system that allows users to integrate and use multiple generative artificial intelligence models (generative AI models). This system allows users to select models through a user interface, analyze their documentation and sample data, understand the functions of each model, generate new prompts, and finally construct a new integrated generative AI model to provide to the user.

[0414] Program Operation

[0415] 1. User model selection

[0416] Through the system's user interface, users can select from multiple generative AI models they wish to use, based on their specific tasks and objectives.

[0417] 2. Collecting model information

[0418] The device sends documents and sample data (example inputs and outputs) for each generative AI model selected by the user to the server.

[0419] 3. Document Analysis

[0420] The server analyzes the received documents using natural language processing (NLP) technology to identify the characteristics and functions of each generative AI model. For example, a document about an AI model that explains a paper will be analyzed to identify its function of explaining the paper.

[0421] 4. Analysis of sample data

[0422] The server analyzes sample data and understands the specific input and output formats of each model. For example, based on the input example of a code auto-generation AI model, it understands that when program specifications are input, code will be output.

[0423] 5. Prompt Generation

[0424] The server generates new prompts based on the capabilities of each identified model, which are designed to reflect the user's intent and integrate the capabilities of multiple generative AI models.

[0425] 6. Building a new generative AI model

[0426] The server then builds a new generative AI model based on the generated prompts, which provides different functions such as paper explanation, code generation, and illustration generation in a single, unified interface.

[0427] 7. Providing a Generative Model

[0428] The server then provides this newly constructed generative AI model to the user's device, where the user can use this integrated model to seamlessly execute a series of tasks.

[0429] Specific examples

[0430] For example, the following scenario is possible:

[0431] A user wants to read a paper, understand its content, generate related code based on the paper, and create an illustration to visually represent it. First, the user logs into the Mix Box platform and selects one of three models: a paper explanation AI model, an automatic code generation AI model, or an illustration generation AI model.

[0432] The device sends the documents and sample data of these models to the server, which analyzes them to understand the characteristics of each model. The server then creates an integrated prompt, such as "generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code." Based on this prompt, the server then builds a new generative AI model and provides it to the user.

[0433] Users input a paper using the new generative AI model, which then generates a summary, along with code and related illustrations based on the summary, seamlessly. This consistent process allows users to efficiently and conveniently leverage the capabilities of multiple generative AI models.

[0434] In this way, the present invention provides a system that allows users to use the functions of multiple generative AI models in an integrated manner, and is realized in a concrete form for sharing context and reducing effort.

[0435] The processing flow will be explained below.

[0436] Step 1:

[0437] Users log in to the Mix Box platform and select the generative AI model they want to use. Specific examples include a paper explanation AI model, an automatic code generation AI model, and an illustration generation AI model.

[0438] Step 2:

[0439] The device collects documentation and sample data (example inputs and outputs) for each generative AI model selected by the user, and sends this data to the server.

[0440] Step 3:

[0441] The server analyzes the received documents using natural language processing (NLP) technology. For example, from a document about an AI model explaining a paper, it understands that the model has the function of explaining a paper. It also analyzes AI models for automatic code generation and illustration generation in the same way.

[0442] Step 4:

[0443] The server analyzes the received sample data. Specifically, it identifies the input and output formats of each generative AI model. For example, it understands that the input example of a code auto-generation AI model is a program specification, and that the output example is code based on that specification.

[0444] Step 5:

[0445] The server generates an integrated prompt based on the analysis results. This prompt combines the functions of multiple generative AI models. For example, it creates a prompt that says, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[0446] Step 6:

[0447] The server uses these prompts to build a new generative AI model that combines the capabilities of paper commentary, code generation, and illustration generation.

[0448] Step 7:

[0449] The server then provides the newly constructed generative AI model to the user's device, allowing the user to seamlessly execute a series of tasks using this integrated model.

[0450] Step 8:

[0451] Users can then use the new generative AI models provided to them to perform specific tasks. For example, by inputting a paper, a summary of the paper is generated, code based on the summary is generated, and then illustrations related to the code are generated. In this way, users can efficiently use the functions of multiple generative AI models in a single system.

[0452] Through these steps, the system enables users to integrate the functions of multiple generative AI models, reducing the need for context sharing and effort.

[0453] Example 1

[0454] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0455] Currently, there is no system for effectively integrating multiple generative AI models. Users must select models individually, input data for each model separately, and obtain results. This process is time-consuming, labor-intensive, and inefficient. Furthermore, it is difficult to integrate data and functions between different generative AI models, making it difficult to obtain consistent results.

[0456] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0457] In this invention, the server includes means for providing a user interface for selecting multiple generative AI models, means for collecting information and sample data on the selected generative AI models, means for analyzing the collected documents and sample data and identifying functions of each generative AI model, means for generating integrated prompts based on the identified functions, means for providing the constructed generative AI model to the user, means for generating prompts that reflect the user's intentions and integrate the functions of the multiple generative AI models, and means for constructing a new generative AI model based on the generated prompts and seamlessly executing a series of tasks. This enables the user to efficiently integrate and use multiple generative AI models and obtain consistent results.

[0458] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates output such as text, images, or program code based on user input.

[0459] A "user interface" refers to a screen or input device that allows a user to interact with a system, and is the means by which a user performs the necessary operations and selections.

[0460] "Sample data" refers to a dataset that contains specific examples to demonstrate the format and content of inputs and outputs to a generative artificial intelligence model.

[0461] "Documentation" refers to documents that describe the functions, specifications, and usage of a generative artificial intelligence model, and includes information essential for understanding and using the model.

[0462] "Natural language processing technology" is a general term for technologies that enable computers to understand and manipulate text data, and includes technologies for analyzing documents and extracting meaning.

[0463] A "prompt" is text given to a generative artificial intelligence model in the form of an instruction or question, and is an input sentence that causes the model to perform a desired action.

[0464] This invention relates to a system that allows users to use multiple generative artificial intelligence models (generative AI models) in an integrated manner. This system mainly consists of a server, a terminal, and a user. The processes and roles of each are explained in detail below.

[0465] System Overview

[0466] The user selects a generative AI model through the system's user interface, after which the server and device work together to collect and analyze the necessary data, ultimately providing the user with an integrated generative AI model.

[0467] Hardware and Software

[0468] server

[0469] A server equipped with high-performance computers and natural language processing technology (e.g., using spaCy or NLTK as an NLP toolkit).

[0470] Terminal

[0471] A computer, tablet, or smartphone with an internet connection.

[0472] User Interface

[0473] The interface is provided using a web browser or a dedicated application.

[0474] Data processing and calculation

[0475] 1. User model selection

[0476] Through the interface, users can select the generative AI model they want to use, such as an AI model for paper commentary, automatic code generation, or illustration generation.

[0477] 2. Collecting model information

[0478] The device collects documentation and sample data about the model selected by the user and sends it to the server, including the model's functionality, specifications, and example input and output data.

[0479] 3. Analyzing Documents and Sample Data

[0480] The server analyzes the collected documents using natural language processing technology and extracts the characteristics and functions of each generative AI model. For example, from documents of an AI model that explains academic papers, it identifies the literature explanation function.

[0481] The server also analyzes sample data to understand the input and output formats of each model. For example, from an example input for a code auto-generation AI model, it identifies the program specification input and the corresponding code output.

[0482] 4. Prompt generation and building a new generative AI model

[0483] The server generates an integrated prompt based on the analysis results. This prompt reflects the user's intent and integrates the functions of multiple generative AI models. For example, it creates a prompt that asks, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[0484] Based on these prompts, we build a new generative AI model that integrates the functions of paper explanation, code generation, and illustration generation.

[0485] 5. Providing a Generative Model

[0486] The server provides the newly constructed generative AI model to the user, who can then use the model to seamlessly execute different tasks.

[0487] Specific examples

[0488] For example, consider a scenario where a user logs in to the Mix Box platform and performs the following steps: First, the user selects an AI model for explaining papers, then an AI model for automatically generating code, and finally an AI model for generating illustrations. The device then sends these documents and sample data to the server, which then analyzes each data and generates an integrated prompt.

[0489] An example prompt might be, "Generate a summary of a paper, generate Python code based on that summary, and generate illustrations related to that code." A new generative AI model is built based on this prompt, allowing users to efficiently and seamlessly utilize multiple AI functions.

[0490] In this way, the present invention provides a system that allows users to integrate the functions of multiple generative AI models and achieve their goals through a consistent process.

[0491] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0492] Step 1:

[0493] User model selection

[0494] Specific actions

[0495] Users access the system's user interface and select from multiple generative artificial intelligence models according to their goals and challenges.

[0496] input

[0497] A list of generative AI models based on user selections (clicks, etc.)

[0498] output

[0499] A list of selected generative AI models

[0500] Step 2:

[0501] Collecting Model Information

[0502] Specific actions

[0503] The device collects documentation and sample data for the generative AI model selected by the user, including API calls and retrieval from a database, and sends the collected data to a server.

[0504] input

[0505] A list of generative AI models selected by the user

[0506] output

[0507] Documentation and sample data for each model

[0508] Step 3:

[0509] Document Parsing

[0510] Specific actions

[0511] The server uses natural language processing (NLP) technology to analyze the documents of each generative AI model received from the device, thereby extracting the characteristics and functions of each AI model.

[0512] input

[0513] Documentation for each generative AI model (PDF, HTML, etc.)

[0514] output

[0515] A list of the characteristics and capabilities of each AI model

[0516] Step 4:

[0517] Analysis of sample data

[0518] Specific actions

[0519] The server analyzes the sample data received from each generative AI model to understand the specific input and output formats, thereby understanding how each model operates.

[0520] input

[0521] Sample data for each generative AI model (JSON, CSV, etc.)

[0522] output

[0523] Details of the input and output formats for each model

[0524] Step 5:

[0525] Prompt Generation

[0526] Specific actions

[0527] The server generates new prompts based on the characteristics and capabilities of each analyzed model, which are designed to reflect the user's intent and integrate the capabilities of multiple AI models.

[0528] input

[0529] Details of the characteristics and functions of each model, as well as input and output formats

[0530] output

[0531] Integrated prompts

[0532] Step 6:

[0533] Building a new generative AI model

[0534] Specific actions

[0535] The server then uses the generated prompts to build a new generative AI model, which is designed to integrate multiple functions and guide users through tasks in a consistent process.

[0536] input

[0537] Integrated prompts

[0538] output

[0539] New generative AI models

[0540] Step 7:

[0541] Providing generative AI models

[0542] Specific actions

[0543] The server provides the constructed generative AI model to the user, who can then use this model to seamlessly execute a series of tasks.

[0544] input

[0545] New generative AI models

[0546] output

[0547] Generative AI models provided by users

[0548] (Application example 1)

[0549] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0550] Conventional electronic payment services have difficulty proposing optimal payment methods, offering promotions, and providing coupons to users. Furthermore, functions such as budget management and fraud detection are provided separately, resulting in the inconvenience of users having to use multiple applications and services. Therefore, there is a need for a system that can provide these multiple functions in a unified manner, improving the user experience.

[0551] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0552] In this invention, the server includes means for providing a user interface for selecting multiple generative artificial intelligence models, means for collecting documents and sample data for the selected generative artificial intelligence models, means for analyzing the collected documents and sample data to identify functions of each generative artificial intelligence model, means for generating integrated prompts based on the identified functions, means for constructing a new generative artificial intelligence model using the generated prompts, means for suggesting an appropriate electronic payment method to the user, means for analyzing past purchase history and market trends to generate and suggest promotions and coupons, means for analyzing input income and expenditure data to provide a budget management plan, means for analyzing transaction data in real time to detect fraudulent transactions, and means for providing the constructed generative artificial intelligence model and various suggestions and notifications to the user. This allows users to receive comprehensive electronic payment support in a single system, improving convenience and safety.

[0553] A "user interface" is an interface that has an operation screen and input means for a user to select from multiple generative artificial intelligence models.

[0554] "Documentation" refers to the specifications, instructions, and technical documentation for the generative artificial intelligence model.

[0555] "Sample data" refers to a dataset that includes example inputs and outputs of a generative artificial intelligence model.

[0556] "Natural language processing technology" is a technology for analyzing text data and understanding human language, including morphological analysis and contextual understanding.

[0557] An "integrated prompt" is a new input instruction that integrates the functions of multiple generative artificial intelligence models and reflects the user's intentions.

[0558] An "electronic payment method" is a means by which a user pays for products online or in a brick-and-mortar store, and includes, for example, credit cards, QR code payments, digital wallets, etc.

[0559] "Promotion" refers to a discount or campaign to promote a particular product or service to users.

[0560] A "Coupon" is electronic or physical evidence of a discount under certain conditions.

[0561] A "budget management plan" is a plan for proper financial management that is created based on the user's income and expenditure data.

[0562] "Unauthorized Transactions" are transactions conducted in an unauthorized manner, including, for example, theft of credit cards and other fraudulent activities.

[0563] "Real-time" refers to a time frame in which data is processed immediately and results are reflected immediately.

[0564] "Generative AI model" is a general term for AI models that have the ability to generate an output corresponding to a specific input.

[0565] MODE FOR CARRYING OUT THE INVENTION

[0566] System Overview

[0567] The present invention is a system that allows users to integrate and use multiple generative AI models, particularly in electronic payment services. This system allows users to select models through a user interface (UI), analyze their documentation and sample data, understand the functions of each model, generate new prompts, and finally construct a new integrated generative AI model to provide to the user.

[0568] Hardware and software used

[0569] Frontend: React Native (Mobile App Development)

[0570] Backend: Node.js (server side), MongoDB (database)

[0571] Using AI models: Python, TensorFlow, GPT models

[0572] Processing flow

[0573] 1. User model selection and document collection:

[0574] The user selects multiple generative AI models (e.g., a payment method suggestion model, a coupon generation model, a budget management model, and a fraudulent transaction detection model) through the UI of a smartphone app.

[0575] Documentation and sample data for the selected model are sent to the server.

[0576] 2. Analyzing the documentation and sample data:

[0577] The server analyzes the received documents using natural language processing (NLP) techniques to identify the characteristics and functions of each generative AI model.

[0578] The server parses the sample data and understands the specific input and output formats for each model.

[0579] 3. Generating new prompts and building an integrated model:

[0580] The server generates new prompts based on the capabilities of each identified model, which are designed to reflect the user's intent and integrate the capabilities of multiple generative AI models.

[0581] Based on the generated prompts, a new generative AI model is built that can suggest appropriate electronic payment methods to users, generate and suggest promotions and coupons, provide budgeting plans, and detect fraudulent transactions.

[0582] 4. User Provision and Notification:

[0583] The server provides this newly constructed generative AI model and various suggestions and notifications to the user's smartphone app, allowing the user to seamlessly perform a series of tasks using this integrated model.

[0584] Specific examples

[0585] Example 1:

[0586] When users try to buy a drink, the app will recommend QR code payment and provide the latest promotional coupons. In addition, the app will suggest appropriate budget management plans and help users manage their overall spending based on their past shopping history.

[0587] Example 2:

[0588] When a user pays for a subscription service, the fraud detection model detects anomalous behavior and immediately alerts the user, allowing them to take immediate action and protect themselves from fraudulent transactions.

[0589] Prompt Sentence Examples

[0590] "Generate recommended coupons based on the user's last five purchases and current market promotions."

[0591] "Based on the income and expense data entered, please suggest the best monthly budget management plan for this user."

[0592] "Detect fraudulent transactions based on the latest transaction data and generate alerts."

[0593] As described above, the present invention enables users to receive comprehensive electronic payment support in one system, improving convenience and security.

[0594] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0595] Step 1:

[0596] The user selects multiple generative AI models through the UI of a smartphone app. Specifically, the user can select a payment method suggestion model, a coupon generation model, a budget management model, or a fraudulent transaction detection model. The input is the ID and name of the AI ​​model selected by the user, and the output is a list of the selected AI models.

[0597] Step 2:

[0598] The terminal collects the documents and sample data of the selected generative AI model and sends them to the server. The specific operation performed by the terminal is to call the API corresponding to the selected AI model to obtain the documents and sample data. The input here is the list of AI models selected in step 1, and the output is a set of documents and sample data for each model.

[0599] Step 3:

[0600] The server analyzes the received documents using natural language processing (NLP) techniques to identify the capabilities of each generative AI model. Specifically, the server performs morphological analysis and contextual understanding to extract important keywords and phrases from the documents. The input is the document obtained in step 2, and the output is a list of the characteristics and capabilities of each AI model.

[0601] Step 4:

[0602] The server analyzes the sample data and understands the specific input and output formats of each model. Specifically, the server analyzes the format of the sample data and identifies typical input data patterns and the corresponding output data formats. The input here is the sample data obtained in step 2, and the output is a list of input and output formats for each model.

[0603] Step 5:

[0604] The server generates a new prompt based on the capabilities of each identified model. This prompt is designed to reflect the user's intent and integrate the capabilities of multiple generative AI models. Specifically, the server optimizes the service by combining appropriate prompts to suit the user's purpose. The input is the list of capabilities and data formats obtained in Step 3 and Step 4, and the output is a new integrated prompt.

[0605] Step 6:

[0606] The server builds a new generative AI model based on the generated prompt. Specifically, the server calls the API of each AI model to send an integrated prompt, and then integrates the output to create a new model. The input is the prompt generated in step 5, and the output is a new integrated generative AI model.

[0607] Step 7:

[0608] The server provides this newly constructed generative AI model and various suggestions and notifications to the user's smartphone app. Specifically, the server provides the generated model as an endpoint and allows the user to access it. The user can use this integrated model to receive electronic payment method suggestions, obtain promotions and coupons, create budget management plans, and monitor fraudulent transactions. The input is the integrated generative AI model constructed in step 6, and the output is various services provided through the user's application interface.

[0609] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0610] Program processing overview

[0611] This invention provides a system that allows users to integrate multiple generative artificial intelligence models (generative AI models), and further combines it with an emotion engine that recognizes the user's emotions and adjusts output accordingly. This system allows users to select models through a user interface, analyzes their documents and sample data, understands the functions of each model, generates new prompts, and finally constructs a new integrated generative AI model to provide to the user.

[0612] Program Operation

[0613] 1. User model selection

[0614] Through the system's user interface, users can select from multiple generative AI models based on their specific tasks and goals. For example, they can choose from an AI model for explaining papers, an AI model for automatically generating code, or an AI model for generating illustrations.

[0615] 2. Collecting model information

[0616] The device collects documentation and sample data (example inputs and outputs) related to the generative AI model selected by the user and sends this to the server.

[0617] 3. Document Analysis

[0618] The server analyzes the received documents using natural language processing (NLP) technology. For example, from a document about an AI model explaining a paper, it understands that the model has the function of explaining a paper. It also analyzes AI models for automatic code generation and illustration generation in the same way.

[0619] 4. Analysis of sample data

[0620] The server analyzes the sample data and understands the specific input and output formats of each model. For example, it understands that the input example of a code auto-generation AI model is a program specification, and the output example is code based on that specification.

[0621] 5. Prompt Generation

[0622] The server generates an integrated prompt based on the analysis results. This prompt combines the functions of multiple generative AI models. For example, it creates a prompt that says, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[0623] 6. Use of Emotion Engines

[0624] The server uses an emotion engine to recognize the user's emotions. The recognized emotions are reflected in the output of the generative AI model. For example, if the user is feeling stressed, the server generates a gentle explanation in accordance with the user's emotions and an illustration to soothe the user's mood.

[0625] 7. Building a new generative AI model

[0626] The server then builds a new generative AI model based on the generated prompts and the user's emotional information. This new model integrates the functions of paper explanation, code generation, and illustration generation, and provides optimal output according to the user's emotions.

[0627] 8. Providing a Generative Model

[0628] The server then provides this newly constructed generative AI model to the user's device, where the user can use this integrated model to seamlessly execute a series of tasks.

[0629] Specific examples

[0630] For example, the following scenario is possible:

[0631] A user wants to read a paper, understand its content, generate related code based on the paper, and create an illustration to visually represent it. First, the user logs into the Mix Box platform and selects one of three models: a paper explanation AI model, an automatic code generation AI model, or an illustration generation AI model.

[0632] The device sends the documents and sample data of these models to the server, which analyzes them to understand the characteristics of each model. The server then creates an integrated prompt that asks, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code." The server then uses an emotion engine to analyze the user's emotions and makes adjustments based on those emotions to build a new generative AI model.

[0633] Users input a paper using the new generative AI model, which then generates a summary of the paper, along with code and related illustrations based on the summary, seamlessly. This consistent process is delivered in a way that takes into account the user's emotional state, allowing users to efficiently and comfortably leverage the capabilities of multiple generative AI models.

[0634] In this way, the present invention provides a system that allows users to use the functions of multiple generative AI models in an integrated manner, and by combining it with an emotion engine, the system reflects the user's emotional state in the output content, thereby realizing a concrete form that improves the user experience while reducing the need for context sharing and effort.

[0635] The processing flow will be explained below.

[0636] Step 1:

[0637] Users log in to the Mix Box platform and select multiple generative AI models they want to use. For example, they can choose an AI model for explaining papers, an AI model for automatically generating code, or an AI model for generating illustrations.

[0638] Step 2:

[0639] The device collects documentation and sample data (example inputs and outputs) for each generative AI model selected by the user and sends this to the server.

[0640] Step 3:

[0641] The server analyzes the received documents using natural language processing (NLP) technology. For example, from a document about an AI model explaining a paper, it understands that this model has the function of explaining a paper. Similarly, it analyzes AI models for automatic code generation and illustration generation.

[0642] Step 4:

[0643] The server analyzes the received sample data and understands the specific input and output formats of each model. For example, it understands that the input example of a code auto-generation AI model is a program specification, and that the output example is code based on that specification.

[0644] Step 5:

[0645] The server generates an integrated prompt based on the analysis results. This prompt combines the functions of multiple generative AI models. For example, it creates a prompt that says, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[0646] Step 6:

[0647] The server uses an emotion engine to recognize the user's emotions, for example, by analyzing the tone and speed of the user's keyboard and voice input to identify their current emotional state (e.g., stress, happiness, fatigue).

[0648] Step 7:

[0649] The server adjusts the generated prompts based on the user's emotional information obtained from the emotion engine. For example, if the user is feeling stressed, the server changes the explanatory text to a gentler tone and adjusts the illustrations to have a more relaxing effect.

[0650] Step 8:

[0651] Based on the tailored prompts, the server builds a new generative AI model that integrates the functions of paper explanation, code generation, and illustration generation, while providing output that responds to the user's emotions.

[0652] Step 9:

[0653] The server then provides the newly constructed generative AI model to the user's device, allowing the user to seamlessly execute a series of tasks using this integrated model.

[0654] Step 10:

[0655] Users use the provided new generative AI model to perform specific tasks. For example, by inputting a paper, the system generates a summary of the paper, generates code based on the summary, and then generates an illustration related to the code. The output content is provided according to the user's emotional state, allowing users to work efficiently and comfortably.

[0656] In this way, this system allows users to integrate the functions of multiple generative AI models, and by combining it with an emotion engine, it reflects the user's emotional state in the output content, improving the user experience.

[0657] Example 2

[0658] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0659] Conventional generative AI models require users to use each individual model, and integrating the outputs of each model requires a lot of time and effort. Furthermore, the results output without considering the user's emotions often do not necessarily respond to the user's needs. This detracts from the overall user experience.

[0660] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0661] In this invention, the server includes means for providing a user interface for selecting multiple generative AI models, means for collecting documents and sample data for the selected generative AI models, means for analyzing the collected documents and sample data and identifying functions of each generative AI model, means for generating an integrated prompt based on the identified functions, means for constructing a new generative AI model based on the analysis results and user emotion information, means for recognizing the user's emotions using an emotion engine and reflecting them in the output of the generative AI model, and means for providing the constructed generative AI model to the user. This makes it possible to efficiently integrate multiple generative AI models and provide optimal output according to the user's emotions.

[0662] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates new information or data based on input data.

[0663] A "user interface" is an interface through which a user interacts with software or a system.

[0664] A "document" is a text or file that contains an explanation or specification of a generative artificial intelligence model.

[0665] "Sample data" refers to specific input and output examples used to demonstrate the functionality of a generative artificial intelligence model.

[0666] "Natural language processing technology" is a technology that enables computers to understand and generate human language.

[0667] A "prompt" is an input instruction or question to a generative artificial intelligence model.

[0668] An "emotion engine" is a technology that recognizes a user's emotions and adjusts responses and output based on the results.

[0669] An "integrated prompt" is a consistent instruction that combines the capabilities of multiple generative artificial intelligence models.

[0670] The "new generative artificial intelligence model" is a new AI model that integrates the functions of multiple existing generative artificial intelligence models to provide optimal output according to the user's emotions.

[0671] The present invention is a system that allows users to use multiple generative AI models in an integrated manner, and also combines an emotion engine that recognizes the user's emotions and adjusts the output accordingly. This system is implemented by combining the following processes.

[0672] First, the user selects the generative AI model they want to use through a user interface, such as a web browser or application on a computer or smart device. The user can choose from a paper explanation AI model, an automatic code generation AI model, an illustration generation AI model, and so on.

[0673] The device then collects documentation and sample data for the selected generative AI model, often in the form of text or JSON files, and transmits the collected data to a server via the HTTPS protocol.

[0674] The server analyzes the received documents using natural language processing (NLP) techniques. Specifically, it uses Python's NLTK library and SpaCy to extract the functions and features of each generative AI model. Sample data is also analyzed using data analysis libraries such as Pandas and NumPy to identify the input and output formats for each model.

[0675] Based on the analysis results, the server generates integrated prompts, which are generated using large-scale language models such as GPT-3. Specifically, the prompts include "generate a summary of the paper, generate code based on that summary, and generate illustrations related to that code."

[0676] Furthermore, the server uses an emotion engine to recognize the user's emotions. The emotion engine uses, for example, IBM Watson's Emotion Analysis. It determines the user's emotional state based on the user's voice and text input obtained from the user interface. The recognized emotion is reflected in the output of the generative AI model. For example, if the user is feeling stressed, a gentle tone of voice corresponding to that emotion is generated.

[0677] The server then builds a new generative AI model based on the generated prompts and the user's emotional information. This new generative AI model is built using machine learning frameworks such as TensorFlow and PyTorch. This makes it possible to integrate functions for paper explanations, code generation, and illustration generation, while providing optimal output based on the user's emotional state.

[0678] Finally, the server provides the constructed generative AI model to the user's device. This process also uses HTTPS to ensure secure communication. The user can then use this integrated model to seamlessly execute a series of tasks.

[0679] As a concrete example, the following scenario can be considered.

[0680] When a user types, "Generate a summary of this paper," the server analyzes the user's input and creates a prompt to generate the summary. Furthermore, if the user is feeling stressed, the server generates a prompt saying, "Generate a summary of this paper in a gentle tone." Then, based on the summarized content, it generates a prompt saying, "Generate appropriate code based on the summarized paper content." Finally, it generates a prompt saying, "Create an illustration related to the generated code." The system seamlessly executes this series of processes.

[0681] Through the above steps, the present invention enables users to use multiple generative artificial intelligence models in an integrated manner, and furthermore, by reflecting the user's emotions in the output content, provides an efficient and appropriate user experience.

[0682] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0683] Step 1: User model selection

[0684] Specific operation: The user selects the generative AI model they want to use through a user interface (e.g., a web browser or application). Available models include an AI model for explaining papers, an AI model for automatically generating code, and an AI model for generating illustrations.

[0685] Input: The model type selected by the user.

[0686] Output: The information of the selected model is saved in the internal database.

[0687] Step 2: Gathering model information

[0688] Specific operations: The terminal collects documents and sample data for the generative artificial intelligence model selected through the user interface, including document files containing descriptions and specifications of the model, as well as example inputs and outputs.

[0689] Input: The model type selected by the user.

[0690] Output: Collected documents and sample data are sent to the server.

[0691] Step 3: Parsing the document

[0692] How it works: The server analyzes the received document files using natural language processing (NLP) techniques to identify the capabilities of each generative AI model. Specifically, it analyzes the content of the documents using Python's NLTK library and SpaCy.

[0693] Input: Collected document files.

[0694] Data processing: Tokenize the text information in the document and extract important keywords and context.

[0695] Output: Data about the capabilities of each model is stored in an internal database.

[0696] Step 4: Analyzing the sample data

[0697] Specific operation: The server analyzes the received sample data and identifies the specific input and output formats for each model. This analysis is performed using data analysis libraries such as Pandas and NumPy.

[0698] Input: Collected sample data.

[0699] Data processing: Analyze sample data to extract input and output formats.

[0700] Output: Data about the input and output formats for each model is stored in an internal database.

[0701] Step 5: Generate prompts

[0702] What it does: The server generates a unified prompt based on the analysis results, using a large-scale language model such as GPT-3.

[0703] Input: Data about the capabilities, input formats, and output formats of each model.

[0704] Data calculation: Combine the functions of each model to generate the optimal prompt.

[0705] Output: The integrated prompt is generated.

[0706] Step 6: Use the Emotion Engine

[0707] Specific operation: The server recognizes the user's emotions using an emotion engine. The emotion engine uses IBM Watson's Emotion Analysis. It analyzes the user's voice and text input obtained from the user interface to identify the user's emotional state.

[0708] Input: User voice and text input.

[0709] Data calculation: The emotion engine analyzes the emotional state.

[0710] Output: Data about the user's emotional state.

[0711] Step 7: Building a new generative AI model

[0712] How it works: The server builds a new generative AI model based on the generated prompts and the user's emotional information, using machine learning frameworks such as TensorFlow and PyTorch.

[0713] Input: Synthesized prompt sentence and user emotional state data.

[0714] Data calculation: Build an optimal generative AI model from prompt sentences and emotional information.

[0715] Output: A new generative artificial intelligence model.

[0716] Step 8: Serving the generative model

[0717] Specific operation: The server provides the constructed generative AI model to the user's device using the HTTPS protocol to ensure secure communication.

[0718] Input: A new generative artificial intelligence model.

[0719] Output: The generative AI model provided to the user's device.

[0720] (Application example 2)

[0721] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0722] In systems that use conventional generative AI models, users have to spend a lot of time and effort integrating multiple models. Furthermore, outputs generated without considering the user's emotional state can degrade the quality of the user experience. This makes it difficult to achieve the desired results for specific applications.

[0723] The specification processing by specification processing unit 290 of data processing device 12 in application example 2 is realized by the following means. In this invention, the server includes means for providing a user interface for selecting multiple generative AI models, means for collecting documents and sample data for the selected generative AI models, means for analyzing the collected documents and sample data and identifying the functions of each generative AI model, means for generating an integrated prompt based on the identified functions, means for constructing a new generative AI model using the generated prompt, means for recognizing a user's emotion and adjusting the output content of the generative AI model based on the recognized emotion, and means for providing the constructed generative AI model to the user. This allows the user to use multiple generative AI models in an integrated manner and further enables output adjustment based on emotion, providing a higher quality user experience.

[0724] A "generative artificial intelligence model" is an artificial intelligence algorithm that has the ability to receive input for a specific task and generate a corresponding output.

[0725] "User interface" is a general term for the screens and operating means that allow users to access and operate systems and software.

[0726] "Documentation" refers to instructions and specifications regarding the functions and usage of generative artificial intelligence models.

[0727] "Sample data" is a set of example inputs and outputs that demonstrate how a generative artificial intelligence model works.

[0728] "Analysis" is the act of examining data or documents in detail to identify meanings and patterns.

[0729] An "integrated prompt" is a command that combines the functions of multiple generative artificial intelligence models.

[0730] "Emotion recognition" refers to analyzing a user's input and behavior to identify their emotional state.

[0731] "Adjusting output" refers to adjusting the information or results generated based on the perceived emotion.

[0732] A "new generative artificial intelligence model" is a new, improved model that integrates the functions and characteristics of multiple existing generative artificial intelligence models.

[0733] "Provision" refers to the act of making a particular service or feature available to users.

[0734] A specific embodiment of the present invention will be described. The present invention is a system that can use multiple generative artificial intelligence models in an integrated manner, and further recognizes the user's emotions and adjusts the output accordingly.

[0735] First, a user interface is provided that allows users to select from multiple generative AI models. Through this interface, users can select the model they need based on their purpose, such as a product recommendation model or a review generation model.

[0736] Next, documents and sample data related to the selected generative AI models are collected and sent to a server, which analyzes the collected documents and sample data using natural language processing techniques (e.g., TensorFlow, PyTorch) to identify the capabilities of each generative AI model.

[0737] The server then generates a unified prompt based on the capabilities of each identified model. This prompt seamlessly combines the capabilities of multiple generative AI models, such as the following prompt:

[0738] "User is feeling stressed. Please recommend some relaxation items."

[0739] Furthermore, the server uses an emotion analysis model (e.g., EmotionAnalyzer) to recognize the user's emotions. Based on the recognized emotions, the output of the generative AI model can be adjusted. For example, if the user is feeling stressed, a gentle tone of voice or relaxation-related products will be recommended to match that emotion.

[0740] Finally, these methods are combined to build a new generative AI model, which is then provided to users. Users can access this integrated model through devices such as smartphones or head-mounted displays (HMDs) to seamlessly perform a series of tasks. This system allows users to receive high-quality services that respond to their emotional state.

[0741] As a concrete example, consider the case where a user is searching for "relaxing products" in a virtual store and enters "I've been very busy and stressed lately." Based on this input, the server generates the following prompt:

[0742] "User is feeling stressed. Please recommend some relaxation items."

[0743] The product recommendation model then uses these prompts to suggest appropriate relaxation items to the user, enabling the user to efficiently and comfortably utilize the capabilities of multiple generative AI models.

[0744] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0745] Step 1:

[0746] User model selection

[0747] The user selects from multiple generative AI models they wish to use through the system's user interface. This interface allows the user to select a specific generative AI model for a purpose, such as automatic product recommendation or review generation. The input is the user's model selection, and the output is a list of the selected models.

[0748] Step 2:

[0749] Collecting Model Information

[0750] The terminal collects documents and sample data related to the generative artificial intelligence models selected by the user, where the input is a list of selected models and the output is the documents and sample data corresponding to those models, and this data is sent to the server.

[0751] Step 3:

[0752] Document Parsing

[0753] The server analyzes the collected documents using natural language processing techniques, such as TensorFlow or PyTorch, to understand the content of the documents and identify the functions of each generative artificial intelligence model. The input here is the collected documents, and the output is a list of the model's functions.

[0754] Step 4:

[0755] Analysis of sample data

[0756] The server analyzes the collected sample data and understands the specific input and output formats of each model, which allows it to determine what data is required for a specific task. The input is the sample data, and the output is the specific information for the input and output formats of each model.

[0757] Step 5:

[0758] Prompt Generation

[0759] The server generates an integrated prompt based on the analysis results. For example, this prompt might be in the form of "The user is feeling stressed. Please recommend some relaxation items." Here, the functions of each model are combined to create a prompt that meets the user's needs. The input is the analysis results (model functions and input / output format), and the output is the integrated prompt.

[0760] Step 6:

[0761] Use of emotion engine

[0762] The server uses an emotion engine to recognize the user's emotions. The recognized emotions are reflected in the output of the generative AI model. For example, EmotionAnalyzer analyzes the user's input, "I've been very busy and stressed lately," and identifies the user's emotional state as stress. The input is the user's text input, and the output is the identified emotional state.

[0763] Step 7:

[0764] Building a new generative artificial intelligence model

[0765] The server constructs a new generative AI model based on the generated prompt and the user's emotional information. This new model integrates the necessary functions to respond to the prompt and can provide optimal output based on the user's emotional state. The input is the prompt and emotional information, and the output is the newly constructed generative AI model.

[0766] Step 8:

[0767] Providing a generative model

[0768] The server provides this newly constructed generative AI model to the user's device. The user can use this integrated model through a device such as a smartphone or head-mounted display (HMD) to seamlessly perform a series of tasks. The input is the new generative AI model, and the output is the services and functions that the user receives.

[0769] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0770] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0771] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0772] [Third embodiment]

[0773] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0774] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0775] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0776] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0777] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0778] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0779] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0780] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0781] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0782] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0783] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0784] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0785] Program processing overview

[0786] The present invention provides a system that allows users to integrate and use multiple generative artificial intelligence models (generative AI models). This system allows users to select models through a user interface, analyze their documentation and sample data, understand the functions of each model, generate new prompts, and finally construct a new integrated generative AI model to provide to the user.

[0787] Program Operation

[0788] 1. User model selection

[0789] Through the system's user interface, users can select from multiple generative AI models they wish to use, based on their specific tasks and objectives.

[0790] 2. Collecting model information

[0791] The device sends documents and sample data (example inputs and outputs) for each generative AI model selected by the user to the server.

[0792] 3. Document Analysis

[0793] The server analyzes the received documents using natural language processing (NLP) technology to identify the characteristics and functions of each generative AI model. For example, a document about an AI model that explains a paper will be analyzed to identify its function of explaining the paper.

[0794] 4. Analysis of sample data

[0795] The server analyzes sample data and understands the specific input and output formats of each model. For example, based on the input example of a code auto-generation AI model, it understands that when program specifications are input, code will be output.

[0796] 5. Prompt Generation

[0797] The server generates new prompts based on the capabilities of each identified model, which are designed to reflect the user's intent and integrate the capabilities of multiple generative AI models.

[0798] 6. Building a new generative AI model

[0799] The server then builds a new generative AI model based on the generated prompts, which provides different functions such as paper explanation, code generation, and illustration generation in a single, unified interface.

[0800] 7. Providing a Generative Model

[0801] The server then provides this newly constructed generative AI model to the user's device, where the user can use this integrated model to seamlessly execute a series of tasks.

[0802] Specific examples

[0803] For example, the following scenario is possible:

[0804] A user wants to read a paper, understand its content, generate related code based on the paper, and create an illustration to visually represent it. First, the user logs into the Mix Box platform and selects one of three models: a paper explanation AI model, an automatic code generation AI model, or an illustration generation AI model.

[0805] The device sends the documents and sample data of these models to the server, which analyzes them to understand the characteristics of each model. The server then creates an integrated prompt, such as "generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code." Based on this prompt, the server then builds a new generative AI model and provides it to the user.

[0806] Users input a paper using the new generative AI model, which then generates a summary, along with code and related illustrations based on the summary, seamlessly. This consistent process allows users to efficiently and conveniently leverage the capabilities of multiple generative AI models.

[0807] In this way, the present invention provides a system that allows users to use the functions of multiple generative AI models in an integrated manner, and is realized in a concrete form for sharing context and reducing effort.

[0808] The processing flow will be explained below.

[0809] Step 1:

[0810] Users log in to the Mix Box platform and select the generative AI model they want to use. Specific examples include a paper explanation AI model, an automatic code generation AI model, and an illustration generation AI model.

[0811] Step 2:

[0812] The device collects documentation and sample data (example inputs and outputs) for each generative AI model selected by the user, and sends this data to the server.

[0813] Step 3:

[0814] The server analyzes the received documents using natural language processing (NLP) technology. For example, from a document about an AI model explaining a paper, it understands that the model has the function of explaining a paper. It also analyzes AI models for automatic code generation and illustration generation in the same way.

[0815] Step 4:

[0816] The server analyzes the received sample data. Specifically, it identifies the input and output formats of each generative AI model. For example, it understands that the input example of a code auto-generation AI model is a program specification, and that the output example is code based on that specification.

[0817] Step 5:

[0818] The server generates an integrated prompt based on the analysis results. This prompt combines the functions of multiple generative AI models. For example, it creates a prompt that says, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[0819] Step 6:

[0820] The server uses these prompts to build a new generative AI model that combines the capabilities of paper commentary, code generation, and illustration generation.

[0821] Step 7:

[0822] The server then provides the newly constructed generative AI model to the user's device, allowing the user to seamlessly execute a series of tasks using this integrated model.

[0823] Step 8:

[0824] Users can then use the new generative AI models provided to them to perform specific tasks. For example, by inputting a paper, a summary of the paper is generated, code based on the summary is generated, and then illustrations related to the code are generated. In this way, users can efficiently use the functions of multiple generative AI models in a single system.

[0825] Through these steps, the system enables users to integrate the functions of multiple generative AI models, reducing the need for context sharing and effort.

[0826] Example 1

[0827] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0828] Currently, there is no system for effectively integrating multiple generative AI models. Users must select models individually, input data for each model separately, and obtain results. This process is time-consuming, labor-intensive, and inefficient. Furthermore, it is difficult to integrate data and functions between different generative AI models, making it difficult to obtain consistent results.

[0829] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0830] In this invention, the server includes means for providing a user interface for selecting multiple generative AI models, means for collecting information and sample data on the selected generative AI models, means for analyzing the collected documents and sample data and identifying functions of each generative AI model, means for generating integrated prompts based on the identified functions, means for providing the constructed generative AI model to the user, means for generating prompts that reflect the user's intentions and integrate the functions of the multiple generative AI models, and means for constructing a new generative AI model based on the generated prompts and seamlessly executing a series of tasks. This enables the user to efficiently integrate and use multiple generative AI models and obtain consistent results.

[0831] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates output such as text, images, or program code based on user input.

[0832] A "user interface" refers to a screen or input device that allows a user to interact with a system, and is the means by which a user performs the necessary operations and selections.

[0833] "Sample data" refers to a dataset that contains specific examples to demonstrate the format and content of inputs and outputs to a generative artificial intelligence model.

[0834] "Documentation" refers to documents that describe the functions, specifications, and usage of a generative artificial intelligence model, and includes information essential for understanding and using the model.

[0835] "Natural language processing technology" is a general term for technologies that enable computers to understand and manipulate text data, and includes technologies for analyzing documents and extracting meaning.

[0836] A "prompt" is text given to a generative artificial intelligence model in the form of an instruction or question, and is an input sentence that causes the model to perform a desired action.

[0837] This invention relates to a system that allows users to use multiple generative artificial intelligence models (generative AI models) in an integrated manner. This system mainly consists of a server, a terminal, and a user. The processes and roles of each are explained in detail below.

[0838] System Overview

[0839] The user selects a generative AI model through the system's user interface, after which the server and device work together to collect and analyze the necessary data, ultimately providing the user with an integrated generative AI model.

[0840] Hardware and Software

[0841] server

[0842] A server equipped with high-performance computers and natural language processing technology (e.g., using spaCy or NLTK as an NLP toolkit).

[0843] Terminal

[0844] A computer, tablet, or smartphone with an internet connection.

[0845] User Interface

[0846] The interface is provided using a web browser or a dedicated application.

[0847] Data processing and calculation

[0848] 1. User model selection

[0849] Through the interface, users can select the generative AI model they want to use, such as an AI model for paper commentary, automatic code generation, or illustration generation.

[0850] 2. Collecting model information

[0851] The device collects documentation and sample data about the model selected by the user and sends it to the server, including the model's features, specifications, and example input and output data.

[0852] 3. Analyzing Documents and Sample Data

[0853] The server analyzes the collected documents using natural language processing technology and extracts the characteristics and functions of each generative AI model. For example, from documents of an AI model that explains academic papers, it identifies the literature explanation function.

[0854] The server also analyzes sample data to understand the input and output formats of each model. For example, from an example input for a code auto-generation AI model, it identifies the program specification input and the corresponding code output.

[0855] 4. Prompt generation and building a new generative AI model

[0856] The server generates an integrated prompt based on the analysis results. This prompt reflects the user's intent and integrates the functions of multiple generative AI models. For example, it creates a prompt that asks, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[0857] Based on these prompts, we build a new generative AI model that integrates the functions of paper explanation, code generation, and illustration generation.

[0858] 5. Providing a Generative Model

[0859] The server provides the newly constructed generative AI model to the user, who can then use the model to seamlessly execute different tasks.

[0860] Specific examples

[0861] For example, consider a scenario where a user logs in to the Mix Box platform and performs the following steps: First, the user selects an AI model for explaining papers, then an AI model for automatically generating code, and finally an AI model for generating illustrations. The device then sends these documents and sample data to the server, which then analyzes each data and generates an integrated prompt.

[0862] An example prompt might be, "Generate a summary of a paper, generate Python code based on that summary, and generate illustrations related to that code." A new generative AI model is built based on this prompt, allowing users to efficiently and seamlessly utilize multiple AI functions.

[0863] In this way, the present invention provides a system that allows users to integrate the functions of multiple generative AI models and achieve their goals through a consistent process.

[0864] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0865] Step 1:

[0866] User model selection

[0867] Specific actions

[0868] Users access the system's user interface and select from multiple generative artificial intelligence models according to their goals and challenges.

[0869] input

[0870] A list of generative AI models based on user selections (clicks, etc.)

[0871] output

[0872] A list of selected generative AI models

[0873] Step 2:

[0874] Collecting Model Information

[0875] Specific actions

[0876] The device collects documentation and sample data for the generative AI model selected by the user, including API calls and retrieval from a database, and sends the collected data to a server.

[0877] input

[0878] A list of generative AI models selected by the user

[0879] output

[0880] Documentation and sample data for each model

[0881] Step 3:

[0882] Document Parsing

[0883] Specific actions

[0884] The server uses natural language processing (NLP) technology to analyze the documents of each generative AI model received from the device, thereby extracting the characteristics and functions of each AI model.

[0885] input

[0886] Documentation for each generative AI model (PDF, HTML, etc.)

[0887] output

[0888] A list of the characteristics and capabilities of each AI model

[0889] Step 4:

[0890] Analysis of sample data

[0891] Specific actions

[0892] The server analyzes the sample data received from each generative AI model and understands the specific input and output formats, thereby understanding how each model operates.

[0893] input

[0894] Sample data for each generative AI model (JSON, CSV, etc.)

[0895] output

[0896] Details of the input and output formats for each model

[0897] Step 5:

[0898] Prompt Generation

[0899] Specific actions

[0900] The server generates new prompts based on the characteristics and capabilities of each analyzed model, which are designed to reflect the user's intent and integrate the capabilities of multiple AI models.

[0901] input

[0902] Details of the characteristics and functions of each model, as well as input and output formats

[0903] output

[0904] Integrated prompts

[0905] Step 6:

[0906] Building a new generative AI model

[0907] Specific actions

[0908] The server then uses the generated prompts to build a new generative AI model, which is designed to integrate multiple functions and guide users through tasks in a consistent process.

[0909] input

[0910] Integrated prompts

[0911] output

[0912] New generative AI models

[0913] Step 7:

[0914] Providing generative AI models

[0915] Specific actions

[0916] The server provides the constructed generative AI model to the user, who can then use this model to seamlessly execute a series of tasks.

[0917] input

[0918] New generative AI models

[0919] output

[0920] Generative AI models provided by users

[0921] (Application example 1)

[0922] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0923] Conventional electronic payment services have difficulty proposing optimal payment methods, offering promotions, and providing coupons to users. Furthermore, functions such as budget management and fraud detection are provided separately, resulting in the inconvenience of users having to use multiple applications and services. Therefore, there is a need for a system that can provide these multiple functions in a unified manner, improving the user experience.

[0924] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0925] In this invention, the server includes means for providing a user interface for selecting multiple generative artificial intelligence models, means for collecting documents and sample data for the selected generative artificial intelligence models, means for analyzing the collected documents and sample data to identify functions of each generative artificial intelligence model, means for generating integrated prompts based on the identified functions, means for constructing a new generative artificial intelligence model using the generated prompts, means for suggesting an appropriate electronic payment method to the user, means for analyzing past purchase history and market trends to generate and suggest promotions and coupons, means for analyzing input income and expenditure data to provide a budget management plan, means for analyzing transaction data in real time to detect fraudulent transactions, and means for providing the constructed generative artificial intelligence model and various suggestions and notifications to the user. This allows users to receive comprehensive electronic payment support in a single system, improving convenience and safety.

[0926] A "user interface" is an interface that has an operation screen and input means for a user to select from multiple generative artificial intelligence models.

[0927] "Documentation" refers to the specifications, instructions, and technical documentation for the generative artificial intelligence model.

[0928] "Sample data" refers to a dataset that includes example inputs and outputs of a generative artificial intelligence model.

[0929] "Natural language processing technology" is a technology for analyzing text data and understanding human language, including morphological analysis and contextual understanding.

[0930] An "integrated prompt" is a new input instruction that integrates the functions of multiple generative artificial intelligence models and reflects the user's intentions.

[0931] An "electronic payment method" is a means by which a user pays for products online or in a brick-and-mortar store, and includes, for example, credit cards, QR code payments, digital wallets, etc.

[0932] "Promotion" refers to a discount or campaign to promote a particular product or service to users.

[0933] A "Coupon" is electronic or physical evidence of a discount under certain conditions.

[0934] A "budget management plan" is a plan for proper financial management that is created based on the user's income and expenditure data.

[0935] "Unauthorized Transactions" are transactions conducted in an unauthorized manner, including, for example, theft of credit cards and other fraudulent activities.

[0936] "Real-time" refers to a time frame in which data is processed immediately and results are reflected immediately.

[0937] "Generative AI model" is a general term for AI models that have the ability to generate an output corresponding to a specific input.

[0938] MODE FOR CARRYING OUT THE INVENTION

[0939] System Overview

[0940] The present invention is a system that allows users to integrate and use multiple generative AI models, particularly in electronic payment services. This system allows users to select models through a user interface (UI), analyze their documentation and sample data, understand the functions of each model, generate new prompts, and finally construct a new integrated generative AI model to provide to the user.

[0941] Hardware and software used

[0942] Frontend: React Native (Mobile App Development)

[0943] Backend: Node.js (server side), MongoDB (database)

[0944] Using AI models: Python, TensorFlow, GPT models

[0945] Processing flow

[0946] 1. User model selection and document collection:

[0947] The user selects multiple generative AI models (e.g., a payment method suggestion model, a coupon generation model, a budget management model, and a fraudulent transaction detection model) through the UI of a smartphone app.

[0948] Documentation and sample data for the selected model are sent to the server.

[0949] 2. Analyzing the documentation and sample data:

[0950] The server analyzes the received documents using natural language processing (NLP) techniques to identify the characteristics and functions of each generative AI model.

[0951] The server parses the sample data and understands the specific input and output formats for each model.

[0952] 3. Generating new prompts and building an integrated model:

[0953] The server generates new prompts based on the capabilities of each identified model, which are designed to reflect the user's intent and integrate the capabilities of multiple generative AI models.

[0954] Based on the generated prompts, a new generative AI model is built that can suggest appropriate electronic payment methods to users, generate and suggest promotions and coupons, provide budgeting plans, and detect fraudulent transactions.

[0955] 4. User Provision and Notification:

[0956] The server provides this newly constructed generative AI model and various suggestions and notifications to the user's smartphone app, allowing the user to seamlessly perform a series of tasks using this integrated model.

[0957] Specific examples

[0958] Example 1:

[0959] When users try to buy a drink, the app will recommend QR code payment and provide the latest promotional coupons. In addition, the app will suggest appropriate budget management plans and help users manage their overall spending based on their past shopping history.

[0960] Example 2:

[0961] When a user pays for a subscription service, the fraud detection model detects anomalous behavior and immediately alerts the user, allowing them to take immediate action and protect themselves from fraudulent transactions.

[0962] Prompt Sentence Examples

[0963] "Generate recommended coupons based on the user's last five purchases and current market promotions."

[0964] "Based on the income and expense data entered, please suggest the best monthly budget management plan for this user."

[0965] "Detect fraudulent transactions based on the latest transaction data and generate alerts."

[0966] As described above, the present invention enables users to receive comprehensive electronic payment support in one system, improving convenience and security.

[0967] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0968] Step 1:

[0969] The user selects multiple generative AI models through the UI of a smartphone app. Specifically, the user can select a payment method suggestion model, a coupon generation model, a budget management model, or a fraudulent transaction detection model. The input is the ID and name of the AI ​​model selected by the user, and the output is a list of the selected AI models.

[0970] Step 2:

[0971] The terminal collects the documents and sample data of the selected generative AI model and sends them to the server. The specific operation performed by the terminal is to call the API corresponding to the selected AI model to obtain the documents and sample data. The input here is the list of AI models selected in step 1, and the output is a set of documents and sample data for each model.

[0972] Step 3:

[0973] The server analyzes the received documents using natural language processing (NLP) techniques to identify the capabilities of each generative AI model. Specifically, the server performs morphological analysis and contextual understanding to extract important keywords and phrases from the documents. The input is the document obtained in step 2, and the output is a list of the characteristics and capabilities of each AI model.

[0974] Step 4:

[0975] The server analyzes the sample data and understands the specific input and output formats of each model. Specifically, the server analyzes the format of the sample data and identifies typical input data patterns and the corresponding output data formats. The input here is the sample data obtained in step 2, and the output is a list of input and output formats for each model.

[0976] Step 5:

[0977] The server generates a new prompt based on the capabilities of each identified model. This prompt is designed to reflect the user's intent and integrate the capabilities of multiple generative AI models. Specifically, the server optimizes the service by combining appropriate prompts to suit the user's purpose. The input is the list of capabilities and data formats obtained in Step 3 and Step 4, and the output is a new integrated prompt.

[0978] Step 6:

[0979] The server builds a new generative AI model based on the generated prompt. Specifically, the server calls the API of each AI model to send an integrated prompt, and then integrates the output to create a new model. The input is the prompt generated in step 5, and the output is a new integrated generative AI model.

[0980] Step 7:

[0981] The server provides this newly constructed generative AI model and various suggestions and notifications to the user's smartphone app. Specifically, the server provides the generated model as an endpoint and allows the user to access it. The user can use this integrated model to receive electronic payment method suggestions, obtain promotions and coupons, create budget management plans, and monitor fraudulent transactions. The input is the integrated generative AI model constructed in step 6, and the output is various services provided through the user's application interface.

[0982] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0983] Program processing overview

[0984] This invention provides a system that allows users to integrate multiple generative artificial intelligence models (generative AI models), and further combines it with an emotion engine that recognizes the user's emotions and adjusts output accordingly. This system allows users to select models through a user interface, analyzes their documents and sample data, understands the functions of each model, generates new prompts, and finally constructs a new integrated generative AI model to provide to the user.

[0985] Program Operation

[0986] 1. User model selection

[0987] Through the system's user interface, users can select from multiple generative AI models based on their specific tasks and goals. For example, they can choose from an AI model for explaining papers, an AI model for automatically generating code, or an AI model for generating illustrations.

[0988] 2. Collecting model information

[0989] The device collects documentation and sample data (example inputs and outputs) related to the generative AI model selected by the user and sends this to the server.

[0990] 3. Document Analysis

[0991] The server analyzes the received documents using natural language processing (NLP) technology. For example, from a document about an AI model explaining a paper, it understands that the model has the function of explaining a paper. It also analyzes AI models for automatic code generation and illustration generation in the same way.

[0992] 4. Analysis of sample data

[0993] The server analyzes the sample data and understands the specific input and output formats of each model. For example, it understands that the input example of a code auto-generation AI model is a program specification, and the output example is code based on that specification.

[0994] 5. Prompt Generation

[0995] The server generates an integrated prompt based on the analysis results. This prompt combines the functions of multiple generative AI models. For example, it creates a prompt that says, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[0996] 6. Use of Emotion Engines

[0997] The server uses an emotion engine to recognize the user's emotions. The recognized emotions are reflected in the output of the generative AI model. For example, if the user is feeling stressed, the server generates a gentle explanation in accordance with the user's emotions and an illustration to soothe the user's mood.

[0998] 7. Building a new generative AI model

[0999] The server then builds a new generative AI model based on the generated prompts and the user's emotional information. This new model integrates the functions of paper explanation, code generation, and illustration generation, and provides optimal output according to the user's emotions.

[1000] 8. Providing a Generative Model

[1001] The server then provides this newly constructed generative AI model to the user's device, where the user can use this integrated model to seamlessly execute a series of tasks.

[1002] Specific examples

[1003] For example, the following scenario is possible:

[1004] A user wants to read a paper, understand its content, generate related code based on the paper, and create an illustration to visually represent it. First, the user logs into the Mix Box platform and selects one of three models: a paper explanation AI model, an automatic code generation AI model, or an illustration generation AI model.

[1005] The device sends the documents and sample data of these models to the server, which analyzes them to understand the characteristics of each model. The server then creates an integrated prompt that asks, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code." The server then uses an emotion engine to analyze the user's emotions and makes adjustments based on those emotions to build a new generative AI model.

[1006] Users input a paper using the new generative AI model, which then generates a summary of the paper, along with code and related illustrations based on the summary, seamlessly. This consistent process is delivered in a way that takes into account the user's emotional state, allowing users to efficiently and comfortably leverage the capabilities of multiple generative AI models.

[1007] In this way, the present invention provides a system that allows users to use the functions of multiple generative AI models in an integrated manner, and by combining it with an emotion engine, the system reflects the user's emotional state in the output content, thereby realizing a concrete form that improves the user experience while reducing the need for context sharing and effort.

[1008] The processing flow will be explained below.

[1009] Step 1:

[1010] Users log in to the Mix Box platform and select multiple generative AI models they want to use. For example, they can choose an AI model for explaining papers, an AI model for automatically generating code, or an AI model for generating illustrations.

[1011] Step 2:

[1012] The device collects documentation and sample data (example inputs and outputs) for each generative AI model selected by the user and sends this to the server.

[1013] Step 3:

[1014] The server analyzes the received documents using natural language processing (NLP) technology. For example, from a document about an AI model explaining a paper, it understands that this model has the function of explaining a paper. Similarly, it analyzes AI models for automatic code generation and illustration generation.

[1015] Step 4:

[1016] The server analyzes the received sample data and understands the specific input and output formats of each model. For example, it understands that the input example of a code auto-generation AI model is a program specification, and that the output example is code based on that specification.

[1017] Step 5:

[1018] The server generates an integrated prompt based on the analysis results. This prompt combines the functions of multiple generative AI models. For example, it creates a prompt that says, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[1019] Step 6:

[1020] The server uses an emotion engine to recognize the user's emotions, for example, by analyzing the tone and speed of the user's keyboard and voice input to identify their current emotional state (e.g., stress, happiness, fatigue).

[1021] Step 7:

[1022] The server adjusts the generated prompts based on the user's emotional information obtained from the emotion engine. For example, if the user is feeling stressed, the server changes the explanatory text to a gentler tone and adjusts the illustrations to have a more relaxing effect.

[1023] Step 8:

[1024] Based on the tailored prompts, the server builds a new generative AI model that integrates the functions of paper explanation, code generation, and illustration generation, while providing output that responds to the user's emotions.

[1025] Step 9:

[1026] The server then provides the newly constructed generative AI model to the user's device, allowing the user to seamlessly execute a series of tasks using this integrated model.

[1027] Step 10:

[1028] Users use the provided new generative AI model to perform specific tasks. For example, by inputting a paper, the system generates a summary of the paper, generates code based on the summary, and then generates an illustration related to the code. The output content is provided according to the user's emotional state, allowing users to work efficiently and comfortably.

[1029] In this way, this system allows users to integrate the functions of multiple generative AI models, and by combining it with an emotion engine, it reflects the user's emotional state in the output content, improving the user experience.

[1030] Example 2

[1031] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1032] Conventional generative AI models require users to use each individual model, and integrating the outputs of each model requires a lot of time and effort. Furthermore, the results output without considering the user's emotions often do not necessarily respond to the user's needs. This detracts from the overall user experience.

[1033] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1034] In this invention, the server includes means for providing a user interface for selecting multiple generative AI models, means for collecting documents and sample data for the selected generative AI models, means for analyzing the collected documents and sample data and identifying functions of each generative AI model, means for generating an integrated prompt based on the identified functions, means for constructing a new generative AI model based on the analysis results and user emotion information, means for recognizing the user's emotions using an emotion engine and reflecting them in the output of the generative AI model, and means for providing the constructed generative AI model to the user. This makes it possible to efficiently integrate multiple generative AI models and provide optimal output according to the user's emotions.

[1035] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates new information or data based on input data.

[1036] A "user interface" is an interface through which a user interacts with software or a system.

[1037] A "document" is a text or file that contains an explanation or specification of a generative artificial intelligence model.

[1038] "Sample data" refers to specific input and output examples used to demonstrate the functionality of a generative artificial intelligence model.

[1039] "Natural language processing technology" is a technology that enables computers to understand and generate human language.

[1040] A "prompt" is an input instruction or question to a generative artificial intelligence model.

[1041] An "emotion engine" is a technology that recognizes a user's emotions and adjusts responses and output based on the results.

[1042] An "integrated prompt" is a consistent instruction that combines the capabilities of multiple generative artificial intelligence models.

[1043] The "new generative artificial intelligence model" is a new AI model that integrates the functions of multiple existing generative artificial intelligence models to provide optimal output according to the user's emotions.

[1044] The present invention is a system that allows users to use multiple generative AI models in an integrated manner, and also combines an emotion engine that recognizes the user's emotions and adjusts the output accordingly. This system is implemented by combining the following processes.

[1045] First, the user selects the generative AI model they want to use through a user interface, such as a web browser or application on a computer or smart device. The user can choose from a paper explanation AI model, an automatic code generation AI model, an illustration generation AI model, and so on.

[1046] The device then collects documentation and sample data for the selected generative AI model, often in the form of text or JSON files, and transmits the collected data to a server via the HTTPS protocol.

[1047] The server analyzes the received documents using natural language processing (NLP) techniques. Specifically, it uses Python's NLTK library and SpaCy to extract the functions and features of each generative AI model. Sample data is also analyzed using data analysis libraries such as Pandas and NumPy to identify the input and output formats for each model.

[1048] Based on the analysis results, the server generates integrated prompts, which are generated using large-scale language models such as GPT-3. Specifically, the prompts include "generate a summary of the paper, generate code based on that summary, and generate illustrations related to that code."

[1049] Furthermore, the server uses an emotion engine to recognize the user's emotions. The emotion engine uses, for example, IBM Watson's Emotion Analysis. It determines the user's emotional state based on the user's voice and text input obtained from the user interface. The recognized emotion is reflected in the output of the generative AI model. For example, if the user is feeling stressed, a gentle tone of voice corresponding to that emotion is generated.

[1050] The server then builds a new generative AI model based on the generated prompts and the user's emotional information. This new generative AI model is built using machine learning frameworks such as TensorFlow and PyTorch. This makes it possible to integrate functions for paper explanations, code generation, and illustration generation, while providing optimal output based on the user's emotional state.

[1051] Finally, the server provides the constructed generative AI model to the user's device. This process also uses HTTPS to ensure secure communication. The user can then use this integrated model to seamlessly execute a series of tasks.

[1052] As a concrete example, the following scenario can be considered.

[1053] When a user types, "Generate a summary of this paper," the server analyzes the user's input and creates a prompt to generate the summary. Furthermore, if the user is feeling stressed, the server generates a prompt saying, "Generate a summary of this paper in a gentle tone." Then, based on the summarized content, it generates a prompt saying, "Generate appropriate code based on the summarized paper content." Finally, it generates a prompt saying, "Create an illustration related to the generated code." The system seamlessly executes this series of processes.

[1054] Through the above steps, the present invention enables users to use multiple generative artificial intelligence models in an integrated manner, and furthermore, by reflecting the user's emotions in the output content, provides an efficient and appropriate user experience.

[1055] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1056] Step 1: User model selection

[1057] Specific operation: The user selects the generative AI model they want to use through a user interface (e.g., a web browser or application). Available models include an AI model for explaining papers, an AI model for automatically generating code, and an AI model for generating illustrations.

[1058] Input: The model type selected by the user.

[1059] Output: The information of the selected model is saved in the internal database.

[1060] Step 2: Gathering model information

[1061] Specific operations: The terminal collects documents and sample data for the generative artificial intelligence model selected through the user interface, including document files containing descriptions and specifications of the model, as well as example inputs and outputs.

[1062] Input: The model type selected by the user.

[1063] Output: Collected documents and sample data are sent to the server.

[1064] Step 3: Parsing the document

[1065] How it works: The server analyzes the received document files using natural language processing (NLP) techniques to identify the capabilities of each generative AI model. Specifically, it analyzes the content of the documents using Python's NLTK library and SpaCy.

[1066] Input: Collected document files.

[1067] Data processing: Tokenize the text information in the document and extract important keywords and context.

[1068] Output: Data about the capabilities of each model is stored in an internal database.

[1069] Step 4: Analyzing the sample data

[1070] Specific operation: The server analyzes the received sample data and identifies the specific input and output formats for each model. This analysis is performed using data analysis libraries such as Pandas and NumPy.

[1071] Input: Collected sample data.

[1072] Data processing: Analyze sample data to extract input and output formats.

[1073] Output: Data about the input and output formats for each model is stored in an internal database.

[1074] Step 5: Generate prompts

[1075] What it does: The server generates a unified prompt based on the analysis results, using a large-scale language model such as GPT-3.

[1076] Input: Data about the capabilities, input formats, and output formats of each model.

[1077] Data calculation: Combine the functions of each model to generate the optimal prompt.

[1078] Output: The integrated prompt is generated.

[1079] Step 6: Use the Emotion Engine

[1080] Specific operation: The server recognizes the user's emotions using an emotion engine. The emotion engine uses IBM Watson's Emotion Analysis. It analyzes the user's voice and text input obtained from the user interface to identify the user's emotional state.

[1081] Input: User voice and text input.

[1082] Data calculation: The emotion engine analyzes the emotional state.

[1083] Output: Data about the user's emotional state.

[1084] Step 7: Building a new generative AI model

[1085] How it works: The server builds a new generative AI model based on the generated prompts and the user's emotional information, using machine learning frameworks such as TensorFlow and PyTorch.

[1086] Input: Synthesized prompt sentence and user emotional state data.

[1087] Data calculation: Build an optimal generative AI model from prompt sentences and emotional information.

[1088] Output: A new generative artificial intelligence model.

[1089] Step 8: Serving the generative model

[1090] Specific operation: The server provides the constructed generative AI model to the user's device using the HTTPS protocol to ensure secure communication.

[1091] Input: A new generative artificial intelligence model.

[1092] Output: The generative AI model provided to the user's device.

[1093] (Application example 2)

[1094] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1095] In systems that use conventional generative AI models, users have to spend a lot of time and effort integrating multiple models. Furthermore, outputs generated without considering the user's emotional state can degrade the quality of the user experience. This makes it difficult to achieve the desired results in certain applications.

[1096] The specification processing by specification processing unit 290 of data processing device 12 in application example 2 is realized by the following means. In this invention, the server includes means for providing a user interface for selecting multiple generative AI models, means for collecting documents and sample data for the selected generative AI models, means for analyzing the collected documents and sample data and identifying the functions of each generative AI model, means for generating an integrated prompt based on the identified functions, means for constructing a new generative AI model using the generated prompt, means for recognizing a user's emotion and adjusting the output content of the generative AI model based on the recognized emotion, and means for providing the constructed generative AI model to the user. This allows the user to use multiple generative AI models in an integrated manner and further enables output adjustment based on emotion, providing a higher quality user experience.

[1097] A "generative artificial intelligence model" is an artificial intelligence algorithm that has the ability to receive input for a specific task and generate a corresponding output.

[1098] "User interface" is a general term for the screens and operating means that allow users to access and operate systems and software.

[1099] "Documentation" refers to instructions and specifications regarding the functions and usage of generative artificial intelligence models.

[1100] "Sample data" is a set of example inputs and outputs that demonstrate how a generative artificial intelligence model works.

[1101] "Analysis" is the act of examining data or documents in detail to identify meanings and patterns.

[1102] An "integrated prompt" is a command that combines the functions of multiple generative artificial intelligence models.

[1103] "Emotion recognition" refers to analyzing a user's input and behavior to identify their emotional state.

[1104] "Adjusting output" refers to adjusting the information or results generated based on the perceived emotion.

[1105] A "new generative artificial intelligence model" is a new, improved model that integrates the functions and characteristics of multiple existing generative artificial intelligence models.

[1106] "Provision" refers to the act of making a particular service or feature available to users.

[1107] A specific embodiment of the present invention will be described. The present invention is a system that can use multiple generative artificial intelligence models in an integrated manner, and further recognizes the user's emotions and adjusts the output accordingly.

[1108] First, a user interface is provided that allows users to select from multiple generative AI models. Through this interface, users can select the model they need based on their purpose, such as a product recommendation model or a review generation model.

[1109] Next, documents and sample data related to the selected generative AI models are collected and sent to a server, which analyzes the collected documents and sample data using natural language processing techniques (e.g., TensorFlow, PyTorch) to identify the capabilities of each generative AI model.

[1110] The server then generates a unified prompt based on the capabilities of each identified model. This prompt seamlessly combines the capabilities of multiple generative AI models, such as the following prompt:

[1111] "User is feeling stressed. Please recommend some relaxation items."

[1112] Furthermore, the server uses an emotion analysis model (e.g., EmotionAnalyzer) to recognize the user's emotions. Based on the recognized emotions, the output of the generative AI model can be adjusted. For example, if the user is feeling stressed, a gentle tone of voice or relaxation-related products will be recommended to match that emotion.

[1113] Finally, these methods are combined to build a new generative AI model, which is then provided to users. Users can access this integrated model through devices such as smartphones or head-mounted displays (HMDs) to seamlessly perform a series of tasks. This system allows users to receive high-quality services that respond to their emotional state.

[1114] As a concrete example, consider the case where a user is searching for "relaxing products" in a virtual store and enters "I've been very busy and stressed lately." Based on this input, the server generates the following prompt:

[1115] "User is feeling stressed. Please recommend some relaxation items."

[1116] The product recommendation model then uses these prompts to suggest appropriate relaxation items to the user, enabling the user to efficiently and comfortably utilize the capabilities of multiple generative AI models.

[1117] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1118] Step 1:

[1119] User model selection

[1120] The user selects from multiple generative AI models they wish to use through the system's user interface. This interface allows the user to select a specific generative AI model for a purpose, such as automatic product recommendation or review generation. The input is the user's model selection, and the output is a list of the selected models.

[1121] Step 2:

[1122] Collecting Model Information

[1123] The terminal collects documents and sample data related to the generative artificial intelligence models selected by the user, where the input is a list of selected models and the output is the documents and sample data corresponding to those models, and this data is sent to the server.

[1124] Step 3:

[1125] Document Parsing

[1126] The server analyzes the collected documents using natural language processing techniques, such as TensorFlow or PyTorch, to understand the content of the documents and identify the functions of each generative artificial intelligence model. The input here is the collected documents, and the output is a list of the model's functions.

[1127] Step 4:

[1128] Analysis of sample data

[1129] The server analyzes the collected sample data and understands the specific input and output formats of each model, which allows it to determine what data is required for a specific task. The input is the sample data, and the output is the specific information for the input and output formats of each model.

[1130] Step 5:

[1131] Prompt Generation

[1132] The server generates an integrated prompt based on the analysis results. For example, this prompt might be in the form of "The user is feeling stressed. Please recommend some relaxation items." Here, the functions of each model are combined to create a prompt that meets the user's needs. The input is the analysis results (model functions and input / output format), and the output is the integrated prompt.

[1133] Step 6:

[1134] Use of emotion engine

[1135] The server uses an emotion engine to recognize the user's emotions. The recognized emotions are reflected in the output of the generative AI model. For example, EmotionAnalyzer analyzes the user's input, "I've been very busy and stressed lately," and identifies the user's emotional state as stress. The input is the user's text input, and the output is the identified emotional state.

[1136] Step 7:

[1137] Building a new generative artificial intelligence model

[1138] The server constructs a new generative AI model based on the generated prompt and the user's emotional information. This new model integrates the necessary functions to respond to the prompt and can provide optimal output based on the user's emotional state. The input is the prompt and emotional information, and the output is the newly constructed generative AI model.

[1139] Step 8:

[1140] Providing a generative model

[1141] The server provides this newly constructed generative AI model to the user's device. The user can use this integrated model through a device such as a smartphone or head-mounted display (HMD) to seamlessly perform a series of tasks. The input is the new generative AI model, and the output is the services and functions that the user receives.

[1142] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1143] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1144] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1145] [Fourth embodiment]

[1146] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1147] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1148] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1149] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1150] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1151] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1152] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1153] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1154] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1155] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1156] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1157] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1158] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1159] Program processing overview

[1160] The present invention provides a system that allows users to integrate and use multiple generative artificial intelligence models (generative AI models). This system allows users to select models through a user interface, analyze their documentation and sample data, understand the functions of each model, generate new prompts, and finally construct a new integrated generative AI model to provide to the user.

[1161] Program Operation

[1162] 1. User model selection

[1163] Through the system's user interface, users can select from multiple generative AI models they wish to use, based on their specific tasks and objectives.

[1164] 2. Collecting model information

[1165] The device sends documents and sample data (example inputs and outputs) for each generative AI model selected by the user to the server.

[1166] 3. Document Analysis

[1167] The server analyzes the received documents using natural language processing (NLP) technology to identify the characteristics and functions of each generative AI model. For example, a document about an AI model that explains a paper will be analyzed to identify its function of explaining the paper.

[1168] 4. Analysis of sample data

[1169] The server analyzes sample data and understands the specific input and output formats of each model. For example, based on the input example of a code auto-generation AI model, it understands that when program specifications are input, code will be output.

[1170] 5. Prompt Generation

[1171] The server generates new prompts based on the capabilities of each identified model, which are designed to reflect the user's intent and integrate the capabilities of multiple generative AI models.

[1172] 6. Building a new generative AI model

[1173] The server then builds a new generative AI model based on the generated prompts, which provides different functions such as paper explanation, code generation, and illustration generation in a single, unified interface.

[1174] 7. Providing a Generative Model

[1175] The server then provides this newly constructed generative AI model to the user's device, where the user can use this integrated model to seamlessly execute a series of tasks.

[1176] Specific examples

[1177] For example, the following scenario is possible:

[1178] A user wants to read a paper, understand its content, generate related code based on the paper, and create an illustration to visually represent it. First, the user logs into the Mix Box platform and selects one of three models: a paper explanation AI model, an automatic code generation AI model, or an illustration generation AI model.

[1179] The device sends the documents and sample data of these models to the server, which analyzes them to understand the characteristics of each model. The server then creates an integrated prompt, such as "generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code." Based on this prompt, the server then builds a new generative AI model and provides it to the user.

[1180] Users input a paper using the new generative AI model, which then generates a summary, along with code and related illustrations based on the summary, seamlessly. This consistent process allows users to efficiently and conveniently leverage the capabilities of multiple generative AI models.

[1181] In this way, the present invention provides a system that allows users to use the functions of multiple generative AI models in an integrated manner, and is realized in a concrete form for sharing context and reducing effort.

[1182] The processing flow will be explained below.

[1183] Step 1:

[1184] Users log in to the Mix Box platform and select the generative AI model they want to use. Specific examples include a paper explanation AI model, an automatic code generation AI model, and an illustration generation AI model.

[1185] Step 2:

[1186] The device collects documentation and sample data (example inputs and outputs) for each generative AI model selected by the user, and sends this data to the server.

[1187] Step 3:

[1188] The server analyzes the received documents using natural language processing (NLP) technology. For example, from a document about an AI model explaining a paper, it understands that the model has the function of explaining a paper. It also analyzes AI models for automatic code generation and illustration generation in the same way.

[1189] Step 4:

[1190] The server analyzes the received sample data. Specifically, it identifies the input and output formats of each generative AI model. For example, it understands that the input example of a code auto-generation AI model is a program specification, and that the output example is code based on that specification.

[1191] Step 5:

[1192] The server generates an integrated prompt based on the analysis results. This prompt combines the functions of multiple generative AI models. For example, it creates a prompt that says, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[1193] Step 6:

[1194] The server uses these prompts to build a new generative AI model that combines the capabilities of paper commentary, code generation, and illustration generation.

[1195] Step 7:

[1196] The server then provides the newly constructed generative AI model to the user's device, allowing the user to seamlessly execute a series of tasks using this integrated model.

[1197] Step 8:

[1198] Users can then use the new generative AI models provided to them to perform specific tasks. For example, by inputting a paper, a summary of the paper is generated, code based on the summary is generated, and then illustrations related to the code are generated. In this way, users can efficiently use the functions of multiple generative AI models in a single system.

[1199] Through these steps, the system enables users to integrate the functions of multiple generative AI models, reducing the need for context sharing and effort.

[1200] Example 1

[1201] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1202] Currently, there is no system for effectively integrating multiple generative AI models. Users must select models individually, input data for each model separately, and obtain results. This process is time-consuming, labor-intensive, and inefficient. Furthermore, it is difficult to integrate data and functions between different generative AI models, making it difficult to obtain consistent results.

[1203] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1204] In this invention, the server includes means for providing a user interface for selecting multiple generative AI models, means for collecting information and sample data on the selected generative AI models, means for analyzing the collected documents and sample data and identifying functions of each generative AI model, means for generating integrated prompts based on the identified functions, means for providing the constructed generative AI model to the user, means for generating prompts that reflect the user's intentions and integrate the functions of the multiple generative AI models, and means for constructing a new generative AI model based on the generated prompts and seamlessly executing a series of tasks. This enables the user to efficiently integrate and use multiple generative AI models and obtain consistent results.

[1205] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates output such as text, images, or program code based on user input.

[1206] A "user interface" refers to a screen or input device that allows a user to interact with a system, and is the means by which a user performs the necessary operations and selections.

[1207] "Sample data" refers to a dataset that contains specific examples to demonstrate the format and content of inputs and outputs to a generative artificial intelligence model.

[1208] "Documentation" refers to documents that describe the functions, specifications, and usage of a generative artificial intelligence model, and includes information essential for understanding and using the model.

[1209] "Natural language processing technology" is a general term for technologies that enable computers to understand and manipulate text data, and includes technologies for analyzing documents and extracting meaning.

[1210] A "prompt" is text given to a generative artificial intelligence model in the form of an instruction or question, and is an input sentence that causes the model to perform a desired action.

[1211] This invention relates to a system that allows users to use multiple generative artificial intelligence models (generative AI models) in an integrated manner. This system mainly consists of a server, a terminal, and a user. The processes and roles of each are explained in detail below.

[1212] System Overview

[1213] The user selects a generative AI model through the system's user interface, after which the server and device work together to collect and analyze the necessary data, ultimately providing the user with an integrated generative AI model.

[1214] Hardware and Software

[1215] server

[1216] A server equipped with high-performance computers and natural language processing technology (e.g., using spaCy or NLTK as an NLP toolkit).

[1217] Terminal

[1218] A computer, tablet, or smartphone with an internet connection.

[1219] User Interface

[1220] The interface is provided using a web browser or a dedicated application.

[1221] Data processing and calculation

[1222] 1. User model selection

[1223] Through the interface, users can select the generative AI model they want to use, such as an AI model for paper commentary, automatic code generation, or illustration generation.

[1224] 2. Collecting model information

[1225] The device collects documentation and sample data about the model selected by the user and sends it to the server, including the model's features, specifications, and example input and output data.

[1226] 3. Analyzing Documents and Sample Data

[1227] The server analyzes the collected documents using natural language processing technology and extracts the characteristics and functions of each generative AI model. For example, from documents of an AI model that explains academic papers, it identifies the literature explanation function.

[1228] The server also analyzes sample data to understand the input and output formats of each model. For example, from an example input for a code auto-generation AI model, it identifies the program specification input and the corresponding code output.

[1229] 4. Prompt generation and building a new generative AI model

[1230] The server generates an integrated prompt based on the analysis results. This prompt reflects the user's intent and integrates the functions of multiple generative AI models. For example, it creates a prompt that asks, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[1231] Based on these prompts, we build a new generative AI model that integrates the functions of paper explanation, code generation, and illustration generation.

[1232] 5. Providing a Generative Model

[1233] The server provides the newly constructed generative AI model to the user, who can then use the model to seamlessly execute different tasks.

[1234] Specific examples

[1235] For example, consider a scenario where a user logs in to the Mix Box platform and performs the following steps: First, the user selects an AI model for explaining papers, then an AI model for automatically generating code, and finally an AI model for generating illustrations. The device then sends these documents and sample data to the server, which then analyzes each data and generates an integrated prompt.

[1236] An example prompt might be, "Generate a summary of a paper, generate Python code based on that summary, and generate illustrations related to that code." A new generative AI model is built based on this prompt, allowing users to efficiently and seamlessly utilize multiple AI functions.

[1237] In this way, the present invention provides a system that allows users to integrate the functions of multiple generative AI models and achieve their goals through a consistent process.

[1238] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1239] Step 1:

[1240] User model selection

[1241] Specific actions

[1242] Users access the system's user interface and select from multiple generative artificial intelligence models according to their goals and challenges.

[1243] input

[1244] A list of generative AI models based on user selections (clicks, etc.)

[1245] output

[1246] A list of selected generative AI models

[1247] Step 2:

[1248] Collecting Model Information

[1249] Specific actions

[1250] The device collects documentation and sample data for the generative AI model selected by the user, including API calls and retrieval from a database, and sends the collected data to a server.

[1251] input

[1252] A list of generative AI models selected by the user

[1253] output

[1254] Documentation and sample data for each model

[1255] Step 3:

[1256] Document Parsing

[1257] Specific actions

[1258] The server uses natural language processing (NLP) technology to analyze the documents of each generative AI model received from the device, thereby extracting the characteristics and functions of each AI model.

[1259] input

[1260] Documentation for each generative AI model (PDF, HTML, etc.)

[1261] output

[1262] A list of the characteristics and capabilities of each AI model

[1263] Step 4:

[1264] Analysis of sample data

[1265] Specific actions

[1266] The server analyzes the sample data received from each generative AI model to understand the specific input and output formats, thereby understanding how each model operates.

[1267] input

[1268] Sample data for each generative AI model (JSON, CSV, etc.)

[1269] output

[1270] Details of the input and output formats for each model

[1271] Step 5:

[1272] Prompt Generation

[1273] Specific actions

[1274] The server generates new prompts based on the characteristics and capabilities of each analyzed model, which are designed to reflect the user's intent and integrate the capabilities of multiple AI models.

[1275] input

[1276] Details of the characteristics and functions of each model, as well as input and output formats

[1277] output

[1278] Integrated prompts

[1279] Step 6:

[1280] Building a new generative AI model

[1281] Specific actions

[1282] The server then uses the generated prompts to build a new generative AI model, which is designed to integrate multiple functions and guide users through tasks in a consistent process.

[1283] input

[1284] Integrated prompts

[1285] output

[1286] New generative AI models

[1287] Step 7:

[1288] Providing generative AI models

[1289] Specific actions

[1290] The server provides the constructed generative AI model to the user, who can then use this model to seamlessly execute a series of tasks.

[1291] input

[1292] New generative AI models

[1293] output

[1294] Generative AI models provided by users

[1295] (Application example 1)

[1296] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1297] Conventional electronic payment services have difficulty proposing optimal payment methods, offering promotions, and providing coupons to users. Furthermore, functions such as budget management and fraud detection are provided separately, resulting in the inconvenience of users having to use multiple applications and services. Therefore, there is a need for a system that can provide these multiple functions in a unified manner, improving the user experience.

[1298] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1299] In this invention, the server includes means for providing a user interface for selecting multiple generative artificial intelligence models, means for collecting documents and sample data for the selected generative artificial intelligence models, means for analyzing the collected documents and sample data to identify functions of each generative artificial intelligence model, means for generating integrated prompts based on the identified functions, means for constructing a new generative artificial intelligence model using the generated prompts, means for suggesting an appropriate electronic payment method to the user, means for analyzing past purchase history and market trends to generate and suggest promotions and coupons, means for analyzing input income and expenditure data to provide a budget management plan, means for analyzing transaction data in real time to detect fraudulent transactions, and means for providing the constructed generative artificial intelligence model and various suggestions and notifications to the user. This allows users to receive comprehensive electronic payment support in a single system, improving convenience and safety.

[1300] A "user interface" is an interface that has an operation screen and input means for a user to select from multiple generative artificial intelligence models.

[1301] "Documentation" refers to the specifications, instructions, and technical documentation for the generative artificial intelligence model.

[1302] "Sample data" refers to a dataset that includes example inputs and outputs of a generative artificial intelligence model.

[1303] "Natural language processing technology" is a technology for analyzing text data and understanding human language, including morphological analysis and contextual understanding.

[1304] An "integrated prompt" is a new input instruction that integrates the functions of multiple generative artificial intelligence models and reflects the user's intentions.

[1305] An "electronic payment method" is a means by which a user pays for products online or in a brick-and-mortar store, and includes, for example, credit cards, QR code payments, digital wallets, etc.

[1306] "Promotion" refers to a discount or campaign to promote a particular product or service to users.

[1307] A "Coupon" is electronic or physical evidence of a discount under certain conditions.

[1308] A "budget management plan" is a plan for proper financial management that is created based on the user's income and expenditure data.

[1309] "Unauthorized Transactions" are transactions conducted in an unauthorized manner, including, for example, theft of credit cards and other fraudulent activities.

[1310] "Real-time" refers to a time frame in which data is processed immediately and results are reflected immediately.

[1311] "Generative AI model" is a general term for AI models that have the ability to generate an output corresponding to a specific input.

[1312] MODE FOR CARRYING OUT THE INVENTION

[1313] System Overview

[1314] The present invention is a system that allows users to integrate and use multiple generative AI models, particularly in electronic payment services. This system allows users to select models through a user interface (UI), analyze their documentation and sample data, understand the functions of each model, generate new prompts, and finally construct a new integrated generative AI model to provide to the user.

[1315] Hardware and software used

[1316] Frontend: React Native (Mobile App Development)

[1317] Backend: Node.js (server side), MongoDB (database)

[1318] Using AI models: Python, TensorFlow, GPT models

[1319] Processing flow

[1320] 1. User model selection and document collection:

[1321] The user selects multiple generative AI models (e.g., a payment method suggestion model, a coupon generation model, a budget management model, and a fraudulent transaction detection model) through the UI of a smartphone app.

[1322] Documentation and sample data for the selected model are sent to the server.

[1323] 2. Analyzing the documentation and sample data:

[1324] The server analyzes the received documents using natural language processing (NLP) techniques to identify the characteristics and functions of each generative AI model.

[1325] The server parses the sample data and understands the specific input and output formats for each model.

[1326] 3. Generating new prompts and building an integrated model:

[1327] The server generates new prompts based on the capabilities of each identified model, which are designed to reflect the user's intent and integrate the capabilities of multiple generative AI models.

[1328] Based on the generated prompts, a new generative AI model is built that can suggest appropriate electronic payment methods to users, generate and suggest promotions and coupons, provide budgeting plans, and detect fraudulent transactions.

[1329] 4. User Provision and Notification:

[1330] The server provides this newly constructed generative AI model and various suggestions and notifications to the user's smartphone app, allowing the user to seamlessly perform a series of tasks using this integrated model.

[1331] Specific examples

[1332] Example 1:

[1333] When users try to buy a drink, the app will recommend QR code payment and provide the latest promotional coupons. In addition, the app will suggest appropriate budget management plans and help users manage their overall spending based on their past shopping history.

[1334] Example 2:

[1335] When a user pays for a subscription service, the fraud detection model detects anomalous behavior and immediately alerts the user, allowing them to take immediate action and protect themselves from fraudulent transactions.

[1336] Prompt Sentence Examples

[1337] "Generate recommended coupons based on the user's last five purchases and current market promotions."

[1338] "Based on the income and expense data entered, please suggest the best monthly budget management plan for this user."

[1339] "Detect fraudulent transactions based on the latest transaction data and generate alerts."

[1340] As described above, the present invention enables users to receive comprehensive electronic payment support in one system, improving convenience and security.

[1341] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1342] Step 1:

[1343] The user selects multiple generative AI models through the UI of a smartphone app. Specifically, the user can select a payment method suggestion model, a coupon generation model, a budget management model, or a fraudulent transaction detection model. The input is the ID and name of the AI ​​model selected by the user, and the output is a list of the selected AI models.

[1344] Step 2:

[1345] The terminal collects the documents and sample data of the selected generative AI model and sends them to the server. The specific operation performed by the terminal is to call the API corresponding to the selected AI model to obtain the documents and sample data. The input here is the list of AI models selected in step 1, and the output is a set of documents and sample data for each model.

[1346] Step 3:

[1347] The server analyzes the received documents using natural language processing (NLP) techniques to identify the capabilities of each generative AI model. Specifically, the server performs morphological analysis and contextual understanding to extract important keywords and phrases from the documents. The input is the document obtained in step 2, and the output is a list of the characteristics and capabilities of each AI model.

[1348] Step 4:

[1349] The server analyzes the sample data and understands the specific input and output formats of each model. Specifically, the server analyzes the format of the sample data and identifies typical input data patterns and the corresponding output data formats. The input here is the sample data obtained in step 2, and the output is a list of input and output formats for each model.

[1350] Step 5:

[1351] The server generates a new prompt based on the capabilities of each identified model. This prompt is designed to reflect the user's intent and integrate the capabilities of multiple generative AI models. Specifically, the server optimizes the service by combining appropriate prompts to suit the user's purpose. The input is the list of capabilities and data formats obtained in Step 3 and Step 4, and the output is a new integrated prompt.

[1352] Step 6:

[1353] The server builds a new generative AI model based on the generated prompt. Specifically, the server calls the API of each AI model to send an integrated prompt, and then integrates the output to create a new model. The input is the prompt generated in step 5, and the output is a new integrated generative AI model.

[1354] Step 7:

[1355] The server provides this newly constructed generative AI model and various suggestions and notifications to the user's smartphone app. Specifically, the server provides the generated model as an endpoint and allows the user to access it. The user can use this integrated model to receive electronic payment method suggestions, obtain promotions and coupons, create budget management plans, and monitor fraudulent transactions. The input is the integrated generative AI model constructed in step 6, and the output is various services provided through the user's application interface.

[1356] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1357] Program processing overview

[1358] This invention provides a system that allows users to integrate multiple generative artificial intelligence models (generative AI models), and further combines it with an emotion engine that recognizes the user's emotions and adjusts output accordingly. This system allows users to select models through a user interface, analyzes their documents and sample data, understands the functions of each model, generates new prompts, and finally constructs a new integrated generative AI model to provide to the user.

[1359] Program Operation

[1360] 1. User model selection

[1361] Through the system's user interface, users can select from multiple generative AI models based on their specific tasks and goals. For example, they can choose from an AI model for explaining papers, an AI model for automatically generating code, or an AI model for generating illustrations.

[1362] 2. Collecting model information

[1363] The device collects documentation and sample data (example inputs and outputs) related to the generative AI model selected by the user and sends this to the server.

[1364] 3. Document Analysis

[1365] The server analyzes the received documents using natural language processing (NLP) technology. For example, from a document about an AI model explaining a paper, it understands that the model has the function of explaining a paper. It also analyzes AI models for automatic code generation and illustration generation in the same way.

[1366] 4. Analysis of sample data

[1367] The server analyzes the sample data and understands the specific input and output formats of each model. For example, it understands that the input example of a code auto-generation AI model is a program specification, and the output example is code based on that specification.

[1368] 5. Prompt Generation

[1369] The server generates an integrated prompt based on the analysis results. This prompt combines the functions of multiple generative AI models. For example, it creates a prompt that says, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[1370] 6. Use of Emotion Engines

[1371] The server uses an emotion engine to recognize the user's emotions. The recognized emotions are reflected in the output of the generative AI model. For example, if the user is feeling stressed, the server generates a gentle explanation in accordance with the user's emotions and an illustration to soothe the user's mood.

[1372] 7. Building a new generative AI model

[1373] The server then builds a new generative AI model based on the generated prompts and the user's emotional information. This new model integrates the functions of paper explanation, code generation, and illustration generation, and provides optimal output according to the user's emotions.

[1374] 8. Providing a Generative Model

[1375] The server then provides this newly constructed generative AI model to the user's device, where the user can use this integrated model to seamlessly execute a series of tasks.

[1376] Specific examples

[1377] For example, the following scenario is possible:

[1378] A user wants to read a paper, understand its content, generate related code based on the paper, and create an illustration to visually represent it. First, the user logs into the Mix Box platform and selects one of three models: a paper explanation AI model, an automatic code generation AI model, or an illustration generation AI model.

[1379] The device sends the documents and sample data of these models to the server, which analyzes them to understand the characteristics of each model. The server then creates an integrated prompt that asks, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code." The server then uses an emotion engine to analyze the user's emotions and makes adjustments based on those emotions to build a new generative AI model.

[1380] Users input a paper using the new generative AI model, which then generates a summary of the paper, along with code and related illustrations based on the summary, seamlessly. This consistent process is delivered in a way that takes into account the user's emotional state, allowing users to efficiently and comfortably leverage the capabilities of multiple generative AI models.

[1381] In this way, the present invention provides a system that allows users to use the functions of multiple generative AI models in an integrated manner, and by combining it with an emotion engine, the system reflects the user's emotional state in the output content, thereby realizing a concrete form that improves the user experience while reducing the need for context sharing and effort.

[1382] The processing flow will be explained below.

[1383] Step 1:

[1384] Users log in to the Mix Box platform and select multiple generative AI models they want to use. For example, they can choose an AI model for explaining papers, an AI model for automatically generating code, or an AI model for generating illustrations.

[1385] Step 2:

[1386] The device collects documentation and sample data (example inputs and outputs) for each generative AI model selected by the user and sends this to the server.

[1387] Step 3:

[1388] The server analyzes the received documents using natural language processing (NLP) technology. For example, from a document about an AI model explaining a paper, it understands that this model has the function of explaining a paper. Similarly, it analyzes AI models for automatic code generation and illustration generation.

[1389] Step 4:

[1390] The server analyzes the received sample data and understands the specific input and output formats of each model. For example, it understands that the input example of a code auto-generation AI model is a program specification, and that the output example is code based on that specification.

[1391] Step 5:

[1392] The server generates an integrated prompt based on the analysis results. This prompt combines the functions of multiple generative AI models. For example, it creates a prompt that says, "Generate a summary of a paper, generate code based on that summary, and generate illustrations related to that code."

[1393] Step 6:

[1394] The server uses an emotion engine to recognize the user's emotions, for example, by analyzing the tone and speed of the user's keyboard and voice input to identify their current emotional state (e.g., stress, happiness, fatigue).

[1395] Step 7:

[1396] The server adjusts the generated prompts based on the user's emotional information obtained from the emotion engine. For example, if the user is feeling stressed, the server changes the explanatory text to a gentler tone and adjusts the illustrations to have a more relaxing effect.

[1397] Step 8:

[1398] Based on the tailored prompts, the server builds a new generative AI model that integrates the functions of paper explanation, code generation, and illustration generation, while providing output that responds to the user's emotions.

[1399] Step 9:

[1400] The server then provides the newly constructed generative AI model to the user's device, allowing the user to seamlessly execute a series of tasks using this integrated model.

[1401] Step 10:

[1402] Users use the provided new generative AI model to perform specific tasks. For example, by inputting a paper, the system generates a summary of the paper, generates code based on the summary, and then generates an illustration related to the code. The output content is provided according to the user's emotional state, allowing users to work efficiently and comfortably.

[1403] In this way, this system allows users to integrate the functions of multiple generative AI models, and by combining it with an emotion engine, it reflects the user's emotional state in the output content, improving the user experience.

[1404] Example 2

[1405] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1406] Conventional generative AI models require users to use each individual model, and integrating the outputs of each model requires a lot of time and effort. Furthermore, the results output without considering the user's emotions often do not necessarily respond to the user's needs. This detracts from the overall user experience.

[1407] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1408] In this invention, the server includes means for providing a user interface for selecting multiple generative AI models, means for collecting documents and sample data for the selected generative AI models, means for analyzing the collected documents and sample data and identifying functions of each generative AI model, means for generating an integrated prompt based on the identified functions, means for constructing a new generative AI model based on the analysis results and user emotion information, means for recognizing the user's emotions using an emotion engine and reflecting them in the output of the generative AI model, and means for providing the constructed generative AI model to the user. This makes it possible to efficiently integrate multiple generative AI models and provide optimal output according to the user's emotions.

[1409] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates new information or data based on input data.

[1410] A "user interface" is an interface through which a user interacts with software or a system.

[1411] A "document" is a text or file that contains an explanation or specification of a generative artificial intelligence model.

[1412] "Sample data" refers to specific input and output examples used to demonstrate the functionality of a generative artificial intelligence model.

[1413] "Natural language processing technology" is a technology that enables computers to understand and generate human language.

[1414] A "prompt" is an input instruction or question to a generative artificial intelligence model.

[1415] An "emotion engine" is a technology that recognizes a user's emotions and adjusts responses and output based on the results.

[1416] An "integrated prompt" is a consistent instruction that combines the capabilities of multiple generative artificial intelligence models.

[1417] The "new generative artificial intelligence model" is a new AI model that integrates the functions of multiple existing generative artificial intelligence models to provide optimal output according to the user's emotions.

[1418] The present invention is a system that allows users to use multiple generative AI models in an integrated manner, and also combines an emotion engine that recognizes the user's emotions and adjusts the output accordingly. This system is implemented by combining the following processes.

[1419] First, the user selects the generative AI model they want to use through a user interface, such as a web browser or application on a computer or smart device. The user can choose from a paper explanation AI model, an automatic code generation AI model, an illustration generation AI model, and so on.

[1420] The device then collects documentation and sample data for the selected generative AI model, often in the form of text or JSON files, and transmits the collected data to a server via the HTTPS protocol.

[1421] The server analyzes the received documents using natural language processing (NLP) techniques. Specifically, it uses Python's NLTK library and SpaCy to extract the functions and features of each generative AI model. Sample data is also analyzed using data analysis libraries such as Pandas and NumPy to identify the input and output formats for each model.

[1422] Based on the analysis results, the server generates integrated prompts, which are generated using large-scale language models such as GPT-3. Specifically, the prompts include "generate a summary of the paper, generate code based on that summary, and generate illustrations related to that code."

[1423] Furthermore, the server uses an emotion engine to recognize the user's emotions. The emotion engine uses, for example, IBM Watson's Emotion Analysis. It determines the user's emotional state based on the user's voice and text input obtained from the user interface. The recognized emotion is reflected in the output of the generative AI model. For example, if the user is feeling stressed, a gentle tone of voice corresponding to that emotion is generated.

[1424] The server then builds a new generative AI model based on the generated prompts and the user's emotional information. This new generative AI model is built using machine learning frameworks such as TensorFlow and PyTorch. This makes it possible to integrate functions for paper explanations, code generation, and illustration generation, while providing optimal output based on the user's emotional state.

[1425] Finally, the server provides the constructed generative AI model to the user's device. This process also uses HTTPS to ensure secure communication. The user can then use this integrated model to seamlessly execute a series of tasks.

[1426] As a concrete example, the following scenario can be considered.

[1427] When a user types, "Generate a summary of this paper," the server analyzes the user's input and creates a prompt to generate the summary. Furthermore, if the user is feeling stressed, the server generates a prompt saying, "Generate a summary of this paper in a gentle tone." Then, based on the summarized content, it generates a prompt saying, "Generate appropriate code based on the summarized paper content." Finally, it generates a prompt saying, "Create an illustration related to the generated code." The system seamlessly executes this series of processes.

[1428] Through the above steps, the present invention enables users to use multiple generative artificial intelligence models in an integrated manner, and furthermore, by reflecting the user's emotions in the output content, provides an efficient and appropriate user experience.

[1429] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1430] Step 1: User model selection

[1431] Specific operation: The user selects the generative AI model they want to use through a user interface (e.g., a web browser or application). Available models include an AI model for explaining papers, an AI model for automatically generating code, and an AI model for generating illustrations.

[1432] Input: The model type selected by the user.

[1433] Output: The information of the selected model is saved in the internal database.

[1434] Step 2: Gathering model information

[1435] Specific operations: The terminal collects documents and sample data for the generative artificial intelligence model selected through the user interface, including document files containing descriptions and specifications of the model, as well as example inputs and outputs.

[1436] Input: The model type selected by the user.

[1437] Output: Collected documents and sample data are sent to the server.

[1438] Step 3: Parsing the document

[1439] How it works: The server analyzes the received document files using natural language processing (NLP) techniques to identify the capabilities of each generative AI model. Specifically, it analyzes the content of the documents using Python's NLTK library and SpaCy.

[1440] Input: Collected document files.

[1441] Data processing: Tokenize the text information in the document and extract important keywords and context.

[1442] Output: Data about the capabilities of each model is stored in an internal database.

[1443] Step 4: Analyzing the sample data

[1444] Specific operation: The server analyzes the received sample data and identifies the specific input and output formats for each model. This analysis is performed using data analysis libraries such as Pandas and NumPy.

[1445] Input: Collected sample data.

[1446] Data processing: Analyze sample data to extract input and output formats.

[1447] Output: Data about the input and output formats for each model is stored in an internal database.

[1448] Step 5: Generate prompts

[1449] What it does: The server generates a unified prompt based on the analysis results, using a large-scale language model such as GPT-3.

[1450] Input: Data about the capabilities, input formats, and output formats of each model.

[1451] Data calculation: Combine the functions of each model to generate the optimal prompt.

[1452] Output: The integrated prompt is generated.

[1453] Step 6: Use the Emotion Engine

[1454] Specific operation: The server recognizes the user's emotions using an emotion engine. The emotion engine uses IBM Watson's Emotion Analysis. It analyzes the user's voice and text input obtained from the user interface to identify the user's emotional state.

[1455] Input: User voice and text input.

[1456] Data calculation: The emotion engine analyzes the emotional state.

[1457] Output: Data about the user's emotional state.

[1458] Step 7: Building a new generative AI model

[1459] How it works: The server builds a new generative AI model based on the generated prompts and the user's emotional information, using machine learning frameworks such as TensorFlow and PyTorch.

[1460] Input: Synthesized prompt sentence and user emotional state data.

[1461] Data calculation: Build an optimal generative AI model from prompt sentences and emotional information.

[1462] Output: A new generative artificial intelligence model.

[1463] Step 8: Serving the generative model

[1464] Specific operation: The server provides the constructed generative AI model to the user's device using the HTTPS protocol to ensure secure communication.

[1465] Input: A new generative artificial intelligence model.

[1466] Output: The generative AI model provided to the user's device.

[1467] (Application example 2)

[1468] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1469] In systems that use conventional generative AI models, users have to spend a lot of time and effort integrating multiple models. Furthermore, outputs generated without considering the user's emotional state can degrade the quality of the user experience. This makes it difficult to achieve the desired results in certain applications.

[1470] The specification processing by specification processing unit 290 of data processing device 12 in application example 2 is realized by the following means. In this invention, the server includes means for providing a user interface for selecting multiple generative AI models, means for collecting documents and sample data for the selected generative AI models, means for analyzing the collected documents and sample data and identifying the functions of each generative AI model, means for generating an integrated prompt based on the identified functions, means for constructing a new generative AI model using the generated prompt, means for recognizing a user's emotion and adjusting the output content of the generative AI model based on the recognized emotion, and means for providing the constructed generative AI model to the user. This allows the user to use multiple generative AI models in an integrated manner and further enables output adjustment based on emotion, providing a higher quality user experience.

[1471] A "generative artificial intelligence model" is an artificial intelligence algorithm that has the ability to receive input for a specific task and generate a corresponding output.

[1472] "User interface" is a general term for the screens and operating means that allow users to access and operate systems and software.

[1473] "Documentation" refers to instructions and specifications regarding the functions and usage of generative artificial intelligence models.

[1474] "Sample data" is a set of example inputs and outputs that demonstrate how a generative artificial intelligence model works.

[1475] "Analysis" is the act of examining data or documents in detail to identify meanings and patterns.

[1476] An "integrated prompt" is a command that combines the functions of multiple generative artificial intelligence models.

[1477] "Emotion recognition" refers to analyzing a user's input and behavior to identify their emotional state.

[1478] "Adjusting output" refers to adjusting the information or results generated based on the perceived emotion.

[1479] A "new generative artificial intelligence model" is a new, improved model that integrates the functions and characteristics of multiple existing generative artificial intelligence models.

[1480] "Provision" refers to the act of making a particular service or feature available to users.

[1481] A specific embodiment of the present invention will be described. The present invention is a system that can use multiple generative artificial intelligence models in an integrated manner, and further recognizes the user's emotions and adjusts the output accordingly.

[1482] First, a user interface is provided that allows users to select from multiple generative AI models. Through this interface, users can select the model they need based on their purpose, such as a product recommendation model or a review generation model.

[1483] Next, documents and sample data related to the selected generative AI models are collected and sent to a server, which analyzes the collected documents and sample data using natural language processing techniques (e.g., TensorFlow, PyTorch) to identify the capabilities of each generative AI model.

[1484] The server then generates a unified prompt based on the capabilities of each identified model. This prompt seamlessly combines the capabilities of multiple generative AI models, such as the following prompt:

[1485] "User is feeling stressed. Please recommend some relaxation items."

[1486] Furthermore, the server uses an emotion analysis model (e.g., EmotionAnalyzer) to recognize the user's emotions. Based on the recognized emotions, the output of the generative AI model can be adjusted. For example, if the user is feeling stressed, a gentle tone of voice or relaxation-related products will be recommended to match that emotion.

[1487] Finally, these methods are combined to build a new generative AI model, which is then provided to users. Users can access this integrated model through devices such as smartphones or head-mounted displays (HMDs) to seamlessly perform a series of tasks. This system allows users to receive high-quality services that respond to their emotional state.

[1488] As a concrete example, consider the case where a user is searching for "relaxing products" in a virtual store and enters "I've been very busy and stressed lately." Based on this input, the server generates the following prompt:

[1489] "User is feeling stressed. Please recommend some relaxation items."

[1490] The product recommendation model then uses these prompts to suggest appropriate relaxation items to the user, enabling the user to efficiently and comfortably utilize the capabilities of multiple generative AI models.

[1491] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1492] Step 1:

[1493] User model selection

[1494] The user selects from multiple generative AI models they wish to use through the system's user interface. This interface allows the user to select a specific generative AI model for a purpose, such as automatic product recommendation or review generation. The input is the user's model selection, and the output is a list of the selected models.

[1495] Step 2:

[1496] Collecting Model Information

[1497] The terminal collects documents and sample data related to the generative artificial intelligence models selected by the user, where the input is a list of selected models and the output is the documents and sample data corresponding to those models, and this data is sent to the server.

[1498] Step 3:

[1499] Document Parsing

[1500] The server analyzes the collected documents using natural language processing techniques, such as TensorFlow or PyTorch, to understand the content of the documents and identify the functions of each generative artificial intelligence model. The input here is the collected documents, and the output is a list of the model's functions.

[1501] Step 4:

[1502] Analysis of sample data

[1503] The server analyzes the collected sample data and understands the specific input and output formats of each model, which allows it to determine what data is required for a specific task. The input is the sample data, and the output is the specific information for the input and output formats of each model.

[1504] Step 5:

[1505] Prompt Generation

[1506] The server generates an integrated prompt based on the analysis results. For example, this prompt might be in the form of "The user is feeling stressed. Please recommend some relaxation items." Here, the functions of each model are combined to create a prompt that meets the user's needs. The input is the analysis results (model functions and input / output format), and the output is the integrated prompt.

[1507] Step 6:

[1508] Use of emotion engine

[1509] The server uses an emotion engine to recognize the user's emotions. The recognized emotions are reflected in the output of the generative AI model. For example, EmotionAnalyzer analyzes the user's input, "I've been very busy and stressed lately," and identifies the user's emotional state as stress. The input is the user's text input, and the output is the identified emotional state.

[1510] Step 7:

[1511] Building a new generative artificial intelligence model

[1512] The server constructs a new generative AI model based on the generated prompt and the user's emotional information. This new model integrates the necessary functions to respond to the prompt and can provide optimal output based on the user's emotional state. The input is the prompt and emotional information, and the output is the newly constructed generative AI model.

[1513] Step 8:

[1514] Providing a generative model

[1515] The server provides this newly constructed generative AI model to the user's device. The user can use this integrated model through a device such as a smartphone or head-mounted display (HMD) to seamlessly perform a series of tasks. The input is the new generative AI model, and the output is the services and functions that the user receives.

[1516] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1517] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1518] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1519] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1520] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1521] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1522] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1523] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1524] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1525] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1526] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1527] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1528] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1529] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1530] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1531] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1532] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1533] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1534] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1535] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1536] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1537] The following is further disclosed regarding the above embodiment.

[1538] (Claim 1)

[1539] means for providing a user interface for selecting a plurality of generative artificial intelligence models;

[1540] means for collecting documentation and sample data for the selected generative artificial intelligence model;

[1541] A means of analyzing the collected documents and sample data to identify the capabilities of each generative artificial intelligence model; and

[1542] a means for generating a unified prompt based on the identified features;

[1543] A means for constructing a new generative artificial intelligence model using the generated prompts;

[1544] A means for providing the constructed generative artificial intelligence model to a user;

[1545] A system including:

[1546] (Claim 2)

[1547] The system according to claim 1, further comprising means for analyzing the document of the generative artificial intelligence model using natural language processing techniques.

[1548] (Claim 3)

[1549] The system of claim 1, further comprising means for analyzing sample data of the generative artificial intelligence model to identify input and output formats.

[1550] (Claim 4)

[1551] 10. The system of claim 1, further comprising means for generating a prompt based on characteristics of a user-selected generative artificial intelligence model and using the prompt to automatically construct a new generative artificial intelligence model.

[1552] (Claim 5)

[1553] 2. The system according to claim 1, further comprising means for providing the constructed new generative artificial intelligence model to a user's terminal.

[1554] "Example 1"

[1555] (Claim 1)

[1556] means for providing a user interface for selecting a plurality of generative artificial intelligence models;

[1557] A means for collecting information and sample data of the selected generative artificial intelligence model;

[1558] A means of analyzing the collected documents and sample data to identify the capabilities of each generative artificial intelligence model; and

[1559] a means for generating a unified prompt based on the identified features;

[1560] A means for constructing a new generative artificial intelligence model using the generated prompts;

[1561] A means for providing the constructed generative artificial intelligence model to a user;

[1562] means for generating prompts that reflect user intent and integrate the capabilities of multiple generative artificial intelligence models;

[1563] A means to build a new generative artificial intelligence model based on the generated prompts and execute a series of tasks seamlessly;

[1564] A system including:

[1565] (Claim 2)

[1566] The system according to claim 1, further comprising means for analyzing the document of the generative artificial intelligence model using natural language processing techniques.

[1567] (Claim 3)

[1568] The system of claim 1, further comprising means for analyzing sample data of the generative artificial intelligence model to identify input and output formats.

[1569] "Application Example 1"

[1570] (Claim 1)

[1571] means for providing a user interface for selecting a plurality of generative artificial intelligence models;

[1572] means for collecting documentation and sample data for the selected generative artificial intelligence model;

[1573] A means of analyzing the collected documents and sample data to identify the capabilities of each generative artificial intelligence model; and

[1574] a means for generating a unified prompt based on the identified features;

[1575] A means for constructing a new generative artificial intelligence model using the generated prompts;

[1576] means for suggesting an appropriate electronic payment method to the user;

[1577] A means of analyzing past purchase history and market trends to generate and suggest promotions and coupons;

[1578] means for analyzing input income and expenditure data and providing a budget management plan;

[1579] A means of analyzing transaction data in real time to detect fraudulent transactions;

[1580] A means for providing the constructed generative artificial intelligence model and various suggestions and notifications to users;

[1581] A system including:

[1582] (Claim 2)

[1583] The system according to claim 1, further comprising means for analyzing the document of the generative artificial intelligence model using natural language processing techniques.

[1584] (Claim 3)

[1585] The system of claim 1, further comprising means for analyzing sample data of the generative artificial intelligence model to identify input and output formats.

[1586] "Example 2: Combining Emotion Engines"

[1587] (Claim 1)

[1588] means for providing a user interface for selecting a plurality of generative artificial intelligence models;

[1589] means for collecting documentation and sample data for the selected generative artificial intelligence model;

[1590] A means of analyzing the collected documents and sample data to identify the capabilities of each generative artificial intelligence model; and

[1591] a means for generating a unified prompt based on the identified features;

[1592] A means for constructing a new generative artificial intelligence model based on the analysis results and user emotional information;

[1593] A means for recognizing a user's emotions using an emotion engine and reflecting the emotions in the output of a generative artificial intelligence model;

[1594] A means for providing the constructed generative artificial intelligence model to a user;

[1595] A system including:

[1596] (Claim 2)

[1597] The system according to claim 1, further comprising means for analyzing documents of said generative artificial intelligence model using natural language processing techniques.

[1598] (Claim 3)

[1599] The system of claim 1, further comprising means for analyzing sample data of the generative artificial intelligence model to identify input and output formats.

[1600] "Application example 2 when combining emotion engines"

[1601] (Claim 1)

[1602] means for providing a user interface for selecting a plurality of generative artificial intelligence models;

[1603] means for collecting documentation and sample data for the selected generative artificial intelligence model;

[1604] A means of analyzing the collected documents and sample data to identify the capabilities of each generative artificial intelligence model; and

[1605] a means for generating a unified prompt based on the identified features;

[1606] A means for constructing a new generative artificial intelligence model using the generated prompts;

[1607] means for recognizing a user's emotion and adjusting the output content of the generative artificial intelligence model based on the recognized emotion;

[1608] A means for providing the constructed generative artificial intelligence model to a user;

[1609] A system including:

[1610] (Claim 2)

[1611] The system according to claim 1, further comprising means for analyzing the document of the generative artificial intelligence model using natural language processing techniques.

[1612] (Claim 3)

[1613] The system of claim 1, further comprising means for analyzing sample data of the generative artificial intelligence model to identify input and output formats. [Explanation of symbols]

[1614] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for providing a user interface for selecting a plurality of generative artificial intelligence models; means for collecting documentation and sample data for the selected generative artificial intelligence model; A means of analyzing the collected documents and sample data to identify the capabilities of each generative artificial intelligence model; and a means for generating a unified prompt based on the identified features; A means for constructing a new generative artificial intelligence model using the generated prompts; A means for providing the constructed generative artificial intelligence model to a user; A system including:

2. The system according to claim 1 , further comprising means for analyzing the document of the generative artificial intelligence model using natural language processing techniques.

3. The system of claim 1 , further comprising means for analyzing sample data of the generative artificial intelligence model to identify input and output formats.

4. 2. The system of claim 1, further comprising means for generating prompts based on characteristics of a user-selected generative artificial intelligence model and for automatically constructing a new generative artificial intelligence model using the prompts.

5. 2. The system according to claim 1, further comprising means for providing the constructed new generative artificial intelligence model to a user terminal.

Images