system

Abstract

We provide the system. [Solution] A means of generating a virtual idle using a generation engine, A means for performing initial settings on the aforementioned virtual idol based on parameters provided by the user, The means by which the virtual idol conducts live streaming and interacts with fans, The aforementioned virtual idol is registered as a digital asset, and a means is provided for exchanging or buying and selling it with other users. A system that includes this.

Description

Technical Field

[0004] , , , ,

[0005] , , , , , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional idol industry, there are many management problems such as idol scandals, slander, and retirement due to age. In addition, after the retirement of a real idol, costs and labor for cultivating an idol again occur, making it difficult to build a sustainable idol market. Furthermore, for fans, the sense of loss due to the retirement of their favorite idol is also a major problem. Means for solving these problems and providing a more sustainable and new form of idol market are required.

Means for Solving the Problems

[0005] This invention provides a system that generates virtual idols using a generation engine and performs initial setup based on parameters provided by the user. The generated virtual idols can perform live streaming and interact with fans, and can also be registered as digital assets that can be exchanged or bought and sold with other users. This system allows users to cultivate and improve the skills of virtual idols and to interact with fans in real time using a natural language processing engine. This solves the management problems of real idols and enables the creation of a sustainable idol market.

[0006] A "generation engine" is an algorithm and software that automatically generates virtual idles based on parameters entered by the user.

[0007] A "virtual idol" is a virtual character created by a generation engine, an avatar that is nurtured and produced by the user.

[0008] "User" refers to an individual or legal entity that creates, develops, and produces virtual idols.

[0009] "Parameters" refer to the information that users provide when creating a virtual idol (e.g., gender, age, voice, character design, etc.).

[0010] "Initial setup" is the process of assigning basic profiles and attributes to the generated virtual idle based on parameters provided by the user.

[0011] "Live streaming" refers to virtual idols performing and interacting with others in real time on the internet.

[0012] "Digital assets" refer to digital content such as virtual idols that are registered using blockchain technology and can be bought, sold, and exchanged.

[0013] A "database" is a system for storing and managing information about generated virtual idols and data from live streams.

[0014] A "natural language processing engine" is an algorithm that analyzes and processes text data to enable dialogue in natural language.

[0015] A "fan" refers to a user who follows and supports the activities of a virtual idol, and participates in conversations and live streams with them.

[0016] "Real-time" refers to instantaneous responses or processing with virtually no delay. [Brief explanation of the drawing]

[0017] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of the data processing device and smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Modes for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] System Overview

[0039] This invention is a system that generates virtual idols (AItubers) using a generation engine, performs initial setup based on parameters provided by the user, and enables live streaming and interaction with fans. Furthermore, it allows the virtual idols to be registered as digital assets, enabling exchange and trading with other users.

[0040] Explanation of the program's processing

[0041] 1. Creating a virtual idol

[0042] user:

[0043] Users enter the necessary information into a virtual idol creation form using a dedicated application or web interface. This includes parameters such as the idol's gender, age, voice, and character design.

[0044] Terminal:

[0045] The terminal sends the parameters entered along with the generation request to the server.

[0046] server:

[0047] The server receives the request and starts the generation engine. Based on the provided parameters, the generation engine generates character data for a virtual idol and saves it to the database. A notification of completion is sent to the terminal, which then displays this to the user.

[0048] Specific example:

[0049] A user expresses a desire to create a virtual idol character who is a young woman with a cute, high-pitched voice. They enter the information into the creation form and press the "Generate" button. The server receives the information sent by the device, and the generation engine creates the character and saves it to the database. Finally, a completion notification is displayed to the user.

[0050] 2. Training and communication with virtual idols

[0051] user:

[0052] Users select training items and skill-building content from the application's training menu and submit requests. They can also interact with virtual idols using the communication menu.

[0053] Terminal:

[0054] The device sends nurturing and communication requests to the server.

[0055] server:

[0056] The server receives the request and updates the virtual idle parameters. The update results are saved to the database and sent to the terminal.

[0057] Specific example:

[0058] A user sends a request to a virtual idol asking for "singing training." The device sends the request to the server, which updates the virtual idol's skill parameters and displays the results to the user.

[0059] 3. Live streaming and interaction with fans

[0060] user:

[0061] Users schedule live streams and submit requests through the application. During the live stream, they can watch as the virtual idol reacts to comments from fans.

[0062] Terminal:

[0063] The device sends schedule information and commands to start and end the live stream to the server. During the live stream, comments from fans are displayed in real time.

[0064] server:

[0065] The server stores the live stream schedule in a database and prepares the appropriate streaming server. During the live stream, comments are processed in real time, and a natural language processing engine is used to ensure that virtual idlers respond. After the live stream ends, the live data is saved.

[0066] Specific example:

[0067] A user sets a schedule, such as "A virtual idol will perform a live show every Friday night at 8 PM." During the live show, comments from fans are displayed in real time, and the virtual idol responds. After the live show ends, the server saves the data.

[0068] 4. Converting virtual idols into NFTs and exchanging them

[0069] user:

[0070] The user enters the settings for creating an NFT of a virtual idle object in the application and submits the request.

[0071] Terminal:

[0072] The terminal sends a request to the server.

[0073] server:

[0074] The server retrieves character data, generates a smart contract to register it on the NFT marketplace, and registers the data on the blockchain. It receives exchange and purchase requests from other users and manages transactions. Successful transactions are recorded in the database, and the results are sent to the terminal.

[0075] Specific example:

[0076] The user enters information to convert their virtual idol into an NFT and submits a request. The server retrieves the character data, generates a smart contract to create the NFT, and registers the data on the blockchain. The transaction is successful, and the result is displayed to the user.

[0077] Through this detailed embodiment, users can efficiently perform a series of operations such as creating, training, live streaming, and converting virtual idols into NFTs.

[0078] The following describes the processing flow.

[0079] Virtual Idol Creation

[0080] Step 1:

[0081] The user opens a dedicated application or web interface and enters the necessary information (gender, age, voice, character design, etc.) into the virtual idol creation form.

[0082] Step 2:

[0083] The user presses the "Generate" button.

[0084] Step 3:

[0085] The terminal sends the parameters entered along with the generation request to the server.

[0086] Step 4:

[0087] The server receives the request and starts the generation engine.

[0088] Step 5:

[0089] The server passes user-provided parameters to the generation engine to generate character data for the virtual idol.

[0090] Step 6:

[0091] The server saves the generated character data to the database.

[0092] Step 7:

[0093] The server sends a notification to the device when generation is complete.

[0094] Step 8:

[0095] The device displays a message or preview to the user indicating that generation is complete.

[0096] Training and communication with virtual idols

[0097] Step 1:

[0098] Users select training items and skill development content from a training menu via the application and submit requests.

[0099] Step 2:

[0100] Users can use the communication menu to interact with virtual idols.

[0101] Step 3:

[0102] The terminal sends a request to the server.

[0103] Step 4:

[0104] The server receives the request and updates the virtual idle's parameters (e.g., skill level).

[0105] Step 5:

[0106] The server saves the updated status to the database.

[0107] Step 6:

[0108] The server sends the update results to the terminal.

[0109] Step 7:

[0110] The device displays the update results to the user.

[0111] Live streaming and interaction with fans

[0112] Step 1:

[0113] Users schedule live streams and submit requests through the application.

[0114] Step 2:

[0115] The terminal sends schedule information to the server.

[0116] Step 3:

[0117] The server saves the schedule to the database and prepares the appropriate delivery server.

[0118] Step 4:

[0119] The user sends the command to start the live stream.

[0120] Step 5:

[0121] The terminal sends a command to the server, and the live stream begins.

[0122] Step 6:

[0123] The server receives comments from fans in real time during the live stream, and a natural language processing engine is used to enable the virtual idol to respond appropriately.

[0124] Step 7:

[0125] The server saves the live data after the live stream ends.

[0126] Step 8:

[0127] The user sends a command to end the live session.

[0128] Step 9:

[0129] The terminal sends a command to the server, and the live session ends.

[0130] NFT conversion and exchange of virtual idols

[0131] Step 1:

[0132] The user enters the settings for creating an NFT of a virtual idle object in the application and submits the request.

[0133] Step 2:

[0134] The terminal sends a request to the server.

[0135] Step 3:

[0136] The server retrieves the character data and generates a smart contract to register it on the NFT marketplace.

[0137] Step 4:

[0138] The server registers the NFT data on the blockchain.

[0139] Step 5:

[0140] The server receives exchange and purchase requests from other users and manages the transactions.

[0141] Step 6:

[0142] The server records successful transactions in the database.

[0143] Step 7:

[0144] The terminal displays the transaction results to the user after the transaction is completed.

[0145] (Example 1)

[0146] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0147] Traditional virtual idol systems lacked an integrated platform that allowed users to smoothly perform a series of operations, such as creating, raising, live streaming, and converting characters into NFTs. Furthermore, they did not adequately provide features for generating real-time responses based on user input or managing character data as digital assets. As a result, users had to manage their virtual idols individually, leading to decreased operational efficiency and a poor user experience.

[0148] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0149] In this invention, the server includes means for the user to input information into a virtual idol generation form using a dedicated application or web interface, means for the terminal to send the input information to the server along with the generation request, and means for the server to start a generation engine, generate character data for the virtual idol based on the provided information, and store it in a database. This allows the user to perform a series of operations, from virtual idol generation and training to live streaming and NFT creation, in a unified manner.

[0150] A "user" refers to an entity that uses the virtual idol system to perform operations such as creating, training, live streaming, and converting virtual idols into NFTs.

[0151] A "terminal" refers to a device used by a user, such as a computer, smartphone, or tablet, that communicates with the virtual idle system.

[0152] A "server" is the central computer system of a virtual idle system, and it is a device that processes user requests and generates, stores, and manages data.

[0153] A "virtual idol" refers to a digitized character created using a generation engine, possessing character design, personality, voice, and other characteristics based on parameters provided by the user.

[0154] A "generation engine" is software or an algorithm used to generate character data for a virtual idol based on user input.

[0155] A "database" is an information management system used to store character data, live streaming data, NFT transaction information, and other data related to the virtual idols that have been created.

[0156] A "natural language processing engine" is software or an algorithm that analyzes text data from users and fans and generates appropriate responses.

[0157] "NFT" stands for "Non-Fungible Token," and refers to a unique digital certificate that registers a virtual idol as a digital asset on the blockchain.

[0158] A "smart contract" is a program that automatically executes transactions on the blockchain and is used for NFT creation of virtual idols and transaction management.

[0159] "Live streaming" refers to virtual idols broadcasting video and audio to viewers in real time via the internet.

[0160] "Training items" refer to specific activities and learning content aimed at improving and developing the skills of virtual idols.

[0161] System Overview

[0162] This invention is a system that generates virtual idols (AItubers) using a generation engine. Based on parameters provided by the user, it performs initial setup, enables live streaming, interaction with fans, training and skill improvement, and can even be converted into NFTs, allowing for exchange and trading with other users as digital assets.

[0163] Virtual Idol Creation

[0164] user:

[0165] Users enter information into a virtual idol creation form using a dedicated application or web interface. Specifically, they enter parameters such as gender, age, voice, and character design.

[0166] Terminal:

[0167] The terminal serializes the input information into JSON format and sends it to the server as a generation request.

[0168] server:

[0169] After the server receives the request, it starts the "AICharacterG engine," which is the generation engine, and generates character data based on the provided parameters. The generated character data is stored in a MySQL® database. A notification of completion of generation is sent to the terminal and is eventually displayed to the user.

[0170] Specific example:

[0171] A user requests to create a virtual idol character who is a young woman with a cute, high-pitched voice. They enter the information and press the "Generate" button. The device sends the request, the server generates the character, and saves it to the database. A notification of completion is displayed to the user.

[0172] Prompt message:

[0173] "We're creating a virtual idol character who is a young woman with a cute, high-pitched voice."

[0174] Training virtual idols

[0175] user:

[0176] Users select training items and skill development content from the application's training menu and submit a request.

[0177] Terminal:

[0178] The device sends a training request to the server.

[0179] server:

[0180] The server receives the request and updates the virtual idle parameters. The update results are saved in the database and notified to the user.

[0181] Specific example:

[0182] A user sends a request to a virtual idol asking for "singing training." The device sends the request to the server, the server updates the virtual idol's skill parameters, and displays the results to the user.

[0183] Prompt message:

[0184] "We have started training to improve the singing skills of our virtual idols."

[0185] Live streaming and interaction with fans

[0186] user:

[0187] Users schedule live streams and submit requests through the application. During the live stream, they can watch as the virtual idol reacts to comments from fans.

[0188] Terminal:

[0189] The device sends schedule information and commands to start and end the live stream to the server. During the live stream, comments from fans are displayed in real time.

[0190] server:

[0191] The server saves the live streaming schedule to a database and prepares the appropriate streaming server. Comments during the live stream are processed in real time by a natural language processing engine, and a virtual idol responds. After the live stream ends, the live data is saved to the database.

[0192] Specific example:

[0193] A user sets a schedule, such as "A virtual idol will perform a live show every Friday night at 8 PM." During the live show, comments from fans are displayed in real time, and the virtual idol responds. After the live show ends, the server saves the data.

[0194] Prompt message:

[0195] "We will be holding a live stream of a virtual idol every Friday night at 8 PM."

[0196] NFT conversion and exchange of virtual idols

[0197] user:

[0198] The user enters the NFT settings for virtual idles in the application and sends the request.

[0199] Terminal:

[0200] The terminal sends a request to the server.

[0201] server:

[0202] The server retrieves character data, generates smart contracts, registers them on the NFT marketplace, and manages transactions. It receives exchange and purchase requests from other users, records successful transactions in a database, and notifies the user of the results.

[0203] Specific example:

[0204] The user enters information to convert their virtual idol into an NFT and submits a request. The server retrieves the character data, generates a smart contract to create the NFT, and registers the data on the blockchain. The transaction is successful, and the result is displayed to the user.

[0205] Prompt message:

[0206] "Turn virtual idols into NFTs and register them on a marketplace."

[0207] Through this detailed embodiment, users can efficiently perform a series of operations such as creating, training, live streaming, and converting virtual idols into NFTs.

[0208] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0209] Virtual Idol Creation

[0210] Step 1:

[0211] The user enters information.

[0212] Users open a dedicated application or web interface and enter information into a form for creating a virtual idol. This input data includes gender, age, voice, character design, and more.

[0213] input:

[0214] Parameters such as gender, age, voice, and character design

[0215] output:

[0216] Form data of the entered information

[0217] Specific actions:

[0218] The user opens the web interface in their browser and enters information into dropdown menus or text boxes.

[0219] Step 2:

[0220] The device sends the request

[0221] The terminal serializes the information entered by the user into JSON format and sends it to the server as a generation request.

[0222] input:

[0223] Form data entered by the user

[0224] output:

[0225] HTTP POST request to the server (data in JSON format)

[0226] Specific actions:

[0227] The terminal serializes the request data and sends an HTTP POST request to the specified endpoint on the server.

[0228] Step 3:

[0229] The server starts the generation engine.

[0230] After the server receives the request, it starts the generation engine (AICharacterG) and generates character data based on the provided parameters. The generated character data is saved in a temporary folder.

[0231] input:

[0232] The server receives a generation request (data in JSON format).

[0233] output:

[0234] Generated character data (file format)

[0235] Specific actions:

[0236] The server starts the generation engine, processes the character data based on the provided parameters, and saves it to a temporary folder.

[0237] Step 4:

[0238] The server saves the generated results and sends notifications.

[0239] The server saves the generated character data to a MySQL database and notifies the terminal when the generation is complete.

[0240] input:

[0241] Generated character data (file format)

[0242] output:

[0243] Character data stored in the database, generation completion notification (HTTP response)

[0244] Specific actions:

[0245] The server saves the character data to the database and sends an HTTP response to the terminal to notify it that generation is complete.

[0246] ---

[0247] Training virtual idols

[0248] Step 1:

[0249] The user selects a training menu.

[0250] Users select training items and skill development content from the application's training menu and submit a request.

[0251] input:

[0252] Training items and skill development content

[0253] output:

[0254] Training request (form data)

[0255] Specific actions:

[0256] The user selects "Singing Training" on the application screen and presses the "Submit" button.

[0257] Step 2:

[0258] The device sends the request

[0259] The device sends the user's selected training request to the server in JSON format.

[0260] input:

[0261] User-selected training request (form data)

[0262] output:

[0263] HTTP POST request to the server (data in JSON format)

[0264] Specific actions:

[0265] The terminal serializes the training request data and sends an HTTP POST request to the server.

[0266] Step 3:

[0267] The server updates the parameters.

[0268] The server receives the request data and updates the character data parameters according to the training details of the corresponding virtual idol. The update results are saved in the database and sent to the terminal as a notification.

[0269] input:

[0270] The server receives a training request (in JSON format).

[0271] output:

[0272] Updated character data stored in the database, update completion notification (HTTP response)

[0273] Specific actions:

[0274] The server updates the character data in the database and saves the new parameters. It then sends an update completion notification to the terminal as an HTTP response.

[0275] ---

[0276] Live streaming and interaction with fans

[0277] Step 1:

[0278] Users set up live streams

[0279] The user sets the live stream schedule in the application and sends the request to the server.

[0280] input:

[0281] Live streaming schedule information (date and time, content)

[0282] Output:

[0283] Live streaming request (form data)

[0284] Specific operations:

[0285] The user sets "Live stream every Friday night at 8:00 PM" in the application and presses the "Reserve" button.

[0286] <00009​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​The server saves the live streaming schedule in the database and prepares the delivery server for live streaming. During the live stream, the natural language processing engine is used to process comments in real time, and the virtual idol responds. After the live stream ends, the live data is saved in the database.

[0298] Input:

[0299] Live schedule information (JSON-formatted data) received by the server

[0300] Output:

[0301] Live schedule information saved in the database, comment data processed in real time, saved live data

[0302] Specific operations:

[0303] The server saves the schedule in the database and configures the delivery server. During the live stream, the comment data is analyzed by the natural language processing engine to generate a response in real time. After the live stream ends, all live data is saved in the database.

[0304] ---

[0305] NFT conversion and exchange of virtual idols

[0306] Step 1:

[0307] The user inputs the NFT conversion settings

[0308] The user uses the application to input the information required for NFT conversion of the virtual idol and send a request.

[0309] Input:

[0310] Required information for NFT conversion (character data, price, etc.)

[0311] Output:

[0312] NFT conversion request (form data)

[0313] Specific actions:

[0314] The user clicks the "Convert to NFT" button, enters the required information, and clicks the "Submit" button.

[0315] Step 2:

[0316] The device sends the request

[0317] The device sends the NFT conversion request entered by the user to the server in JSON format.

[0318] input:

[0319] NFT conversion request (form data) entered by the user

[0320] output:

[0321] HTTP POST request to the server (data in JSON format)

[0322] Specific actions:

[0323] The device serializes the NFT data and sends an HTTP POST request to the server.

[0324] Step 3:

[0325] The server registers NFTs and manages transactions.

[0326] The server receives the request data, retrieves the virtual idol's character data, generates a smart contract, and converts it into an NFT. The generated NFT is registered on the blockchain, where exchange and purchase requests from other users are managed. Successful transactions are recorded in the database, and notifications are sent to the device.

[0327] input:

[0328] NFT conversion request (JSON format data) received by the server

[0329] output:

[0330] Generated smart contract, NFT registered on the blockchain, and transaction success notification (HTTP response).

[0331] Specific actions:

[0332] The server retrieves character data, generates a smart contract, and converts it into an NFT. The data is registered on the blockchain, and if the transaction is successful, the transaction information is recorded in the database. Finally, the transaction result is notified to the terminal.

[0333] (Application Example 1)

[0334] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0335] To enhance the user experience, there is a need for a system that integrates many different functions, such as the creation and training of virtual idols, live streaming, and their management as digital assets. However, there is a lack of technical means to smoothly and efficiently implement these functions. In particular, a technical foundation is required to provide the advanced experiences that users expect, such as real-time responses during live streaming and blockchain-based management of virtual idols as digital assets.

[0336] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0337] In this invention, the server includes means for generating virtual idols using a generation engine, means for performing initial settings on the virtual idols based on parameters provided by the user, means for the virtual idols to perform live streaming and interact with fans, means for registering the virtual idols as digital assets and enabling them to be exchanged or bought and sold with other users, means for registering the digital assets of the virtual idols on the blockchain, and means for setting the live streaming schedule of the virtual idols and providing real-time responses. This makes it possible for users to easily generate and nurture virtual idols, enjoy real-time interaction with fans during live streaming, and manage and trade them as digital assets.

[0338] A "generation engine" is a software or hardware component that automatically generates virtual idles based on parameters provided by the user.

[0339] A "virtual idol" is a character created using digital technology, a digital entity capable of live streaming and interacting with fans.

[0340] "User-provided parameters" refer to the set of information that users input to the generation engine, such as the gender, age, voice, and character design of a virtual idol.

[0341] "Initial setup" refers to the process of configuring the basic characteristics and functions of a generated virtual idol based on parameters provided by the user.

[0342] "Live streaming" is a feature that allows virtual idols to deliver content in real time and engage in interactive communication with viewers.

[0343] "Fan interaction" refers to a feature that allows virtual idols to interact with fans in real time or remotely through live streaming or other communication methods.

[0344] "Digital assets" refer to digital content, such as virtual idols, that can be treated as objects of ownership and trading.

[0345] Blockchain is a distributed ledger technology used to manage and trade digital currencies and digital assets.

[0346] "Real-time response" is a feature that allows virtual idols to instantly respond to comments and questions from fans during live streams.

[0347] This invention provides a system for users to create and nurture virtual idols and interact with fans through live streaming. The mechanism is described in detail below.

[0348] 1. System Configuration

[0349] Hardware and software:

[0350] Smartphone: A device used by users to control a virtual idle.

[0351] Mobile operating systems (iOS / ANDROID®): These are operating systems that run on smartphones.

[0352] Backend Server (Node.js, Express): This server is responsible for managing virtual idols, live streaming, and handling interactions with fans.

[0353] Database (MongoDB): This database is used to store virtual idol data and user data.

[0354] Blockchain (Ethereum, Smart Contracts): A distributed ledger technology for managing and trading virtual idols as digital assets.

[0355] Natural Language Processing (NLP) Engine (Google® Cloud NLP): A natural language processing tool for generating real-time responses in communication with fans.

[0356] 2. Creating a virtual idol

[0357] User actions:

[0358] Users input parameters such as the gender, age, voice, and character design of a virtual idol through a smartphone app. This sends a generation request to the backend server.

[0359] Server processing:

[0360] The server starts the generation engine based on the received parameters and generates character data for the virtual idol. This data is stored in the database, and a notification of completion is sent to the smartphone app.

[0361] 3. Training and communication with virtual idols

[0362] User actions:

[0363] Users select training items and skill-up content via a smartphone app and submit training requests. They can also interact with virtual idols using the communication menu.

[0364] Server processing:

[0365] The server receives training and communication requests and updates the virtual idol's parameters. This result is stored in a database, and the updated information is sent to the smartphone.

[0366] 4. Live streaming and interaction with fans

[0367] User actions:

[0368] Users can schedule live streams and submit requests using a smartphone app. During the live stream, comments from fans are displayed in real time.

[0369] Server processing:

[0370] The server stores the live stream schedule in a database and prepares the appropriate streaming server. During the stream, comments are processed in real time, and a virtual idler responds using a natural language processing (NLP) engine. After the stream ends, the live data is stored in the database.

[0371] 5. Converting virtual idols into NFTs and exchanging them.

[0372] User actions:

[0373] Users submit an NFT request for a virtual idol via a smartphone app and enter the necessary information.

[0374] Server processing:

[0375] The server retrieves character data and generates a smart contract to register it on the NFT marketplace. The data is then registered on the blockchain. Transaction requests are received, successful transactions are recorded in the database, and the results are sent to the user.

[0376] Specific example

[0377] Example 1:

[0378] User A uses a smartphone app to create a young female character with an "idol cafe" theme and requests singing training.

[0379] Prompt message:

[0380] "I'm a 19-year-old female character, please create a virtual idol with a high-pitched voice."

[0381] Example 2:

[0382] User B sets up a live stream every Friday night at 8 PM to interact with fans in real time.

[0383] Prompt message:

[0384] "Please set up a live stream every Friday night at 8 PM and respond to fan comments in real time."

[0385] Example 3:

[0386] User C registers the virtual idol they have trained as an NFT on the marketplace and trades it there.

[0387] Prompt message:

[0388] "Please register my virtual idol as an NFT on the blockchain and make it tradable."

[0389] This will allow users to easily create and nurture virtual idols, enjoy real-time interaction with fans, and manage and trade them as digital assets.

[0390] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0391] Step 1:

[0392] User submits a request to generate a virtual idol.

[0393] The user launches a smartphone app and enters parameters such as gender, age, voice, and character design into a virtual idol creation form. The device sends the entered parameters along with the creation request to the backend server.

[0394] Input: Parameters entered by the user, such as gender, age, voice, and character design.

[0395] Output: The generation request is sent to the backend server.

[0396] Step 2:

[0397] Server-driven generation of virtual idles

[0398] The server starts the generation engine based on the received generation request and generates character data for a virtual idol based on the provided parameters. This data is stored in the database.

[0399] Input: Parameters included in the generation request

[0400] Output: The virtual idol's character data and a generation completion notification are saved to the database and sent to the terminal.

[0401] Step 3:

[0402] User submission of a request to train a virtual idol

[0403] Users select training items and skill development content from the training menu in the smartphone app and send training requests through the app. The device then sends the request to the backend server.

[0404] Input: Training items and skill development content selected by the user.

[0405] Output: Training request is sent to the backend server.

[0406] Step 4:

[0407] Server updates of virtual idol training data

[0408] The server receives the training request and updates the virtual idol's parameters. The update results are saved in the database and sent to the terminal.

[0409] Input: Training items and skill-up content included in the training request

[0410] Output: Updated virtual idle parameters, saved to the database and sent to the terminal.

[0411] Step 5:

[0412] User-controlled live streaming scheduling

[0413] The user sets a live stream schedule using a smartphone app and sends a schedule request. The device then sends the request to the backend server.

[0414] Input: User-defined live streaming schedule

[0415] Output: The schedule request is sent to the backend server.

[0416] Step 6:

[0417] Preparing the server for live streaming

[0418] The server stores the schedule request in the database and prepares the appropriate streaming server. During the live stream, comments from fans are displayed in real time, and the virtual idol responds using a natural language processing engine.

[0419] Input: Live stream schedule

[0420] Output: Live stream preparation, real-time display and response to fan comments.

[0421] Step 7:

[0422] User submits NFT request

[0423] The user submits an NFT request for a virtual idol via a smartphone app. After entering the necessary information, the device sends the request to the backend server.

[0424] Input: Information of the virtual idol to be converted to NFT.

[0425] Output: An NFT request is sent to the backend server.

[0426] Step 8:

[0427] NFT creation and transaction management via server

[0428] The server retrieves character data and generates and executes a smart contract to register it on the NFT marketplace. It registers the data on the blockchain, receives transaction requests, records successful transactions in the database, and sends the results to the terminal.

[0429] Input: Information about the virtual idol included in the NFT request

[0430] Output: Execution of NFT creation via smart contract, registration of data to the blockchain, notification of transaction results.

[0431] This allows users to easily create and nurture virtual idols, enjoy real-time interaction with fans, and even manage and trade them as digital assets.

[0432] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0433] System Overview

[0434] This invention is a system that generates virtual idols using a generation engine, performs initial setup based on user-provided parameters, and enables live streaming and interaction with fans. Furthermore, it allows virtual idols to be registered as digital assets, enabling exchange and trading with other users. In addition, by combining it with an emotion engine that recognizes user emotions, the system can adjust the virtual idol's responses and actions based on the user's emotions.

[0435] Explanation of the program's processing

[0436] 1. Creating a virtual idol

[0437] user:

[0438] The user opens a dedicated application or web interface and enters the necessary information (gender, age, voice, character design, etc.) into the virtual idol creation form. This includes parameters such as the idol's gender, age, voice, and character design.

[0439] Terminal:

[0440] The terminal sends the parameters entered along with the generation request to the server.

[0441] server:

[0442] The server receives the request and starts the generation engine. The generation engine generates character data for a virtual idol based on the provided parameters and saves it to the database. A notification of completion of generation is sent to the terminal, and the terminal displays this to the user.

[0443] Specific example:

[0444] A user expresses a desire to create a virtual idol character who is a young woman with a cute, high-pitched voice. They enter the information into the creation form and press the "Generate" button. The server receives the information sent by the device, and the generation engine creates the character and saves it to the database. Finally, a completion notification is displayed to the user.

[0445] 2. Training and communication with virtual idols

[0446] user:

[0447] Users select training items and skill-building content from the application's training menu and submit requests. They can also interact with virtual idols using the communication menu.

[0448] Terminal:

[0449] The device sends nurturing and communication requests to the server.

[0450] server:

[0451] The server receives the request and updates the virtual idle's parameters (e.g., skill level). It saves the update results to the database and sends them to the terminal.

[0452] Specific example:

[0453] A user sends a request to a virtual idol asking for "singing training." The device sends the request to the server, which updates the virtual idol's skill parameters and displays the results to the user.

[0454] 3. Live streaming and interaction with fans

[0455] user:

[0456] Users schedule live streams and submit requests through the application. During the live stream, they can watch as the virtual idol reacts to comments from fans.

[0457] Terminal:

[0458] The device sends schedule information and commands to start and end the live stream to the server. During the live stream, comments from fans are displayed in real time.

[0459] server:

[0460] The server stores the live stream schedule in a database and prepares the appropriate streaming server. During the live stream, comments are processed in real time, and a natural language processing engine is used to ensure that virtual idlers respond. After the live stream ends, the live data is saved.

[0461] Specific example:

[0462] A user sets a schedule, such as "A virtual idol will perform a live show every Friday night at 8 PM." During the live show, comments from fans are displayed in real time, and the virtual idol responds. After the live show ends, the server saves the data.

[0463] 4. Converting virtual idols into NFTs and exchanging them

[0464] user:

[0465] The user enters the settings for creating an NFT of a virtual idle object in the application and submits the request.

[0466] Terminal:

[0467] The terminal sends a request to the server.

[0468] server:

[0469] The server retrieves character data, generates a smart contract to register it on the NFT marketplace, and registers the data on the blockchain. It receives exchange and purchase requests from other users and manages transactions. Successful transactions are recorded in the database, and the results are sent to the terminal.

[0470] Specific example:

[0471] The user enters information to convert their virtual idol into an NFT and submits a request. The server retrieves the character data, generates a smart contract to create the NFT, and registers the data on the blockchain. The transaction is successful, and the result is displayed to the user.

[0472] 5. Implementation of the emotion engine

[0473] user:

[0474] Users interact and communicate with virtual idols through the application.

[0475] Terminal:

[0476] The device transmits conversation and interaction data to the server in real time.

[0477] server:

[0478] The server uses an emotion engine to analyze the user's emotions from their conversations and interactions. The emotion engine analyzes input data (voice, text, facial expressions, etc.) to identify the user's emotional state.

[0479] server:

[0480] The server adjusts the virtual idle's responses based on the emotion engine's analysis results. Specifically, it provides encouraging messages if the user is sad, and jokes or positive topics if the user is happy.

[0481] server:

[0482] The server saves the sentiment analysis results to a database and sends them to the application.

[0483] Terminal:

[0484] The device displays the sentiment analysis results and adjusted responses to the user. This is provided to the user as a real-time response from a virtual idler.

[0485] Specific example:

[0486] During a chat session with a virtual idol, the user types, "I'm a little tired today." The device sends this message to the server, where an emotion engine analyzes the user's emotions, such as "tired" or "depressed." Based on this, the server adjusts the response to the virtual idol, sending an encouraging message such as, "Take it easy today!" The result is displayed to the user through the device, and the user receives words of encouragement from the virtual idol.

[0487] Through this detailed embodiment, users can efficiently perform a series of operations, including generating, training, live streaming, NFT creation, and interaction using emotion recognition for virtual idols.

[0488] The following describes the processing flow.

[0489] Virtual Idol Creation

[0490] Step 1:

[0491] The user opens a dedicated application or web interface and enters the necessary information (gender, age, voice, character design, etc.) into the virtual idol creation form.

[0492] Step 2:

[0493] The user presses the "Generate" button.

[0494] Step 3:

[0495] The terminal sends the parameters entered along with the generation request to the server.

[0496] Step 4:

[0497] The server receives the request and starts the generation engine.

[0498] Step 5:

[0499] The server passes user-provided parameters to the generation engine to generate character data for the virtual idol.

[0500] Step 6:

[0501] The server saves the generated character data to the database.

[0502] Step 7:

[0503] The server sends a notification to the device when generation is complete.

[0504] Step 8:

[0505] The device displays a message or preview to the user indicating that generation is complete.

[0506] Training and communication with virtual idols

[0507] Step 1:

[0508] Users select training items and skill development content from the application's training menu and submit a request.

[0509] Step 2:

[0510] Users can use the communication menu to interact with virtual idols.

[0511] Step 3:

[0512] The device sends nurturing and communication requests to the server.

[0513] Step 4:

[0514] The server receives the request and updates the virtual idle's parameters (e.g., skill level).

[0515] Step 5:

[0516] The server saves the updated status to the database.

[0517] Step 6:

[0518] The server sends the update results to the terminal.

[0519] Step 7:

[0520] The device displays the update results to the user.

[0521] Live streaming and interaction with fans

[0522] Step 1:

[0523] Users schedule live streams and submit requests through the application.

[0524] Step 2:

[0525] The terminal sends schedule information to the server.

[0526] Step 3:

[0527] The server saves the schedule to the database and prepares the appropriate delivery server.

[0528] Step 4:

[0529] The user sends the command to start the live stream.

[0530] Step 5:

[0531] The terminal sends a command to the server, and the live stream begins.

[0532] Step 6:

[0533] The server receives comments from fans in real time during the live stream, and a natural language processing engine is used to enable the virtual idol to respond appropriately.

[0534] Step 7:

[0535] The server saves the live data after the live stream ends.

[0536] Step 8:

[0537] The user sends a command to end the live session.

[0538] Step 9:

[0539] The terminal sends a command to the server, and the live session ends.

[0540] NFT conversion and exchange of virtual idols

[0541] Step 1:

[0542] The user enters the settings for creating an NFT of a virtual idle object in the application and submits the request.

[0543] Step 2:

[0544] The terminal sends a request to the server.

[0545] Step 3:

[0546] The server retrieves the character data and generates a smart contract to register it on the NFT marketplace.

[0547] Step 4:

[0548] The server registers the NFT data on the blockchain.

[0549] Step 5:

[0550] The server receives exchange and purchase requests from other users and manages the transactions.

[0551] Step 6:

[0552] The server records successful transactions in the database.

[0553] Step 7:

[0554] The terminal displays the transaction results to the user after the transaction is completed.

[0555] Implementation of an emotion engine

[0556] Step 1:

[0557] Users interact and communicate with virtual idols through the application.

[0558] Step 2:

[0559] The device transmits conversation and interaction data to the server in real time.

[0560] Step 3:

[0561] The server uses an emotion engine to analyze user emotions from their conversations and interactions.

[0562] Step 4:

[0563] The server adjusts the virtual idle's response based on the analysis results of the emotion engine.

[0564] Step 5:

[0565] The server saves the emotion analysis results to a database.

[0566] Step 6:

[0567] The device displays the sentiment analysis results and adjusted responses to the user.

[0568] Specific example:

[0569] During a chat session with a virtual idol, the user types, "I'm a little tired today." The device sends this message to the server, where an emotion engine analyzes the user's emotions, such as "tired" or "depressed." Based on this, the server adjusts the response to the virtual idol, sending an encouraging message such as, "Take it easy today!" The result is displayed to the user through the device, and the user receives words of encouragement from the virtual idol.

[0570] (Example 2)

[0571] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0572] Modern users demand more personalized entertainment experiences, with interactive experiences through live streaming and conversations with virtual idols being particularly important. However, current systems struggle to provide consistent management and high flexibility in areas such as virtual idol creation, training, live streaming, fan interaction, and even handling as digital assets. Furthermore, recognizing user emotions and generating appropriate responses is difficult. This results in a limited user experience and a reduced appeal of the system.

[0573] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for generating virtual idols using a generation engine, means for initializing the terminal based on parameters provided by the user, means for providing live streaming services and interacting with the user, means for registering them as digital assets and enabling exchange or trading with other users, means for receiving generation requests and sending notifications of generation completion, and means for analyzing the user's emotions using an emotion recognition engine and adjusting the response content. This makes it possible for the user to consistently generate, nurture, live stream, handle virtual idols as digital assets, and engage in interactive dialogue using emotion recognition.

[0574] A "generation engine" is a software or hardware means for generating character data for a virtual idol based on parameters provided by the user.

[0575] A "virtual idol" is a digital character generated based on user input, providing an interactive entertainment experience through live streaming and dialogue.

[0576] "Parameters" are the information that users provide when creating a virtual idol, and examples include gender, age, voice, and character design.

[0577] "Initial setup" is the process of configuring the basic attributes and characteristics of a virtual idle based on parameters provided by the user.

[0578] "Live streaming" refers to a type of streaming service where virtual idols interact with users and fans in real time.

[0579] "Dialogue" refers to the act of a virtual idol interacting with users or fans using text or voice.

[0580] "Digital assets" refer to virtual idols and other digital items that are registered in a form that can be owned and traded as digital information.

[0581] An "emotion recognition engine" is a software or hardware system that analyzes emotions from user input data (such as voice, text, and facial expressions) and generates an appropriate response based on that analysis.

[0582] "Real-time" means that processing and responses are performed instantly with minimal delay.

[0583] This invention is a system that generates virtual idols using a generation engine and an emotion recognition engine, performs initial setup based on parameters provided by the user, and conducts live streaming and interactions with fans. Furthermore, it allows the virtual idols to be registered as digital assets, enabling exchange and trading with other users.

[0584] System Configuration

[0585] 1. Creating a virtual idol

[0586] User:

[0587] Users enter the necessary information into a virtual idol creation form using a dedicated application or web interface. This information includes gender, age, voice, and character design. A specific example of user input might be, "I want to create a virtual idol who is a young woman with a cute, high-pitched voice."

[0588] Terminal:

[0589] The terminal sends the entered parameters to the server as a generation request. This can be done using either an HTTP request or a WebSocket.

[0590] server:

[0591] When the server receives a generation request, it starts the generation engine. The generation engine generates character data for a virtual idol based on the provided parameters and saves it to the database. It also sends a notification to the terminal when generation is complete, and the terminal displays this to the user.

[0592] Specific example:

[0593] The user enters "I want to create a virtual idol character who is a young woman with a cute, high-pitched voice" and presses the "Generate" button. The server receives the information sent by the terminal, the generation engine generates the character, and saves it to the database. A notification of completion is displayed to the user.

[0594] 2. Training and communication with virtual idols

[0595] User:

[0596] Users select training items and skill-building content from the application's training menu and submit requests. They can also interact with virtual idols using the communication menu.

[0597] Terminal:

[0598] The device sends training and communication requests to the server. These requests include the data ID and training items of the target virtual idol.

[0599] server:

[0600] The server receives the request and updates the virtual idle parameters based on the specified training items. A database write operation is performed, the update results are sent to the terminal, and the terminal displays them to the user.

[0601] Specific example:

[0602] A user sends a request to a virtual idol asking for "singing training." The device sends the request to the server, the server updates the virtual idol's skill parameters, and displays the results to the user.

[0603] 3. Live streaming and interaction with fans

[0604] User:

[0605] Users can schedule live streams and submit requests through the application. For example, they might set up a live stream every Friday at 8 PM.

[0606] Terminal:

[0607] The terminal sends schedule information to the server. The request includes the date and time and the virtual idol's data ID.

[0608] server:

[0609] The server stores schedule information in a database and prepares for live streaming. It starts streaming at the scheduled time, processes fan comments in real time, and the virtual idol responds using a natural language processing engine. After the live stream ends, it saves the live data.

[0610] Specific example:

[0611] A user sets a schedule, such as "A virtual idol will perform a live show every Friday night at 8 PM." During the live show, comments from fans are displayed in real time, and the virtual idol responds. After the live show ends, the server saves the data.

[0612] 4. Converting virtual idols into NFTs and exchanging them

[0613] User:

[0614] The user enters the settings for creating an NFT of a virtual idle character in the application and submits the request. The provided settings include the price and description of the NFT.

[0615] Terminal:

[0616] The device sends an NFT request to the server. This request includes the virtual idol's data ID and detailed information about the NFT creation.

[0617] server:

[0618] The server receives the request, retrieves the character data, and generates a smart contract to register it on the NFT marketplace. The generated data is then registered on the blockchain to manage exchange and purchase requests.

[0619] Specific example:

[0620] The user enters information to convert their virtual idol into an NFT and submits a request. The server retrieves the character data, generates a smart contract to create the NFT, and registers the data on the blockchain. The transaction is successful, and the result is displayed to the user.

[0621] 5. Implementation of the emotion engine

[0622] User:

[0623] Users interact and converse with virtual idols through the application. For example, they might type a message like, "I'm a little tired today."

[0624] Terminal:

[0625] The terminal transmits conversation and interaction data to the server in real time. Specifically, it transfers the entered message data to the server.

[0626] server:

[0627] The server uses an emotion engine to analyze user emotions from their conversations and interactions. This analysis includes various input data such as voice, text, and facial expressions. Based on the emotion analysis results, the server adjusts the virtual idol's responses and sends the results to the terminal.

[0628] Specific example:

[0629] During a chat session with a virtual idol, the user types, "I'm a little tired today." The device sends this message to the server, where the emotion engine analyzes emotions such as "tired" and "depressed." Based on this, the server adjusts the response to the virtual idol to send an encouraging message such as, "Take it easy today!" The result is displayed to the user through the device, and the user receives words of encouragement from the virtual idol.

[0630] Example prompts for generative AI models:

[0631] "Please create a virtual idol character who is a young woman with a cute, high-pitched voice."

[0632] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0633] Step 1:

[0634] User:

[0635] The user opens a dedicated application or web interface and enters the necessary information (gender, age, voice, character design, etc.) into the virtual idol creation form.

[0636] Input: Gender, age, voice, character design

[0637] Output: The user's input information is reflected in the form.

[0638] Specific actions:

[0639] The user expresses a desire to create a virtual idol character who is a young woman with a cute, high-pitched voice, and enters specific information in each field.

[0640] Step 2:

[0641] Terminal:

[0642] The terminal sends the parameters entered by the user to the server as a generation request.

[0643] Input: User input parameters

[0644] Output: The generation request is sent to the server.

[0645] Specific actions:

[0646] The terminal sends user input information to the server using an HTTP request or WebSocket.

[0647] Step 3:

[0648] server:

[0649] The server receives the generation request and starts the generation engine. The generation engine generates character data for a virtual idol based on the provided parameters and saves it to the database.

[0650] Input: Generation request and user input parameters

[0651] Output: Virtual idol character data

[0652] Specific actions:

[0653] The server starts the generation engine and generates character data for a virtual idol based on the parameters provided by the user. The character data is then saved to the database.

[0654] Step 4:

[0655] server:

[0656] The server sends a notification to the device when generation is complete.

[0657] Input: Generation completion information

[0658] Output: Generation completion notification

[0659] Specific actions:

[0660] After the generation engine finishes, the server creates a generation completion notification and sends it to the terminal.

[0661] Step 5:

[0662] Terminal:

[0663] The terminal displays a generation completion notification received from the server to the user.

[0664] Input: Generation completion notification

[0665] Output: Display of generation complete messages

[0666] Specific actions:

[0667] The device displays a message to the user such as, "Virtual idle creation complete."

[0668] Step 6:

[0669] User:

[0670] Users select training items and skill development content from the application's training menu and submit a request.

[0671] Input: Training items and skill development content

[0672] Output: Generation of training requests

[0673] Specific actions:

[0674] The user selects and submits a specific training request, such as "I would like singing training."

[0675] Step 7:

[0676] Terminal:

[0677] The device sends a training request to the server.

[0678] Input: Training Request

[0679] Output: Sending a request

[0680] Specific actions:

[0681] The device sends a training request to the server using an HTTP request or WebSocket.

[0682] Step 8:

[0683] server:

[0684] The server receives a training request and updates the virtual idol's parameters based on the specified training items. The update results are saved to the database.

[0685] Input: Training Request

[0686] Output: Updated parameters

[0687] Specific actions:

[0688] The server updates the skill level of the virtual idlers based on the training items and saves the results to the database.

[0689] Step 9:

[0690] server:

[0691] The server sends the update results to the terminal.

[0692] Input: Updated parameters

[0693] Output: Update result notification

[0694] Specific actions:

[0695] The server sends an update result notification to the terminal.

[0696] Step 10:

[0697] Terminal:

[0698] The device displays the update results to the user.

[0699] Input: Update result notification

[0700] Output: Display of update complete messages

[0701] Specific actions:

[0702] The device displays a message to the user such as, "Singing training is complete, and your skill level has increased."

[0703] Step 11:

[0704] User:

[0705] Users schedule live streams and submit requests through the application.

[0706] Input: Live Streaming Schedule

[0707] Output: Generation of schedule request

[0708] Specific actions:

[0709] The user enters a schedule, stating, "A virtual idol will perform live every Friday night at 8 PM," and submits it.

[0710] Step 12:

[0711] Terminal:

[0712] The terminal sends schedule information to the server.

[0713] Input: Schedule information

[0714] Output: Sending a schedule request

[0715] Specific actions:

[0716] The terminal sends schedule information to the server using an HTTP request or WebSocket.

[0717] Step 13:

[0718] server:

[0719] The server stores the schedule information in a database and then prepares the appropriate distribution server.

[0720] Input: Schedule information

[0721] Output: Saved schedule information, delivery readiness status

[0722] Specific actions:

[0723] The server saves the schedule information to the database and prepares for the live stream.

[0724] Step 14:

[0725] server:

[0726] The server will begin live streaming at the scheduled start time. During the live stream, comments from fans will be processed in real time, and the virtual idol will respond using a natural language processing engine.

[0727] Input: Live stream comments

[0728] Output: Real-time response

[0729] Specific actions:

[0730] The server starts streaming, analyzing comments from fans in real time and generating responses.

[0731] Step 15:

[0732] Terminal:

[0733] The device displays comments from fans and responses from the virtual idol in real time.

[0734] Input: Real-time response

[0735] Output: Display of comments and responses

[0736] Specific actions:

[0737] The device displays comments and virtual idle responses to the user in real time.

[0738] Step 16:

[0739] server:

[0740] After the live stream ends, the server saves the live data.

[0741] Input: Live data

[0742] Output: Saved live data

[0743] Specific actions:

[0744] The server saves the live stream data to the database.

[0745] Step 17:

[0746] User:

[0747] The user enters the settings for creating an NFT of a virtual idle in the application and submits the request.

[0748] Input: NFT configuration information

[0749] Output: Generation of NFT request

[0750] Specific actions:

[0751] The user enters and submits the necessary information to convert the virtual idle into an NFT.

[0752] Step 18:

[0753] Terminal:

[0754] The device sends an NFT request to the server.

[0755] Input: NFT configuration information

[0756] Output: Sending an NFT request

[0757] Specific actions:

[0758] The device sends an NFT creation request to the server using an HTTP request or WebSocket.

[0759] Step 19:

[0760] server:

[0761] The server receives the request, retrieves the character data, and generates a smart contract to register it on the NFT marketplace. The generated data is then registered on the blockchain.

[0762] Input: NFT request

[0763] Output: Smart contracts, data registered on the blockchain

[0764] Specific actions:

[0765] The server generates a smart contract, converts the virtual idle into an NFT, and registers it on the blockchain.

[0766] Step 20:

[0767] server:

[0768] It receives exchange and purchase requests from other users and manages transactions. If a transaction is successful, it records the result in the database and sends the result to the terminal.

[0769] Input: Exchange request, Purchase request

[0770] Output: Transaction results

[0771] Specific actions:

[0772] The server processes the transaction request, records successful transactions in the database, and sends the results to the terminal.

[0773] Step 21:

[0774] Terminal:

[0775] Display the transaction results to the user.

[0776] Input: Transaction result

[0777] Output: Display of results

[0778] Specific actions:

[0779] The terminal displays the transaction results to the user, and if successful, ownership of the virtual idol is transferred to the user.

[0780] Step 22:

[0781] User:

[0782] Users interact and communicate with virtual idols through the application.

[0783] Input: User interactions and messages

[0784] Output: Dialogue data

[0785] Specific actions:

[0786] The user enters a message through the application such as, "I'm a little tired today."

[0787] Step 23:

[0788] Terminal:

[0789] The device transmits conversation and interaction data to the server in real time.

[0790] Input: User input data

[0791] Output: Sending dialogue data

[0792] Specific actions:

[0793] The terminal sends interaction data to the server using an HTTP request or WebSocket.

[0794] Step 24:

[0795] server:

[0796] The server uses an emotion engine to analyze user emotions from their conversations and interactions. Based on the analysis results, it adjusts the virtual idle's responses.

[0797] Input: Dialogue data

[0798] Output: Sentiment analysis results, adjusted response content

[0799] Specific actions:

[0800] The server activates the emotion engine, analyzes the user's dialogue data to identify emotions, and adjusts the virtual idle's response based on the analysis results.

[0801] Step 25:

[0802] server:

[0803] The server sends the adjusted response to the application.

[0804] Input: Adjusted response content

[0805] Output: Sending response data

[0806] Specific actions:

[0807] The server sends the generated response to the application.

[0808] Step 26:

[0809] Terminal:

[0810] The terminal displays the response to the user.

[0811] Input: Response data

[0812] Output: Display response

[0813] Specific actions:

[0814] The device displays a response message to the user such as, "Take a good rest today!"

[0815] (Application Example 2)

[0816] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0817] Conventional virtual character systems, while capable of generating and distributing characters and interacting with viewers, lacked the ability to recognize the user's emotional state in real time and provide appropriate responses based on that state. As a result, virtual character responses were uniform and fixed, leading to a decline in the quality of user interaction. Furthermore, when registering virtual characters as digital assets and exchanging or trading them with other users, the lack of systems incorporating emotion recognition meant that users were not adequately provided with a sense of presence or personalization.

[0818] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating a virtual character using a generation engine, means for performing initial settings on the virtual character based on attributes provided by the user, means for the virtual character to distribute information in real time within a virtual environment and interact with viewers, means for registering the virtual character as a digital asset and enabling it to be exchanged or bought and sold with other users, means for analyzing the user's emotional state, and means for adjusting the virtual character's response based on the emotional state. As a result, it becomes possible to recognize the user's emotional state in real time and provide an appropriate response in accordance with that state, thereby improving the quality of interaction with the user and enabling the provision of a more immersive and personalized experience.

[0819] A "generation engine" is a system that has algorithms and functions for generating virtual characters based on attribute data provided by the user.

[0820] A "virtual character" is a digital character created by a user using a generation engine, and is an entity that can operate within a virtual environment.

[0821] "User" refers to an individual or group that uses this system to create, develop, and manage virtual characters.

[0822] "Attributes" refer to basic information about a virtual character, such as gender, age, voice, and character design.

[0823] "Initial setup" refers to the process of setting the basic properties and characteristics of a virtual character based on attributes provided by the user.

[0824] A "virtual environment" is a digital space provided on the internet or other digital platforms for virtual characters to operate.

[0825] "Real time" refers to the fact that the activities and communications of virtual characters occur simultaneously with real time.

[0826] "Digital assets" are assets in which virtual characters are recorded as data, possess a specific value, and can be bought, sold, or exchanged.

[0827] "Emotional state" refers to the emotional state of a user as perceived based on their comments and actions.

[0828] "Adjusting responses" means controlling the virtual character to return reactions and messages that are appropriate to the user's emotional state.

[0829] The system for implementing this invention utilizes a combination of a generation engine, an emotion analysis engine, and a natural language processing engine. It generates a virtual character based on user-provided attributes, and this character delivers information in real time, interacts with viewers, and responds based on the user's emotional state. Furthermore, the generated virtual character can be registered as a digital asset and exchanged or bought and sold with other users. This system uses hardware such as a smartphone, head-mounted display (HMD), server, and database, along with software such as a generation AI model, a natural language processing engine, and an emotion analysis engine.

[0830] First, the user opens a dedicated application and enters the necessary information into the virtual character generation form. This information includes attributes such as gender, age, voice, and character design. The information entered by the user is sent from the terminal to the server, which then starts the generation engine. The generation engine generates a virtual character based on the provided attribute data and saves that data to a database. After generation is complete, the result is sent to the terminal and displayed to the user.

[0831] Next, when developing or improving the skills of a virtual character, the user selects training items and skill-up content from the application's development menu and sends a request. This request is sent from the terminal to the server, which updates the virtual character's parameters (e.g., skill level). The update results are stored in the database and displayed to the user through the terminal.

[0832] Furthermore, when live streaming, users schedule the live stream within the application and submit a request. During the live stream, a virtual character reacts to comments from viewers. The server processes comments in real time, and an emotion analysis engine analyzes the user's emotions from the comments. Based on this analysis, a natural language processing engine generates an appropriate response, which the virtual character displays.

[0833] Finally, when registering a virtual character as a digital asset, the user enters the necessary information in the application and sends a request to the server. The server retrieves the character data, generates a smart contract to register it on the NFT marketplace, and registers the data on the blockchain. It receives exchange and purchase requests from other users, records successful transactions in the database, and sends the results to the user.

[0834] For example, when a user performs the action of "Create a new virtual character," it sends a prompt to the generation engine saying, "Create a virtual character that is a young woman with a cute, high-pitched voice." The generation engine generates a character based on this prompt and saves it to the database. The generated character is then displayed on the user's device. If the virtual character comments "I'm a little tired today" during a live stream, the sentiment analysis engine identifies emotions such as "tired" and "depressed" and sends back an encouraging message such as "Take it easy today!"

[0835] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0836] Step 1:

[0837] The user enters attributes into a virtual character generation form and submits a generation request. Specifically, the user fills in information such as gender, age, voice, and character design into the application's input form and presses the submit button. The input data is sent from the terminal to the server. The attribute data entered by the user is received as input and sent to the server as output.

[0838] Step 2:

[0839] The server starts the generation engine and generates a virtual character based on the provided attributes. The server receives attribute data sent by the user and sends prompt messages to the generation AI model based on that data. An example of a prompt message would be, "Create a virtual character that is a young woman with a cute, high-pitched voice." The generation engine generates the virtual character and saves its data to the database. The system takes user attribute data as input and saves the generated virtual character data to the database as output.

[0840] Step 3:

[0841] The terminal displays a generation completion notification to the user. A generation completion notification is sent from the server to the terminal, and the terminal displays it to the user. It receives a generation completion notification as input and displays the notification to the user as output.

[0842] Step 4:

[0843] Users submit requests for training and skill improvement of their virtual characters. Users select their desired training items and skill development content from the application's training menu and submit their requests. The application receives user training requests as input and sends them to the server as output.

[0844] Step 5:

[0845] The server updates the virtual character's parameters. The server receives requests from users and updates the virtual character data in the database. For example, if the skill level increases. It receives training requests as input and the updated character data is saved to the database as output.

[0846] Step 6:

[0847] The terminal displays the update results to the user. The server sends the update results to the terminal, and the terminal displays them to the user. It receives the update results as input and displays them to the user as output.

[0848] Step 7:

[0849] The user schedules a live stream and submits a request. The user sets the schedule for the live stream in the application and submits the request. The application receives the schedule information as input and sends it to the server as output.

[0850] Step 8:

[0851] The server prepares for live streaming and saves schedule information to the database. The server receives schedule information sent by the user and prepares the appropriate streaming server. Schedule information is received as input, and the streaming server configuration and its saving to the database are output.

[0852] Step 9:

[0853] During a live stream, the server receives comments from users, and an emotion analysis engine analyzes them. The server sends the received comments to the emotion analysis engine in real time. The emotion analysis engine analyzes the emotions from the comments and returns the results to the server. User comments are received as input, and the emotion analysis results are returned as output.

[0854] Step 10:

[0855] The server uses a natural language processing engine to generate a response from a virtual character based on the sentiment analysis results. The server sends a prompt to the natural language processing engine based on the sentiment analysis results, and generates an appropriate response. For example, if the comment is "I'm a little tired today," the sentiment analysis engine identifies "tiredness," and the natural language processing engine generates an encouraging message such as "Take it easy today!" The server receives the sentiment analysis results as input and returns the generated response message as output.

[0856] Step 11:

[0857] The terminal displays the generated response to the user. A response message is sent from the server to the terminal, and the terminal displays it to the user in real time. It receives a response message as input and displays it to the user as output.

[0858] Step 12:

[0859] A user submits a request to convert a virtual character to an NFT. The user enters the NFT conversion settings for the virtual character in the application and submits the request. The application receives the NFT conversion request as input and sends it to the server as output.

[0860] Step 13:

[0861] The server retrieves character data, generates a smart contract, and registers it on the NFT marketplace. The server receives requests from users, retrieves character data, and generates a smart contract. It receives an NFT conversion request as input, generates a smart contract as output, and registers it on the marketplace.

[0862] Step 14:

[0863] The server manages transaction requests and records successful transactions in a database. It receives exchange and purchase requests from other users, and if a transaction is successful, it records the result in the database and sends it to the user. Transaction requests are received as input, and transaction results are recorded in the database as output.

[0864] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0865] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0866] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0867] [Second Embodiment]

[0868] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0869] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0870] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0871] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0872] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0873] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0874] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0875] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0876] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0877] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0878] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0879] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0880] System Overview

[0881] This invention is a system that generates virtual idols (AItubers) using a generation engine, performs initial setup based on parameters provided by the user, and enables live streaming and interaction with fans. Furthermore, it allows the virtual idols to be registered as digital assets, enabling exchange and trading with other users.

[0882] Explanation of the program's processing

[0883] 1. Creating a virtual idol

[0884] user:

[0885] Users enter the necessary information into a virtual idol creation form using a dedicated application or web interface. This includes parameters such as the idol's gender, age, voice, and character design.

[0886] Terminal:

[0887] The terminal sends the parameters entered along with the generation request to the server.

[0888] server:

[0889] The server receives the request and starts the generation engine. Based on the provided parameters, the generation engine generates character data for a virtual idol and saves it to the database. A notification of completion is sent to the terminal, which then displays this to the user.

[0890] Specific example:

[0891] A user expresses a desire to create a virtual idol character who is a young woman with a cute, high-pitched voice. They enter the information into the creation form and press the "Generate" button. The server receives the information sent by the device, and the generation engine creates the character and saves it to the database. Finally, a completion notification is displayed to the user.

[0892] 2. Training and communication with virtual idols

[0893] user:

[0894] Users select training items and skill-building content from the application's training menu and submit requests. They can also interact with virtual idols using the communication menu.

[0895] Terminal:

[0896] The device sends nurturing and communication requests to the server.

[0897] server:

[0898] The server receives the request and updates the virtual idle parameters. The update results are saved to the database and sent to the terminal.

[0899] Specific example:

[0900] A user sends a request to a virtual idol asking for "singing training." The device sends the request to the server, which updates the virtual idol's skill parameters and displays the results to the user.

[0901] 3. Live streaming and interaction with fans

[0902] user:

[0903] Users schedule live streams and submit requests through the application. During the live stream, they can watch as the virtual idol reacts to comments from fans.

[0904] Terminal:

[0905] The device sends schedule information and commands to start and end the live stream to the server. During the live stream, comments from fans are displayed in real time.

[0906] server:

[0907] The server stores the live stream schedule in a database and prepares the appropriate streaming server. During the live stream, comments are processed in real time, and a natural language processing engine is used to ensure that virtual idlers respond. After the live stream ends, the live data is saved.

[0908] Specific example:

[0909] A user sets a schedule, such as "A virtual idol will perform a live show every Friday night at 8 PM." During the live show, comments from fans are displayed in real time, and the virtual idol responds. After the live show ends, the server saves the data.

[0910] 4. Converting virtual idols into NFTs and exchanging them

[0911] user:

[0912] The user enters the settings for creating an NFT of a virtual idle object in the application and submits the request.

[0913] Terminal:

[0914] The terminal sends a request to the server.

[0915] server:

[0916] The server retrieves character data, generates a smart contract to register it on the NFT marketplace, and registers the data on the blockchain. It receives exchange and purchase requests from other users and manages transactions. Successful transactions are recorded in the database, and the results are sent to the terminal.

[0917] Specific example:

[0918] The user enters information to convert their virtual idol into an NFT and submits a request. The server retrieves the character data, generates a smart contract to create the NFT, and registers the data on the blockchain. The transaction is successful, and the result is displayed to the user.

[0919] Through this detailed embodiment, users can efficiently perform a series of operations such as creating, training, live streaming, and converting virtual idols into NFTs.

[0920] The following describes the processing flow.

[0921] Virtual Idol Creation

[0922] Step 1:

[0923] The user opens a dedicated application or web interface and enters the necessary information (gender, age, voice, character design, etc.) into the virtual idol creation form.

[0924] Step 2:

[0925] The user presses the "Generate" button.

[0926] Step 3:

[0927] The terminal sends the parameters entered along with the generation request to the server.

[0928] Step 4:

[0929] The server receives the request and starts the generation engine.

[0930] Step 5:

[0931] The server passes user-provided parameters to the generation engine to generate character data for the virtual idol.

[0932] Step 6:

[0933] The server saves the generated character data to the database.

[0934] Step 7:

[0935] The server sends a notification to the device when generation is complete.

[0936] Step 8:

[0937] The device displays a message or preview to the user indicating that generation is complete.

[0938] Training and communication with virtual idols

[0939] Step 1:

[0940] Users select training items and skill development content from a training menu via the application and submit requests.

[0941] Step 2:

[0942] Users can use the communication menu to interact with virtual idols.

[0943] Step 3:

[0944] The terminal sends a request to the server.

[0945] Step 4:

[0946] The server receives the request and updates the virtual idle's parameters (e.g., skill level).

[0947] Step 5:

[0948] The server saves the updated status to the database.

[0949] Step 6:

[0950] The server sends the update results to the terminal.

[0951] Step 7:

[0952] The device displays the update results to the user.

[0953] Live streaming and interaction with fans

[0954] Step 1:

[0955] Users schedule live streams and submit requests through the application.

[0956] Step 2:

[0957] The terminal sends schedule information to the server.

[0958] Step 3:

[0959] The server saves the schedule to the database and prepares the appropriate delivery server.

[0960] Step 4:

[0961] The user sends the command to start the live stream.

[0962] Step 5:

[0963] The terminal sends a command to the server, and the live stream begins.

[0964] Step 6:

[0965] The server receives comments from fans in real time during the live stream, and a natural language processing engine is used to enable the virtual idol to respond appropriately.

[0966] Step 7:

[0967] The server saves the live data after the live stream ends.

[0968] Step 8:

[0969] The user sends a command to end the live session.

[0970] Step 9:

[0971] The terminal sends a command to the server, and the live session ends.

[0972] NFT conversion and exchange of virtual idols

[0973] Step 1:

[0974] The user enters the settings for creating an NFT of a virtual idle object in the application and submits the request.

[0975] Step 2:

[0976] The terminal sends a request to the server.

[0977] Step 3:

[0978] The server retrieves the character data and generates a smart contract to register it on the NFT marketplace.

[0979] Step 4:

[0980] The server registers the NFT data on the blockchain.

[0981] Step 5:

[0982] The server receives exchange and purchase requests from other users and manages the transactions.

[0983] Step 6:

[0984] The server records successful transactions in the database.

[0985] Step 7:

[0986] The terminal displays the transaction results to the user after the transaction is completed.

[0987] (Example 1)

[0988] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0989] Traditional virtual idol systems lacked an integrated platform that allowed users to smoothly perform a series of operations, such as creating, raising, live streaming, and converting characters into NFTs. Furthermore, they did not adequately provide features for generating real-time responses based on user input or managing character data as digital assets. As a result, users had to manage their virtual idols individually, leading to decreased operational efficiency and a poor user experience.

[0990] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0991] In this invention, the server includes means for the user to input information into a virtual idol generation form using a dedicated application or web interface, means for the terminal to send the input information to the server along with the generation request, and means for the server to start a generation engine, generate character data for the virtual idol based on the provided information, and store it in a database. This allows the user to perform a series of operations, from virtual idol generation and training to live streaming and NFT creation, in a unified manner.

[0992] A "user" refers to an entity that uses the virtual idol system to perform operations such as creating, training, live streaming, and converting virtual idols into NFTs.

[0993] A "terminal" refers to a device used by a user, such as a computer, smartphone, or tablet, that communicates with the virtual idle system.

[0994] A "server" is the central computer system of a virtual idle system, and it is a device that processes user requests and generates, stores, and manages data.

[0995] A "virtual idol" refers to a digitized character created using a generation engine, possessing character design, personality, voice, and other characteristics based on parameters provided by the user.

[0996] A "generation engine" is software or an algorithm used to generate character data for a virtual idol based on user input.

[0997] A "database" is an information management system used to store character data, live streaming data, NFT transaction information, and other data related to the virtual idols that have been created.

[0998] A "natural language processing engine" is software or an algorithm that analyzes text data from users and fans and generates appropriate responses.

[0999] "NFT" stands for "Non-Fungible Token," and refers to a unique digital certificate that registers a virtual idol as a digital asset on the blockchain.

[1000] A "smart contract" is a program that automatically executes transactions on the blockchain and is used for NFT creation of virtual idols and transaction management.

[1001] "Live streaming" refers to virtual idols broadcasting video and audio to viewers in real time via the internet.

[1002] "Training items" refer to specific activities and learning content aimed at improving and developing the skills of virtual idols.

[1003] System Overview

[1004] This invention is a system that generates virtual idols (AItubers) using a generation engine. Based on parameters provided by the user, it performs initial setup, enables live streaming, interaction with fans, training and skill improvement, and can even be converted into NFTs, allowing for exchange and trading with other users as digital assets.

[1005] Virtual Idol Creation

[1006] user:

[1007] Users enter information into a virtual idol creation form using a dedicated application or web interface. Specifically, they enter parameters such as gender, age, voice, and character design.

[1008] Terminal:

[1009] The terminal serializes the input information into JSON format and sends it to the server as a generation request.

[1010] server:

[1011] After the server receives the request, it starts the "AICharacterG engine," which is the generation engine, and generates character data based on the provided parameters. The generated character data is stored in a MySQL database. A notification of completion of generation is sent to the terminal and is eventually displayed to the user.

[1012] Specific example:

[1013] A user requests to create a virtual idol character who is a young woman with a cute, high-pitched voice. They enter the information and press the "Generate" button. The device sends the request, the server generates the character, and saves it to the database. A notification of completion is displayed to the user.

[1014] Prompt message:

[1015] "We're creating a virtual idol character who is a young woman with a cute, high-pitched voice."

[1016] Training virtual idols

[1017] user:

[1018] Users select training items and skill development content from the application's training menu and submit a request.

[1019] Terminal:

[1020] The device sends a training request to the server.

[1021] server:

[1022] The server receives the request and updates the virtual idle parameters. The update results are saved in the database and notified to the user.

[1023] Specific example:

[1024] A user sends a request to a virtual idol asking for "singing training." The device sends the request to the server, the server updates the virtual idol's skill parameters, and displays the results to the user.

[1025] Prompt message:

[1026] "We have started training to improve the singing skills of our virtual idols."

[1027] Live streaming and interaction with fans

[1028] user:

[1029] Users schedule live streams and submit requests through the application. During the live stream, they can watch as the virtual idol reacts to comments from fans.

[1030] Terminal:

[1031] The device sends schedule information and commands to start and end the live stream to the server. During the live stream, comments from fans are displayed in real time.

[1032] server:

[1033] The server saves the live streaming schedule to a database and prepares the appropriate streaming server. Comments during the live stream are processed in real time by a natural language processing engine, and a virtual idol responds. After the live stream ends, the live data is saved to the database.

[1034] Specific example:

[1035] A user sets a schedule, such as "A virtual idol will perform a live show every Friday night at 8 PM." During the live show, comments from fans are displayed in real time, and the virtual idol responds. After the live show ends, the server saves the data.

[1036] Prompt message:

[1037] "We will be holding a live stream of a virtual idol every Friday night at 8 PM."

[1038] NFT conversion and exchange of virtual idols

[1039] user:

[1040] The user enters the NFT settings for virtual idles in the application and sends the request.

[1041] Terminal:

[1042] The terminal sends a request to the server.

[1043] server:

[1044] The server retrieves character data, generates smart contracts, registers them on the NFT marketplace, and manages transactions. It receives exchange and purchase requests from other users, records successful transactions in a database, and notifies the user of the results.

[1045] Specific example:

[1046] The user enters information to convert their virtual idol into an NFT and submits a request. The server retrieves the character data, generates a smart contract to create the NFT, and registers the data on the blockchain. The transaction is successful, and the result is displayed to the user.

[1047] Prompt message:

[1048] "Turn virtual idols into NFTs and register them on a marketplace."

[1049] Through this detailed embodiment, users can efficiently perform a series of operations such as creating, training, live streaming, and converting virtual idols into NFTs.

[1050] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1051] Virtual Idol Creation

[1052] Step 1:

[1053] The user enters information.

[1054] Users open a dedicated application or web interface and enter information into a form for creating a virtual idol. This input data includes gender, age, voice, character design, and more.

[1055] input:

[1056] Parameters such as gender, age, voice, and character design

[1057] output:

[1058] Form data of the entered information

[1059] Specific actions:

[1060] The user opens the web interface in their browser and enters information into dropdown menus or text boxes.

[1061] Step 2:

[1062] The device sends the request

[1063] The terminal serializes the information entered by the user into JSON format and sends it to the server as a generation request.

[1064] input:

[1065] Form data entered by the user

[1066] output:

[1067] HTTP POST request to the server (data in JSON format)

[1068] Specific actions:

[1069] The terminal serializes the request data and sends an HTTP POST request to the specified endpoint on the server.

[1070] Step 3:

[1071] The server starts the generation engine.

[1072] After the server receives the request, it starts the generation engine (AICharacterG) and generates character data based on the provided parameters. The generated character data is saved in a temporary folder.

[1073] input:

[1074] The server receives a generation request (data in JSON format).

[1075] output:

[1076] Generated character data (file format)

[1077] Specific actions:

[1078] The server starts the generation engine, processes the character data based on the provided parameters, and saves it to a temporary folder.

[1079] Step 4:

[1080] The server saves the generated results and sends notifications.

[1081] The server saves the generated character data to a MySQL database and notifies the terminal when the generation is complete.

[1082] input:

[1083] Generated character data (file format)

[1084] output:

[1085] Character data stored in the database, generation completion notification (HTTP response)

[1086] Specific actions:

[1087] The server saves the character data to the database and sends an HTTP response to the terminal to notify it that generation is complete.

[1088] ---

[1089] Training virtual idols

[1090] Step 1:

[1091] The user selects a training menu.

[1092] Users select training items and skill development content from the application's training menu and submit a request.

[1093] input:

[1094] Training items and skill development content

[1095] output:

[1096] Training request (form data)

[1097] Specific actions:

[1098] The user selects "Singing Training" on the application screen and presses the "Submit" button.

[1099] Step 2:

[1100] The device sends the request

[1101] The device sends the user's selected training request to the server in JSON format.

[1102] input:

[1103] User-selected training request (form data)

[1104] output:

[1105] HTTP POST request to the server (data in JSON format)

[1106] Specific actions:

[1107] The terminal serializes the training request data and sends an HTTP POST request to the server.

[1108] Step 3:

[1109] The server updates the parameters.

[1110] The server receives the request data and updates the character data parameters according to the training details of the corresponding virtual idol. The update results are saved in the database and sent to the terminal as a notification.

[1111] input:

[1112] The server receives a training request (in JSON format).

[1113] output:

[1114] Updated character data stored in the database, update completion notification (HTTP response)

[1115] Specific actions:

[1116] The server updates the character data in the database and saves the new parameters. It then sends an update completion notification to the terminal as an HTTP response.

[1117] ---

[1118] Live streaming and interaction with fans

[1119] Step 1:

[1120] Users set up live streams

[1121] The user sets the live stream schedule in the application and sends the request to the server.

[1122] input:

[1123] Live stream schedule information (date, time, content)

[1124] output:

[1125] Live stream request (form data)

[1126] Specific actions:

[1127] The user sets "Live stream every Friday night at 8 PM" in the application and presses the "Schedule" button.

[1128] Step 2:

[1129] The device sends the request

[1130] The device sends the user's configured schedule information to the server in JSON format.

[1131] input:

[1132] Live schedule information set by the user (form data)

[1133] output:

[1134] HTTP POST request to the server (data in JSON format)

[1135] Specific actions:

[1136] The terminal serializes the live schedule information and sends an HTTP POST request to the server.

[1137] Step 3:

[1138] The server saves the schedule and starts delivery.

[1139] The server saves the live streaming schedule to the database and prepares the streaming server for live streaming. During the live stream, a natural language processing engine is used to process comments in real time, and a virtual idol responds. After the live stream ends, the live data is saved to the database.

[1140] input:

[1141] Live schedule information (in JSON format) received by the server.

[1142] output:

[1143] Live schedule information stored in the database, real-time processed comment data, and stored live data.

[1144] Specific actions:

[1145] The server saves the schedule to the database and configures the distribution server. During the live broadcast, comment data is analyzed using a natural language processing engine, and responses are generated in real time. After the live broadcast ends, all live data is saved to the database.

[1146] ---

[1147] NFT conversion and exchange of virtual idols

[1148] Step 1:

[1149] The user enters the NFT settings.

[1150] The user uses the application to input the information necessary to convert the virtual idle into an NFT and submits the request.

[1151] input:

[1152] Information required for NFT creation (character data, price, etc.)

[1153] output:

[1154] NFT conversion request (form data)

[1155] Specific actions:

[1156] The user clicks the "Convert to NFT" button, enters the required information, and clicks the "Submit" button.

[1157] Step 2:

[1158] The device sends the request

[1159] The device sends the NFT conversion request entered by the user to the server in JSON format.

[1160] input:

[1161] NFT conversion request (form data) entered by the user

[1162] output:

[1163] HTTP POST request to the server (data in JSON format)

[1164] Specific actions:

[1165] The device serializes the NFT data and sends an HTTP POST request to the server.

[1166] Step 3:

[1167] The server registers NFTs and manages transactions.

[1168] The server receives the request data, retrieves the virtual idol's character data, generates a smart contract, and converts it into an NFT. The generated NFT is registered on the blockchain, where exchange and purchase requests from other users are managed. Successful transactions are recorded in the database, and notifications are sent to the device.

[1169] input:

[1170] NFT conversion request (JSON format data) received by the server

[1171] output:

[1172] Generated smart contract, NFT registered on the blockchain, and transaction success notification (HTTP response).

[1173] Specific actions:

[1174] The server retrieves character data, generates a smart contract, and converts it into an NFT. The data is registered on the blockchain, and if the transaction is successful, the transaction information is recorded in the database. Finally, the transaction result is notified to the terminal.

[1175] (Application Example 1)

[1176] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1177] To enhance the user experience, there is a need for a system that integrates many different functions, such as the creation and training of virtual idols, live streaming, and their management as digital assets. However, there is a lack of technical means to smoothly and efficiently implement these functions. In particular, a technical foundation is required to provide the advanced experiences that users expect, such as real-time responses during live streaming and blockchain-based management of virtual idols as digital assets.

[1178] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1179] In this invention, the server includes means for generating virtual idols using a generation engine, means for performing initial settings on the virtual idols based on parameters provided by the user, means for the virtual idols to perform live streaming and interact with fans, means for registering the virtual idols as digital assets and enabling them to be exchanged or bought and sold with other users, means for registering the digital assets of the virtual idols on the blockchain, and means for setting the live streaming schedule of the virtual idols and providing real-time responses. This makes it possible for users to easily generate and nurture virtual idols, enjoy real-time interaction with fans during live streaming, and manage and trade them as digital assets.

[1180] A "generation engine" is a software or hardware component that automatically generates virtual idles based on parameters provided by the user.

[1181] A "virtual idol" is a character created using digital technology, a digital entity capable of live streaming and interacting with fans.

[1182] "User-provided parameters" refer to the set of information that users input to the generation engine, such as the gender, age, voice, and character design of a virtual idol.

[1183] "Initial setup" refers to the process of configuring the basic characteristics and functions of a generated virtual idol based on parameters provided by the user.

[1184] "Live streaming" is a feature that allows virtual idols to deliver content in real time and engage in interactive communication with viewers.

[1185] "Fan interaction" refers to a feature that allows virtual idols to interact with fans in real time or remotely through live streaming or other communication methods.

[1186] "Digital assets" refer to digital content, such as virtual idols, that can be treated as objects of ownership and trading.

[1187] Blockchain is a distributed ledger technology used to manage and trade digital currencies and digital assets.

[1188] "Real-time response" is a feature that allows virtual idols to instantly respond to comments and questions from fans during live streams.

[1189] This invention provides a system for users to create and nurture virtual idols and interact with fans through live streaming. The mechanism is described in detail below.

[1190] 1. System Configuration

[1191] Hardware and software:

[1192] Smartphone: A device used by users to control a virtual idle.

[1193] Mobile operating systems (iOS / Android): These are operating systems that run on smartphones.

[1194] Backend Server (Node.js, Express): This server is responsible for managing virtual idols, live streaming, and handling interactions with fans.

[1195] Database (MongoDB): This database is used to store virtual idol data and user data.

[1196] Blockchain (Ethereum, Smart Contracts): A distributed ledger technology for managing and trading virtual idols as digital assets.

[1197] Natural Language Processing (NLP) Engine (Google Cloud NLP): A natural language processing tool for generating real-time responses in communication with fans.

[1198] 2. Creating a virtual idol

[1199] User actions:

[1200] Users input parameters such as the gender, age, voice, and character design of a virtual idol through a smartphone app. This sends a generation request to the backend server.

[1201] Server processing:

[1202] The server starts the generation engine based on the received parameters and generates character data for the virtual idol. This data is stored in the database, and a notification of completion is sent to the smartphone app.

[1203] 3. Training and communication with virtual idols

[1204] User actions:

[1205] Users select training items and skill-up content via a smartphone app and submit training requests. They can also interact with virtual idols using the communication menu.

[1206] Server processing:

[1207] The server receives training and communication requests and updates the virtual idol's parameters. This result is stored in a database, and the updated information is sent to the smartphone.

[1208] 4. Live streaming and interaction with fans

[1209] User actions:

[1210] Users can schedule live streams and submit requests using a smartphone app. During the live stream, comments from fans are displayed in real time.

[1211] Server processing:

[1212] The server stores the live stream schedule in a database and prepares the appropriate streaming server. During the stream, comments are processed in real time, and a virtual idler responds using a natural language processing (NLP) engine. After the stream ends, the live data is stored in the database.

[1213] 5. Converting virtual idols into NFTs and exchanging them.

[1214] User actions:

[1215] Users submit an NFT request for a virtual idol via a smartphone app and enter the necessary information.

[1216] Server processing:

[1217] The server retrieves character data and generates a smart contract to register it on the NFT marketplace. The data is then registered on the blockchain. Transaction requests are received, successful transactions are recorded in the database, and the results are sent to the user.

[1218] Specific example

[1219] Example 1:

[1220] User A uses a smartphone app to create a young female character with an "idol cafe" theme and requests singing training.

[1221] Prompt message:

[1222] "I'm a 19-year-old female character, please create a virtual idol with a high-pitched voice."

[1223] Example 2:

[1224] User B sets up a live stream every Friday night at 8 PM to interact with fans in real time.

[1225] Prompt message:

[1226] "Please set up a live stream every Friday night at 8 PM and respond to fan comments in real time."

[1227] Example 3:

[1228] User C registers the virtual idol they have trained as an NFT on the marketplace and trades it there.

[1229] Prompt message:

[1230] "Please register my virtual idol as an NFT on the blockchain and make it tradable."

[1231] This will allow users to easily create and nurture virtual idols, enjoy real-time interaction with fans, and manage and trade them as digital assets.

[1232] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1233] Step 1:

[1234] User submits a request to generate a virtual idol.

[1235] The user launches a smartphone app and enters parameters such as gender, age, voice, and character design into a virtual idol creation form. The device sends the entered parameters along with the creation request to the backend server.

[1236] Input: Parameters entered by the user, such as gender, age, voice, and character design.

[1237] Output: The generation request is sent to the backend server.

[1238] Step 2:

[1239] Server-driven generation of virtual idles

[1240] The server starts the generation engine based on the received generation request and generates character data for a virtual idol based on the provided parameters. This data is stored in the database.

[1241] Input: Parameters included in the generation request

[1242] Output: The virtual idol's character data and a generation completion notification are saved to the database and sent to the terminal.

[1243] Step 3:

[1244] User submission of a request to train a virtual idol

[1245] Users select training items and skill development content from the training menu in the smartphone app and send training requests through the app. The device then sends the request to the backend server.

[1246] Input: Training items and skill development content selected by the user.

[1247] Output: Training request is sent to the backend server.

[1248] Step 4:

[1249] Server updates of virtual idol training data

[1250] The server receives the training request and updates the virtual idol's parameters. The update results are saved in the database and sent to the terminal.

[1251] Input: Training items and skill-up content included in the training request

[1252] Output: Updated virtual idle parameters, saved to the database and sent to the terminal.

[1253] Step 5:

[1254] User-controlled live streaming scheduling

[1255] The user sets a live stream schedule using a smartphone app and sends a schedule request. The device then sends the request to the backend server.

[1256] Input: User-defined live streaming schedule

[1257] Output: The schedule request is sent to the backend server.

[1258] Step 6:

[1259] Preparing the server for live streaming

[1260] The server stores the schedule request in the database and prepares the appropriate streaming server. During the live stream, comments from fans are displayed in real time, and the virtual idol responds using a natural language processing engine.

[1261] Input: Live stream schedule

[1262] Output: Live stream preparation, real-time display and response to fan comments.

[1263] Step 7:

[1264] User submits NFT request

[1265] The user submits an NFT request for a virtual idol via a smartphone app. After entering the necessary information, the device sends the request to the backend server.

[1266] Input: Information of the virtual idol to be converted to NFT.

[1267] Output: An NFT request is sent to the backend server.

[1268] Step 8:

[1269] NFT creation and transaction management via server

[1270] The server retrieves character data and generates and executes a smart contract to register it on the NFT marketplace. It registers the data on the blockchain, receives transaction requests, records successful transactions in the database, and sends the results to the terminal.

[1271] Input: Information about the virtual idol included in the NFT request

[1272] Output: Execution of NFT creation via smart contract, registration of data to the blockchain, notification of transaction results.

[1273] This allows users to easily create and nurture virtual idols, enjoy real-time interaction with fans, and even manage and trade them as digital assets.

[1274] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1275] System Overview

[1276] This invention is a system that generates virtual idols using a generation engine, performs initial setup based on user-provided parameters, and enables live streaming and interaction with fans. Furthermore, it allows virtual idols to be registered as digital assets, enabling exchange and trading with other users. In addition, by combining it with an emotion engine that recognizes user emotions, the system can adjust the virtual idol's responses and actions based on the user's emotions.

[1277] Explanation of the program's processing

[1278] 1. Creating a virtual idol

[1279] user:

[1280] The user opens a dedicated application or web interface and enters the necessary information (gender, age, voice, character design, etc.) into the virtual idol creation form. This includes parameters such as the idol's gender, age, voice, and character design.

[1281] Terminal:

[1282] The terminal sends the parameters entered along with the generation request to the server.

[1283] server:

[1284] The server receives the request and starts the generation engine. The generation engine generates character data for a virtual idol based on the provided parameters and saves it to the database. A notification of completion of generation is sent to the terminal, and the terminal displays this to the user.

[1285] Specific example:

[1286] A user expresses a desire to create a virtual idol character who is a young woman with a cute, high-pitched voice. They enter the information into the creation form and press the "Generate" button. The server receives the information sent by the device, and the generation engine creates the character and saves it to the database. Finally, a completion notification is displayed to the user.

[1287] 2. Training and communication with virtual idols

[1288] user:

[1289] Users select training items and skill-building content from the application's training menu and submit requests. They can also interact with virtual idols using the communication menu.

[1290] Terminal:

[1291] The device sends nurturing and communication requests to the server.

[1292] server:

[1293] The server receives the request and updates the virtual idle's parameters (e.g., skill level). It saves the update results to the database and sends them to the terminal.

[1294] Specific example:

[1295] A user sends a request to a virtual idol asking for "singing training." The device sends the request to the server, which updates the virtual idol's skill parameters and displays the results to the user.

[1296] 3. Live streaming and interaction with fans

[1297] user:

[1298] Users schedule live streams and submit requests through the application. During the live stream, they can watch as the virtual idol reacts to comments from fans.

[1299] Terminal:

[1300] The device sends schedule information and commands to start and end the live stream to the server. During the live stream, comments from fans are displayed in real time.

[1301] server:

[1302] The server stores the live stream schedule in a database and prepares the appropriate streaming server. During the live stream, comments are processed in real time, and a natural language processing engine is used to ensure that virtual idlers respond. After the live stream ends, the live data is saved.

[1303] Specific example:

[1304] A user sets a schedule, such as "A virtual idol will perform a live show every Friday night at 8 PM." During the live show, comments from fans are displayed in real time, and the virtual idol responds. After the live show ends, the server saves the data.

[1305] 4. Converting virtual idols into NFTs and exchanging them

[1306] user:

[1307] The user enters the settings for creating an NFT of a virtual idle object in the application and submits the request.

[1308] Terminal:

[1309] The terminal sends a request to the server.

[1310] server:

[1311] The server retrieves character data, generates a smart contract to register it on the NFT marketplace, and registers the data on the blockchain. It receives exchange and purchase requests from other users and manages transactions. Successful transactions are recorded in the database, and the results are sent to the terminal.

[1312] Specific example:

[1313] The user enters information to convert their virtual idol into an NFT and submits a request. The server retrieves the character data, generates a smart contract to create the NFT, and registers the data on the blockchain. The transaction is successful, and the result is displayed to the user.

[1314] 5. Implementation of the emotion engine

[1315] user:

[1316] Users interact and communicate with virtual idols through the application.

[1317] Terminal:

[1318] The device transmits conversation and interaction data to the server in real time.

[1319] server:

[1320] The server uses an emotion engine to analyze the user's emotions from their conversations and interactions. The emotion engine analyzes input data (voice, text, facial expressions, etc.) to identify the user's emotional state.

[1321] server:

[1322] The server adjusts the virtual idle's responses based on the emotion engine's analysis results. Specifically, it provides encouraging messages if the user is sad, and jokes or positive topics if the user is happy.

[1323] server:

[1324] The server saves the sentiment analysis results to a database and sends them to the application.

[1325] Terminal:

[1326] The device displays the sentiment analysis results and adjusted responses to the user. This is provided to the user as a real-time response from a virtual idler.

[1327] Specific example:

[1328] During a chat session with a virtual idol, the user types, "I'm a little tired today." The device sends this message to the server, where an emotion engine analyzes the user's emotions, such as "tired" or "depressed." Based on this, the server adjusts the response to the virtual idol, sending an encouraging message such as, "Take it easy today!" The result is displayed to the user through the device, and the user receives words of encouragement from the virtual idol.

[1329] Through this detailed embodiment, users can efficiently perform a series of operations, including generating, training, live streaming, NFT creation, and interaction using emotion recognition for virtual idols.

[1330] The following describes the processing flow.

[1331] Virtual Idol Creation

[1332] Step 1:

[1333] The user opens a dedicated application or web interface and enters the necessary information (gender, age, voice, character design, etc.) into the virtual idol creation form.

[1334] Step 2:

[1335] The user presses the "Generate" button.

[1336] Step 3:

[1337] The terminal sends the parameters entered along with the generation request to the server.

[1338] Step 4:

[1339] The server receives the request and starts the generation engine.

[1340] Step 5:

[1341] The server passes user-provided parameters to the generation engine to generate character data for the virtual idol.

[1342] Step 6:

[1343] The server saves the generated character data to the database.

[1344] Step 7:

[1345] The server sends a notification to the device when generation is complete.

[1346] Step 8:

[1347] The device displays a message or preview to the user indicating that generation is complete.

[1348] Training and communication with virtual idols

[1349] Step 1:

[1350] Users select training items and skill development content from the application's training menu and submit a request.

[1351] Step 2:

[1352] Users can use the communication menu to interact with virtual idols.

[1353] Step 3:

[1354] The device sends nurturing and communication requests to the server.

[1355] Step 4:

[1356] The server receives the request and updates the virtual idle's parameters (e.g., skill level).

[1357] Step 5:

[1358] The server saves the updated status to the database.

[1359] Step 6:

[1360] The server sends the update results to the terminal.

[1361] Step 7:

[1362] The device displays the update results to the user.

[1363] Live streaming and interaction with fans

[1364] Step 1:

[1365] Users schedule live streams and submit requests through the application.

[1366] Step 2:

[1367] The terminal sends schedule information to the server.

[1368] Step 3:

[1369] The server saves the schedule to the database and prepares the appropriate delivery server.

[1370] Step 4:

[1371] The user sends the command to start the live stream.

[1372] Step 5:

[1373] The terminal sends a command to the server, and the live stream begins.

[1374] Step 6:

[1375] The server receives comments from fans in real time during the live stream, and a natural language processing engine is used to enable the virtual idol to respond appropriately.

[1376] Step 7:

[1377] The server saves the live data after the live stream ends.

[1378] Step 8:

[1379] The user sends a command to end the live session.

[1380] Step 9:

[1381] The terminal sends a command to the server, and the live session ends.

[1382] NFT conversion and exchange of virtual idols

[1383] Step 1:

[1384] The user enters the settings for creating an NFT of a virtual idle object in the application and submits the request.

[1385] Step 2:

[1386] The terminal sends a request to the server.

[1387] Step 3:

[1388] The server retrieves the character data and generates a smart contract to register it on the NFT marketplace.

[1389] Step 4:

[1390] The server registers the NFT data on the blockchain.

[1391] Step 5:

[1392] The server receives exchange and purchase requests from other users and manages the transactions.

[1393] Step 6:

[1394] The server records successful transactions in the database.

[1395] Step 7:

[1396] The terminal displays the transaction results to the user after the transaction is completed.

[1397] Implementation of an emotion engine

[1398] Step 1:

[1399] Users interact and communicate with virtual idols through the application.

[1400] Step 2:

[1401] The device transmits conversation and interaction data to the server in real time.

[1402] Step 3:

[1403] The server uses an emotion engine to analyze user emotions from their conversations and interactions.

[1404] Step 4:

[1405] The server adjusts the virtual idle's response based on the analysis results of the emotion engine.

[1406] Step 5:

[1407] The server saves the emotion analysis results to a database.

[1408] Step 6:

[1409] The device displays the sentiment analysis results and adjusted responses to the user.

[1410] Specific example:

[1411] During a chat session with a virtual idol, the user types, "I'm a little tired today." The device sends this message to the server, where an emotion engine analyzes the user's emotions, such as "tired" or "depressed." Based on this, the server adjusts the response to the virtual idol, sending an encouraging message such as, "Take it easy today!" The result is displayed to the user through the device, and the user receives words of encouragement from the virtual idol.

[1412] (Example 2)

[1413] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1414] Modern users demand more personalized entertainment experiences, with interactive experiences through live streaming and conversations with virtual idols being particularly important. However, current systems struggle to provide consistent management and high flexibility in areas such as virtual idol creation, training, live streaming, fan interaction, and even handling as digital assets. Furthermore, recognizing user emotions and generating appropriate responses is difficult. This results in a limited user experience and a reduced appeal of the system.

[1415] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for generating virtual idols using a generation engine, means for initializing the terminal based on parameters provided by the user, means for providing live streaming services and interacting with the user, means for registering them as digital assets and enabling exchange or trading with other users, means for receiving generation requests and sending notifications of generation completion, and means for analyzing the user's emotions using an emotion recognition engine and adjusting the response content. This makes it possible for the user to consistently generate, nurture, live stream, handle virtual idols as digital assets, and engage in interactive dialogue using emotion recognition.

[1416] A "generation engine" is a software or hardware means for generating character data for a virtual idol based on parameters provided by the user.

[1417] A "virtual idol" is a digital character generated based on user input, providing an interactive entertainment experience through live streaming and dialogue.

[1418] "Parameters" are the information that users provide when creating a virtual idol, and examples include gender, age, voice, and character design.

[1419] "Initial setup" is the process of configuring the basic attributes and characteristics of a virtual idle based on parameters provided by the user.

[1420] "Live streaming" refers to a type of streaming service where virtual idols interact with users and fans in real time.

[1421] "Dialogue" refers to the act of a virtual idol interacting with users or fans using text or voice.

[1422] "Digital assets" refer to virtual idols and other digital items that are registered in a form that can be owned and traded as digital information.

[1423] An "emotion recognition engine" is a software or hardware system that analyzes emotions from user input data (such as voice, text, and facial expressions) and generates an appropriate response based on that analysis.

[1424] "Real-time" means that processing and responses are performed instantly with minimal delay.

[1425] This invention is a system that generates virtual idols using a generation engine and an emotion recognition engine, performs initial setup based on parameters provided by the user, and conducts live streaming and interactions with fans. Furthermore, it allows the virtual idols to be registered as digital assets, enabling exchange and trading with other users.

[1426] System Configuration

[1427] 1. Creating a virtual idol

[1428] User:

[1429] Users enter the necessary information into a virtual idol creation form using a dedicated application or web interface. This information includes gender, age, voice, and character design. A specific example of user input might be, "I want to create a virtual idol who is a young woman with a cute, high-pitched voice."

[1430] Terminal:

[1431] The terminal sends the entered parameters to the server as a generation request. This can be done using either an HTTP request or a WebSocket.

[1432] server:

[1433] When the server receives a generation request, it starts the generation engine. The generation engine generates character data for a virtual idol based on the provided parameters and saves it to the database. It also sends a notification to the terminal when generation is complete, and the terminal displays this to the user.

[1434] Specific example:

[1435] The user enters "I want to create a virtual idol character who is a young woman with a cute, high-pitched voice" and presses the "Generate" button. The server receives the information sent by the terminal, the generation engine generates the character, and saves it to the database. A notification of completion is displayed to the user.

[1436] 2. Training and communication with virtual idols

[1437] User:

[1438] Users select training items and skill-building content from the application's training menu and submit requests. They can also interact with virtual idols using the communication menu.

[1439] Terminal:

[1440] The device sends training and communication requests to the server. These requests include the data ID and training items of the target virtual idol.

[1441] server:

[1442] The server receives the request and updates the virtual idle parameters based on the specified training items. A database write operation is performed, the update results are sent to the terminal, and the terminal displays them to the user.

[1443] Specific example:

[1444] A user sends a request to a virtual idol asking for "singing training." The device sends the request to the server, the server updates the virtual idol's skill parameters, and displays the results to the user.

[1445] 3. Live streaming and interaction with fans

[1446] User:

[1447] Users can schedule live streams and submit requests through the application. For example, they might set up a live stream every Friday at 8 PM.

[1448] Terminal:

[1449] The terminal sends schedule information to the server. The request includes the date and time and the virtual idol's data ID.

[1450] server:

[1451] The server stores schedule information in a database and prepares for live streaming. It starts streaming at the scheduled time, processes fan comments in real time, and the virtual idol responds using a natural language processing engine. After the live stream ends, it saves the live data.

[1452] Specific example:

[1453] A user sets a schedule, such as "A virtual idol will perform a live show every Friday night at 8 PM." During the live show, comments from fans are displayed in real time, and the virtual idol responds. After the live show ends, the server saves the data.

[1454] 4. Converting virtual idols into NFTs and exchanging them

[1455] User:

[1456] The user enters the settings for creating an NFT of a virtual idle character in the application and submits the request. The provided settings include the price and description of the NFT.

[1457] Terminal:

[1458] The device sends an NFT request to the server. This request includes the virtual idol's data ID and detailed information about the NFT creation.

[1459] server:

[1460] The server receives the request, retrieves the character data, and generates a smart contract to register it on the NFT marketplace. The generated data is then registered on the blockchain to manage exchange and purchase requests.

[1461] Specific example:

[1462] The user enters information to convert their virtual idol into an NFT and submits a request. The server retrieves the character data, generates a smart contract to create the NFT, and registers the data on the blockchain. The transaction is successful, and the result is displayed to the user.

[1463] 5. Implementation of the emotion engine

[1464] User:

[1465] Users interact and converse with virtual idols through the application. For example, they might type a message like, "I'm a little tired today."

[1466] Terminal:

[1467] The terminal transmits conversation and interaction data to the server in real time. Specifically, it transfers the entered message data to the server.

[1468] server:

[1469] The server uses an emotion engine to analyze user emotions from their conversations and interactions. This analysis includes various input data such as voice, text, and facial expressions. Based on the emotion analysis results, the server adjusts the virtual idol's responses and sends the results to the terminal.

[1470] Specific example:

[1471] During a chat session with a virtual idol, the user types, "I'm a little tired today." The device sends this message to the server, where the emotion engine analyzes emotions such as "tired" and "depressed." Based on this, the server adjusts the response to the virtual idol to send an encouraging message such as, "Take it easy today!" The result is displayed to the user through the device, and the user receives words of encouragement from the virtual idol.

[1472] Example prompts for generative AI models:

[1473] "Please create a virtual idol character who is a young woman with a cute, high-pitched voice."

[1474] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1475] Step 1:

[1476] User:

[1477] The user opens a dedicated application or web interface and enters the necessary information (gender, age, voice, character design, etc.) into the virtual idol creation form.

[1478] Input: Gender, age, voice, character design

[1479] Output: The user's input information is reflected in the form.

[1480] Specific actions:

[1481] The user expresses a desire to create a virtual idol character who is a young woman with a cute, high-pitched voice, and enters specific information in each field.

[1482] Step 2:

[1483] Terminal:

[1484] The terminal sends the parameters entered by the user to the server as a generation request.

[1485] Input: User input parameters

[1486] Output: The generation request is sent to the server.

[1487] Specific actions:

[1488] The terminal sends user input information to the server using an HTTP request or WebSocket.

[1489] Step 3:

[1490] server:

[1491] The server receives the generation request and starts the generation engine. The generation engine generates character data for a virtual idol based on the provided parameters and saves it to the database.

[1492] Input: Generation request and user input parameters

[1493] Output: Virtual idol character data

[1494] Specific actions:

[1495] The server starts the generation engine and generates character data for a virtual idol based on the parameters provided by the user. The character data is then saved to the database.

[1496] Step 4:

[1497] server:

[1498] The server sends a notification to the device when generation is complete.

[1499] Input: Generation completion information

[1500] Output: Generation completion notification

[1501] Specific actions:

[1502] After the generation engine finishes, the server creates a generation completion notification and sends it to the terminal.

[1503] Step 5:

[1504] Terminal:

[1505] The terminal displays a generation completion notification received from the server to the user.

[1506] Input: Generation completion notification

[1507] Output: Display of generation complete messages

[1508] Specific actions:

[1509] The device displays a message to the user such as, "Virtual idle creation complete."

[1510] Step 6:

[1511] User:

[1512] Users select training items and skill development content from the application's training menu and submit a request.

[1513] Input: Training items and skill development content

[1514] Output: Generation of training requests

[1515] Specific actions:

[1516] The user selects and submits a specific training request, such as "I would like singing training."

[1517] Step 7:

[1518] Terminal:

[1519] The device sends a training request to the server.

[1520] Input: Training Request

[1521] Output: Sending a request

[1522] Specific actions:

[1523] The device sends a training request to the server using an HTTP request or WebSocket.

[1524] Step 8:

[1525] server:

[1526] The server receives a training request and updates the virtual idol's parameters based on the specified training items. The update results are saved to the database.

[1527] Input: Training Request

[1528] Output: Updated parameters

[1529] Specific actions:

[1530] The server updates the skill level of the virtual idlers based on the training items and saves the results to the database.

[1531] Step 9:

[1532] server:

[1533] The server sends the update results to the terminal.

[1534] Input: Updated parameters

[1535] Output: Update result notification

[1536] Specific actions:

[1537] The server sends an update result notification to the terminal.

[1538] Step 10:

[1539] Terminal:

[1540] The device displays the update results to the user.

[1541] Input: Update result notification

[1542] Output: Display of update complete messages

[1543] Specific actions:

[1544] The device displays a message to the user such as, "Singing training is complete, and your skill level has increased."

[1545] Step 11:

[1546] User:

[1547] Users schedule live streams and submit requests through the application.

[1548] Input: Live Streaming Schedule

[1549] Output: Generation of schedule request

[1550] Specific actions:

[1551] The user enters a schedule, stating, "A virtual idol will perform live every Friday night at 8 PM," and submits it.

[1552] Step 12:

[1553] Terminal:

[1554] The terminal sends schedule information to the server.

[1555] Input: Schedule information

[1556] Output: Sending a schedule request

[1557] Specific actions:

[1558] The terminal sends schedule information to the server using an HTTP request or WebSocket.

[1559] Step 13:

[1560] server:

[1561] The server stores the schedule information in a database and then prepares the appropriate distribution server.

[1562] Input: Schedule information

[1563] Output: Saved schedule information, delivery readiness status

[1564] Specific actions:

[1565] The server saves the schedule information to the database and prepares for the live stream.

[1566] Step 14:

[1567] server:

[1568] The server will begin live streaming at the scheduled start time. During the live stream, comments from fans will be processed in real time, and the virtual idol will respond using a natural language processing engine.

[1569] Input: Live stream comments

[1570] Output: Real-time response

[1571] Specific actions:

[1572] The server starts streaming, analyzing comments from fans in real time and generating responses.

[1573] Step 15:

[1574] Terminal:

[1575] The device displays comments from fans and responses from the virtual idol in real time.

[1576] Input: Real-time response

[1577] Output: Display of comments and responses

[1578] Specific actions:

[1579] The device displays comments and virtual idle responses to the user in real time.

[1580] Step 16:

[1581] server:

[1582] After the live stream ends, the server saves the live data.

[1583] Input: Live data

[1584] Output: Saved live data

[1585] Specific actions:

[1586] The server saves the live stream data to the database.

[1587] Step 17:

[1588] User:

[1589] The user enters the settings for creating an NFT of a virtual idle in the application and submits the request.

[1590] Input: NFT configuration information

[1591] Output: Generation of NFT request

[1592] Specific actions:

[1593] The user enters and submits the necessary information to convert the virtual idle into an NFT.

[1594] Step 18:

[1595] Terminal:

[1596] The device sends an NFT request to the server.

[1597] Input: NFT configuration information

[1598] Output: Sending an NFT request

[1599] Specific actions:

[1600] The device sends an NFT creation request to the server using an HTTP request or WebSocket.

[1601] Step 19:

[1602] server:

[1603] The server receives the request, retrieves the character data, and generates a smart contract to register it on the NFT marketplace. The generated data is then registered on the blockchain.

[1604] Input: NFT request

[1605] Output: Smart contracts, data registered on the blockchain

[1606] Specific actions:

[1607] The server generates a smart contract, converts the virtual idle into an NFT, and registers it on the blockchain.

[1608] Step 20:

[1609] server:

[1610] It receives exchange and purchase requests from other users and manages transactions. If a transaction is successful, it records the result in the database and sends the result to the terminal.

[1611] Input: Exchange request, Purchase request

[1612] Output: Transaction results

[1613] Specific actions:

[1614] The server processes the transaction request, records successful transactions in the database, and sends the results to the terminal.

[1615] Step 21:

[1616] Terminal:

[1617] Display the transaction results to the user.

[1618] Input: Transaction result

[1619] Output: Display of results

[1620] Specific actions:

[1621] The terminal displays the transaction results to the user, and if successful, ownership of the virtual idol is transferred to the user.

[1622] Step 22:

[1623] User:

[1624] Users interact and communicate with virtual idols through the application.

[1625] Input: User interactions and messages

[1626] Output: Dialogue data

[1627] Specific actions:

[1628] The user enters a message through the application such as, "I'm a little tired today."

[1629] Step 23:

[1630] Terminal:

[1631] The device transmits conversation and interaction data to the server in real time.

[1632] Input: User input data

[1633] Output: Sending dialogue data

[1634] Specific actions:

[1635] The terminal sends interaction data to the server using an HTTP request or WebSocket.

[1636] Step 24:

[1637] server:

[1638] The server uses an emotion engine to analyze user emotions from their conversations and interactions. Based on the analysis results, it adjusts the virtual idle's responses.

[1639] Input: Dialogue data

[1640] Output: Sentiment analysis results, adjusted response content

[1641] Specific actions:

[1642] The server activates the emotion engine, analyzes the user's dialogue data to identify emotions, and adjusts the virtual idle's response based on the analysis results.

[1643] Step 25:

[1644] server:

[1645] The server sends the adjusted response to the application.

[1646] Input: Adjusted response content

[1647] Output: Sending response data

[1648] Specific actions:

[1649] The server sends the generated response to the application.

[1650] Step 26:

[1651] Terminal:

[1652] The terminal displays the response to the user.

[1653] Input: Response data

[1654] Output: Display response

[1655] Specific actions:

[1656] The device displays a response message to the user such as, "Take a good rest today!"

[1657] (Application Example 2)

[1658] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1659] Conventional virtual character systems, while capable of generating and distributing characters and interacting with viewers, lacked the ability to recognize the user's emotional state in real time and provide appropriate responses based on that state. As a result, virtual character responses were uniform and fixed, leading to a decline in the quality of user interaction. Furthermore, when registering virtual characters as digital assets and exchanging or trading them with other users, the lack of systems incorporating emotion recognition meant that users were not adequately provided with a sense of presence or personalization.

[1660] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating a virtual character using a generation engine, means for performing initial settings on the virtual character based on attributes provided by the user, means for the virtual character to distribute information in real time within a virtual environment and interact with viewers, means for registering the virtual character as a digital asset and enabling it to be exchanged or bought and sold with other users, means for analyzing the user's emotional state, and means for adjusting the virtual character's response based on the emotional state. As a result, it becomes possible to recognize the user's emotional state in real time and provide an appropriate response in accordance with that state, thereby improving the quality of interaction with the user and enabling the provision of a more immersive and personalized experience.

[1661] A "generation engine" is a system that has algorithms and functions for generating virtual characters based on attribute data provided by the user.

[1662] A "virtual character" is a digital character created by a user using a generation engine, and is an entity that can operate within a virtual environment.

[1663] "User" refers to an individual or group that uses this system to create, develop, and manage virtual characters.

[1664] "Attributes" refer to basic information about a virtual character, such as gender, age, voice, and character design.

[1665] "Initial setup" refers to the process of setting the basic properties and characteristics of a virtual character based on attributes provided by the user.

[1666] A "virtual environment" is a digital space provided on the internet or other digital platforms for virtual characters to operate.

[1667] "Real time" refers to the fact that the activities and communications of virtual characters occur simultaneously with real time.

[1668] "Digital assets" are assets in which virtual characters are recorded as data, possess a specific value, and can be bought, sold, or exchanged.

[1669] "Emotional state" refers to the emotional state of a user as perceived based on their comments and actions.

[1670] "Adjusting responses" means controlling the virtual character to return reactions and messages that are appropriate to the user's emotional state.

[1671] The system for implementing this invention utilizes a combination of a generation engine, an emotion analysis engine, and a natural language processing engine. It generates a virtual character based on user-provided attributes, and this character delivers information in real time, interacts with viewers, and responds based on the user's emotional state. Furthermore, the generated virtual character can be registered as a digital asset and exchanged or bought and sold with other users. This system uses hardware such as a smartphone, head-mounted display (HMD), server, and database, along with software such as a generation AI model, a natural language processing engine, and an emotion analysis engine.

[1672] First, the user opens a dedicated application and enters the necessary information into the virtual character generation form. This information includes attributes such as gender, age, voice, and character design. The information entered by the user is sent from the terminal to the server, which then starts the generation engine. The generation engine generates a virtual character based on the provided attribute data and saves that data to a database. After generation is complete, the result is sent to the terminal and displayed to the user.

[1673] Next, when developing or improving the skills of a virtual character, the user selects training items and skill-up content from the application's development menu and sends a request. This request is sent from the terminal to the server, which updates the virtual character's parameters (e.g., skill level). The update results are stored in the database and displayed to the user through the terminal.

[1674] Furthermore, when live streaming, users schedule the live stream within the application and submit a request. During the live stream, a virtual character reacts to comments from viewers. The server processes comments in real time, and an emotion analysis engine analyzes the user's emotions from the comments. Based on this analysis, a natural language processing engine generates an appropriate response, which the virtual character displays.

[1675] Finally, when registering a virtual character as a digital asset, the user enters the necessary information in the application and sends a request to the server. The server retrieves the character data, generates a smart contract to register it on the NFT marketplace, and registers the data on the blockchain. It receives exchange and purchase requests from other users, records successful transactions in the database, and sends the results to the user.

[1676] For example, when a user performs the action of "Create a new virtual character," it sends a prompt to the generation engine saying, "Create a virtual character that is a young woman with a cute, high-pitched voice." The generation engine generates a character based on this prompt and saves it to the database. The generated character is then displayed on the user's device. If the virtual character comments "I'm a little tired today" during a live stream, the sentiment analysis engine identifies emotions such as "tired" and "depressed" and sends back an encouraging message such as "Take it easy today!"

[1677] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1678] Step 1:

[1679] The user enters attributes into a virtual character generation form and submits a generation request. Specifically, the user fills in information such as gender, age, voice, and character design into the application's input form and presses the submit button. The input data is sent from the terminal to the server. The attribute data entered by the user is received as input and sent to the server as output.

[1680] Step 2:

[1681] The server starts the generation engine and generates a virtual character based on the provided attributes. The server receives attribute data sent by the user and sends prompt messages to the generation AI model based on that data. An example of a prompt message would be, "Create a virtual character that is a young woman with a cute, high-pitched voice." The generation engine generates the virtual character and saves its data to the database. The system takes user attribute data as input and saves the generated virtual character data to the database as output.

[1682] Step 3:

[1683] The terminal displays a generation completion notification to the user. A generation completion notification is sent from the server to the terminal, and the terminal displays it to the user. It receives a generation completion notification as input and displays the notification to the user as output.

[1684] Step 4:

[1685] Users submit requests for training and skill improvement of their virtual characters. Users select their desired training items and skill development content from the application's training menu and submit their requests. The application receives user training requests as input and sends them to the server as output.

[1686] Step 5:

[1687] The server updates the virtual character's parameters. The server receives requests from users and updates the virtual character data in the database. For example, if the skill level increases. It receives training requests as input and the updated character data is saved to the database as output.

[1688] Step 6:

[1689] The terminal displays the update results to the user. The server sends the update results to the terminal, and the terminal displays them to the user. It receives the update results as input and displays them to the user as output.

[1690] Step 7:

[1691] The user schedules a live stream and submits a request. The user sets the schedule for the live stream in the application and submits the request. The application receives the schedule information as input and sends it to the server as output.

[1692] Step 8:

[1693] The server prepares for live streaming and saves schedule information to the database. The server receives schedule information sent by the user and prepares the appropriate streaming server. Schedule information is received as input, and the streaming server configuration and its saving to the database are output.

[1694] Step 9:

[1695] During a live stream, the server receives comments from users, and an emotion analysis engine analyzes them. The server sends the received comments to the emotion analysis engine in real time. The emotion analysis engine analyzes the emotions from the comments and returns the results to the server. User comments are received as input, and the emotion analysis results are returned as output.

[1696] Step 10:

[1697] The server uses a natural language processing engine to generate a response from a virtual character based on the sentiment analysis results. The server sends a prompt to the natural language processing engine based on the sentiment analysis results, and generates an appropriate response. For example, if the comment is "I'm a little tired today," the sentiment analysis engine identifies "tiredness," and the natural language processing engine generates an encouraging message such as "Take it easy today!" The server receives the sentiment analysis results as input and returns the generated response message as output.

[1698] Step 11:

[1699] The terminal displays the generated response to the user. A response message is sent from the server to the terminal, and the terminal displays it to the user in real time. It receives a response message as input and displays it to the user as output.

[1700] Step 12:

[1701] A user submits a request to convert a virtual character to an NFT. The user enters the NFT conversion settings for the virtual character in the application and submits the request. The application receives the NFT conversion request as input and sends it to the server as output.

[1702] Step 13:

[1703] The server retrieves character data, generates a smart contract, and registers it on the NFT marketplace. The server receives requests from users, retrieves character data, and generates a smart contract. It receives an NFT conversion request as input, generates a smart contract as output, and registers it on the marketplace.

[1704] Step 14:

[1705] The server manages transaction requests and records successful transactions in a database. It receives exchange and purchase requests from other users, and if a transaction is successful, it records the result in the database and sends it to the user. Transaction requests are received as input, and transaction results are recorded in the database as output.

[1706] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1707] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1708] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1709] [Third Embodiment]

[1710] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1711] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1712] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1713] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1714] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1715] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1716] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1717] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1718] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1719] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1720] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1721] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1722] System Overview

[1723] This invention is a system that generates virtual idols (AItubers) using a generation engine, performs initial setup based on parameters provided by the user, and enables live streaming and interaction with fans. Furthermore, it allows the virtual idols to be registered as digital assets, enabling exchange and trading with other users.

[1724] Explanation of the program's processing

[1725] 1. Creating a virtual idol

[1726] user:

[1727] Users enter the necessary information into a virtual idol creation form using a dedicated application or web interface. This includes parameters such as the idol's gender, age, voice, and character design.

[1728] Terminal:

[1729] The terminal sends the parameters entered along with the generation request to the server.

[1730] server:

[1731] The server receives the request and starts the generation engine. Based on the provided parameters, the generation engine generates character data for a virtual idol and saves it to the database. A notification of completion is sent to the terminal, which then displays this to the user.

[1732] Specific example:

[1733] A user expresses a desire to create a virtual idol character who is a young woman with a cute, high-pitched voice. They enter the information into the creation form and press the "Generate" button. The server receives the information sent by the device, and the generation engine creates the character and saves it to the database. Finally, a completion notification is displayed to the user.

[1734] 2. Training and communication with virtual idols

[1735] user:

[1736] Users select training items and skill-building content from the application's training menu and submit requests. They can also interact with virtual idols using the communication menu.

[1737] Terminal:

[1738] The device sends nurturing and communication requests to the server.

[1739] server:

[1740] The server receives the request and updates the virtual idle parameters. The update results are saved to the database and sent to the terminal.

[1741] Specific example:

[1742] A user sends a request to a virtual idol asking for "singing training." The device sends the request to the server, which updates the virtual idol's skill parameters and displays the results to the user.

[1743] 3. Live streaming and interaction with fans

[1744] user:

[1745] Users schedule live streams and submit requests through the application. During the live stream, they can watch as the virtual idol reacts to comments from fans.

[1746] Terminal:

[1747] The device sends schedule information and commands to start and end the live stream to the server. During the live stream, comments from fans are displayed in real time.

[1748] server:

[1749] The server stores the live stream schedule in a database and prepares the appropriate streaming server. During the live stream, comments are processed in real time, and a natural language processing engine is used to ensure that virtual idlers respond. After the live stream ends, the live data is saved.

[1750] Specific example:

[1751] A user sets a schedule, such as "A virtual idol will perform a live show every Friday night at 8 PM." During the live show, comments from fans are displayed in real time, and the virtual idol responds. After the live show ends, the server saves the data.

[1752] 4. Converting virtual idols into NFTs and exchanging them

[1753] user:

[1754] The user enters the settings for creating an NFT of a virtual idle object in the application and submits the request.

[1755] Terminal:

[1756] The terminal sends a request to the server.

[1757] server:

[1758] The server retrieves character data, generates a smart contract to register it on the NFT marketplace, and registers the data on the blockchain. It receives exchange and purchase requests from other users and manages transactions. Successful transactions are recorded in the database, and the results are sent to the terminal.

[1759] Specific example:

[1760] The user enters information to convert their virtual idol into an NFT and submits a request. The server retrieves the character data, generates a smart contract to create the NFT, and registers the data on the blockchain. The transaction is successful, and the result is displayed to the user.

[1761] Through this detailed embodiment, users can efficiently perform a series of operations such as creating, training, live streaming, and converting virtual idols into NFTs.

[1762] The following describes the processing flow.

[1763] Virtual Idol Creation

[1764] Step 1:

[1765] The user opens a dedicated application or web interface and enters the necessary information (gender, age, voice, character design, etc.) into the virtual idol creation form.

[1766] Step 2:

[1767] The user presses the "Generate" button.

[1768] Step 3:

[1769] The terminal sends the parameters entered along with the generation request to the server.

[1770] Step 4:

[1771] The server receives the request and starts the generation engine.

[1772] Step 5:

[1773] The server passes user-provided parameters to the generation engine to generate character data for the virtual idol.

[1774] Step 6:

[1775] The server saves the generated character data to the database.

[1776] Step 7:

[1777] The server sends a notification to the device when generation is complete.

[1778] Step 8:

[1779] The device displays a message or preview to the user indicating that generation is complete.

[1780] Training and communication with virtual idols

[1781] Step 1:

[1782] Users select training items and skill development content from a training menu via the application and submit requests.

[1783] Step 2:

[1784] Users can use the communication menu to interact with virtual idols.

[1785] Step 3:

[1786] The terminal sends a request to the server.

[1787] Step 4:

[1788] The server receives the request and updates the virtual idle's parameters (e.g., skill level).

[1789] Step 5:

[1790] The server saves the updated status to the database.

[1791] Step 6:

[1792] The server sends the update results to the terminal.

[1793] Step 7:

[1794] The device displays the update results to the user.

[1795] Live streaming and interaction with fans

[1796] Step 1:

[1797] Users schedule live streams and submit requests through the application.

[1798] Step 2:

[1799] The terminal sends schedule information to the server.

[1800] Step 3:

[1801] The server saves the schedule to the database and prepares the appropriate delivery server.

[1802] Step 4:

[1803] The user sends the command to start the live stream.

[1804] Step 5:

[1805] The terminal sends a command to the server, and the live stream begins.

[1806] Step 6:

[1807] The server receives comments from fans in real time during the live stream, and a natural language processing engine is used to enable the virtual idol to respond appropriately.

[1808] Step 7:

[1809] The server saves the live data after the live stream ends.

[1810] Step 8:

[1811] The user sends a command to end the live session.

[1812] Step 9:

[1813] The terminal sends a command to the server, and the live session ends.

[1814] NFT conversion and exchange of virtual idols

[1815] Step 1:

[1816] The user enters the settings for creating an NFT of a virtual idle object in the application and submits the request.

[1817] Step 2:

[1818] The terminal sends a request to the server.

[1819] Step 3:

[1820] The server retrieves the character data and generates a smart contract to register it on the NFT marketplace.

[1821] Step 4:

[1822] The server registers the NFT data on the blockchain.

[1823] Step 5:

[1824] The server receives exchange and purchase requests from other users and manages the transactions.

[1825] Step 6:

[1826] The server records successful transactions in the database.

[1827] Step 7:

[1828] The terminal displays the transaction results to the user after the transaction is completed.

[1829] (Example 1)

[1830] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1831] Traditional virtual idol systems lacked an integrated platform that allowed users to smoothly perform a series of operations, such as creating, raising, live streaming, and converting characters into NFTs. Furthermore, they did not adequately provide features for generating real-time responses based on user input or managing character data as digital assets. As a result, users had to manage their virtual idols individually, leading to decreased operational efficiency and a poor user experience.

[1832] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1833] In this invention, the server includes means for the user to input information into a virtual idol generation form using a dedicated application or web interface, means for the terminal to send the input information to the server along with the generation request, and means for the server to start a generation engine, generate character data for the virtual idol based on the provided information, and store it in a database. This allows the user to perform a series of operations, from virtual idol generation and training to live streaming and NFT creation, in a unified manner.

[1834] A "user" refers to an entity that uses the virtual idol system to perform operations such as creating, training, live streaming, and converting virtual idols into NFTs.

[1835] A "terminal" refers to a device used by a user, such as a computer, smartphone, or tablet, that communicates with the virtual idle system.

[1836] A "server" is the central computer system of a virtual idle system, and it is a device that processes user requests and generates, stores, and manages data.

[1837] A "virtual idol" refers to a digitized character created using a generation engine, possessing character design, personality, voice, and other characteristics based on parameters provided by the user.

[1838] A "generation engine" is software or an algorithm used to generate character data for a virtual idol based on user input.

[1839] A "database" is an information management system used to store character data, live streaming data, NFT transaction information, and other data related to the virtual idols that have been created.

[1840] A "natural language processing engine" is software or an algorithm that analyzes text data from users and fans and generates appropriate responses.

[1841] "NFT" stands for "Non-Fungible Token," and refers to a unique digital certificate that registers a virtual idol as a digital asset on the blockchain.

[1842] A "smart contract" is a program that automatically executes transactions on the blockchain and is used for NFT creation of virtual idols and transaction management.

[1843] "Live streaming" refers to virtual idols broadcasting video and audio to viewers in real time via the internet.

[1844] "Training items" refer to specific activities and learning content aimed at improving and developing the skills of virtual idols.

[1845] System Overview

[1846] This invention is a system that generates virtual idols (AItubers) using a generation engine. Based on parameters provided by the user, it performs initial setup, enables live streaming, interaction with fans, training and skill improvement, and can even be converted into NFTs, allowing for exchange and trading with other users as digital assets.

[1847] Virtual Idol Creation

[1848] user:

[1849] Users enter information into a virtual idol creation form using a dedicated application or web interface. Specifically, they enter parameters such as gender, age, voice, and character design.

[1850] Terminal:

[1851] The terminal serializes the input information into JSON format and sends it to the server as a generation request.

[1852] server:

[1853] After the server receives the request, it starts the "AICharacterG engine," which is the generation engine, and generates character data based on the provided parameters. The generated character data is stored in a MySQL database. A notification of completion of generation is sent to the terminal and is eventually displayed to the user.

[1854] Specific example:

[1855] A user requests to create a virtual idol character who is a young woman with a cute, high-pitched voice. They enter the information and press the "Generate" button. The device sends the request, the server generates the character, and saves it to the database. A notification of completion is displayed to the user.

[1856] Prompt message:

[1857] "We're creating a virtual idol character who is a young woman with a cute, high-pitched voice."

[1858] Training virtual idols

[1859] user:

[1860] Users select training items and skill development content from the application's training menu and submit a request.

[1861] Terminal:

[1862] The device sends a training request to the server.

[1863] server:

[1864] The server receives the request and updates the virtual idle parameters. The update results are saved in the database and notified to the user.

[1865] Specific example:

[1866] A user sends a request to a virtual idol asking for "singing training." The device sends the request to the server, the server updates the virtual idol's skill parameters, and displays the results to the user.

[1867] Prompt message:

[1868] "We have started training to improve the singing skills of our virtual idols."

[1869] Live streaming and interaction with fans

[1870] user:

[1871] Users schedule live streams and submit requests through the application. During the live stream, they can watch as the virtual idol reacts to comments from fans.

[1872] Terminal:

[1873] The device sends schedule information and commands to start and end the live stream to the server. During the live stream, comments from fans are displayed in real time.

[1874] server:

[1875] The server saves the live streaming schedule to a database and prepares the appropriate streaming server. Comments during the live stream are processed in real time by a natural language processing engine, and a virtual idol responds. After the live stream ends, the live data is saved to the database.

[1876] Specific example:

[1877] A user sets a schedule, such as "A virtual idol will perform a live show every Friday night at 8 PM." During the live show, comments from fans are displayed in real time, and the virtual idol responds. After the live show ends, the server saves the data.

[1878] Prompt message:

[1879] "We will be holding a live stream of a virtual idol every Friday night at 8 PM."

[1880] NFT conversion and exchange of virtual idols

[1881] user:

[1882] The user enters the NFT settings for virtual idles in the application and sends the request.

[1883] Terminal:

[1884] The terminal sends a request to the server.

[1885] server:

[1886] The server retrieves character data, generates smart contracts, registers them on the NFT marketplace, and manages transactions. It receives exchange and purchase requests from other users, records successful transactions in a database, and notifies the user of the results.

[1887] Specific example:

[1888] The user enters information to convert their virtual idol into an NFT and submits a request. The server retrieves the character data, generates a smart contract to create the NFT, and registers the data on the blockchain. The transaction is successful, and the result is displayed to the user.

[1889] Prompt message:

[1890] "Turn virtual idols into NFTs and register them on a marketplace."

[1891] Through this detailed embodiment, users can efficiently perform a series of operations such as creating, training, live streaming, and converting virtual idols into NFTs.

[1892] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1893] Virtual Idol Creation

[1894] Step 1:

[1895] The user enters information.

[1896] Users open a dedicated application or web interface and enter information into a form for creating a virtual idol. This input data includes gender, age, voice, character design, and more.

[1897] input:

[1898] Parameters such as gender, age, voice, and character design

[1899] output:

[1900] Form data of the entered information

[1901] Specific actions:

[1902] The user opens the web interface in their browser and enters information into dropdown menus or text boxes.

[1903] Step 2:

[1904] The device sends the request

[1905] The terminal serializes the information entered by the user into JSON format and sends it to the server as a generation request.

[1906] input:

[1907] Form data entered by the user

[1908] output:

[1909] HTTP POST request to the server (data in JSON format)

[1910] Specific actions:

[1911] The terminal serializes the request data and sends an HTTP POST request to the specified endpoint on the server.

[1912] Step 3:

[1913] The server starts the generation engine.

[1914] After the server receives the request, it starts the generation engine (AICharacterG) and generates character data based on the provided parameters. The generated character data is saved in a temporary folder.

[1915] input:

[1916] The server receives a generation request (data in JSON format).

[1917] output:

[1918] Generated character data (file format)

[1919] Specific actions:

[1920] The server starts the generation engine, processes the character data based on the provided parameters, and saves it to a temporary folder.

[1921] Step 4:

[1922] The server saves the generated results and sends notifications.

[1923] The server saves the generated character data to a MySQL database and notifies the terminal when the generation is complete.

[1924] input:

[1925] Generated character data (file format)

[1926] output:

[1927] Character data stored in the database, generation completion notification (HTTP response)

[1928] Specific actions:

[1929] The server saves the character data to the database and sends an HTTP response to the terminal to notify it that generation is complete.

[1930] ---

[1931] Training virtual idols

[1932] Step 1:

[1933] The user selects a training menu.

[1934] Users select training items and skill development content from the application's training menu and submit a request.

[1935] input:

[1936] Training items and skill development content

[1937] output:

[1938] Training request (form data)

[1939] Specific actions:

[1940] The user selects "Singing Training" on the application screen and presses the "Submit" button.

[1941] Step 2:

[1942] The device sends the request

[1943] The device sends the user's selected training request to the server in JSON format.

[1944] input:

[1945] User-selected training request (form data)

[1946] output:

[1947] HTTP POST request to the server (data in JSON format)

[1948] Specific actions:

[1949] The terminal serializes the training request data and sends an HTTP POST request to the server.

[1950] Step 3:

[1951] The server updates the parameters.

[1952] The server receives the request data and updates the character data parameters according to the training details of the corresponding virtual idol. The update results are saved in the database and sent to the terminal as a notification.

[1953] input:

[1954] The server receives a training request (in JSON format).

[1955] output:

[1956] Updated character data stored in the database, update completion notification (HTTP response)

[1957] Specific actions:

[1958] The server updates the character data in the database and saves the new parameters. It then sends an update completion notification to the terminal as an HTTP response.

[1959] ---

[1960] Live streaming and interaction with fans

[1961] Step 1:

[1962] Users set up live streams

[1963] The user sets the live stream schedule in the application and sends the request to the server.

[1964] input:

[1965] Live stream schedule information (date, time, content)

[1966] output:

[1967] Live stream request (form data)

[1968] Specific actions:

[1969] The user sets "Live stream every Friday night at 8 PM" in the application and presses the "Schedule" button.

[1970] Step 2:

[1971] The device sends the request

[1972] The device sends the user's configured schedule information to the server in JSON format.

[1973] input:

[1974] Live schedule information set by the user (form data)

[1975] output:

[1976] HTTP POST request to the server (data in JSON format)

[1977] Specific actions:

[1978] The terminal serializes the live schedule information and sends an HTTP POST request to the server.

[1979] Step 3:

[1980] The server saves the schedule and starts delivery.

[1981] The server saves the live streaming schedule to the database and prepares the streaming server for live streaming. During the live stream, a natural language processing engine is used to process comments in real time, and a virtual idol responds. After the live stream ends, the live data is saved to the database.

[1982] input:

[1983] Live schedule information (in JSON format) received by the server.

[1984] output:

[1985] Live schedule information stored in the database, real-time processed comment data, and stored live data.

[1986] Specific actions:

[1987] The server saves the schedule to the database and configures the distribution server. During the live broadcast, comment data is analyzed using a natural language processing engine, and responses are generated in real time. After the live broadcast ends, all live data is saved to the database.

[1988] ---

[1989] NFT conversion and exchange of virtual idols

[1990] Step 1:

[1991] The user enters the NFT settings.

[1992] The user uses the application to input the information necessary to convert the virtual idle into an NFT and submits the request.

[1993] input:

[1994] Information required for NFT creation (character data, price, etc.)

[1995] output:

[1996] NFT conversion request (form data)

[1997] Specific actions:

[1998] The user clicks the "Convert to NFT" button, enters the required information, and clicks the "Submit" button.

[1999] Step 2:

[2000] The device sends the request

[2001] The device sends the NFT conversion request entered by the user to the server in JSON format.

[2002] input:

[2003] NFT conversion request (form data) entered by the user

[2004] output:

[2005] HTTP POST request to the server (data in JSON format)

[2006] Specific actions:

[2007] The device serializes the NFT data and sends an HTTP POST request to the server.

[2008] Step 3:

[2009] The server registers NFTs and manages transactions.

[2010] The server receives the request data, retrieves the virtual idol's character data, generates a smart contract, and converts it into an NFT. The generated NFT is registered on the blockchain, where exchange and purchase requests from other users are managed. Successful transactions are recorded in the database, and notifications are sent to the device.

[2011] input:

[2012] NFT conversion request (JSON format data) received by the server

[2013] output:

[2014] Generated smart contract, NFT registered on the blockchain, and transaction success notification (HTTP response).

[2015] Specific actions:

[2016] The server retrieves character data, generates a smart contract, and converts it into an NFT. The data is registered on the blockchain, and if the transaction is successful, the transaction information is recorded in the database. Finally, the transaction result is notified to the terminal.

[2017] (Application Example 1)

[2018] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[2019] To enhance the user experience, there is a need for a system that integrates many different functions, such as the creation and training of virtual idols, live streaming, and their management as digital assets. However, there is a lack of technical means to smoothly and efficiently implement these functions. In particular, a technical foundation is required to provide the advanced experiences that users expect, such as real-time responses during live streaming and blockchain-based management of virtual idols as digital assets.

[2020] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[2021] In this invention, the server includes means for generating virtual idols using a generation engine, means for performing initial settings on the virtual idols based on parameters provided by the user, means for the virtual idols to perform live streaming and interact with fans, means for registering the virtual idols as digital assets and enabling them to be exchanged or bought and sold with other users, means for registering the digital assets of the virtual idols on the blockchain, and means for setting the live streaming schedule of the virtual idols and providing real-time responses. This makes it possible for users to easily generate and nurture virtual idols, enjoy real-time interaction with fans during live streaming, and manage and trade them as digital assets.

[2022] A "generation engine" is a software or hardware component that automatically generates virtual idles based on parameters provided by the user.

[2023] A "virtual idol" is a character created using digital technology, a digital entity capable of live streaming and interacting with fans.

[2024] "User-provided parameters" refer to the set of information that users input to the generation engine, such as the gender, age, voice, and character design of a virtual idol.

[2025] "Initial setup" refers to the process of configuring the basic characteristics and functions of a generated virtual idol based on parameters provided by the user.

[2026] "Live streaming" is a feature that allows virtual idols to deliver content in real time and engage in interactive communication with viewers.

[2027] "Fan interaction" refers to a feature that allows virtual idols to interact with fans in real time or remotely through live streaming or other communication methods.

[2028] "Digital assets" refer to digital content, such as virtual idols, that can be treated as objects of ownership and trading.

[2029] Blockchain is a distributed ledger technology used to manage and trade digital currencies and digital assets.

[2030] "Real-time response" is a feature that allows virtual idols to instantly respond to comments and questions from fans during live streams.

[2031] This invention provides a system for users to create and nurture virtual idols and interact with fans through live streaming. The mechanism is described in detail below.

[2032] 1. System Configuration

[2033] Hardware and software:

[2034] Smartphone: A device used by users to control a virtual idle.

[2035] Mobile operating systems (iOS / Android): These are operating systems that run on smartphones.

[2036] Backend Server (Node.js, Express): This server is responsible for managing virtual idols, live streaming, and handling interactions with fans.

[2037] Database (MongoDB): This database is used to store virtual idol data and user data.

[2038] Blockchain (Ethereum, Smart Contracts): A distributed ledger technology for managing and trading virtual idols as digital assets.

[2039] Natural Language Processing (NLP) Engine (Google Cloud NLP): A natural language processing tool for generating real-time responses in communication with fans.

[2040] 2. Creating a virtual idol

[2041] User actions:

[2042] Users input parameters such as the gender, age, voice, and character design of a virtual idol through a smartphone app. This sends a generation request to the backend server.

[2043] Server processing:

[2044] The server starts the generation engine based on the received parameters and generates character data for the virtual idol. This data is stored in the database, and a notification of completion is sent to the smartphone app.

[2045] 3. Training and communication with virtual idols

[2046] User actions:

[2047] Users select training items and skill-up content via a smartphone app and submit training requests. They can also interact with virtual idols using the communication menu.

[2048] Server processing:

[2049] The server receives training and communication requests and updates the virtual idol's parameters. This result is stored in a database, and the updated information is sent to the smartphone.

[2050] 4. Live streaming and interaction with fans

[2051] User actions:

[2052] Users can schedule live streams and submit requests using a smartphone app. During the live stream, comments from fans are displayed in real time.

[2053] Server processing:

[2054] The server stores the live stream schedule in a database and prepares the appropriate streaming server. During the stream, comments are processed in real time, and a virtual idler responds using a natural language processing (NLP) engine. After the stream ends, the live data is stored in the database.

[2055] 5. Converting virtual idols into NFTs and exchanging them.

[2056] User actions:

[2057] Users submit an NFT request for a virtual idol via a smartphone app and enter the necessary information.

[2058] Server processing:

[2059] The server retrieves character data and generates a smart contract to register it on the NFT marketplace. The data is then registered on the blockchain. Transaction requests are received, successful transactions are recorded in the database, and the results are sent to the user.

[2060] Specific example

[2061] Example 1:

[2062] User A uses a smartphone app to create a young female character with an "idol cafe" theme and requests singing training.

[2063] Prompt message:

[2064] "I'm a 19-year-old female character, please create a virtual idol with a high-pitched voice."

[2065] Example 2:

[2066] User B sets up a live stream every Friday night at 8 PM to interact with fans in real time.

[2067] Prompt message:

[2068] "Please set up a live stream every Friday night at 8 PM and respond to fan comments in real time."

[2069] Example 3:

[2070] User C registers the virtual idol they have trained as an NFT on the marketplace and trades it there.

[2071] Prompt message:

[2072] "Please register my virtual idol as an NFT on the blockchain and make it tradable."

[2073] This will allow users to easily create and nurture virtual idols, enjoy real-time interaction with fans, and manage and trade them as digital assets.

[2074] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[2075] Step 1:

[2076] User submits a request to generate a virtual idol.

[2077] The user launches a smartphone app and enters parameters such as gender, age, voice, and character design into a virtual idol creation form. The device sends the entered parameters along with the creation request to the backend server.

[2078] Input: Parameters entered by the user, such as gender, age, voice, and character design.

[2079] Output: The generation request is sent to the backend server.

[2080] Step 2:

[2081] Server-driven generation of virtual idles

[2082] The server starts the generation engine based on the received generation request and generates character data for a virtual idol based on the provided parameters. This data is stored in the database.

[2083] Input: Parameters included in the generation request

[2084] Output: The virtual idol's character data and a generation completion notification are saved to the database and sent to the terminal.

[2085] Step 3:

[2086] User submission of a request to train a virtual idol

[2087] Users select training items and skill development content from the training menu in the smartphone app and send training requests through the app. The device then sends the request to the backend server.

[2088] Input: Training items and skill development content selected by the user.

[2089] Output: Training request is sent to the backend server.

[2090] Step 4:

[2091] Server updates of virtual idol training data

[2092] The server receives the training request and updates the virtual idol's parameters. The update results are saved in the database and sent to the terminal.

[2093] Input: Training items and skill-up content included in the training request

[2094] Output: Updated virtual idle parameters, saved to the database and sent to the terminal.

[2095] Step 5:

[2096] User-controlled live streaming scheduling

[2097] The user sets a live stream schedule using a smartphone app and sends a schedule request. The device then sends the request to the backend server.

[2098] Input: User-defined live streaming schedule

[2099] Output: The schedule request is sent to the backend server.

[2100] Step 6:

[2101] Preparing the server for live streaming

[2102] The server stores the schedule request in the database and prepares the appropriate streaming server. During the live stream, comments from fans are displayed in real time, and the virtual idol responds using a natural language processing engine.

[2103] Input: Live stream schedule

[2104] Output: Live stream preparation, real-time display and response to fan comments.

[2105] Step 7:

[2106] User submits NFT request

[2107] The user submits an NFT request for a virtual idol via a smartphone app. After entering the necessary information, the device sends the request to the backend server.

[2108] Input: Information of the virtual idol to be converted to NFT.

[2109] Output: An NFT request is sent to the backend server.

[2110] Step 8:

[2111] NFT creation and transaction management via server

[2112] The server retrieves character data and generates and executes a smart contract to register it on the NFT marketplace. It registers the data on the blockchain, receives transaction requests, records successful transactions in the database, and sends the results to the terminal.

[2113] Input: Information about the virtual idol included in the NFT request

[2114] Output: Execution of NFT creation via smart contract, registration of data to the blockchain, notification of transaction results.

[2115] This allows users to easily create and nurture virtual idols, enjoy real-time interaction with fans, and even manage and trade them as digital assets.

[2116] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[2117] System Overview

[2118] This invention is a system that generates virtual idols using a generation engine, performs initial setup based on user-provided parameters, and enables live streaming and interaction with fans. Furthermore, it allows virtual idols to be registered as digital assets, enabling exchange and trading with other users. In addition, by combining it with an emotion engine that recognizes user emotions, the system can adjust the virtual idol's responses and actions based on the user's emotions.

[2119] Explanation of the program's processing

[2120] 1. Creating a virtual idol

[2121] user:

[2122] The user opens a dedicated application or web interface and enters the necessary information (gender, age, voice, character design, etc.) into the virtual idol creation form. This includes parameters such as the idol's gender, age, voice, and character design.

[2123] Terminal:

[2124] The terminal sends the parameters entered along with the generation request to the server.

[2125] server:

[2126] The server receives the request and starts the generation engine. The generation engine generates character data for a virtual idol based on the provided parameters and saves it to the database. A notification of completion of generation is sent to the terminal, and the terminal displays this to the user.

[2127] Specific example:

[2128] A user expresses a desire to create a virtual idol character who is a young woman with a cute, high-pitched voice. They enter the information into the creation form and press the "Generate" button. The server receives the information sent by the device, and the generation engine creates the character and saves it to the database. Finally, a completion notification is displayed to the user.

[2129] 2. Training and communication with virtual idols

[2130] user:

[2131] Users select training items and skill-building content from the application's training menu and submit requests. They can also interact with virtual idols using the communication menu.

[2132] Terminal:

[2133] The device sends nurturing and communication requests to the server.

[2134] server:

[2135] The server receives the request and updates the virtual idle's parameters (e.g., skill level). It saves the update results to the database and sends them to the terminal.

[2136] Specific example:

[2137] A user sends a request to a virtual idol asking for "singing training." The device sends the request to the server, which updates the virtual idol's skill parameters and displays the results to the user.

[2138] 3. Live streaming and interaction with fans

[2139] user:

[2140] Users schedule live streams and submit requests through the application. During the live stream, they can watch as the virtual idol reacts to comments from fans.

[2141] Terminal:

[2142] The device sends schedule information and commands to start and end the live stream to the server. During the live stream, comments from fans are displayed in real time.

[2143] server:

[2144] The server stores the live stream schedule in a database and prepares the appropriate streaming server. During the live stream, comments are processed in real time, and a natural language processing engine is used to ensure that virtual idlers respond. After the live stream ends, the live data is saved.

[2145] Specific example:

[2146] A user sets a schedule, such as "A virtual idol will perform a live show every Friday night at 8 PM." During the live show, comments from fans are displayed in real time, and the virtual idol responds. After the live show ends, the server saves the data.

[2147] 4. Converting virtual idols into NFTs and exchanging them

[2148] user:

[2149] The user enters the settings for creating an NFT of a virtual idle object in the application and submits the request.

[2150] Terminal:

[2151] The terminal sends a request to the server.

[2152] server:

[2153] The server retrieves character data, generates a smart contract to register it on the NFT marketplace, and registers the data on the blockchain. It receives exchange and purchase requests from other users and manages transactions. Successful transactions are recorded in the database, and the results are sent to the terminal.

[2154] Specific example:

[2155] The user enters information to convert their virtual idol into an NFT and submits a request. The server retrieves the character data, generates a smart contract to create the NFT, and registers the data on the blockchain. The transaction is successful, and the result is displayed to the user.

[2156] 5. Implementation of the emotion engine

[2157] user:

[2158] Users interact and communicate with virtual idols through the application.

[2159] Terminal:

[2160] The device transmits conversation and interaction data to the server in real time.

[2161] server:

[2162] The server uses an emotion engine to analyze the user's emotions from their conversations and interactions. The emotion engine analyzes input data (voice, text, facial expressions, etc.) to identify the user's emotional state.

[2163] server:

[2164] The server adjusts the virtual idle's responses based on the emotion engine's analysis results. Specifically, it provides encouraging messages if the user is sad, and jokes or positive topics if the user is happy.

[2165] server:

[2166] The server saves the sentiment analysis results to a database and sends them to the application.

[2167] Terminal:

[2168] The device displays the sentiment analysis results and adjusted responses to the user. This is provided to the user as a real-time response from a virtual idler.

[2169] Specific example:

[2170] During a chat session with a virtual idol, the user types, "I'm a little tired today." The device sends this message to the server, where an emotion engine analyzes the user's emotions, such as "tired" or "depressed." Based on this, the server adjusts the response to the virtual idol, sending an encouraging message such as, "Take it easy today!" The result is displayed to the user through the device, and the user receives words of encouragement from the virtual idol.

[2171] Through this detailed embodiment, users can efficiently perform a series of operations, including generating, training, live streaming, NFT creation, and interaction using emotion recognition for virtual idols.

[2172] The following describes the processing flow.

[2173] Virtual Idol Creation

[2174] Step 1:

[2175] The user opens a dedicated application or web interface and enters the necessary information (gender, age, voice, character design, etc.) into the virtual idol creation form.

[2176] Step 2:

[2177] The user presses the "Generate" button.

[2178] Step 3:

[2179] The terminal sends the parameters entered along with the generation request to the server.

[2180] Step 4:

[2181] The server receives the request and starts the generation engine.

[2182] Step 5:

[2183] The server passes user-provided parameters to the generation engine to generate character data for the virtual idol.

[2184] Step 6:

[2185] The server saves the generated character data to the database.

[2186] Step 7:

[2187] The server sends a notification to the device when generation is complete.

[2188] Step 8:

[2189] The device displays a message or preview to the user indicating that generation is complete.

[2190] Training and communication with virtual idols

[2191] Step 1:

[2192] Users select training items and skill development content from the application's training menu and submit a request.

[2193] Step 2:

[2194] Users can use the communication menu to interact with virtual idols.

[2195] Step 3:

[2196] The device sends nurturing and communication requests to the server.

[2197] Step 4:

[2198] The server receives the request and updates the virtual idle's parameters (e.g., skill level).

[2199] Step 5:

[2200] The server saves the updated status to the database.

[2201] Step 6:

[2202] The server sends the update results to the terminal.

[2203] Step 7:

[2204] The device displays the update results to the user.

[2205] Live streaming and interaction with fans

[2206] Step 1:

[2207] Users schedule live streams and submit requests through the application.

[2208] Step 2:

[2209] The terminal sends schedule information to the server.

[2210] Step 3:

[2211] The server saves the schedule to the database and prepares the appropriate delivery server.

[2212] Step 4:

[2213] The user sends the command to start the live stream.

[2214] Step 5:

[2215] The terminal sends a command to the server, and the live stream begins.

[2216] Step 6:

[2217] The server receives comments from fans in real time during the live stream, and a natural language processing engine is used to enable the virtual idol to respond appropriately.

[2218] Step 7:

[2219] The server saves the live data after the live stream ends.

[2220] Step 8:

[2221] The user sends a command to end the live session.

[2222] Step 9:

[2223] The terminal sends a command to the server, and the live session ends.

[2224] NFT conversion and exchange of virtual idols

[2225] Step 1:

[2226] The user enters the settings for creating an NFT of a virtual idle object in the application and submits the request.

[2227] Step 2:

[2228] The terminal sends a request to the server.

[2229] Step 3:

[2230] The server retrieves the character data and generates a smart contract to register it on the NFT marketplace.

[2231] Step 4:

[2232] The server registers the NFT data on the blockchain.

[2233] Step 5:

[2234] The server receives exchange and purchase requests from other users and manages the transactions.

[2235] Step 6:

[2236] The server records successful transactions in the database.

[2237] Step 7:

[2238] The terminal displays the transaction results to the user after the transaction is completed.

[2239] Implementation of an emotion engine

[2240] Step 1:

[2241] Users interact and communicate with virtual idols through the application.

[2242] Step 2:

[2243] The device transmits conversation and interaction data to the server in real time.

[2244] Step 3:

[2245] The server uses an emotion engine to analyze user emotions from their conversations and interactions.

[2246] Step 4:

[2247] The server adjusts the virtual idle's response based on the analysis results of the emotion engine.

[2248] Step 5:

[2249] The server saves the emotion analysis results to a database.

[2250] Step 6:

[2251] The device displays the sentiment analysis results and adjusted responses to the user.

[2252] Specific example:

[2253] During a chat session with a virtual idol, the user types, "I'm a little tired today." The device sends this message to the server, where an emotion engine analyzes the user's emotions, such as "tired" or "depressed." Based on this, the server adjusts the response to the virtual idol, sending an encouraging message such as, "Take it easy today!" The result is displayed to the user through the device, and the user receives words of encouragement from the virtual idol.

[2254] (Example 2)

[2255] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[2256] Modern users demand more personalized entertainment experiences, with interactive experiences through live streaming and conversations with virtual idols being particularly important. However, current systems struggle to provide consistent management and high flexibility in areas such as virtual idol creation, training, live streaming, fan interaction, and even handling as digital assets. Furthermore, recognizing user emotions and generating appropriate responses is difficult. This results in a limited user experience and a reduced appeal of the system.

[2257] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for generating virtual idols using a generation engine, means for initializing the terminal based on parameters provided by the user, means for providing live streaming services and interacting with the user, means for registering them as digital assets and enabling exchange or trading with other users, means for receiving generation requests and sending notifications of generation completion, and means for analyzing the user's emotions using an emotion recognition engine and adjusting the response content. This makes it possible for the user to consistently generate, nurture, live stream, handle virtual idols as digital assets, and engage in interactive dialogue using emotion recognition.

[2258] A "generation engine" is a software or hardware means for generating character data for a virtual idol based on parameters provided by the user.

[2259] A "virtual idol" is a digital character generated based on user input, providing an interactive entertainment experience through live streaming and dialogue.

[2260] "Parameters" are the information that users provide when creating a virtual idol, and examples include gender, age, voice, and character design.

[2261] "Initial setup" is the process of configuring the basic attributes and characteristics of a virtual idle based on parameters provided by the user.

[2262] "Live streaming" refers to a type of streaming service where virtual idols interact with users and fans in real time.

[2263] "Dialogue" refers to the act of a virtual idol interacting with users or fans using text or voice.

[2264] "Digital assets" refer to virtual idols and other digital items that are registered in a form that can be owned and traded as digital information.

[2265] An "emotion recognition engine" is a software or hardware system that analyzes emotions from user input data (such as voice, text, and facial expressions) and generates an appropriate response based on that analysis.

[2266] "Real-time" means that processing and responses are performed instantly with minimal delay.

[2267] This invention is a system that generates virtual idols using a generation engine and an emotion recognition engine, performs initial setup based on parameters provided by the user, and conducts live streaming and interactions with fans. Furthermore, it allows the virtual idols to be registered as digital assets, enabling exchange and trading with other users.

[2268] System Configuration

[2269] 1. Creating a virtual idol

[2270] User:

[2271] Users enter the necessary information into a virtual idol creation form using a dedicated application or web interface. This information includes gender, age, voice, and character design. A specific example of user input might be, "I want to create a virtual idol who is a young woman with a cute, high-pitched voice."

[2272] Terminal:

[2273] The terminal sends the entered parameters to the server as a generation request. This can be done using either an HTTP request or a WebSocket.

[2274] server:

[2275] When the server receives a generation request, it starts the generation engine. The generation engine generates character data for a virtual idol based on the provided parameters and saves it to the database. It also sends a notification to the terminal when generation is complete, and the terminal displays this to the user.

[2276] Specific example:

[2277] The user enters "I want to create a virtual idol character who is a young woman with a cute, high-pitched voice" and presses the "Generate" button. The server receives the information sent by the terminal, the generation engine generates the character, and saves it to the database. A notification of completion is displayed to the user.

[2278] 2. Training and communication with virtual idols

[2279] User:

[2280] Users select training items and skill-building content from the application's training menu and submit requests. They can also interact with virtual idols using the communication menu.

[2281] Terminal:

[2282] The device sends training and communication requests to the server. These requests include the data ID and training items of the target virtual idol.

[2283] server:

[2284] The server receives the request and updates the virtual idle parameters based on the specified training items. A database write operation is performed, the update results are sent to the terminal, and the terminal displays them to the user.

[2285] Specific example:

[2286] A user sends a request to a virtual idol asking for "singing training." The device sends the request to the server, the server updates the virtual idol's skill parameters, and displays the results to the user.

[2287] 3. Live streaming and interaction with fans

[2288] User:

[2289] Users can schedule live streams and submit requests through the application. For example, they might set up a live stream every Friday at 8 PM.

[2290] Terminal:

[2291] The terminal sends schedule information to the server. The request includes the date and time and the virtual idol's data ID.

[2292] server:

[2293] The server stores schedule information in a database and prepares for live streaming. It starts streaming at the scheduled time, processes fan comments in real time, and the virtual idol responds using a natural language processing engine. After the live stream ends, it saves the live data.

[2294] Specific example:

[2295] A user sets a schedule, such as "A virtual idol will perform a live show every Friday night at 8 PM." During the live show, comments from fans are displayed in real time, and the virtual idol responds. After the live show ends, the server saves the data.

[2296] 4. Converting virtual idols into NFTs and exchanging them

[2297] User:

[2298] The user enters the settings for creating an NFT of a virtual idle character in the application and submits the request. The provided settings include the price and description of the NFT.

[2299] Terminal:

[2300] The device sends an NFT request to the server. This request includes the virtual idol's data ID and detailed information about the NFT creation.

[2301] server:

[2302] The server receives the request, retrieves the character data, and generates a smart contract to register it on the NFT marketplace. The generated data is then registered on the blockchain to manage exchange and purchase requests.

[2303] Specific example:

[2304] The user enters information to convert their virtual idol into an NFT and submits a request. The server retrieves the character data, generates a smart contract to create the NFT, and registers the data on the blockchain. The transaction is successful, and the result is displayed to the user.

[2305] 5. Implementation of the emotion engine

[2306] User:

[2307] Users interact and converse with virtual idols through the application. For example, they might type a message like, "I'm a little tired today."

[2308] Terminal:

[2309] The terminal transmits conversation and interaction data to the server in real time. Specifically, it transfers the entered message data to the server.

[2310] server:

[2311] The server uses an emotion engine to analyze user emotions from their conversations and interactions. This analysis includes various input data such as voice, text, and facial expressions. Based on the emotion analysis results, the server adjusts the virtual idol's responses and sends the results to the terminal.

[2312] Specific example:

[2313] During a chat session with a virtual idol, the user types, "I'm a little tired today." The device sends this message to the server, where the emotion engine analyzes emotions such as "tired" and "depressed." Based on this, the server adjusts the response to the virtual idol to send an encouraging message such as, "Take it easy today!" The result is displayed to the user through the device, and the user receives words of encouragement from the virtual idol.

[2314] Example prompts for generative AI models:

[2315] "Please create a virtual idol character who is a young woman with a cute, high-pitched voice."

[2316] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2317] Step 1:

[2318] User:

[2319] The user opens a dedicated application or web interface and enters the necessary information (gender, age, voice, character design, etc.) into the virtual idol creation form.

[2320] Input: Gender, age, voice, character design

[2321] Output: The user's input information is reflected in the form.

[2322] Specific actions:

[2323] The user expresses a desire to create a virtual idol character who is a young woman with a cute, high-pitched voice, and enters specific information in each field.

[2324] Step 2:

[2325] Terminal:

[2326] The terminal sends the parameters entered by the user to the server as a generation request.

[2327] Input: User input parameters

[2328] Output: The generation request is sent to the server.

[2329] Specific actions:

[2330] The terminal sends user input information to the server using an HTTP request or WebSocket.

[2331] Step 3:

[2332] server:

[2333] The server receives the generation request and starts the generation engine. The generation engine generates character data for a virtual idol based on the provided parameters and saves it to the database.

[2334] Input: Generation request and user input parameters

[2335] Output: Virtual idol character data

[2336] Specific actions:

[2337] The server starts the generation engine and generates character data for a virtual idol based on the parameters provided by the user. The character data is then saved to the database.

[2338] Step 4:

[2339] server:

[2340] The server sends a notification to the device when generation is complete.

[2341] Input: Generation completion information

[2342] Output: Generation completion notification

[2343] Specific actions:

[2344] After the generation engine finishes, the server creates a generation completion notification and sends it to the terminal.

[2345] Step 5:

[2346] Terminal:

[2347] The terminal displays a generation completion notification received from the server to the user.

[2348] Input: Generation completion notification

[2349] Output: Display of generation complete messages

[2350] Specific actions:

[2351] The device displays a message to the user such as, "Virtual idle creation complete."

[2352] Step 6:

[2353] User:

[2354] Users select training items and skill development content from the application's training menu and submit a request.

[2355] Input: Training items and skill development content

[2356] Output: Generation of training requests

[2357] Specific actions:

[2358] The user selects and submits a specific training request, such as "I would like singing training."

[2359] Step 7:

[2360] Terminal:

[2361] The device sends a training request to the server.

[2362] Input: Training Request

[2363] Output: Sending a request

[2364] Specific actions:

[2365] The device sends a training request to the server using an HTTP request or WebSocket.

[2366] Step 8:

[2367] server:

[2368] The server receives a training request and updates the virtual idol's parameters based on the specified training items. The update results are saved to the database.

[2369] Input: Training Request

[2370] Output: Updated parameters

[2371] Specific actions:

[2372] The server updates the skill level of the virtual idlers based on the training items and saves the results to the database.

[2373] Step 9:

[2374] server:

[2375] The server sends the update results to the terminal.

[2376] Input: Updated parameters

[2377] Output: Update result notification

[2378] Specific actions:

[2379] The server sends an update result notification to the terminal.

[2380] Step 10:

[2381] Terminal:

[2382] The device displays the update results to the user.

[2383] Input: Update result notification

[2384] Output: Display of update complete messages

[2385] Specific actions:

[2386] The device displays a message to the user such as, "Singing training is complete, and your skill level has increased."

[2387] Step 11:

[2388] User:

[2389] Users schedule live streams and submit requests through the application.

[2390] Input: Live Streaming Schedule

[2391] Output: Generation of schedule request

[2392] Specific actions:

[2393] The user enters a schedule, stating, "A virtual idol will perform live every Friday night at 8 PM," and submits it.

[2394] Step 12:

[2395] Terminal:

[2396] The terminal sends schedule information to the server.

[2397] Input: Schedule information

[2398] Output: Sending a schedule request

[2399] Specific actions:

[2400] The terminal sends schedule information to the server using an HTTP request or WebSocket.

[2401] Step 13:

[2402] server:

[2403] The server stores the schedule information in a database and then prepares the appropriate distribution server.

[2404] Input: Schedule information

[2405] Output: Saved schedule information, delivery readiness status

[2406] Specific actions:

[2407] The server saves the schedule information to the database and prepares for the live stream.

[2408] Step 14:

[2409] server:

[2410] The server will begin live streaming at the scheduled start time. During the live stream, comments from fans will be processed in real time, and the virtual idol will respond using a natural language processing engine.

[2411] Input: Live stream comments

[2412] Output: Real-time response

[2413] Specific actions:

[2414] The server starts streaming, analyzing comments from fans in real time and generating responses.

[2415] Step 15:

[2416] Terminal:

[2417] The device displays comments from fans and responses from the virtual idol in real time.

[2418] Input: Real-time response

[2419] Output: Display of comments and responses

[2420] Specific actions:

[2421] The device displays comments and virtual idle responses to the user in real time.

[2422] Step 16:

[2423] server:

[2424] After the live stream ends, the server saves the live data.

[2425] Input: Live data

[2426] Output: Saved live data

[2427] Specific actions:

[2428] The server saves the live stream data to the database.

[2429] Step 17:

[2430] User:

[2431] The user enters the settings for creating an NFT of a virtual idle in the application and submits the request.

[2432] Input: NFT configuration information

[2433] Output: Generation of NFT request

[2434] Specific actions:

[2435] The user enters and submits the necessary information to convert the virtual idle into an NFT.

[2436] Step 18:

[2437] Terminal:

[2438] The device sends an NFT request to the server.

[2439] Input: NFT configuration information

[2440] Output: Sending an NFT request

[2441] Specific actions:

[2442] The device sends an NFT creation request to the server using an HTTP request or WebSocket.

[2443] Step 19:

[2444] server:

[2445] The server receives the request, retrieves the character data, and generates a smart contract to register it on the NFT marketplace. The generated data is then registered on the blockchain.

[2446] Input: NFT request

[2447] Output: Smart contracts, data registered on the blockchain

[2448] Specific actions:

[2449] The server generates a smart contract, converts the virtual idle into an NFT, and registers it on the blockchain.

[2450] Step 20:

[2451] server:

[2452] It receives exchange and purchase requests from other users and manages transactions. If a transaction is successful, it records the result in the database and sends the result to the terminal.

[2453] Input: Exchange request, Purchase request

[2454] Output: Transaction results

[2455] Specific actions:

[2456] The server processes the transaction request, records successful transactions in the database, and sends the results to the terminal.

[2457] Step 21:

[2458] Terminal:

[2459] Display the transaction results to the user.

[2460] Input: Transaction result

[2461] Output: Display of results

[2462] Specific actions:

[2463] The terminal displays the transaction results to the user, and if successful, ownership of the virtual idol is transferred to the user.

[2464] Step 22:

[2465] User:

[2466] Users interact and communicate with virtual idols through the application.

[2467] Input: User interactions and messages

[2468] Output: Dialogue data

[2469] Specific actions:

[2470] The user enters a message through the application such as, "I'm a little tired today."

[2471] Step 23:

[2472] Terminal:

[2473] The device transmits conversation and interaction data to the server in real time.

[2474] Input: User input data

[2475] Output: Sending dialogue data

[2476] Specific actions:

[2477] The terminal sends interaction data to the server using an HTTP request or WebSocket.

[2478] Step 24:

[2479] server:

[2480] The server uses an emotion engine to analyze user emotions from their conversations and interactions. Based on the analysis results, it adjusts the virtual idle's responses.

[2481] Input: Dialogue data

[2482] Output: Sentiment analysis results, adjusted response content

[2483] Specific actions:

[2484] The server activates the emotion engine, analyzes the user's dialogue data to identify emotions, and adjusts the virtual idle's response based on the analysis results.

[2485] Step 25:

[2486] server:

[2487] The server sends the adjusted response to the application.

[2488] Input: Adjusted response content

[2489] Output: Sending response data

[2490] Specific actions:

[2491] The server sends the generated response to the application.

[2492] Step 26:

[2493] Terminal:

[2494] The terminal displays the response to the user.

[2495] Input: Response data

[2496] Output: Display response

[2497] Specific actions:

[2498] The device displays a response message to the user such as, "Take a good rest today!"

[2499] (Application Example 2)

[2500] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[2501] Conventional virtual character systems, while capable of generating and distributing characters and interacting with viewers, lacked the ability to recognize the user's emotional state in real time and provide appropriate responses based on that state. As a result, virtual character responses were uniform and fixed, leading to a decline in the quality of user interaction. Furthermore, when registering virtual characters as digital assets and exchanging or trading them with other users, the lack of systems incorporating emotion recognition meant that users were not adequately provided with a sense of presence or personalization.

[2502] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating a virtual character using a generation engine, means for performing initial settings on the virtual character based on attributes provided by the user, means for the virtual character to distribute information in real time within a virtual environment and interact with viewers, means for registering the virtual character as a digital asset and enabling it to be exchanged or bought and sold with other users, means for analyzing the user's emotional state, and means for adjusting the virtual character's response based on the emotional state. As a result, it becomes possible to recognize the user's emotional state in real time and provide an appropriate response in accordance with that state, thereby improving the quality of interaction with the user and enabling the provision of a more immersive and personalized experience.

[2503] A "generation engine" is a system that has algorithms and functions for generating virtual characters based on attribute data provided by the user.

[2504] A "virtual character" is a digital character created by a user using a generation engine, and is an entity that can operate within a virtual environment.

[2505] "User" refers to an individual or group that uses this system to create, develop, and manage virtual characters.

[2506] "Attributes" refer to basic information about a virtual character, such as gender, age, voice, and character design.

[2507] "Initial setup" refers to the process of setting the basic properties and characteristics of a virtual character based on attributes provided by the user.

[2508] A "virtual environment" is a digital space provided on the internet or other digital platforms for virtual characters to operate.

[2509] "Real time" refers to the fact that the activities and communications of virtual characters occur simultaneously with real time.

[2510] "Digital assets" are assets in which virtual characters are recorded as data, possess a specific value, and can be bought, sold, or exchanged.

[2511] "Emotional state" refers to the emotional state of a user as perceived based on their comments and actions.

[2512] "Adjusting responses" means controlling the virtual character to return reactions and messages that are appropriate to the user's emotional state.

[2513] The system for implementing this invention utilizes a combination of a generation engine, an emotion analysis engine, and a natural language processing engine. It generates a virtual character based on user-provided attributes, and this character delivers information in real time, interacts with viewers, and responds based on the user's emotional state. Furthermore, the generated virtual character can be registered as a digital asset and exchanged or bought and sold with other users. This system uses hardware such as a smartphone, head-mounted display (HMD), server, and database, along with software such as a generation AI model, a natural language processing engine, and an emotion analysis engine.

[2514] First, the user opens a dedicated application and enters the necessary information into the virtual character generation form. This information includes attributes such as gender, age, voice, and character design. The information entered by the user is sent from the terminal to the server, which then starts the generation engine. The generation engine generates a virtual character based on the provided attribute data and saves that data to a database. After generation is complete, the result is sent to the terminal and displayed to the user.

[2515] Next, when developing or improving the skills of a virtual character, the user selects training items and skill-up content from the application's development menu and sends a request. This request is sent from the terminal to the server, which updates the virtual character's parameters (e.g., skill level). The update results are stored in the database and displayed to the user through the terminal.

[2516] Furthermore, when live streaming, users schedule the live stream within the application and submit a request. During the live stream, a virtual character reacts to comments from viewers. The server processes comments in real time, and an emotion analysis engine analyzes the user's emotions from the comments. Based on this analysis, a natural language processing engine generates an appropriate response, which the virtual character displays.

[2517] Finally, when registering a virtual character as a digital asset, the user enters the necessary information in the application and sends a request to the server. The server retrieves the character data, generates a smart contract to register it on the NFT marketplace, and registers the data on the blockchain. It receives exchange and purchase requests from other users, records successful transactions in the database, and sends the results to the user.

[2518] For example, when a user performs the action of "Create a new virtual character," it sends a prompt to the generation engine saying, "Create a virtual character that is a young woman with a cute, high-pitched voice." The generation engine generates a character based on this prompt and saves it to the database. The generated character is then displayed on the user's device. If the virtual character comments "I'm a little tired today" during a live stream, the sentiment analysis engine identifies emotions such as "tired" and "depressed" and sends back an encouraging message such as "Take it easy today!"

[2519] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2520] Step 1:

[2521] The user enters attributes into a virtual character generation form and submits a generation request. Specifically, the user fills in information such as gender, age, voice, and character design into the application's input form and presses the submit button. The input data is sent from the terminal to the server. The attribute data entered by the user is received as input and sent to the server as output.

[2522] Step 2:

[2523] The server starts the generation engine and generates a virtual character based on the provided attributes. The server receives attribute data sent by the user and sends prompt messages to the generation AI model based on that data. An example of a prompt message would be, "Create a virtual character that is a young woman with a cute, high-pitched voice." The generation engine generates the virtual character and saves its data to the database. The system takes user attribute data as input and saves the generated virtual character data to the database as output.

[2524] Step 3:

[2525] The terminal displays a generation completion notification to the user. A generation completion notification is sent from the server to the terminal, and the terminal displays it to the user. It receives a generation completion notification as input and displays the notification to the user as output.

[2526] Step 4:

[2527] Users submit requests for training and skill improvement of their virtual characters. Users select their desired training items and skill development content from the application's training menu and submit their requests. The application receives user training requests as input and sends them to the server as output.

[2528] Step 5:

[2529] The server updates the virtual character's parameters. The server receives requests from users and updates the virtual character data in the database. For example, if the skill level increases. It receives training requests as input and the updated character data is saved to the database as output.

[2530] Step 6:

[2531] The terminal displays the update results to the user. The server sends the update results to the terminal, and the terminal displays them to the user. It receives the update results as input and displays them to the user as output.

[2532] Step 7:

[2533] The user schedules a live stream and submits a request. The user sets the schedule for the live stream in the application and submits the request. The application receives the schedule information as input and sends it to the server as output.

[2534] Step 8:

[2535] The server prepares for live streaming and saves schedule information to the database. The server receives schedule information sent by the user and prepares the appropriate streaming server. Schedule information is received as input, and the streaming server configuration and its saving to the database are output.

[2536] Step 9:

[2537] During a live stream, the server receives comments from users, and an emotion analysis engine analyzes them. The server sends the received comments to the emotion analysis engine in real time. The emotion analysis engine analyzes the emotions from the comments and returns the results to the server. User comments are received as input, and the emotion analysis results are returned as output.

[2538] Step 10:

[2539] The server uses a natural language processing engine to generate a response from a virtual character based on the sentiment analysis results. The server sends a prompt to the natural language processing engine based on the sentiment analysis results, and generates an appropriate response. For example, if the comment is "I'm a little tired today," the sentiment analysis engine identifies "tiredness," and the natural language processing engine generates an encouraging message such as "Take it easy today!" The server receives the sentiment analysis results as input and returns the generated response message as output.

[2540] Step 11:

[2541] The terminal displays the generated response to the user. A response message is sent from the server to the terminal, and the terminal displays it to the user in real time. It receives a response message as input and displays it to the user as output.

[2542] Step 12:

[2543] A user submits a request to convert a virtual character to an NFT. The user enters the NFT conversion settings for the virtual character in the application and submits the request. The application receives the NFT conversion request as input and sends it to the server as output.

[2544] Step 13:

[2545] The server retrieves character data, generates a smart contract, and registers it on the NFT marketplace. The server receives requests from users, retrieves character data, and generates a smart contract. It receives an NFT conversion request as input, generates a smart contract as output, and registers it on the marketplace.

[2546] Step 14:

[2547] The server manages transaction requests and records successful transactions in a database. It receives exchange and purchase requests from other users, and if a transaction is successful, it records the result in the database and sends it to the user. Transaction requests are received as input, and transaction results are recorded in the database as output.

[2548] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2549] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2550] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[2551] [Fourth Embodiment]

[2552] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[2553] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2554] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2555] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[2556] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[2557] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[2558] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[2559] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[2560] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[2561] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2562] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2563] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[2564] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2565] System Overview

[2566] This invention is a system that generates virtual idols (AItubers) using a generation engine, performs initial setup based on parameters provided by the user, and enables live streaming and interaction with fans. Furthermore, it allows the virtual idols to be registered as digital assets, enabling exchange and trading with other users.

[2567] Explanation of the program's processing

[2568] 1. Creating a virtual idol

[2569] user:

[2570] Users enter the necessary information into a virtual idol creation form using a dedicated application or web interface. This includes parameters such as the idol's gender, age, voice, and character design.

[2571] Terminal:

[2572] The terminal sends the parameters entered along with the generation request to the server.

[2573] server:

[2574] The server receives the request and starts the generation engine. Based on the provided parameters, the generation engine generates character data for a virtual idol and saves it to the database. A notification of completion is sent to the terminal, which then displays this to the user.

[2575] Specific example:

[2576] A user expresses a desire to create a virtual idol character who is a young woman with a cute, high-pitched voice. They enter the information into the creation form and press the "Generate" button. The server receives the information sent by the device, and the generation engine creates the character and saves it to the database. Finally, a completion notification is displayed to the user.

[2577] 2. Training and communication with virtual idols

[2578] user:

[2579] Users select training items and skill-building content from the application's training menu and submit requests. They can also interact with virtual idols using the communication menu.

[2580] Terminal:

[2581] The device sends nurturing and communication requests to the server.

[2582] server:

[2583] The server receives the request and updates the virtual idle parameters. The update results are saved to the database and sent to the terminal.

[2584] Specific example:

[2585] A user sends a request to a virtual idol asking for "singing training." The device sends the request to the server, which updates the virtual idol's skill parameters and displays the results to the user.

[2586] 3. Live streaming and interaction with fans

[2587] user:

[2588] Users schedule live streams and submit requests through the application. During the live stream, they can watch as the virtual idol reacts to comments from fans.

[2589] Terminal:

[2590] The device sends schedule information and commands to start and end the live stream to the server. During the live stream, comments from fans are displayed in real time.

[2591] server:

[2592] The server stores the live stream schedule in a database and prepares the appropriate streaming server. During the live stream, comments are processed in real time, and a natural language processing engine is used to ensure that virtual idlers respond. After the live stream ends, the live data is saved.

[2593] Specific example:

[2594] A user sets a schedule, such as "A virtual idol will perform a live show every Friday night at 8 PM." During the live show, comments from fans are displayed in real time, and the virtual idol responds. After the live show ends, the server saves the data.

[2595] 4. Converting virtual idols into NFTs and exchanging them

[2596] user:

[2597] The user enters the settings for creating an NFT of a virtual idle object in the application and submits the request.

[2598] Terminal:

[2599] The terminal sends a request to the server.

[2600] server:

[2601] The server retrieves character data, generates a smart contract to register it on the NFT marketplace, and registers the data on the blockchain. It receives exchange and purchase requests from other users and manages transactions. Successful transactions are recorded in the database, and the results are sent to the terminal.

[2602] Specific example:

[2603] The user enters information to convert their virtual idol into an NFT and submits a request. The server retrieves the character data, generates a smart contract to create the NFT, and registers the data on the blockchain. The transaction is successful, and the result is displayed to the user.

[2604] Through this detailed embodiment, users can efficiently perform a series of operations such as creating, training, live streaming, and converting virtual idols into NFTs.

[2605] The following describes the processing flow.

[2606] Virtual Idol Creation

[2607] Step 1:

[2608] The user opens a dedicated application or web interface and enters the necessary information (gender, age, voice, character design, etc.) into the virtual idol creation form.

[2609] Step 2:

[2610] The user presses the "Generate" button.

[2611] Step 3:

[2612] The terminal sends the parameters entered along with the generation request to the server.

[2613] Step 4:

[2614] The server receives the request and starts the generation engine.

[2615] Step 5:

[2616] The server passes user-provided parameters to the generation engine to generate character data for the virtual idol.

[2617] Step 6:

[2618] The server saves the generated character data to the database.

[2619] Step 7:

[2620] The server sends a notification to the device when generation is complete.

[2621] Step 8:

[2622] The device displays a message or preview to the user indicating that generation is complete.

[2623] Training and communication with virtual idols

[2624] Step 1:

[2625] Users select training items and skill development content from a training menu via the application and submit requests.

[2626] Step 2:

[2627] Users can use the communication menu to interact with virtual idols.

[2628] Step 3:

[2629] The terminal sends a request to the server.

[2630] Step 4:

[2631] The server receives the request and updates the virtual idle's parameters (e.g., skill level).

[2632] Step 5:

[2633] The server saves the updated status to the database.

[2634] Step 6:

[2635] The server sends the update results to the terminal.

[2636] Step 7:

[2637] The device displays the update results to the user.

[2638] Live streaming and interaction with fans

[2639] Step 1:

[2640] Users schedule live streams and submit requests through the application.

[2641] Step 2:

[2642] The terminal sends schedule information to the server.

[2643] Step 3:

[2644] The server saves the schedule to the database and prepares the appropriate delivery server.

[2645] Step 4:

[2646] The user sends the command to start the live stream.

[2647] Step 5:

[2648] The terminal sends a command to the server, and the live stream begins.

[2649] Step 6:

[2650] The server receives comments from fans in real time during the live stream, and a natural language processing engine is used to enable the virtual idol to respond appropriately.

[2651] Step 7:

[2652] The server saves the live data after the live stream ends.

[2653] Step 8:

[2654] The user sends a command to end the live session.

[2655] Step 9:

[2656] The terminal sends a command to the server, and the live session ends.

[2657] NFT conversion and exchange of virtual idols

[2658] Step 1:

[2659] The user enters the settings for creating an NFT of a virtual idle object in the application and submits the request.

[2660] Step 2:

[2661] The terminal sends a request to the server.

[2662] Step 3:

[2663] The server retrieves the character data and generates a smart contract to register it on the NFT marketplace.

[2664] Step 4:

[2665] The server registers the NFT data on the blockchain.

[2666] Step 5:

[2667] The server receives exchange and purchase requests from other users and manages the transactions.

[2668] Step 6:

[2669] The server records successful transactions in the database.

[2670] Step 7:

[2671] The terminal displays the transaction results to the user after the transaction is completed.

[2672] (Example 1)

[2673] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2674] Traditional virtual idol systems lacked an integrated platform that allowed users to smoothly perform a series of operations, such as creating, raising, live streaming, and converting characters into NFTs. Furthermore, they did not adequately provide features for generating real-time responses based on user input or managing character data as digital assets. As a result, users had to manage their virtual idols individually, leading to decreased operational efficiency and a poor user experience.

[2675] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[2676] In this invention, the server includes means for the user to input information into a virtual idol generation form using a dedicated application or web interface, means for the terminal to send the input information to the server along with the generation request, and means for the server to start a generation engine, generate character data for the virtual idol based on the provided information, and store it in a database. This allows the user to perform a series of operations, from virtual idol generation and training to live streaming and NFT creation, in a unified manner.

[2677] A "user" refers to an entity that uses the virtual idol system to perform operations such as creating, training, live streaming, and converting virtual idols into NFTs.

[2678] A "terminal" refers to a device used by a user, such as a computer, smartphone, or tablet, that communicates with the virtual idle system.

[2679] A "server" is the central computer system of a virtual idle system, and it is a device that processes user requests and generates, stores, and manages data.

[2680] A "virtual idol" refers to a digitized character created using a generation engine, possessing character design, personality, voice, and other characteristics based on parameters provided by the user.

[2681] A "generation engine" is software or an algorithm used to generate character data for a virtual idol based on user input.

[2682] A "database" is an information management system used to store character data, live streaming data, NFT transaction information, and other data related to the virtual idols that have been created.

[2683] A "natural language processing engine" is software or an algorithm that analyzes text data from users and fans and generates appropriate responses.

[2684] "NFT" stands for "Non-Fungible Token," and refers to a unique digital certificate that registers a virtual idol as a digital asset on the blockchain.

[2685] A "smart contract" is a program that automatically executes transactions on the blockchain and is used for NFT creation of virtual idols and transaction management.

[2686] "Live streaming" refers to virtual idols broadcasting video and audio to viewers in real time via the internet.

[2687] "Training items" refer to specific activities and learning content aimed at improving and developing the skills of virtual idols.

[2688] System Overview

[2689] This invention is a system that generates virtual idols (AItubers) using a generation engine. Based on parameters provided by the user, it performs initial setup, enables live streaming, interaction with fans, training and skill improvement, and can even be converted into NFTs, allowing for exchange and trading with other users as digital assets.

[2690] Virtual Idol Creation

[2691] user:

[2692] Users enter information into a virtual idol creation form using a dedicated application or web interface. Specifically, they enter parameters such as gender, age, voice, and character design.

[2693] Terminal:

[2694] The terminal serializes the input information into JSON format and sends it to the server as a generation request.

[2695] server:

[2696] After the server receives the request, it starts the "AICharacterG engine," which is the generation engine, and generates character data based on the provided parameters. The generated character data is stored in a MySQL database. A notification of completion of generation is sent to the terminal and is eventually displayed to the user.

[2697] Specific example:

[2698] A user requests to create a virtual idol character who is a young woman with a cute, high-pitched voice. They enter the information and press the "Generate" button. The device sends the request, the server generates the character, and saves it to the database. A notification of completion is displayed to the user.

[2699] Prompt message:

[2700] "We're creating a virtual idol character who is a young woman with a cute, high-pitched voice."

[2701] Training virtual idols

[2702] user:

[2703] Users select training items and skill development content from the application's training menu and submit a request.

[2704] Terminal:

[2705] The device sends a training request to the server.

[2706] server:

[2707] The server receives the request and updates the virtual idle parameters. The update results are saved in the database and notified to the user.

[2708] Specific example:

[2709] A user sends a request to a virtual idol asking for "singing training." The device sends the request to the server, the server updates the virtual idol's skill parameters, and displays the results to the user.

[2710] Prompt message:

[2711] "We have started training to improve the singing skills of our virtual idols."

[2712] Live streaming and interaction with fans

[2713] user:

[2714] Users schedule live streams and submit requests through the application. During the live stream, they can watch as the virtual idol reacts to comments from fans.

[2715] Terminal:

[2716] The device sends schedule information and commands to start and end the live stream to the server. During the live stream, comments from fans are displayed in real time.

[2717] server:

[2718] The server saves the live streaming schedule to a database and prepares the appropriate streaming server. Comments during the live stream are processed in real time by a natural language processing engine, and a virtual idol responds. After the live stream ends, the live data is saved to the database.

[2719] Specific example:

[2720] A user sets a schedule, such as "A virtual idol will perform a live show every Friday night at 8 PM." During the live show, comments from fans are displayed in real time, and the virtual idol responds. After the live show ends, the server saves the data.

[2721] Prompt message:

[2722] "We will be holding a live stream of a virtual idol every Friday night at 8 PM."

[2723] NFT conversion and exchange of virtual idols

[2724] user:

[2725] The user enters the NFT settings for virtual idles in the application and sends the request.

[2726] Terminal:

[2727] The terminal sends a request to the server.

[2728] server:

[2729] The server retrieves character data, generates smart contracts, registers them on the NFT marketplace, and manages transactions. It receives exchange and purchase requests from other users, records successful transactions in a database, and notifies the user of the results.

[2730] Specific example:

[2731] The user enters information to convert their virtual idol into an NFT and submits a request. The server retrieves the character data, generates a smart contract to create the NFT, and registers the data on the blockchain. The transaction is successful, and the result is displayed to the user.

[2732] Prompt message:

[2733] "Turn virtual idols into NFTs and register them on a marketplace."

[2734] Through this detailed embodiment, users can efficiently perform a series of operations such as creating, training, live streaming, and converting virtual idols into NFTs.

[2735] The flow of the specific processing in Example 1 will be explained using Figure 11.

[2736] Virtual Idol Creation

[2737] Step 1:

[2738] The user enters information.

[2739] Users open a dedicated application or web interface and enter information into a form for creating a virtual idol. This input data includes gender, age, voice, character design, and more.

[2740] input:

[2741] Parameters such as gender, age, voice, and character design

[2742] output:

[2743] Form data of the entered information

[2744] Specific actions:

[2745] The user opens the web interface in their browser and enters information into dropdown menus or text boxes.

[2746] Step 2:

[2747] The device sends the request

[2748] The terminal serializes the information entered by the user into JSON format and sends it to the server as a generation request.

[2749] input:

[2750] Form data entered by the user

[2751] output:

[2752] HTTP POST request to the server (data in JSON format)

[2753] Specific actions:

[2754] The terminal serializes the request data and sends an HTTP POST request to the specified endpoint on the server.

[2755] Step 3:

[2756] The server starts the generation engine.

[2757] After the server receives the request, it starts the generation engine (AICharacterG) and generates character data based on the provided parameters. The generated character data is saved in a temporary folder.

[2758] input:

[2759] The server receives a generation request (data in JSON format).

[2760] output:

[2761] Generated character data (file format)

[2762] Specific actions:

[2763] The server starts the generation engine, processes the character data based on the provided parameters, and saves it to a temporary folder.

[2764] Step 4:

[2765] The server saves the generated results and sends notifications.

[2766] The server saves the generated character data to a MySQL database and notifies the terminal when the generation is complete.

[2767] input:

[2768] Generated character data (file format)

[2769] output:

[2770] Character data stored in the database, generation completion notification (HTTP response)

[2771] Specific actions:

[2772] The server saves the character data to the database and sends an HTTP response to the terminal to notify it that generation is complete.

[2773] ---

[2774] Training virtual idols

[2775] Step 1:

[2776] The user selects a training menu.

[2777] Users select training items and skill development content from the application's training menu and submit a request.

[2778] input:

[2779] Training items and skill development content

[2780] output:

[2781] Training request (form data)

[2782] Specific actions:

[2783] The user selects "Singing Training" on the application screen and presses the "Submit" button.

[2784] Step 2:

[2785] The device sends the request

[2786] The device sends the user's selected training request to the server in JSON format.

[2787] input:

[2788] User-selected training request (form data)

[2789] output:

[2790] HTTP POST request to the server (data in JSON format)

[2791] Specific actions:

[2792] The terminal serializes the training request data and sends an HTTP POST request to the server.

[2793] Step 3:

[2794] The server updates the parameters.

[2795] The server receives the request data and updates the character data parameters according to the training details of the corresponding virtual idol. The update results are saved in the database and sent to the terminal as a notification.

[2796] input:

[2797] The server receives a training request (in JSON format).

[2798] output:

[2799] Updated character data stored in the database, update completion notification (HTTP response)

[2800] Specific actions:

[2801] The server updates the character data in the database and saves the new parameters. It then sends an update completion notification to the terminal as an HTTP response.

[2802] ---

[2803] Live streaming and interaction with fans

[2804] Step 1:

[2805] Users set up live streams

[2806] The user sets the live stream schedule in the application and sends the request to the server.

[2807] input:

[2808] Live stream schedule information (date, time, content)

[2809] output:

[2810] Live stream request (form data)

[2811] Specific actions:

[2812] The user sets "Live stream every Friday night at 8 PM" in the application and presses the "Schedule" button.

[2813] Step 2:

[2814] The device sends the request

[2815] The device sends the user's configured schedule information to the server in JSON format.

[2816] input:

[2817] Live schedule information set by the user (form data)

[2818] output:

[2819] HTTP POST request to the server (data in JSON format)

[2820] Specific actions:

[2821] The terminal serializes the live schedule information and sends an HTTP POST request to the server.

[2822] Step 3:

[2823] The server saves the schedule and starts delivery.

[2824] The server saves the live streaming schedule to the database and prepares the streaming server for live streaming. During the live stream, a natural language processing engine is used to process comments in real time, and a virtual idol responds. After the live stream ends, the live data is saved to the database.

[2825] input:

[2826] Live schedule information (in JSON format) received by the server.

[2827] output:

[2828] Live schedule information stored in the database, real-time processed comment data, and stored live data.

[2829] Specific actions:

[2830] The server saves the schedule to the database and configures the distribution server. During the live broadcast, comment data is analyzed using a natural language processing engine, and responses are generated in real time. After the live broadcast ends, all live data is saved to the database.

[2831] ---

[2832] NFT conversion and exchange of virtual idols

[2833] Step 1:

[2834] The user enters the NFT settings.

[2835] The user uses the application to input the information necessary to convert the virtual idle into an NFT and submits the request.

[2836] input:

[2837] Information required for NFT creation (character data, price, etc.)

[2838] output:

[2839] NFT conversion request (form data)

[2840] Specific actions:

[2841] The user clicks the "Convert to NFT" button, enters the required information, and clicks the "Submit" button.

[2842] Step 2:

[2843] The device sends the request

[2844] The device sends the NFT conversion request entered by the user to the server in JSON format.

[2845] input:

[2846] NFT conversion request (form data) entered by the user

[2847] output:

[2848] HTTP POST request to the server (data in JSON format)

[2849] Specific actions:

[2850] The device serializes the NFT data and sends an HTTP POST request to the server.

[2851] Step 3:

[2852] The server registers NFTs and manages transactions.

[2853] The server receives the request data, retrieves the virtual idol's character data, generates a smart contract, and converts it into an NFT. The generated NFT is registered on the blockchain, where exchange and purchase requests from other users are managed. Successful transactions are recorded in the database, and notifications are sent to the device.

[2854] input:

[2855] NFT conversion request (JSON format data) received by the server

[2856] output:

[2857] Generated smart contract, NFT registered on the blockchain, and transaction success notification (HTTP response).

[2858] Specific actions:

[2859] The server retrieves character data, generates a smart contract, and converts it into an NFT. The data is registered on the blockchain, and i...

Claims

1. A means of generating a virtual idle using a generation engine, A means for performing initial settings on the aforementioned virtual idol based on parameters provided by the user, The means by which the virtual idol conducts live streaming and interacts with fans, The aforementioned virtual idol is registered as a digital asset, and a means is provided for exchanging or buying and selling it with other users. A system that includes this.

2. The system according to claim 1, further comprising means for training and improving the skills of the virtual idol.

3. The system according to claim 1, further comprising means for using a natural language processing engine in communication with the user to generate a response in real time.

Images