System

Abstract

A system is provided.SOLUTION: The system includes a means for accepting application information of a model and registering it in a database, a means for acquiring the registered model information and providing it to a user, a means for customizing an idol based on the model information selected by the user, a means for entering the customized idol and holding a virtual live event, a means for accepting voting in the virtual live event and totaling the result, and a means for providing the totaling result to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional idol training simulation games and virtual live performance systems lack the ability for users to freely create and customize their favorite idols and incorporate a competitive element into the idols' live performances. Furthermore, they lack the functionality for virtual live events that users can participate in and enjoy in real time. This makes it difficult to maintain user interest. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for accepting model application information and registering it in a database, a means for acquiring the registered model information and providing it to the user, a means for customizing an idol based on the model information selected by the user, a means for entering the customized idol and holding a virtual live event, a means for accepting votes in the virtual live event and tallying the results, and a means for providing the tallying results to the user, thereby allowing users to create their own idol and adding a competitive element to the idol performing live shows. Also, by providing virtual live events in which users can participate in real time, a system that can continue to attract users' interest is realized.

[0006] A "model" is a person who provides the basic information for an idol through an application by a user.

[0007] The "database" refers to a system for centrally managing model information, customization information, and data related to live events.

[0008] "User" means an individual who uses the system to create, customize, and participate in live events with idols.

[0009] An "idol" is a virtual being that is customized based on user-selected model information.

[0010] "Customization" refers to the user selecting and changing elements of a model, such as hairstyle, clothing, and accessories.

[0011] A "virtual live event" is a live performance event in a virtual space where users can enter their created idols and compete with other users.

[0012] "Voting" is an action taken by a user to show support for their favorite idol during a live event.

[0013] "Counting" refers to the process of compiling and calculating the results of votes received during a virtual live event. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The system of the present invention allows users to create original idols in a virtual space and have them compete with other users in terms of their performance. The system has a variety of functions, and specific examples thereof will be described below.

[0036] 1. Model Recruitment

[0037] server:

[0038] 1. Prepare form data for recruiting models and provide an API for devices to access it.

[0039] 2. Receive model application information sent from the device and register it in the database.

[0040] Device:

[0041] 1. Use the server's API to retrieve the form data and display the model recruitment page.

[0042] 2. The model application information entered by the user is sent to the server.

[0043] User:

[0044] 1. Enter the required information such as your name, profile, and photo on the model recruitment page and submit.

[0045] 2. Selecting an idol body

[0046] server:

[0047] 1. Extract registered model information from the database and provide an API in a form that can be accessed by the device.

[0048] Device:

[0049] 1. Call the server's API to obtain model information and display it as a list to the user.

[0050] User:

[0051] 1. Select the model you like from the presented models.

[0052] 3. Idol Customization

[0053] server:

[0054] 1. Store customization options (hairstyle, clothing, accessories, etc.) in a database and provide an API to provide them.

[0055] 2. Save the customization information changed by the user to the database.

[0056] Device:

[0057] 1. Use the server's API to get the customization options and display the customization screen to the user.

[0058] 2. Send the customization information selected by the user to the server.

[0059] User:

[0060] 1. Choose your favorite customization options, such as hairstyle, clothing, and accessories.

[0061] 4. Idol Production

[0062] server:

[0063] 1. Information about the live event (date and time, participation method, voting system, etc.) is retrieved from the database and provided to the device via API.

[0064] 2. Manage the information and results of idols participating in the event and save the results in a database.

[0065] Device:

[0066] 1. Call the server's API to get the live event information and display it to the user.

[0067] 2. The idle information entered by the user is sent to the server.

[0068] User:

[0069] 1. Check the live event information and enter your customized idol.

[0070] 5. Live Events and Voting

[0071] server:

[0072] 1. Prepare and hold a virtual live event based on the entry information.

[0073] 2. Receive and tally voting information from the terminals in real time.

[0074] 3. After the event is over, the results are saved in the database and notified to the user.

[0075] Device:

[0076] 1. Use the server's API to retrieve data from the virtual live event and display the live video to the user.

[0077] 2. The voting page accepts votes in real time and sends them to the server.

[0078] User:

[0079] 1. Browse live events and vote for your favorite idol.

[0080] 2. Check the results after the event is over.

[0081] Specific examples

[0082] When User A applies to be a new model, he / she first accesses the model recruitment page on his / her device, enters the necessary information such as name, profile, photo, etc., and submits it. The server saves this information in the database and completes the model registration.

[0083] User B logs in to the system, retrieves a list of models on their device, and selects their favorite model. Based on the selected model information, they can freely customize hairstyle, clothing, accessories, etc.

[0084] A few days later, a live event is held. User B enters the idol he created into the event and competes with other users. During the live event, users vote in real time, and the server tallies the votes.

[0085] Through this series of steps, users can raise their own original idol in a virtual space and enjoy watching it grow.

[0086] The processing flow will be explained below.

[0087] Program processing steps

[0088] Model recruitment process steps

[0089] Step 1:

[0090] Server: Prepares form data for model recruitment and provides it to the device via API.

[0091] Step 2:

[0092] Device: Calls the API and displays the model recruitment page based on the retrieved form data.

[0093] Step 3:

[0094] User: Enter the required information such as name, profile, photo, etc. on the model recruitment page and submit.

[0095] Step 4:

[0096] Terminal: Sends the entered model application information to the server.

[0097] Step 5:

[0098] Server: Registers the received model application information in a database.

[0099] Idol element selection process steps

[0100] Step 1:

[0101] Server: Extracts registered model information from the database and provides an API in a format that can be accessed by the device.

[0102] Step 2:

[0103] Terminal: Calls the API to obtain model information and displays it to the user.

[0104] Step 3:

[0105] User: Select the model of their choice from the presented models.

[0106] Step 4:

[0107] Terminal: Sends the selected model information to the server.

[0108] Step 5:

[0109] Server: Stores the received model information in a database.

[0110] Idol Customization Process Steps

[0111] Step 1:

[0112] Server: Stores customization options (hairstyle, clothing, accessories, etc.) in a database and provides an API to provide them.

[0113] Step 2:

[0114] Device: Uses the API to retrieve customization options and displays the customization screen to the user.

[0115] Step 3:

[0116] User: Select your preferred customization options, such as hairstyle, clothing, and accessories.

[0117] Step 4:

[0118] Terminal: Sends the selected customization information to the server.

[0119] Step 5:

[0120] Server: Stores the received customization information in a database.

[0121] Idol production process steps

[0122] Step 1:

[0123] Server: Retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it to the device via API.

[0124] Step 2:

[0125] Terminal: Calls the API to retrieve live event information and displays it to the user.

[0126] Step 3:

[0127] User: Check live event information and enter your customized idol.

[0128] Step 4:

[0129] Terminal: Sends the entered idle information to the server.

[0130] Step 5:

[0131] Server: Stores the received entry information in a database.

[0132] Live event execution and voting processing steps

[0133] Step 1:

[0134] Server: Generates data for the virtual live event based on the entry information and starts the live event according to the event time.

[0135] Step 2:

[0136] Terminal: Uses APIs to retrieve data from virtual live events and display live footage to users.

[0137] Step 3:

[0138] Users: View live events and vote for their favorite idols.

[0139] Step 4:

[0140] Terminal: Sends voting information to the server in real time.

[0141] Step 5:

[0142] Server: Counts the received voting information and stores the results in a database.

[0143] Step 6:

[0144] Server: After the event ends, the results are provided to the terminal and displayed to the user.

[0145] Through these modularized processing steps, users can create their own original idol and experience the entire process of having that idol compete against other users in a virtual live performance.

[0146] Example 1

[0147] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0148] Conventional virtual idol creation and live event systems have difficulty allowing users to create original idols and easily compete with each other in performance. Real-time voting and tallying are also difficult, resulting in a poor user experience. Furthermore, customized idol information cannot be easily reused, resulting in a lack of convenience for users who wish to use the system repeatedly.

[0149] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0150] In this invention, the server includes means for accepting model application information and registering it in a database, means for acquiring the registered model information and providing it to a terminal, means for customizing an idol based on the model information selected by the user, means for entering the customized idol and holding a virtual live event, means for accepting votes for the virtual live event and tallying them in real time, means for storing the tally results in a database and providing them to the user, means for storing the customized idol information in the database and making it available for future live events, and means for providing an interface for users to view the idol's performance in real time and vote during the virtual live event. This allows users to easily create and customize original idols in a virtual space and compete against other users in their performances. Real-time voting and tallying are also performed smoothly, improving the user experience. Furthermore, reusing customized idol information improves convenience during repeated use.

[0151] "Model application information" refers to information submitted by a user to apply to be a model, and specifically includes the user's name, profile, photo, etc.

[0152] A "database" is a system that stores information in an organized manner and allows for efficient searching and updating.

[0153] A "terminal" is an electronic device such as a computer or smartphone used by a user.

[0154] An "API" is an interface that allows different software components to communicate with each other.

[0155] "Idols" are characters created and customized by users in the virtual space, and are participants in live events.

[0156] "Customization" refers to the act of a user setting or changing an idol's appearance, such as hairstyle, clothing, and accessories.

[0157] A "virtual live event" is a live performance event that takes place in a virtual space and can be viewed and participated in by multiple users.

[0158] "Voting" is an evaluation action that a user takes for their favorite idol during or after a live event.

[0159] "Interface" refers to the screen and input means that users use to operate the system.

[0160] "Counting" is the process of calculating and analyzing the collected voting data and calculating the results.

[0161] "Storage" is the act of recording information in a database so that it can be reused at a later time.

[0162] The system of the present invention allows users to create original idols in a virtual space and compete with other users for their performance. The system operates in cooperation with a server, terminals, and users. Here, we will explain a specific embodiment of the system, including what hardware and software are used to process and calculate data.

[0163] 1. Model Recruitment

[0164] The server prepares form data for model recruitment and provides an API that the device can access (e.g., https: / / example.com / api / model-form). This allows the device to call the server's API and display the model recruitment page based on the obtained form data. On this page, the user enters application information such as name, profile, and photo and submits it. The device sends the information entered by the user to the server, and the server registers the application information in a database.

[0165] For example, the prompt text might look like this:

[0166] "Enter your name, profile, and photo and submit."

[0167] 2. Selecting an idol body

[0168] The server extracts registered model information from the database and provides it as an API (e.g., https: / / example.com / api / models) that can be retrieved by the device. The device calls the server's API, retrieves the model information, and displays it as a list to the user. The user can then select their preferred model from the list.

[0169] For example, the prompt text might look like this:

[0170] Choose your favorite model from the list.

[0171] 3. Idol Customization

[0172] The server stores customization option data (hairstyle, clothing, accessories, etc.) in a database and provides an API (e.g., https: / / example.com / api / customize-options) that provides this data. The device uses the server's API to obtain the customization options and displays a customization screen to the user. The user selects customization options such as hairstyle, clothing, and accessories, and sends the selected customization information to the server via the device. The server stores this customization information in a database.

[0173] For example, the prompt text might look like this:

[0174] "Choose your hairstyle, outfit, and accessories."

[0175] 4. Idol Production

[0176] The server retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it to the device via API (e.g. https: / / example.com / api / event-info). The device calls the server's API to retrieve the live event information and displays it to the user. The user checks this information and enters their customized idol. The device sends the entry information to the server, which receives it and stores it in the database.

[0177] For example, the prompt text might look like this:

[0178] "Please select the idol you would like to enter in the live event."

[0179] 5. Live Events and Voting

[0180] The server prepares and runs the virtual live event based on the entry information (for example, using game engines such as Unity or Unreal Engine). The device uses the server's API to obtain data about the virtual live event and displays live footage to the user. Users view the live event and vote for their favorite idol. The device accepts votes in real time on the voting page and sends them to the server. The server receives this voting information in real time and tallys it. After the event ends, the server saves the results in a database and notifies the user.

[0181] For example, the prompt text might look like this:

[0182] "Vote for your idol during the live event"

[0183] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0184] Step 1: Model recruitment

[0185] server:

[0186] Prepare the form data and provide the API (e.g., https: / / example.com / api / model-form), which allows devices to access the model recruitment form.

[0187] Input: Server preferences, form data

[0188] Output: API endpoint containing form data

[0189] Device:

[0190] Call the server API to get the form data.

[0191] The model recruitment page is displayed to the user.

[0192] Input: Form data obtained from the API

[0193] Output: Model recruitment page shown to the user

[0194] User:

[0195] On the model recruitment page, enter your application information such as your name, profile, and photo, and click the submit button.

[0196] Input: User's application information (name, profile, photo)

[0197] Output: Submitted application information

[0198] Device:

[0199] The application information entered by the user is sent to the server.

[0200] Input: Application information provided by the user

[0201] Output: Application information sent to the server

[0202] server:

[0203] The received application information is registered in a database.

[0204] Input: Application information sent from the device

[0205] Output: Application information stored in a database

[0206] Step 2: Select the idol body

[0207] server:

[0208] Extract model information from the database and provide an API in a format that can be retrieved by the device (e.g. https: / / example.com / api / models).

[0209] Input: Model information in the database

[0210] Output: API endpoint containing model information

[0211] Device:

[0212] Call the server's API to obtain model information and display it as a list to the user.

[0213] Input: Model information obtained from the API

[0214] Output: A list of models displayed to the user

[0215] User:

[0216] Select the model you like from the ones presented.

[0217] Input: List of models

[0218] Output: Selected model information

[0219] Step 3: Customize your idol

[0220] server:

[0221] Store customization options (hairstyles, clothing, accessories, etc.) in a database and provide an API to serve them (e.g., https: / / example.com / api / customize-options).

[0222] Input: Customization options data in the database

[0223] Output: API endpoint with customization options

[0224] Device:

[0225] Use the server's API to get the customization options and display the customization screen to the user.

[0226] Input: Customization options obtained from the API

[0227] Output: The customized screen that is displayed to the user

[0228] User:

[0229] Choose customization options like hairstyle, clothing, accessories, etc.

[0230] Input: Customization options

[0231] Output: Selected customization information

[0232] Device:

[0233] The customization information selected by the user is sent to the server.

[0234] Input: Customization information selected by the user

[0235] Output: Customization information sent to the server

[0236] server:

[0237] The received customization information is stored in a database.

[0238] Input: Customization information sent from the device

[0239] Output: Customization information stored in the database

[0240] Step 4: Producing idols

[0241] server:

[0242] Retrieve live event information (date and time, participation method, voting system, etc.) from the database and provide it to the device via an API (e.g. https: / / example.com / api / event-info).

[0243] Input: Live event information in the database

[0244] Output: API endpoint containing live event information

[0245] Device:

[0246] Call the server's API to get live event information and display it to the user.

[0247] Input: Live event information obtained from the API

[0248] Output: Live event information displayed to the user

[0249] User:

[0250] Check the live event information and enter your customized idol.

[0251] Input: Live Event Information

[0252] Output: Entry information

[0253] Device:

[0254] The idle information entered by the user is sent to the server.

[0255] Input: User entry information

[0256] Output: Entry information sent to the server

[0257] server:

[0258] The received entry information is stored in a database.

[0259] Input: Entry information sent from the terminal

[0260] Output: Entry information stored in the database

[0261] Step 5: Live Event and Voting

[0262] server:

[0263] Prepare and conduct a virtual live event based on the entry information (e.g., using game engines such as Unity or Unreal Engine).

[0264] Input: Entry information stored in the database

[0265] Output: Virtual live event

[0266] Device:

[0267] Use the server's API to retrieve data from the virtual live event and display live footage to the user.

[0268] Input: Live event data retrieved from the API

[0269] Output: Live video displayed to the user

[0270] User:

[0271] Watch live events and vote for your favorite idol.

[0272] Input: Live video

[0273] Output: Voting information

[0274] Device:

[0275] The voting page accepts votes in real time and sends them to the server.

[0276] Input: Voting information from users

[0277] Output: Voting information sent to the server

[0278] server:

[0279] Voting information from the terminals is received in real time and tallied.

[0280] After the event is over, the results are stored in a database and notified to the user.

[0281] Input: Voting information sent from the device

[0282] Output: Aggregation results, saving to database, notifying user

[0283] (Application example 1)

[0284] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0285] Previously, systems existed that allowed users to compete using the personalities and customization elements of idols created in a virtual space. However, they lacked functionality, particularly functionality for linking with virtual stores to try on fashion items and hold fashion shows. Therefore, there is a need for a system that allows users to enjoy a more comprehensive experience, not only by customizing the appearance of their virtual idols, but also by selecting fashion items and participating in fashion show events. There is also a need for a system that allows users to try on fashion items in real time and compete with other users on their evaluations, thereby improving interaction between users.

[0286] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0287] In this invention, the server includes means for accepting application information for models and registering it in a database, means for acquiring the registered model information and providing it to the user, means for customizing an idol based on the model information selected by the user, means for entering the customized idol and holding a virtual live event, means for accepting votes in the virtual live event and tallying the results, means for providing the tallying results to the user, means for providing a virtual store application and displaying customization options for the idol's appearance and clothing, and means for entering the idol customized by the user in a fashion show event and having it compete with other users. This enables users to customize the appearance and fashion of idols created in a virtual space and compete with other users in real time.

[0288] "Model application information" refers to personal information such as name, profile, and photo that a user provides to the system.

[0289] "Model information" refers to all information such as personal information and appearance data of users registered in the database.

[0290] An "idol" is a character that a user designs and customizes in a virtual space.

[0291] "Customization" refers to the process of changing appearance elements such as hairstyle, clothing, and accessories according to the user's preferences.

[0292] A "virtual live event" is a live performance or fashion show held in a virtual space.

[0293] "Voting" is the act of users rating different idols in real time during a live event.

[0294] "Counting" refers to the process of processing the voting results after the live event has ended and calculating the final ratings.

[0295] A "virtual store" is a virtual shop or boutique operated on the Internet.

[0296] An "application" is software that users use on their smartphones or other devices.

[0297] "Customization options" are various options for changing an idol's appearance.

[0298] A "fashion show event" is a virtual event where users can compete with idols they have customized.

[0299] The present invention is a system that allows users to create original idols in a virtual space and have them compete with other users in terms of performance and fashion sense. Specific examples of the system are described below.

[0300] The server first accepts the model's application information and registers it in a database, including information such as name, profile, and photo. It then retrieves the registered model information and provides it to the device via API.

[0301] The device uses the server's API to obtain model information and displays it to the user. The user can then select their preferred model from this list. Based on the selected model information, the user can customize the idol. Customization options include hairstyle, clothing, and accessories.

[0302] After users have completed customization, they can enter their customized idols into virtual live events and fashion show events. In virtual live events, users can view their idols' performances in real time and compete against other users. Voting is also conducted in real time, and the results are tallied on the server. The tallied results are provided to users after the event ends.

[0303] A virtual store application is also provided, giving users access to customization options for the idol's appearance and clothing, allowing them to enjoy the idol's appearance and fashion created in the virtual space and compete with other users.

[0304] The hardware used is a smartphone, and the software used is an application framework such as React Native or Flutter (registered trademark), with Node.js or Python's Flask used as the backend.

[0305] For example, a user opens the app, enters their name, profile, and photo on the model recruitment page, and submits them. This information is then stored in a database by the server. Next, the user selects their preferred model from a list of models presented, and selects customization options such as hairstyle, clothing, and accessories. After that, the user enters their customized idol in a fashion show event scheduled on the app, competing against other users. During the live event, users can vote in real time with other users.

[0306] Examples of prompts for generative AI models include:

[0307] "How do I apply as a new model?"

[0308] "I want to change my idol's hairstyle. How can I customize it?"

[0309] "Please let me know about the next live event."

[0310] "How do I vote for my idol against other users' idols?"

[0311] In this way, the system of the present invention provides a forum where users can customize original idols in a virtual space and compete in fashion sense and performance.

[0312] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0313] Step 1:

[0314] The user opens the model recruitment page, enters application information such as name, profile, and photo, and submits it.

[0315] Input: Personal information you enter, such as your name, profile, and photo.

[0316] Data processing: The terminal sends the input information to the server via the API.

[0317] Output: The application information is sent to the server and stored in a database.

[0318] Specific operation: The terminal displays the application information as an input form, the user enters the required information in the form, and clicks the "Submit" button.

[0319] Step 2:

[0320] The server retrieves model information from the database and provides it to the terminal.

[0321] Input: Model information stored on the server.

[0322] Data processing: The server executes database queries to extract model information and provides it to the terminal in API format.

[0323] Output: The model information arrives on the terminal and is displayed in a list to the user.

[0324] Specific operation: The server extracts the necessary model information from the database and sends that information to the terminal as a JSON format API response.

[0325] Step 3:

[0326] The user selects model information and selects customization options on the customization page.

[0327] Input: Model information and customization options selected by the user (hairstyle, clothing, accessories, etc.).

[0328] Data processing: The device sends the selected model information and customization options to the server via API.

[0329] Output: The customized idle information is saved on the server.

[0330] Specific operation: The user selects the model of their choice from the model list, selects customization options, and clicks the "Apply" button.

[0331] Step 4:

[0332] The user enters the customized idol into a fashion show event.

[0333] Input: Customized idle information and selected event information.

[0334] Data processing: The device sends idle information and event information to the server via API.

[0335] Output: The idle is entered into the selected event.

[0336] Specific operation: The user checks the customized idol, selects the event they want to participate in from the list of fashion show events, and clicks the "Enter" button.

[0337] Step 5:

[0338] Users compete against other users in virtual live events.

[0339] Input: Virtual live event configuration information and user votes.

[0340] Data processing: The server tally the voting information in real time and calculate the rankings.

[0341] Output: The server notifies the user of the results of the live event.

[0342] Specific operation: The server tally the voting results in real time and display the rankings and ratings to users after the event ends.

[0343] Step 6:

[0344] A virtual store allows users to access customization options for their idols' appearance and clothing.

[0345] Input: Virtual store customization options information.

[0346] Data processing: The server provides customization option information to the terminal via the API.

[0347] Output: The customization options are displayed on the device and can be selected by the user.

[0348] Specific operation: The server extracts customization option information from the virtual store and sends it to the terminal in API format. The user can then check and select the options on the terminal.

[0349] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0350] The system of the present invention allows users to create original idols in a virtual space and compete with other users in their performances. Furthermore, by combining it with an emotion engine, it has the function of recognizing the user's emotions and reflecting them in the virtual live performance. A specific example of this is described below.

[0351] 1. Model Recruitment

[0352] server:

[0353] 1. Prepare form data for recruiting models and provide an API for devices to access it.

[0354] 2. Receive model application information sent from the device and register it in the database.

[0355] Device:

[0356] 1. Use the server's API to retrieve the form data and display the model recruitment page.

[0357] 2. The model application information entered by the user is sent to the server.

[0358] User:

[0359] 1. Enter the required information such as your name, profile, and photo on the model recruitment page and submit.

[0360] 2. Selecting an idol body

[0361] server:

[0362] 1. Extract registered model information from the database and provide an API in a form that can be accessed by the device.

[0363] Device:

[0364] 1. Call the API to get model information and display it to the user.

[0365] User:

[0366] 1. Select the model you like from the presented models.

[0367] 3. Idol Customization

[0368] server:

[0369] 1. Store customization options (hairstyle, clothing, accessories, etc.) in a database and provide an API to provide them.

[0370] 2. Save the customization information changed by the user to the database.

[0371] Device:

[0372] 1. Use the API to get the customization options and display the customization screen to the user.

[0373] 2. Send the customization information selected by the user to the server.

[0374] User:

[0375] 1. Choose your favorite customization options, such as hairstyle, clothing, and accessories.

[0376] 4. Idol Production

[0377] server:

[0378] 1. Information about the live event (date and time, participation method, voting system, etc.) is retrieved from the database and provided to the device via API.

[0379] 2. Manage the information and results of idols participating in the event and save the results in a database.

[0380] Device:

[0381] 1. Call the server's API to get the live event information and display it to the user.

[0382] 2. The idle information entered by the user is sent to the server.

[0383] User:

[0384] 1. Check the live event information and enter your customized idol.

[0385] 5. Live Events and Voting

[0386] server:

[0387] 1. Prepare and hold a virtual live event based on the entry information.

[0388] 2. Receive and tally voting information from the terminals in real time.

[0389] 3. After the event is over, the results are saved in the database and notified to the user.

[0390] Device:

[0391] 1. Use the server's API to retrieve data from the virtual live event and display the live video to the user.

[0392] 2. The voting page accepts votes in real time and sends them to the server.

[0393] User:

[0394] 1. Browse live events and vote for your favorite idol.

[0395] 2. Check the results after the event is over.

[0396] 6. Implementing the Emotion Engine

[0397] 6.1 Emotion recognition

[0398] server:

[0399] 1. Analyze user emotional data (facial expressions, voice, text) through an emotion engine and provide an API that reflects the results in real time.

[0400] Device:

[0401] 1. Use the API to obtain emotion recognition results and incorporate them into a virtual live performance.

[0402] User:

[0403] 1. Your emotions are recognized during live events.

[0404] 6.2 Performance Tuning

[0405] server:

[0406] 1. Implement an algorithm to adjust the virtual live performance based on the recognition results of the emotion engine.

[0407] 2. The recognized emotion data is stored in a database and used to adjust performance in future applications.

[0408] Device:

[0409] 1. Use the server's API to retrieve the adjusted live performance data and display it to the user.

[0410] Specific examples

[0411] For example, if User A applies to be a new model, he or she first accesses the model recruitment page on his or her device, enters the necessary information such as name, profile, and photo, and submits it. The server saves this information in the database and completes the model registration.

[0412] User B logs in to the system, retrieves a list of models on their device, and selects their favorite model. Based on the selected model information, they can freely customize hairstyle, clothing, accessories, etc.

[0413] A few days later, a live event is held. User B enters the idol he created into the event and competes against other users. During the live event, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions, and the results are reflected in their performance in real time.

[0414] Through this series of steps, users can raise their own original idol in a virtual space and enjoy watching it grow and react emotionally.

[0415] The processing flow will be explained below.

[0416] Program processing steps

[0417] Model recruitment process steps

[0418] Step 1:

[0419] Server: Prepares form data for model recruitment and provides it to the device via API.

[0420] Step 2:

[0421] Device: Calls the API and displays the model recruitment page based on the retrieved form data.

[0422] Step 3:

[0423] User: Enter the required information such as name, profile, photo, etc. on the model recruitment page and submit.

[0424] Step 4:

[0425] Terminal: Sends the entered model application information to the server.

[0426] Step 5:

[0427] Server: Registers the received model application information in a database.

[0428] Idol element selection process steps

[0429] Step 1:

[0430] Server: Extracts registered model information from the database and provides an API in a format that can be accessed by the device.

[0431] Step 2:

[0432] Terminal: Calls the API to obtain model information and displays it to the user.

[0433] Step 3:

[0434] User: Select the model of their choice from the presented models.

[0435] Step 4:

[0436] Terminal: Sends the selected model information to the server.

[0437] Step 5:

[0438] Server: Stores the received model information in a database.

[0439] Idol Customization Process Steps

[0440] Step 1:

[0441] Server: Stores customization options (hairstyle, clothing, accessories, etc.) in a database and provides an API to provide them.

[0442] Step 2:

[0443] Device: Uses the API to retrieve customization options and displays the customization screen to the user.

[0444] Step 3:

[0445] User: Select your preferred customization options, such as hairstyle, clothing, and accessories.

[0446] Step 4:

[0447] Terminal: Sends the selected customization information to the server.

[0448] Step 5:

[0449] Server: Stores the received customization information in a database.

[0450] Idol production process steps

[0451] Step 1:

[0452] Server: Retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it to the device via API.

[0453] Step 2:

[0454] Terminal: Calls the API to retrieve live event information and displays it to the user.

[0455] Step 3:

[0456] User: Check live event information and enter your customized idol.

[0457] Step 4:

[0458] Terminal: Sends the entered idle information to the server.

[0459] Step 5:

[0460] Server: Stores the received entry information in a database.

[0461] Live event execution and voting processing steps

[0462] Step 1:

[0463] Server: Generates data for the virtual live event based on the entry information and starts the live event according to the event time.

[0464] Step 2:

[0465] Terminal: Uses APIs to retrieve data from virtual live events and display live footage to users.

[0466] Step 3:

[0467] Users: View live events and vote for their favorite idols.

[0468] Step 4:

[0469] Terminal: Sends voting information to the server in real time.

[0470] Step 5:

[0471] Server: Counts the received voting information and stores the results in a database.

[0472] Step 6:

[0473] Server: After the event ends, the results are provided to the terminal and displayed to the user.

[0474] Processing steps for implementing an emotion engine

[0475] Step 1:

[0476] Server: Prepares the emotion engine and provides an API for analyzing user emotion data (facial expressions, voice, text).

[0477] Step 2:

[0478] Terminal: Uses API to obtain emotion data analysis results and presents them to the user.

[0479] Step 3:

[0480] Users: Ensure their emotions are recognized while participating in live events.

[0481] Step 4:

[0482] Server: Based on the analyzed emotion recognition data, the virtual live performance is adjusted in real time.

[0483] Step 5:

[0484] Server: The recognition results are stored in a database and used for performance adjustments in the future.

[0485] Step 6:

[0486] Terminal: Uses the server's API to retrieve the adjusted live performance data and display it to the user.

[0487] Specific examples

[0488] For example, if User A applies to be a new model, he or she first accesses the model recruitment page on his or her device, enters the necessary information such as name, profile, and photo, and submits it. The server saves this information in the database and completes the model registration.

[0489] User B logs in to the system, retrieves a list of models on their device, and selects their favorite model. Based on the selected model information, they can freely customize hairstyle, clothing, accessories, etc.

[0490] A few days later, a live event is held. User B enters the idol he created into the event and competes against other users. During the live event, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions, and the results are reflected in their performance in real time.

[0491] Through this series of steps, users can raise their own original idol in a virtual space and enjoy watching it grow and react emotionally.

[0492] Example 2

[0493] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0494] In conventional virtual character development systems, customizing user-created characters and performing live events was cumbersome, and it was not possible to reflect user emotions in real time. Furthermore, it was difficult to efficiently analyze and automate user input data, leaving challenges in improving the user experience.

[0495] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0496] In this invention, the server includes means for accepting application information and registering it in a database, means for acquiring the registered information and providing it to the user, means for customizing a virtual character based on information selected by the user, means for entering the customized virtual character and holding a virtual event, means for accepting votes for the virtual event and tallying the results, and means for providing the tallying results to the user. This allows users to easily customize their virtual characters and participate in events and vote in real time. Furthermore, by including means for analyzing user emotional data and reflecting this in performance during the virtual event in real time, and means for adjusting the performance of the virtual event based on the analyzed emotional data, it is possible to provide a performance that reflects the user's emotions. Furthermore, by including means for analyzing user input data using a generative AI model and automating operations at each step, the user experience can be improved.

[0497] "Application information" refers to information such as name, profile, and photo that a user provides to create a virtual character.

[0498] "Database" refers to an information system for storing application information, registered information, customized virtual character information, emotional data, voting results, and the like.

[0499] A "virtual character" is a digital agent or avatar that a user can create and customize in a digital environment.

[0500] A "virtual event" is an event in a digital environment where virtual characters perform and users can participate and vote in real time.

[0501] A "vote" is an act in which a user expresses their opinion or support for a particular virtual character during a virtual event.

[0502] "Emotional data" refers to information about emotions extracted from a user's facial expressions, voice, and text, and is reflected in their performance during the virtual event.

[0503] "Performance" refers to the movements, actions, and performances made by a virtual character during a virtual event.

[0504] A "generative AI model" is an artificial intelligence model used to analyze input data from a user and automate operations at each step.

[0505] An "interface" is a screen or operating means that allows a user to interact with the system, enabling operations such as voting and customization.

[0506] The system of the present invention allows users to create original virtual characters in a virtual space, customize those characters, and participate in virtual events. It also has the function of recognizing the user's emotions and reflecting those emotions in the performance of the virtual event. A specific embodiment of the system is described below.

[0507] Model Wanted

[0508] server:

[0509] The server uses a web framework (e.g., Django, Flask) to prepare form data for model recruitment, provides an API endpoint for this (e.g., " / api / model recruitment form"), and receives application information (name, profile, photo, etc.) sent from the device and registers it in a database (e.g., MySQL (registered trademark), PostgreSQL).

[0510] Device:

[0511] The device sends an HTTP GET request to the server's API, retrieves form data in JSON format, displays the model recruitment page using an HTML template (e.g., React, Vue.js), captures the model application information entered by the user, and sends it to the server as an HTTP POST request.

[0512] (Example)

[0513] When User A applies to be a new model, he / she accesses the model recruitment page on his / her device, enters information such as the name "John Doe", profile "singer-songwriter", and photo "john_doe.jpg", and submits it. The server receives this information and stores it in a database.

[0514] Idol Body Selection

[0515] server:

[0516] The server extracts registered model information from the database and provides it at the API endpoint " / api / models".

[0517] Device:

[0518] The device sends an HTTP GET request to the API and displays a list of models to the user using the retrieved information. Model cards are dynamically generated using React and Vue.js, allowing users to select their preferred model.

[0519] (Example)

[0520] User B logs in to the system and selects "John Doe" from the list of registered models.

[0521] Idol Customization

[0522] server:

[0523] The server prepares customization option data (hairstyle, clothing, accessories) and provides it at the API endpoint " / api / customize_options". It also saves the customization information changed by the user in the database.

[0524] Device:

[0525] The device sends an HTTP GET request to the API, displays the customization screen using the retrieved option data, captures the customization information selected by the user, and sends it to the server as an HTTP POST request.

[0526] (Example)

[0527] User B changes "John Doe's" hairstyle to "short hair", clothing to "casual", and accessories to "sunglasses", and sends this information to the server, which stores this information in a database.

[0528] Idol production

[0529] server:

[0530] The server retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it via the API endpoint " / api / live_events". It also saves entry information in the database and prepares and executes the virtual event.

[0531] Device:

[0532] The device sends an HTTP GET request to the API to retrieve live event information and display it to the user. The device also sends the idle information entered by the user to the server as an HTTP POST request.

[0533] (Example)

[0534] User B enters his virtual character into a live event and competes against other users.

[0535] Live Events and Voting

[0536] server:

[0537] The server prepares a virtual live event based on the entry information, generates live video using game engines such as Unity or Unreal Engine, and receives and tally vote information in real time.

[0538] Device:

[0539] The device retrieves streaming data from the API, displays live video to the user using WebRTC or HLS, and captures voting information and sends it to the server as an HTTP POST request.

[0540] (Example)

[0541] Users watch a live event and vote for "John Doe." The server tallies the votes and notifies the results after the event ends.

[0542] Implementing the Emotion Engine

[0543] server:

[0544] The server uses an emotion engine (e.g., Microsoft® Azure® Emotion API, Google® Cloud Speech-to-Text API) to analyze users' emotional data, provides an API that reflects the results in the virtual event performance, implements an algorithm that adjusts the virtual performance based on the emotional data, and stores the data in a database.

[0545] Device:

[0546] The device obtains emotion recognition results from the API and reflects them in real time in the virtual live performance.

[0547] (Example)

[0548] During the live event, the user's facial expression data is analyzed, and if the emotion is recognized as "joy," that emotion is reflected in the virtual character's performance in real time.

[0549] Example of generative AI model and prompt used

[0550] Generative AI models:

[0551] It uses OpenAI (registered trademark) GPT-4 (registered trademark) to analyze input data from users and automate operations at each step.

[0552] Example prompt sentence:

[0553] Based on the system's processing steps, please provide a detailed explanation of the steps a user takes to create an original idol in a virtual space and have that idol participate in a live event. Also, please provide a detailed explanation of how the emotion engine reflects the user's emotions in the live performance.

[0554] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0555] Step 1:

[0556] Process: Preparation and provision of form data for model recruitment.

[0557] Subject and action: The server uses a web framework to prepare form data for model recruitment and provides an API endpoint for this purpose: " / api / model recruitment form".

[0558] Input: Template data for model submission on the server.

[0559] Data processing / calculation: The server converts the template data into JSON format.

[0560] Output: The form data in JSON format provided by the API endpoint.

[0561] Specific operation: For example, define a model recruitment form using Django, convert the form data to JSON, and provide it via an API endpoint.

[0562] Step 2:

[0563] Processing content: Display model recruitment page.

[0564] Subject and Action: The device sends an HTTP GET request to the server's API, retrieves form data in JSON format, and displays a model recruitment page using an HTML template.

[0565] Input: JSON formatted form data retrieved from an API endpoint.

[0566] Data processing / calculation: The terminal parses the JSON data and embeds it into an HTML template.

[0567] Output: The model recruitment page shown to the user.

[0568] Specific operation: Generate form elements based on JSON data obtained using React and display them in the browser.

[0569] Step 3:

[0570] What it does: Collect and submit model application information.

[0571] Subject and Action: The user enters the required information such as name, profile, and photo on the model recruitment page, and the device sends this to the server as an HTTP POST request.

[0572] Input: Data entered by the user, such as name, profile, and photo.

[0573] Data processing / calculation: The terminal converts the form data into JSON format.

[0574] Output: JSON formatted application information sent to the server.

[0575] What it does: When a user enters information into a form and clicks the "Submit" button, the device captures the information and sends it to the server.

[0576] Step 4:

[0577] Process: Receiving and registering model application information.

[0578] Subject and Action: The server receives the model application information sent from the terminal and registers it in the database.

[0579] Input: Model application information received from the device in JSON format.

[0580] Data processing / calculation: The server parses the JSON data and inserts it into the database.

[0581] Output: Application information stored in a database.

[0582] Specific behavior: For example, use a Django model to save the received application information to a database.

[0583] Step 5:

[0584] Processing: Extracts and provides registered model information.

[0585] Subject and action: The server extracts registered model information from the database and provides it at the API endpoint " / api / models".

[0586] Input: A request for model information from the database.

[0587] Data processing / computation: The server executes the database query and converts the results into JSON format.

[0588] Output: Model information in JSON format provided by the API endpoint.

[0589] What happens: The server executes a MySQL query to get the model information and provides it to the endpoint in JSON format.

[0590] Step 6:

[0591] Process: Retrieves and displays model information.

[0592] Subject and Action: The device sends an HTTP GET request to the API endpoint to retrieve model information and display it to the user.

[0593] Input: Model information in JSON format obtained from the API endpoint.

[0594] Data processing / calculation: The terminal parses the JSON data and embeds it into an HTML template.

[0595] Output: A page listing the model information that is displayed to the user.

[0596] Specific operation: Generate a list based on the JSON data obtained using React and display it in the browser.

[0597] Step 7:

[0598] What it does: Select a model.

[0599] Subject and Action: The user selects the model of their choice from the list of models presented.

[0600] Input: A user-chosen model ID.

[0601] Data processing / calculation: The device captures the selected model ID.

[0602] Output: The selected model ID.

[0603] Specific behavior: The user clicks on "John Doe" from the list of models.

[0604] Step 8:

[0605] What it does: Provides customization options.

[0606] Subject and Action: The server provides customization option data (hairstyle, clothing, accessories) at the API endpoint " / api / customize_options".

[0607] Input: Customization options request from database.

[0608] Data processing / computation: The server executes the database query and converts the results into JSON format.

[0609] Output: Customization options provided in JSON format at the API endpoint.

[0610] Specific operation: The server obtains the customization data and provides it to the endpoint in JSON format.

[0611] Step 9:

[0612] Processing content: Display the customization screen.

[0613] Subject and Action: The device sends an HTTP GET request to the API endpoint to retrieve the customization options and display the customization screen.

[0614] Input: Customization options in JSON format retrieved from the API endpoint.

[0615] Data processing / calculation: The terminal parses the JSON data and embeds it into an HTML template.

[0616] Output: The customized screen that is displayed to the user.

[0617] What it does: Uses React to display customization options on the screen and allow the user to select them.

[0618] Step 10:

[0619] Process: Sending customization information.

[0620] Subject and Action: The user selects customization options (hairstyle, clothing, accessories), and the device sends that information to the server as an HTTP POST request.

[0621] Input: The customization options selected by the user (e.g. hairstyle ID, outfit ID, accessory ID).

[0622] Data processing / calculation: The terminal converts the customization data into JSON format.

[0623] Output: Customization information in JSON format sent to the server.

[0624] What happens: When a user saves a customization, the device sends that information to the server.

[0625] Step 11:

[0626] What it does: Saves customization information.

[0627] Subject and Action: The server receives the customization information sent from the terminal and registers it in the database.

[0628] Input: Customization information received from the terminal in JSON format.

[0629] Data processing / calculation: The server parses the JSON data and inserts it into the database.

[0630] Output: Customization information stored in a database.

[0631] What it does: Uses Django models to store customization information in the database.

[0632] Step 12:

[0633] What it does: Providing live event information.

[0634] Subject and Action: The server retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it at the API endpoint " / api / live_events".

[0635] Input: A request for live event information from the database.

[0636] Data processing / computation: The server executes the database query and converts the results into JSON format.

[0637] Output: Live event information in JSON format provided by the API endpoint.

[0638] Specific behavior: The server retrieves live event information and provides it to the endpoint in JSON format.

[0639] Step 13:

[0640] What it does: Displays live event information.

[0641] Subject and Action: The device sends an HTTP GET request to the API endpoint to retrieve live event information and display it to the user.

[0642] Input: Live event information in JSON format retrieved from the API endpoint.

[0643] Data processing / calculation: The terminal parses the JSON data and embeds it into an HTML template.

[0644] Output: The live event information page that is displayed to the user.

[0645] Specific operation: Display live event information obtained using React in the browser.

[0646] Step 14:

[0647] Processing: Idle entry.

[0648] Subject and Action: A user enters a customized virtual character into a live event, and the device sends this information to the server as an HTTP POST request.

[0649] Input: The ID of the virtual character the user is entering.

[0650] Data processing / calculation: The terminal converts the entry information into JSON format.

[0651] Output: Entry information in JSON format sent to the server.

[0652] Specific operation: When the user presses the entry button, the terminal sends the information to the server.

[0653] Step 15:

[0654] What it does: Store your entry information and conduct the virtual live event.

[0655] Subject and Action: The server stores the entry information in a database and prepares and executes the virtual live event.

[0656] Input: Entry information received from the device in JSON format.

[0657] Data processing / computation: The server parses the JSON data, inserts it into a database, and runs algorithms to generate the live performance.

[0658] Output: Entry information stored in a database and live performances generated.

[0659] How it works: Live video is generated using Unity or Unreal Engine and distributed via a streaming server.

[0660] Step 16:

[0661] What it does: Show live events and take votes.

[0662] Subject and Action: The device retrieves live streaming data from the API and displays it to the user. It also captures voting information and sends it to the server as an HTTP POST request.

[0663] Input: Streaming data retrieved from API endpoints and user voting information.

[0664] Data processing / calculation: The device plays back the streaming data and converts the voting information into JSON format.

[0665] Output: The live video displayed to the user and the voting information sent to the server.

[0666] Specific operation: Live video is displayed using WebRTC and HLS, and when a user presses the voting button, the information is sent to the server.

[0667] Step 17:

[0668] Process: Counting voting information and notifying results.

[0669] Subject and Action: The server receives and tally vote information from the terminals in real time. After the event ends, it saves the results in a database and notifies the users.

[0670] Input: Voting information received from the device in JSON format.

[0671] Data processing / calculation: The server aggregates the voting data and converts the results into JSON format.

[0672] Output: Voting results stored in a database and notification to the user.

[0673] Specific operation: The server tally the voting information, calculate the rankings, and notify the results after the event ends.

[0674] Step 18:

[0675] What it does: Analyzes emotion data and adjusts performance.

[0676] Subject and Action: The server uses an emotion engine to analyze the user's facial expressions and voice data, and reflects the results in the virtual event performance in real time. It also adjusts the virtual performance based on the analysis results and stores the data in a database.

[0677] Input: User's facial and voice data.

[0678] Data processing / calculation: Analyze data using the emotion engine and run algorithms that reflect the results in performance.

[0679] Output: Emotion data stored in a database and adjusted performance.

[0680] What it does: Analyzes emotion data and adjusts performance using Microsoft Azure's Emotion API and Google Cloud's Speech-to-Text API.

[0681] (Application example 2)

[0682] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0683] Conventional systems for cultivating and performing idols in virtual spaces lacked features such as performances that reflected the user's emotions in real time and promotional activities within virtual stores. This limited the user experience and made it difficult to provide more interactive and engaging content. Furthermore, there was a need for an effective system that could utilize emotional data to adjust idols' performances and realize continuous promotional activities.

[0684] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0685] In this invention, the server includes means for accepting model application information and registering it in a database, means for acquiring the registered model information and providing it to the user, means for customizing an idol based on the model information selected by the user, means for entering the customized idol and holding a virtual live event, means for accepting votes in the virtual live event and tallying the results, means for providing the tallying results to the user, means for users to implement a promotion in which the idol introduces products in a virtual store, and means for recognizing the user's emotions and reflecting the emotion data in the idol's performance. This makes it possible to create, train, and adjust the performance of idols in a virtual space, and furthermore, promotional activities in the virtual store are continuously carried out, making it possible to provide an interactive experience that reflects the user's emotions.

[0686] "Model application information" refers to information such as name, profile, and photo entered by a user who wishes to create an idol in a virtual space.

[0687] "Database" refers to a storage device for storing and managing model application information, customized idol information, and data on virtual live events and promotional activities.

[0688] A "user" is a person who uses the system to create, customize, and enter idols in a virtual space and participate in virtual live events.

[0689] "Customization" is an operation in which the user changes the appearance and attributes of the selected model, such as hairstyle, clothing, and accessories, to create an individual idol.

[0690] "Entry" refers to an operation to allow a customized idol to participate in a virtual live event or promotional activity.

[0691] A "virtual live event" is a live performance competition event held in a virtual space, in which idols entered by users compete in performance against other users.

[0692] "Voting" is an act by which a user shows support for their favorite idol at a virtual live event, and the results are tallied.

[0693] "Counting" is the process of compiling the voting results received at the virtual live event and calculating the final score.

[0694] A "virtual store" is a fictitious store space that exists on the Internet, where virtual idols introduce products and engage in promotional activities.

[0695] "Promotional activities" are the act of idols introducing and advertising products within a virtual store.

[0696] "Emotion recognition" is a technology that analyzes emotions from a user's facial expressions, voice, etc. and incorporates those emotions into the system.

[0697] "Reflecting in performance" means adjusting the virtual idol's movements and words in real time based on the user's emotions obtained through emotion recognition, resulting in a more interactive performance.

[0698] The system for implementing the present invention includes a server, a user terminal, and an emotion engine as its main components. The system operates as follows.

[0699] The server accepts application information for models and registers it in a database. When accessed from a user's device, the server provides an API that provides registered model information. The idol is customized based on the model information obtained by the user via their device. The customized idol is then sent to the server as entry information, and the virtual live event is prepared.

[0700] During the virtual live event, the server processes data in real time, accepts and tally votes, and provides the results to users. The user terminal also displays the virtual live performance in real time and provides an interface for users to vote.

[0701] Furthermore, users can use idols to promote products in virtual stores. The emotion engine analyzes users' emotional data, such as facial expressions and voice, and reflects that data in real time in the performance of the virtual idols, providing users with an interactive and engaging experience.

[0702] The hardware used includes smartphones and head-mounted displays. These devices are equipped with camera modules and microphones to capture the user's facial expressions and voice. The software uses EmotionEngine for emotion recognition, VirtualIdol for managing virtual idols, and the Requests library for API communication.

[0703] As a concrete example, a user accesses a model recruitment page using a smartphone, enters information such as name, profile, and photo, and submits it. The server stores this information in a database, completing the model registration. When the user promotes a new product in a virtual store, the emotion engine detects the user's joy and excitement and reflects this in real time in the idol's words, actions, and performance.

[0704] An example prompt for a generative AI model is:

[0705] "Create a smartphone app for promoting virtual idols. Include a process where users can create a virtual idol and introduce products in a virtual store. Implement a function where the idol adjusts their performance based on the user's emotions. Also, consider a voting system where users can compete with other users, and describe how to synchronize data with the server."

[0706] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0707] Step 1:

[0708] A user accesses the model recruitment page, enters model application information such as name, profile, and photo, and submits it.

[0709] Input: Data entered by users, such as name, profile, and photo.

[0710] Output: The input data is sent to the server.

[0711] Specific behavior:

[0712] Users access the model recruitment form on their smartphone or head-mounted display screen, enter the necessary information, and then press the send button to send the application information to the server.

[0713] Step 2:

[0714] The server accepts model application information and registers it in a database.

[0715] Input: Model application information submitted by the user.

[0716] Output: Application information registered in the database.

[0717] Specific behavior:

[0718] The server receives application information from users via API communication, stores the data in a database, and then returns a registration completion response to the user.

[0719] Step 3:

[0720] The user accesses the virtual idol's customization page using a terminal and customizes the virtual idol by selecting hairstyle, clothes, accessories, etc.

[0721] Input: Model information stored in the database, user customization selections.

[0722] Output: Customized idle information.

[0723] Specific behavior:

[0724] The user adjusts the appearance of the virtual idol using the customization options provided, and once selections are complete, the customization information is sent to the server.

[0725] Step 4:

[0726] The server stores the customized idol in a database and registers the entry information for the virtual live event.

[0727] Input: Customization information submitted by the user.

[0728] Output: Customized idle information stored in the database, and entry information.

[0729] Specific behavior:

[0730] The server receives the customization information and stores it in a database, while also registering the entry information for participating in the virtual live event.

[0731] Step 5:

[0732] During a virtual live event, the emotion engine analyzes the user's facial expressions and voice to generate emotion data.

[0733] Input: User's facial expression data, voice data.

[0734] Output: User emotion data.

[0735] Specific behavior:

[0736] The emotion engine analyzes the user's facial expressions and voice in real time, generating emotional data that is then reflected in the virtual idol's performance.

[0737] Step 6:

[0738] The device adjusts the virtual idol's performance based on emotional data and displays live footage.

[0739] Input: Parsed emotion data.

[0740] Output: Performance footage of the virtual idol after adjustments.

[0741] Specific behavior:

[0742] The device adjusts the movements and behavior of the virtual idol based on the emotional data sent from the emotion engine, and displays live footage in real time to provide the user with a more interactive experience.

[0743] Step 7:

[0744] A server accepts and tallys voting data during the virtual live event.

[0745] Input: Voting information from users.

[0746] Output: The tallied vote results.

[0747] Specific behavior:

[0748] The server receives the voting data sent by users in real time, tallying it up and calculating the final results, and stores the results in a database after the live event ends.

[0749] Step 8:

[0750] A promotion is carried out in which a user introduces products using idols in a virtual store.

[0751] Input: User promotion request information, customized idle information.

[0752] Output: Product introduction video by a virtual idol.

[0753] Specific behavior:

[0754] Users can request product promotions in the virtual store, and customized idols will generate and display videos introducing the products.

[0755] Step 9:

[0756] The terminal reflects the emotional data generated in real time in the promotional performance of the virtual idol.

[0757] Input: Emotion data, promotional video data.

[0758] Output: Adjusted promotional performance footage.

[0759] Specific behavior:

[0760] The terminal reflects the emotional data acquired from the emotion engine in the promotional video and adjusts the virtual idol's movements and speech to correspond to the user's emotions.

[0761] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0762] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0763] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0764] [Second embodiment]

[0765] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0766] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0767] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0768] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0769] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0770] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0771] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0772] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0773] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0774] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0775] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0776] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0777] The system of the present invention allows users to create original idols in a virtual space and have them compete with other users in terms of their performance. The system has a variety of functions, and specific examples thereof will be described below.

[0778] 1. Model Recruitment

[0779] server:

[0780] 1. Prepare form data for recruiting models and provide an API for devices to access it.

[0781] 2. Receive model application information sent from the device and register it in the database.

[0782] Device:

[0783] 1. Use the server's API to retrieve the form data and display the model recruitment page.

[0784] 2. The model application information entered by the user is sent to the server.

[0785] User:

[0786] 1. Enter the required information such as your name, profile, and photo on the model recruitment page and submit.

[0787] 2. Selecting an idol body

[0788] server:

[0789] 1. Extract registered model information from the database and provide an API in a form that can be accessed by the device.

[0790] Device:

[0791] 1. Call the server's API to obtain model information and display it as a list to the user.

[0792] User:

[0793] 1. Select the model you like from the presented models.

[0794] 3. Idol Customization

[0795] server:

[0796] 1. Store customization options (hairstyle, clothing, accessories, etc.) in a database and provide an API to provide them.

[0797] 2. Save the customization information changed by the user to the database.

[0798] Device:

[0799] 1. Use the server's API to get the customization options and display the customization screen to the user.

[0800] 2. Send the customization information selected by the user to the server.

[0801] User:

[0802] 1. Choose your favorite customization options, such as hairstyle, clothing, and accessories.

[0803] 4. Idol Production

[0804] server:

[0805] 1. Information about the live event (date and time, participation method, voting system, etc.) is retrieved from the database and provided to the device via API.

[0806] 2. Manage the information and results of idols participating in the event and save the results in a database.

[0807] Device:

[0808] 1. Call the server's API to get the live event information and display it to the user.

[0809] 2. The idle information entered by the user is sent to the server.

[0810] User:

[0811] 1. Check the live event information and enter your customized idol.

[0812] 5. Live Events and Voting

[0813] server:

[0814] 1. Prepare and hold a virtual live event based on the entry information.

[0815] 2. Receive and tally voting information from the terminals in real time.

[0816] 3. After the event is over, the results are saved in the database and notified to the user.

[0817] Device:

[0818] 1. Use the server's API to retrieve data from the virtual live event and display the live video to the user.

[0819] 2. The voting page accepts votes in real time and sends them to the server.

[0820] User:

[0821] 1. Browse live events and vote for your favorite idol.

[0822] 2. Check the results after the event is over.

[0823] Specific examples

[0824] When User A applies to be a new model, he / she first accesses the model recruitment page on his / her device, enters the necessary information such as name, profile, photo, etc., and submits it. The server saves this information in the database and completes the model registration.

[0825] User B logs in to the system, retrieves a list of models on their device, and selects their favorite model. Based on the selected model information, they can freely customize hairstyle, clothing, accessories, etc.

[0826] A few days later, a live event is held. User B enters the idol he created into the event and competes with other users. During the live event, users vote in real time, and the server tallies the votes.

[0827] Through this series of steps, users can raise their own original idol in a virtual space and enjoy watching it grow.

[0828] The processing flow will be explained below.

[0829] Program processing steps

[0830] Model recruitment process steps

[0831] Step 1:

[0832] Server: Prepares form data for model recruitment and provides it to the device via API.

[0833] Step 2:

[0834] Device: Calls the API and displays the model recruitment page based on the retrieved form data.

[0835] Step 3:

[0836] User: Enter the required information such as name, profile, photo, etc. on the model recruitment page and submit.

[0837] Step 4:

[0838] Terminal: Sends the entered model application information to the server.

[0839] Step 5:

[0840] Server: Registers the received model application information in a database.

[0841] Idol element selection process steps

[0842] Step 1:

[0843] Server: Extracts registered model information from the database and provides an API in a format that can be accessed by the device.

[0844] Step 2:

[0845] Terminal: Calls the API to obtain model information and displays it to the user.

[0846] Step 3:

[0847] User: Select the model of their choice from the presented models.

[0848] Step 4:

[0849] Terminal: Sends the selected model information to the server.

[0850] Step 5:

[0851] Server: Stores the received model information in a database.

[0852] Idol Customization Process Steps

[0853] Step 1:

[0854] Server: Stores customization options (hairstyle, clothing, accessories, etc.) in a database and provides an API to provide them.

[0855] Step 2:

[0856] Device: Uses the API to retrieve customization options and displays the customization screen to the user.

[0857] Step 3:

[0858] User: Select your preferred customization options, such as hairstyle, clothing, and accessories.

[0859] Step 4:

[0860] Terminal: Sends the selected customization information to the server.

[0861] Step 5:

[0862] Server: Stores the received customization information in a database.

[0863] Idol production process steps

[0864] Step 1:

[0865] Server: Retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it to the device via API.

[0866] Step 2:

[0867] Terminal: Calls the API to retrieve live event information and displays it to the user.

[0868] Step 3:

[0869] User: Check live event information and enter your customized idol.

[0870] Step 4:

[0871] Terminal: Sends the entered idle information to the server.

[0872] Step 5:

[0873] Server: Stores the received entry information in a database.

[0874] Live event execution and voting processing steps

[0875] Step 1:

[0876] Server: Generates data for the virtual live event based on the entry information and starts the live event according to the event time.

[0877] Step 2:

[0878] Terminal: Uses APIs to retrieve data from virtual live events and display live footage to users.

[0879] Step 3:

[0880] Users: View live events and vote for their favorite idols.

[0881] Step 4:

[0882] Terminal: Sends voting information to the server in real time.

[0883] Step 5:

[0884] Server: Counts the received voting information and stores the results in a database.

[0885] Step 6:

[0886] Server: After the event ends, the results are provided to the terminal and displayed to the user.

[0887] Through these modularized processing steps, users can create their own original idol and experience the entire process of having that idol compete against other users in a virtual live performance.

[0888] Example 1

[0889] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0890] Conventional virtual idol creation and live event systems have difficulty allowing users to create original idols and easily compete with each other in performance. Real-time voting and tallying are also difficult, resulting in a poor user experience. Furthermore, customized idol information cannot be easily reused, resulting in a lack of convenience for users who wish to use the system repeatedly.

[0891] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0892] In this invention, the server includes means for accepting model application information and registering it in a database, means for acquiring the registered model information and providing it to a terminal, means for customizing an idol based on the model information selected by the user, means for entering the customized idol and holding a virtual live event, means for accepting votes for the virtual live event and tallying them in real time, means for storing the tally results in a database and providing them to the user, means for storing the customized idol information in the database and making it available for future live events, and means for providing an interface for users to view the idol's performance in real time and vote during the virtual live event. This allows users to easily create and customize original idols in a virtual space and compete against other users in their performances. Real-time voting and tallying are also performed smoothly, improving the user experience. Furthermore, reusing customized idol information improves convenience during repeated use.

[0893] "Model application information" refers to information submitted by a user to apply to be a model, and specifically includes the user's name, profile, photo, etc.

[0894] A "database" is a system that stores information in an organized manner and allows for efficient searching and updating.

[0895] A "terminal" is an electronic device such as a computer or smartphone used by a user.

[0896] An "API" is an interface that allows different software components to communicate with each other.

[0897] "Idols" are characters created and customized by users in the virtual space, and are participants in live events.

[0898] "Customization" refers to the act of a user setting or changing an idol's appearance, such as hairstyle, clothing, and accessories.

[0899] A "virtual live event" is a live performance event that takes place in a virtual space and can be viewed and participated in by multiple users.

[0900] "Voting" is an evaluation action that a user takes for their favorite idol during or after a live event.

[0901] "Interface" refers to the screen and input means that users use to operate the system.

[0902] "Counting" is the process of calculating and analyzing the collected voting data and calculating the results.

[0903] "Storage" is the act of recording information in a database so that it can be reused at a later time.

[0904] The system of the present invention allows users to create original idols in a virtual space and compete with other users for their performance. The system operates in cooperation with a server, terminals, and users. Here, we will explain a specific embodiment of the system, including what hardware and software are used to process and calculate data.

[0905] 1. Model Recruitment

[0906] The server prepares form data for model recruitment and provides an API that the device can access (e.g., https: / / example.com / api / model-form). This allows the device to call the server's API and display the model recruitment page based on the obtained form data. On this page, the user enters application information such as name, profile, and photo and submits it. The device sends the information entered by the user to the server, and the server registers the application information in a database.

[0907] For example, the prompt text might look like this:

[0908] "Enter your name, profile, and photo and submit."

[0909] 2. Selecting an idol body

[0910] The server extracts registered model information from the database and provides it as an API (e.g., https: / / example.com / api / models) that can be retrieved by the device. The device calls the server's API, retrieves the model information, and displays it as a list to the user. The user can then select their preferred model from the list.

[0911] For example, the prompt text might look like this:

[0912] Choose your favorite model from the list.

[0913] 3. Idol Customization

[0914] The server stores customization option data (hairstyle, clothing, accessories, etc.) in a database and provides an API (e.g., https: / / example.com / api / customize-options) that provides this data. The device uses the server's API to obtain the customization options and displays a customization screen to the user. The user selects customization options such as hairstyle, clothing, and accessories, and sends the selected customization information to the server via the device. The server stores this customization information in a database.

[0915] For example, the prompt text might look like this:

[0916] "Choose your hairstyle, outfit, and accessories."

[0917] 4. Idol Production

[0918] The server retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it to the device via API (e.g. https: / / example.com / api / event-info). The device calls the server's API to retrieve the live event information and displays it to the user. The user checks this information and enters their customized idol. The device sends the entry information to the server, which receives it and stores it in the database.

[0919] For example, the prompt text might look like this:

[0920] "Please select the idol you would like to enter in the live event."

[0921] 5. Live Events and Voting

[0922] The server prepares and runs the virtual live event based on the entry information (for example, using game engines such as Unity or Unreal Engine). The device uses the server's API to obtain data about the virtual live event and displays live footage to the user. Users view the live event and vote for their favorite idol. The device accepts votes in real time on the voting page and sends them to the server. The server receives this voting information in real time and tallys it. After the event ends, the server saves the results in a database and notifies the user.

[0923] For example, the prompt text might look like this:

[0924] "Vote for your idol during the live event"

[0925] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0926] Step 1: Model recruitment

[0927] server:

[0928] Prepare the form data and provide the API (e.g., https: / / example.com / api / model-form), which allows devices to access the model recruitment form.

[0929] Input: Server preferences, form data

[0930] Output: API endpoint containing form data

[0931] Device:

[0932] Call the server API to get the form data.

[0933] The model recruitment page is displayed to the user.

[0934] Input: Form data obtained from the API

[0935] Output: Model recruitment page shown to the user

[0936] User:

[0937] On the model recruitment page, enter your application information such as your name, profile, and photo, and click the submit button.

[0938] Input: User's application information (name, profile, photo)

[0939] Output: Submitted application information

[0940] Device:

[0941] The application information entered by the user is sent to the server.

[0942] Input: Application information provided by the user

[0943] Output: Application information sent to the server

[0944] server:

[0945] The received application information is registered in a database.

[0946] Input: Application information sent from the device

[0947] Output: Application information stored in a database

[0948] Step 2: Select the idol body

[0949] server:

[0950] Extract model information from the database and provide an API in a format that can be retrieved by the device (e.g. https: / / example.com / api / models).

[0951] Input: Model information in the database

[0952] Output: API endpoint containing model information

[0953] Device:

[0954] Call the server's API to obtain model information and display it as a list to the user.

[0955] Input: Model information obtained from the API

[0956] Output: A list of models displayed to the user

[0957] User:

[0958] Select the model you like from the ones presented.

[0959] Input: List of models

[0960] Output: Selected model information

[0961] Step 3: Customize your idol

[0962] server:

[0963] Store customization options (hairstyles, clothing, accessories, etc.) in a database and provide an API to serve them (e.g., https: / / example.com / api / customize-options).

[0964] Input: Customization options data in the database

[0965] Output: API endpoint with customization options

[0966] Device:

[0967] Use the server's API to get the customization options and display the customization screen to the user.

[0968] Input: Customization options obtained from the API

[0969] Output: The customized screen that is displayed to the user

[0970] User:

[0971] Choose customization options like hairstyle, clothing, accessories, etc.

[0972] Input: Customization options

[0973] Output: Selected customization information

[0974] Device:

[0975] The customization information selected by the user is sent to the server.

[0976] Input: Customization information selected by the user

[0977] Output: Customization information sent to the server

[0978] server:

[0979] The received customization information is stored in a database.

[0980] Input: Customization information sent from the device

[0981] Output: Customization information stored in the database

[0982] Step 4: Producing idols

[0983] server:

[0984] Retrieve live event information (date and time, participation method, voting system, etc.) from the database and provide it to the device via an API (e.g. https: / / example.com / api / event-info).

[0985] Input: Live event information in the database

[0986] Output: API endpoint containing live event information

[0987] Device:

[0988] Call the server's API to get live event information and display it to the user.

[0989] Input: Live event information obtained from the API

[0990] Output: Live event information displayed to the user

[0991] User:

[0992] Check the live event information and enter your customized idol.

[0993] Input: Live Event Information

[0994] Output: Entry information

[0995] Device:

[0996] The idle information entered by the user is sent to the server.

[0997] Input: User entry information

[0998] Output: Entry information sent to the server

[0999] server:

[1000] The received entry information is stored in a database.

[1001] Input: Entry information sent from the terminal

[1002] Output: Entry information stored in the database

[1003] Step 5: Live Event and Voting

[1004] server:

[1005] Prepare and conduct a virtual live event based on the entry information (e.g., using game engines such as Unity or Unreal Engine).

[1006] Input: Entry information stored in the database

[1007] Output: Virtual live event

[1008] Device:

[1009] Use the server's API to retrieve data from the virtual live event and display live footage to the user.

[1010] Input: Live event data retrieved from the API

[1011] Output: Live video displayed to the user

[1012] User:

[1013] Watch live events and vote for your favorite idol.

[1014] Input: Live video

[1015] Output: Voting information

[1016] Device:

[1017] The voting page accepts votes in real time and sends them to the server.

[1018] Input: Voting information from users

[1019] Output: Voting information sent to the server

[1020] server:

[1021] Voting information from the terminals is received in real time and tallied.

[1022] After the event is over, the results are stored in a database and notified to the user.

[1023] Input: Voting information sent from the device

[1024] Output: Aggregation results, saving to database, notifying user

[1025] (Application example 1)

[1026] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1027] Previously, systems existed that allowed users to compete using the personalities and customization elements of idols created in a virtual space. However, they lacked functionality, particularly functionality for linking with virtual stores to try on fashion items and hold fashion shows. Therefore, there is a need for a system that allows users to enjoy a more comprehensive experience, not only by customizing the appearance of their virtual idols, but also by selecting fashion items and participating in fashion show events. There is also a need for a system that allows users to try on fashion items in real time and compete with other users on their evaluations, thereby improving interaction between users.

[1028] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1029] In this invention, the server includes means for accepting application information for models and registering it in a database, means for acquiring the registered model information and providing it to the user, means for customizing an idol based on the model information selected by the user, means for entering the customized idol and holding a virtual live event, means for accepting votes in the virtual live event and tallying the results, means for providing the tallying results to the user, means for providing a virtual store application and displaying customization options for the idol's appearance and clothing, and means for entering the idol customized by the user in a fashion show event and having it compete with other users. This enables users to customize the appearance and fashion of idols created in a virtual space and compete with other users in real time.

[1030] "Model application information" refers to personal information such as name, profile, and photo that a user provides to the system.

[1031] "Model information" refers to all information such as personal information and appearance data of users registered in the database.

[1032] An "idol" is a character that a user designs and customizes in a virtual space.

[1033] "Customization" refers to the process of changing appearance elements such as hairstyle, clothing, and accessories according to the user's preferences.

[1034] A "virtual live event" is a live performance or fashion show held in a virtual space.

[1035] "Voting" is the act of users rating different idols in real time during a live event.

[1036] "Counting" refers to the process of processing the voting results after the live event has ended and calculating the final ratings.

[1037] A "virtual store" is a virtual shop or boutique operated on the Internet.

[1038] An "application" is software that users use on their smartphones or other devices.

[1039] "Customization options" are various options for changing an idol's appearance.

[1040] A "fashion show event" is a virtual event where users can compete with idols they have customized.

[1041] The present invention is a system that allows users to create original idols in a virtual space and have them compete with other users in terms of performance and fashion sense. Specific examples of the system are described below.

[1042] The server first accepts the model's application information and registers it in a database, including information such as name, profile, and photo. It then retrieves the registered model information and provides it to the device via API.

[1043] The device uses the server's API to obtain model information and displays it to the user. The user can then select their preferred model from this list. Based on the selected model information, the user can customize the idol. Customization options include hairstyle, clothing, and accessories.

[1044] After users have completed customization, they can enter their customized idols into virtual live events and fashion show events. In virtual live events, users can view their idols' performances in real time and compete against other users. Voting is also conducted in real time, and the results are tallied on the server. The tallied results are provided to users after the event ends.

[1045] A virtual store application is also provided, giving users access to customization options for the idol's appearance and clothing, allowing them to enjoy the idol's appearance and fashion created in the virtual space and compete with other users.

[1046] The hardware used is a smartphone, and the software used is an application framework such as React Native or Flutter, with Node.js or Python's Flask as the backend.

[1047] For example, a user opens the app, enters their name, profile, and photo on the model recruitment page, and submits them. This information is then stored in a database by the server. Next, the user selects their preferred model from a list of models presented, and selects customization options such as hairstyle, clothing, and accessories. After that, the user enters their customized idol in a fashion show event scheduled on the app, competing against other users. During the live event, users can vote in real time with other users.

[1048] Examples of prompts for generative AI models include:

[1049] "How do I apply as a new model?"

[1050] "I want to change my idol's hairstyle. How can I customize it?"

[1051] "Please let me know about the next live event."

[1052] "How do I vote for my idol against other users' idols?"

[1053] In this way, the system of the present invention provides a forum where users can customize original idols in a virtual space and compete in fashion sense and performance.

[1054] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1055] Step 1:

[1056] The user opens the model recruitment page, enters application information such as name, profile, and photo, and submits it.

[1057] Input: Personal information you enter, such as your name, profile, and photo.

[1058] Data processing: The terminal sends the input information to the server via the API.

[1059] Output: The application information is sent to the server and stored in a database.

[1060] Specific operation: The terminal displays the application information as an input form, the user enters the required information in the form, and clicks the "Submit" button.

[1061] Step 2:

[1062] The server retrieves model information from the database and provides it to the terminal.

[1063] Input: Model information stored on the server.

[1064] Data processing: The server executes database queries to extract model information and provides it to the terminal in API format.

[1065] Output: The model information arrives on the terminal and is displayed in a list to the user.

[1066] Specific operation: The server extracts the necessary model information from the database and sends that information to the terminal as a JSON format API response.

[1067] Step 3:

[1068] The user selects model information and selects customization options on the customization page.

[1069] Input: Model information and customization options selected by the user (hairstyle, clothing, accessories, etc.).

[1070] Data processing: The device sends the selected model information and customization options to the server via API.

[1071] Output: The customized idle information is saved on the server.

[1072] Specific operation: The user selects the model of their choice from the model list, selects customization options, and clicks the "Apply" button.

[1073] Step 4:

[1074] The user enters the customized idol into a fashion show event.

[1075] Input: Customized idle information and selected event information.

[1076] Data processing: The device sends idle information and event information to the server via API.

[1077] Output: The idle is entered into the selected event.

[1078] Specific operation: The user checks the customized idol, selects the event they want to participate in from the list of fashion show events, and clicks the "Enter" button.

[1079] Step 5:

[1080] Users compete against other users in virtual live events.

[1081] Input: Virtual live event configuration information and user votes.

[1082] Data processing: The server tally the voting information in real time and calculate the rankings.

[1083] Output: The server notifies the user of the results of the live event.

[1084] Specific operation: The server tally the voting results in real time and display the rankings and ratings to users after the event ends.

[1085] Step 6:

[1086] A virtual store allows users to access customization options for their idols' appearance and clothing.

[1087] Input: Virtual store customization options information.

[1088] Data processing: The server provides customization option information to the terminal via the API.

[1089] Output: The customization options are displayed on the device and can be selected by the user.

[1090] Specific operation: The server extracts customization option information from the virtual store and sends it to the terminal in API format. The user can then check and select the options on the terminal.

[1091] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1092] The system of the present invention allows users to create original idols in a virtual space and compete with other users in their performances. Furthermore, by combining it with an emotion engine, it has the function of recognizing the user's emotions and reflecting them in the virtual live performance. A specific example of this is described below.

[1093] 1. Model Recruitment

[1094] server:

[1095] 1. Prepare form data for recruiting models and provide an API for devices to access it.

[1096] 2. Receive model application information sent from the device and register it in the database.

[1097] Device:

[1098] 1. Use the server's API to retrieve the form data and display the model recruitment page.

[1099] 2. The model application information entered by the user is sent to the server.

[1100] User:

[1101] 1. Enter the required information such as your name, profile, and photo on the model recruitment page and submit.

[1102] 2. Selecting an idol body

[1103] server:

[1104] 1. Extract registered model information from the database and provide an API in a form that can be accessed by the device.

[1105] Device:

[1106] 1. Call the API to get model information and display it to the user.

[1107] User:

[1108] 1. Select the model you like from the presented models.

[1109] 3. Idol Customization

[1110] server:

[1111] 1. Store customization options (hairstyle, clothing, accessories, etc.) in a database and provide an API to provide them.

[1112] 2. Save the customization information changed by the user to the database.

[1113] Device:

[1114] 1. Use the API to get the customization options and display the customization screen to the user.

[1115] 2. Send the customization information selected by the user to the server.

[1116] User:

[1117] 1. Choose your favorite customization options, such as hairstyle, clothing, and accessories.

[1118] 4. Idol Production

[1119] server:

[1120] 1. Information about the live event (date and time, participation method, voting system, etc.) is retrieved from the database and provided to the device via API.

[1121] 2. Manage the information and results of idols participating in the event and save the results in a database.

[1122] Device:

[1123] 1. Call the server's API to get the live event information and display it to the user.

[1124] 2. The idle information entered by the user is sent to the server.

[1125] User:

[1126] 1. Check the live event information and enter your customized idol.

[1127] 5. Live Events and Voting

[1128] server:

[1129] 1. Prepare and hold a virtual live event based on the entry information.

[1130] 2. Receive and tally voting information from the terminals in real time.

[1131] 3. After the event is over, the results are saved in the database and notified to the user.

[1132] Device:

[1133] 1. Use the server's API to retrieve data from the virtual live event and display the live video to the user.

[1134] 2. The voting page accepts votes in real time and sends them to the server.

[1135] User:

[1136] 1. Browse live events and vote for your favorite idol.

[1137] 2. Check the results after the event is over.

[1138] 6. Implementing the Emotion Engine

[1139] 6.1 Emotion recognition

[1140] server:

[1141] 1. Analyze user emotional data (facial expressions, voice, text) through an emotion engine and provide an API that reflects the results in real time.

[1142] Device:

[1143] 1. Use the API to obtain emotion recognition results and incorporate them into a virtual live performance.

[1144] User:

[1145] 1. Your emotions are recognized during live events.

[1146] 6.2 Performance Tuning

[1147] server:

[1148] 1. Implement an algorithm to adjust the virtual live performance based on the recognition results of the emotion engine.

[1149] 2. The recognized emotion data is stored in a database and used to adjust performance in future applications.

[1150] Device:

[1151] 1. Use the server's API to retrieve the adjusted live performance data and display it to the user.

[1152] Specific examples

[1153] For example, if User A applies to be a new model, he or she first accesses the model recruitment page on his or her device, enters the necessary information such as name, profile, and photo, and submits it. The server saves this information in the database and completes the model registration.

[1154] User B logs in to the system, retrieves a list of models on their device, and selects their favorite model. Based on the selected model information, they can freely customize hairstyle, clothing, accessories, etc.

[1155] A few days later, a live event is held. User B enters the idol he created into the event and competes against other users. During the live event, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions, and the results are reflected in their performance in real time.

[1156] Through this series of steps, users can raise their own original idol in a virtual space and enjoy watching it grow and react emotionally.

[1157] The processing flow will be explained below.

[1158] Program processing steps

[1159] Model recruitment process steps

[1160] Step 1:

[1161] Server: Prepares form data for model recruitment and provides it to the device via API.

[1162] Step 2:

[1163] Device: Calls the API and displays the model recruitment page based on the retrieved form data.

[1164] Step 3:

[1165] User: Enter the required information such as name, profile, photo, etc. on the model recruitment page and submit.

[1166] Step 4:

[1167] Terminal: Sends the entered model application information to the server.

[1168] Step 5:

[1169] Server: Registers the received model application information in a database.

[1170] Idol element selection process steps

[1171] Step 1:

[1172] Server: Extracts registered model information from the database and provides an API in a format that can be accessed by the device.

[1173] Step 2:

[1174] Terminal: Calls the API to obtain model information and displays it to the user.

[1175] Step 3:

[1176] User: Select the model of their choice from the presented models.

[1177] Step 4:

[1178] Terminal: Sends the selected model information to the server.

[1179] Step 5:

[1180] Server: Stores the received model information in a database.

[1181] Idol Customization Process Steps

[1182] Step 1:

[1183] Server: Stores customization options (hairstyle, clothing, accessories, etc.) in a database and provides an API to provide them.

[1184] Step 2:

[1185] Device: Uses the API to retrieve customization options and displays the customization screen to the user.

[1186] Step 3:

[1187] User: Select your preferred customization options, such as hairstyle, clothing, and accessories.

[1188] Step 4:

[1189] Terminal: Sends the selected customization information to the server.

[1190] Step 5:

[1191] Server: Stores the received customization information in a database.

[1192] Idol production process steps

[1193] Step 1:

[1194] Server: Retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it to the device via API.

[1195] Step 2:

[1196] Terminal: Calls the API to retrieve live event information and displays it to the user.

[1197] Step 3:

[1198] User: Check live event information and enter your customized idol.

[1199] Step 4:

[1200] Terminal: Sends the entered idle information to the server.

[1201] Step 5:

[1202] Server: Stores the received entry information in a database.

[1203] Live event execution and voting processing steps

[1204] Step 1:

[1205] Server: Generates data for the virtual live event based on the entry information and starts the live event according to the event time.

[1206] Step 2:

[1207] Terminal: Uses APIs to retrieve data from virtual live events and display live footage to users.

[1208] Step 3:

[1209] Users: View live events and vote for their favorite idols.

[1210] Step 4:

[1211] Terminal: Sends voting information to the server in real time.

[1212] Step 5:

[1213] Server: Counts the received voting information and stores the results in a database.

[1214] Step 6:

[1215] Server: After the event ends, the results are provided to the terminal and displayed to the user.

[1216] Processing steps for implementing an emotion engine

[1217] Step 1:

[1218] Server: Prepares the emotion engine and provides an API for analyzing user emotion data (facial expressions, voice, text).

[1219] Step 2:

[1220] Terminal: Uses API to obtain emotion data analysis results and presents them to the user.

[1221] Step 3:

[1222] Users: Ensure their emotions are recognized while participating in live events.

[1223] Step 4:

[1224] Server: Based on the analyzed emotion recognition data, the virtual live performance is adjusted in real time.

[1225] Step 5:

[1226] Server: The recognition results are stored in a database and used for performance adjustments in the future.

[1227] Step 6:

[1228] Terminal: Uses the server's API to retrieve the adjusted live performance data and display it to the user.

[1229] Specific examples

[1230] For example, if User A applies to be a new model, he or she first accesses the model recruitment page on his or her device, enters the necessary information such as name, profile, and photo, and submits it. The server saves this information in the database and completes the model registration.

[1231] User B logs in to the system, retrieves a list of models on their device, and selects their favorite model. Based on the selected model information, they can freely customize hairstyle, clothing, accessories, etc.

[1232] A few days later, a live event is held. User B enters the idol he created into the event and competes against other users. During the live event, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions, and the results are reflected in their performance in real time.

[1233] Through this series of steps, users can raise their own original idol in a virtual space and enjoy watching it grow and react emotionally.

[1234] Example 2

[1235] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1236] In conventional virtual character development systems, customizing user-created characters and performing live events was cumbersome, and it was not possible to reflect user emotions in real time. Furthermore, it was difficult to efficiently analyze and automate user input data, leaving challenges in improving the user experience.

[1237] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1238] In this invention, the server includes means for accepting application information and registering it in a database, means for acquiring the registered information and providing it to the user, means for customizing a virtual character based on information selected by the user, means for entering the customized virtual character and holding a virtual event, means for accepting votes for the virtual event and tallying the results, and means for providing the tallying results to the user. This allows users to easily customize their virtual characters and participate in events and vote in real time. Furthermore, by including means for analyzing user emotional data and reflecting this in performance during the virtual event in real time, and means for adjusting the performance of the virtual event based on the analyzed emotional data, it is possible to provide a performance that reflects the user's emotions. Furthermore, by including means for analyzing user input data using a generative AI model and automating operations at each step, the user experience can be improved.

[1239] "Application information" refers to information such as name, profile, and photo that a user provides to create a virtual character.

[1240] "Database" refers to an information system for storing application information, registered information, customized virtual character information, emotional data, voting results, and the like.

[1241] A "virtual character" is a digital agent or avatar that a user can create and customize in a digital environment.

[1242] A "virtual event" is an event in a digital environment where virtual characters perform and users can participate and vote in real time.

[1243] A "vote" is an act in which a user expresses their opinion or support for a particular virtual character during a virtual event.

[1244] "Emotional data" refers to information about emotions extracted from a user's facial expressions, voice, and text, and is reflected in their performance during the virtual event.

[1245] "Performance" refers to the movements, actions, and performances made by a virtual character during a virtual event.

[1246] A "generative AI model" is an artificial intelligence model used to analyze input data from a user and automate operations at each step.

[1247] An "interface" is a screen or operating means that allows a user to interact with the system, enabling operations such as voting and customization.

[1248] The system of the present invention allows users to create original virtual characters in a virtual space, customize those characters, and participate in virtual events. It also has the function of recognizing the user's emotions and reflecting those emotions in the performance of the virtual event. A specific embodiment of the system is described below.

[1249] Model Wanted

[1250] server:

[1251] The server uses a web framework (e.g., Django, Flask) to prepare form data for model recruitment, provides an API endpoint for this (e.g., " / api / model recruitment form"), and receives application information (name, profile, photo, etc.) sent from the device and registers it in a database (e.g., MySQL, PostgreSQL).

[1252] Device:

[1253] The device sends an HTTP GET request to the server's API, retrieves form data in JSON format, displays the model recruitment page using an HTML template (e.g., React, Vue.js), captures the model application information entered by the user, and sends it to the server as an HTTP POST request.

[1254] (Example)

[1255] When User A applies to be a new model, he / she accesses the model recruitment page on his / her device, enters information such as the name "John Doe", profile "singer-songwriter", and photo "john_doe.jpg", and submits it. The server receives this information and stores it in a database.

[1256] Idol Body Selection

[1257] server:

[1258] The server extracts registered model information from the database and provides it at the API endpoint " / api / models".

[1259] Device:

[1260] The device sends an HTTP GET request to the API and displays a list of models to the user using the retrieved information. Model cards are dynamically generated using React and Vue.js, allowing users to select their preferred model.

[1261] (Example)

[1262] User B logs in to the system and selects "John Doe" from the list of registered models.

[1263] Idol Customization

[1264] server:

[1265] The server prepares customization option data (hairstyle, clothing, accessories) and provides it at the API endpoint " / api / customize_options". It also saves the customization information changed by the user in the database.

[1266] Device:

[1267] The device sends an HTTP GET request to the API, displays the customization screen using the retrieved option data, captures the customization information selected by the user, and sends it to the server as an HTTP POST request.

[1268] (Example)

[1269] User B changes "John Doe's" hairstyle to "short hair", clothing to "casual", and accessories to "sunglasses", and sends this information to the server, which stores this information in a database.

[1270] Idol production

[1271] server:

[1272] The server retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it via the API endpoint " / api / live_events". It also saves entry information in the database and prepares and executes the virtual event.

[1273] Device:

[1274] The device sends an HTTP GET request to the API to retrieve live event information and display it to the user. The device also sends the idle information entered by the user to the server as an HTTP POST request.

[1275] (Example)

[1276] User B enters his virtual character into a live event and competes against other users.

[1277] Live Events and Voting

[1278] server:

[1279] The server prepares a virtual live event based on the entry information, generates live video using game engines such as Unity or Unreal Engine, and receives and tally vote information in real time.

[1280] Device:

[1281] The device retrieves streaming data from the API, displays live video to the user using WebRTC or HLS, and captures voting information and sends it to the server as an HTTP POST request.

[1282] (Example)

[1283] Users watch a live event and vote for "John Doe." The server tallies the votes and notifies the results after the event ends.

[1284] Implementing the Emotion Engine

[1285] server:

[1286] The server uses an emotion engine (e.g., Microsoft Azure's Emotion API or Google Cloud's Speech-to-Text API) to analyze users' emotional data, provides an API that reflects the results in the virtual event performance, implements an algorithm that adjusts the virtual performance based on the emotional data, and stores the data in a database.

[1287] Device:

[1288] The device obtains emotion recognition results from the API and reflects them in real time in the virtual live performance.

[1289] (Example)

[1290] During the live event, the user's facial expression data is analyzed, and if the emotion is recognized as "joy," that emotion is reflected in the virtual character's performance in real time.

[1291] Example of generative AI model and prompt used

[1292] Generative AI models:

[1293] It uses OpenAI GPT-4 to analyze input data from users and automate operations at each step.

[1294] Example prompt sentence:

[1295] Based on the system's processing steps, please provide a detailed explanation of the steps a user takes to create an original idol in a virtual space and have that idol participate in a live event. Also, please provide a detailed explanation of how the emotion engine reflects the user's emotions in the live performance.

[1296] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1297] Step 1:

[1298] Process: Preparation and provision of form data for model recruitment.

[1299] Subject and action: The server uses a web framework to prepare form data for model recruitment and provides an API endpoint for this purpose: " / api / model recruitment form".

[1300] Input: Template data for model submission on the server.

[1301] Data processing / calculation: The server converts the template data into JSON format.

[1302] Output: The form data in JSON format provided by the API endpoint.

[1303] Specific operation: For example, define a model recruitment form using Django, convert the form data to JSON, and provide it via an API endpoint.

[1304] Step 2:

[1305] Processing content: Display model recruitment page.

[1306] Subject and Action: The device sends an HTTP GET request to the server's API, retrieves form data in JSON format, and displays a model recruitment page using an HTML template.

[1307] Input: JSON formatted form data retrieved from an API endpoint.

[1308] Data processing / calculation: The terminal parses the JSON data and embeds it into an HTML template.

[1309] Output: The model recruitment page shown to the user.

[1310] Specific operation: Generate form elements based on JSON data obtained using React and display them in the browser.

[1311] Step 3:

[1312] What it does: Collect and submit model application information.

[1313] Subject and Action: The user enters the required information such as name, profile, and photo on the model recruitment page, and the device sends this to the server as an HTTP POST request.

[1314] Input: Data entered by the user, such as name, profile, and photo.

[1315] Data processing / calculation: The terminal converts the form data into JSON format.

[1316] Output: JSON formatted application information sent to the server.

[1317] What it does: When a user enters information into a form and clicks the "Submit" button, the device captures the information and sends it to the server.

[1318] Step 4:

[1319] Process: Receiving and registering model application information.

[1320] Subject and Action: The server receives the model application information sent from the terminal and registers it in the database.

[1321] Input: Model application information received from the device in JSON format.

[1322] Data processing / calculation: The server parses the JSON data and inserts it into the database.

[1323] Output: Application information stored in a database.

[1324] Specific behavior: For example, use a Django model to save the received application information to a database.

[1325] Step 5:

[1326] Processing: Extracts and provides registered model information.

[1327] Subject and action: The server extracts registered model information from the database and provides it at the API endpoint " / api / models".

[1328] Input: A request for model information from the database.

[1329] Data processing / computation: The server executes the database query and converts the results into JSON format.

[1330] Output: Model information in JSON format provided by the API endpoint.

[1331] What happens: The server executes a MySQL query to get the model information and provides it to the endpoint in JSON format.

[1332] Step 6:

[1333] Process: Retrieves and displays model information.

[1334] Subject and Action: The device sends an HTTP GET request to the API endpoint to retrieve model information and display it to the user.

[1335] Input: Model information in JSON format obtained from the API endpoint.

[1336] Data processing / calculation: The terminal parses the JSON data and embeds it into an HTML template.

[1337] Output: A page listing the model information that is displayed to the user.

[1338] Specific operation: Generate a list based on the JSON data obtained using React and display it in the browser.

[1339] Step 7:

[1340] What it does: Select a model.

[1341] Subject and Action: The user selects the model of their choice from the list of models presented.

[1342] Input: A user-chosen model ID.

[1343] Data processing / calculation: The device captures the selected model ID.

[1344] Output: The selected model ID.

[1345] Specific behavior: The user clicks on "John Doe" from the list of models.

[1346] Step 8:

[1347] What it does: Provides customization options.

[1348] Subject and Action: The server provides customization option data (hairstyle, clothing, accessories) at the API endpoint " / api / customize_options".

[1349] Input: Customization options request from database.

[1350] Data processing / computation: The server executes the database query and converts the results into JSON format.

[1351] Output: Customization options provided in JSON format at the API endpoint.

[1352] Specific operation: The server obtains the customization data and provides it to the endpoint in JSON format.

[1353] Step 9:

[1354] Processing content: Display the customization screen.

[1355] Subject and Action: The device sends an HTTP GET request to the API endpoint to retrieve the customization options and display the customization screen.

[1356] Input: Customization options in JSON format retrieved from the API endpoint.

[1357] Data processing / calculation: The terminal parses the JSON data and embeds it into an HTML template.

[1358] Output: The customized screen that is displayed to the user.

[1359] What it does: Uses React to display customization options on the screen and allow the user to select them.

[1360] Step 10:

[1361] Process: Sending customization information.

[1362] Subject and Action: The user selects customization options (hairstyle, clothing, accessories), and the device sends that information to the server as an HTTP POST request.

[1363] Input: The customization options selected by the user (e.g. hairstyle ID, outfit ID, accessory ID).

[1364] Data processing / calculation: The terminal converts the customization data into JSON format.

[1365] Output: Customization information in JSON format sent to the server.

[1366] What happens: When a user saves a customization, the device sends that information to the server.

[1367] Step 11:

[1368] What it does: Saves customization information.

[1369] Subject and Action: The server receives the customization information sent from the terminal and registers it in the database.

[1370] Input: Customization information received from the terminal in JSON format.

[1371] Data processing / calculation: The server parses the JSON data and inserts it into the database.

[1372] Output: Customization information stored in a database.

[1373] What it does: Uses Django models to store customization information in the database.

[1374] Step 12:

[1375] What it does: Providing live event information.

[1376] Subject and Action: The server retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it at the API endpoint " / api / live_events".

[1377] Input: A request for live event information from the database.

[1378] Data processing / computation: The server executes the database query and converts the results into JSON format.

[1379] Output: Live event information in JSON format provided by the API endpoint.

[1380] Specific behavior: The server retrieves live event information and provides it to the endpoint in JSON format.

[1381] Step 13:

[1382] What it does: Displays live event information.

[1383] Subject and Action: The device sends an HTTP GET request to the API endpoint to retrieve live event information and display it to the user.

[1384] Input: Live event information in JSON format retrieved from the API endpoint.

[1385] Data processing / calculation: The terminal parses the JSON data and embeds it into an HTML template.

[1386] Output: The live event information page that is displayed to the user.

[1387] Specific operation: Display live event information obtained using React in the browser.

[1388] Step 14:

[1389] Processing: Idle entry.

[1390] Subject and Action: A user enters a customized virtual character into a live event, and the device sends this information to the server as an HTTP POST request.

[1391] Input: The ID of the virtual character the user is entering.

[1392] Data processing / calculation: The terminal converts the entry information into JSON format.

[1393] Output: Entry information in JSON format sent to the server.

[1394] Specific operation: When the user presses the entry button, the terminal sends the information to the server.

[1395] Step 15:

[1396] What it does: Store your entry information and conduct the virtual live event.

[1397] Subject and Action: The server stores the entry information in a database and prepares and executes the virtual live event.

[1398] Input: Entry information received from the device in JSON format.

[1399] Data processing / computation: The server parses the JSON data, inserts it into a database, and runs algorithms to generate the live performance.

[1400] Output: Entry information stored in a database and live performances generated.

[1401] How it works: Live video is generated using Unity or Unreal Engine and distributed via a streaming server.

[1402] Step 16:

[1403] What it does: Show live events and take votes.

[1404] Subject and Action: The device retrieves live streaming data from the API and displays it to the user. It also captures voting information and sends it to the server as an HTTP POST request.

[1405] Input: Streaming data retrieved from API endpoints and user voting information.

[1406] Data processing / calculation: The device plays back the streaming data and converts the voting information into JSON format.

[1407] Output: The live video displayed to the user and the voting information sent to the server.

[1408] Specific operation: Live video is displayed using WebRTC and HLS, and when a user presses the voting button, the information is sent to the server.

[1409] Step 17:

[1410] Process: Counting voting information and notifying results.

[1411] Subject and Action: The server receives and tally vote information from the terminals in real time. After the event ends, it saves the results in a database and notifies the users.

[1412] Input: Voting information received from the device in JSON format.

[1413] Data processing / calculation: The server aggregates the voting data and converts the results into JSON format.

[1414] Output: Voting results stored in a database and notification to the user.

[1415] Specific operation: The server tally the voting information, calculate the rankings, and notify the results after the event ends.

[1416] Step 18:

[1417] What it does: Analyzes emotion data and adjusts performance.

[1418] Subject and Action: The server uses an emotion engine to analyze the user's facial expressions and voice data, and reflects the results in the virtual event performance in real time. It also adjusts the virtual performance based on the analysis results and stores the data in a database.

[1419] Input: User's facial and voice data.

[1420] Data processing / calculation: Analyze data using the emotion engine and run algorithms that reflect the results in performance.

[1421] Output: Emotion data stored in a database and adjusted performance.

[1422] What it does: Analyzes emotion data and adjusts performance using Microsoft Azure's Emotion API and Google Cloud's Speech-to-Text API.

[1423] (Application example 2)

[1424] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1425] Conventional systems for cultivating and performing idols in virtual spaces lacked features such as performances that reflected the user's emotions in real time and promotional activities within virtual stores. This limited the user experience and made it difficult to provide more interactive and engaging content. Furthermore, there was a need for an effective system that could utilize emotional data to adjust idols' performances and realize continuous promotional activities.

[1426] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1427] In this invention, the server includes means for accepting model application information and registering it in a database, means for acquiring the registered model information and providing it to the user, means for customizing an idol based on the model information selected by the user, means for entering the customized idol and holding a virtual live event, means for accepting votes in the virtual live event and tallying the results, means for providing the tallying results to the user, means for users to implement a promotion in which the idol introduces products in a virtual store, and means for recognizing the user's emotions and reflecting the emotion data in the idol's performance. This makes it possible to create, train, and adjust the performance of idols in a virtual space, and furthermore, promotional activities in the virtual store are continuously carried out, making it possible to provide an interactive experience that reflects the user's emotions.

[1428] "Model application information" refers to information such as name, profile, and photo entered by a user who wishes to create an idol in a virtual space.

[1429] "Database" refers to a storage device for storing and managing model application information, customized idol information, and data on virtual live events and promotional activities.

[1430] A "user" is a person who uses the system to create, customize, and enter idols in a virtual space and participate in virtual live events.

[1431] "Customization" is an operation in which the user changes the appearance and attributes of the selected model, such as hairstyle, clothing, and accessories, to create an individual idol.

[1432] "Entry" refers to an operation to allow a customized idol to participate in a virtual live event or promotional activity.

[1433] A "virtual live event" is a live performance competition event held in a virtual space, in which idols entered by users compete in performance against other users.

[1434] "Voting" is an act by which a user shows support for their favorite idol at a virtual live event, and the results are tallied.

[1435] "Counting" is the process of compiling the voting results received at the virtual live event and calculating the final score.

[1436] A "virtual store" is a fictitious store space that exists on the Internet, where virtual idols introduce products and engage in promotional activities.

[1437] "Promotional activities" are the act of idols introducing and advertising products within a virtual store.

[1438] "Emotion recognition" is a technology that analyzes emotions from a user's facial expressions, voice, etc. and incorporates those emotions into the system.

[1439] "Reflecting in performance" means adjusting the virtual idol's movements and words in real time based on the user's emotions obtained through emotion recognition, resulting in a more interactive performance.

[1440] The system for implementing the present invention includes a server, a user terminal, and an emotion engine as its main components. The system operates as follows.

[1441] The server accepts application information for models and registers it in a database. When accessed from a user's device, the server provides an API that provides registered model information. The idol is customized based on the model information obtained by the user via their device. The customized idol is then sent to the server as entry information, and the virtual live event is prepared.

[1442] During the virtual live event, the server processes data in real time, accepts and tally votes, and provides the results to users. The user terminal also displays the virtual live performance in real time and provides an interface for users to vote.

[1443] Furthermore, users can use idols to promote products in virtual stores. The emotion engine analyzes users' emotional data, such as facial expressions and voice, and reflects that data in real time in the performance of the virtual idols, providing users with an interactive and engaging experience.

[1444] The hardware used includes smartphones and head-mounted displays. These devices are equipped with camera modules and microphones to capture the user's facial expressions and voice. The software uses EmotionEngine for emotion recognition, VirtualIdol for managing virtual idols, and the Requests library for API communication.

[1445] As a concrete example, a user accesses a model recruitment page using a smartphone, enters information such as name, profile, and photo, and submits it. The server stores this information in a database, completing the model registration. When the user promotes a new product in a virtual store, the emotion engine detects the user's joy and excitement and reflects this in real time in the idol's words, actions, and performance.

[1446] An example prompt for a generative AI model is:

[1447] "Create a smartphone app for promoting virtual idols. Include a process where users can create a virtual idol and introduce products in a virtual store. Implement a function where the idol adjusts their performance based on the user's emotions. Also, consider a voting system where users can compete with other users, and describe how to synchronize data with the server."

[1448] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1449] Step 1:

[1450] A user accesses the model recruitment page, enters model application information such as name, profile, and photo, and submits it.

[1451] Input: Data entered by users, such as name, profile, and photo.

[1452] Output: The input data is sent to the server.

[1453] Specific behavior:

[1454] Users access the model recruitment form on their smartphone or head-mounted display screen, enter the necessary information, and then press the send button to send the application information to the server.

[1455] Step 2:

[1456] The server accepts model application information and registers it in a database.

[1457] Input: Model application information submitted by the user.

[1458] Output: Application information registered in the database.

[1459] Specific behavior:

[1460] The server receives application information from users via API communication, stores the data in a database, and then returns a registration completion response to the user.

[1461] Step 3:

[1462] The user accesses the virtual idol's customization page using a terminal and customizes the virtual idol by selecting hairstyle, clothes, accessories, etc.

[1463] Input: Model information stored in the database, user customization selections.

[1464] Output: Customized idle information.

[1465] Specific behavior:

[1466] The user adjusts the appearance of the virtual idol using the customization options provided, and once selections are complete, the customization information is sent to the server.

[1467] Step 4:

[1468] The server stores the customized idol in a database and registers the entry information for the virtual live event.

[1469] Input: Customization information submitted by the user.

[1470] Output: Customized idle information stored in the database, and entry information.

[1471] Specific behavior:

[1472] The server receives the customization information and stores it in a database, while also registering the entry information for participating in the virtual live event.

[1473] Step 5:

[1474] During a virtual live event, the emotion engine analyzes the user's facial expressions and voice to generate emotion data.

[1475] Input: User's facial expression data, voice data.

[1476] Output: User emotion data.

[1477] Specific behavior:

[1478] The emotion engine analyzes the user's facial expressions and voice in real time, generating emotional data that is then reflected in the virtual idol's performance.

[1479] Step 6:

[1480] The device adjusts the virtual idol's performance based on emotional data and displays live footage.

[1481] Input: Parsed emotion data.

[1482] Output: Performance footage of the virtual idol after adjustments.

[1483] Specific behavior:

[1484] The device adjusts the movements and behavior of the virtual idol based on the emotional data sent from the emotion engine, and displays live footage in real time to provide the user with a more interactive experience.

[1485] Step 7:

[1486] A server accepts and tallys voting data during the virtual live event.

[1487] Input: Voting information from users.

[1488] Output: The tallied vote results.

[1489] Specific behavior:

[1490] The server receives the voting data sent by users in real time, tallying it up and calculating the final results, and stores the results in a database after the live event ends.

[1491] Step 8:

[1492] A promotion is carried out in which a user introduces products using idols in a virtual store.

[1493] Input: User promotion request information, customized idle information.

[1494] Output: Product introduction video by a virtual idol.

[1495] Specific behavior:

[1496] Users can request product promotions in the virtual store, and customized idols will generate and display videos introducing the products.

[1497] Step 9:

[1498] The terminal reflects the emotional data generated in real time in the promotional performance of the virtual idol.

[1499] Input: Emotion data, promotional video data.

[1500] Output: Adjusted promotional performance footage.

[1501] Specific behavior:

[1502] The terminal reflects the emotional data acquired from the emotion engine in the promotional video and adjusts the virtual idol's movements and speech to correspond to the user's emotions.

[1503] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1504] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1505] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1506] [Third embodiment]

[1507] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1508] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1509] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1510] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1511] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1512] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1513] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1514] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1515] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1516] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1517] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1518] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1519] The system of the present invention allows users to create original idols in a virtual space and have them compete with other users in terms of their performance. The system has a variety of functions, and specific examples thereof will be described below.

[1520] 1. Model Recruitment

[1521] server:

[1522] 1. Prepare form data for recruiting models and provide an API for devices to access it.

[1523] 2. Receive model application information sent from the device and register it in the database.

[1524] Device:

[1525] 1. Use the server's API to retrieve the form data and display the model recruitment page.

[1526] 2. The model application information entered by the user is sent to the server.

[1527] User:

[1528] 1. Enter the required information such as your name, profile, and photo on the model recruitment page and submit.

[1529] 2. Selecting an idol body

[1530] server:

[1531] 1. Extract registered model information from the database and provide an API in a form that can be accessed by the device.

[1532] Device:

[1533] 1. Call the server's API to obtain model information and display it as a list to the user.

[1534] User:

[1535] 1. Select the model you like from the presented models.

[1536] 3. Idol Customization

[1537] server:

[1538] 1. Store customization options (hairstyle, clothing, accessories, etc.) in a database and provide an API to provide them.

[1539] 2. Save the customization information changed by the user to the database.

[1540] Device:

[1541] 1. Use the server's API to get the customization options and display the customization screen to the user.

[1542] 2. Send the customization information selected by the user to the server.

[1543] User:

[1544] 1. Choose your favorite customization options, such as hairstyle, clothing, and accessories.

[1545] 4. Idol Production

[1546] server:

[1547] 1. Information about the live event (date and time, participation method, voting system, etc.) is retrieved from the database and provided to the device via API.

[1548] 2. Manage the information and results of idols participating in the event and save the results in a database.

[1549] Device:

[1550] 1. Call the server's API to get the live event information and display it to the user.

[1551] 2. The idle information entered by the user is sent to the server.

[1552] User:

[1553] 1. Check the live event information and enter your customized idol.

[1554] 5. Live Events and Voting

[1555] server:

[1556] 1. Prepare and hold a virtual live event based on the entry information.

[1557] 2. Receive and tally voting information from the terminals in real time.

[1558] 3. After the event is over, the results are saved in the database and notified to the user.

[1559] Device:

[1560] 1. Use the server's API to retrieve data from the virtual live event and display the live video to the user.

[1561] 2. The voting page accepts votes in real time and sends them to the server.

[1562] User:

[1563] 1. Browse live events and vote for your favorite idol.

[1564] 2. Check the results after the event is over.

[1565] Specific examples

[1566] When User A applies to be a new model, he / she first accesses the model recruitment page on his / her device, enters the necessary information such as name, profile, photo, etc., and submits it. The server saves this information in the database and completes the model registration.

[1567] User B logs in to the system, retrieves a list of models on their device, and selects their favorite model. Based on the selected model information, they can freely customize hairstyle, clothing, accessories, etc.

[1568] A few days later, a live event is held. User B enters the idol he created into the event and competes with other users. During the live event, users vote in real time, and the server tallies the votes.

[1569] Through this series of steps, users can raise their own original idol in a virtual space and enjoy watching it grow.

[1570] The processing flow will be explained below.

[1571] Program processing steps

[1572] Model recruitment process steps

[1573] Step 1:

[1574] Server: Prepares form data for model recruitment and provides it to the device via API.

[1575] Step 2:

[1576] Device: Calls the API and displays the model recruitment page based on the retrieved form data.

[1577] Step 3:

[1578] User: Enter the required information such as name, profile, photo, etc. on the model recruitment page and submit.

[1579] Step 4:

[1580] Terminal: Sends the entered model application information to the server.

[1581] Step 5:

[1582] Server: Registers the received model application information in a database.

[1583] Idol element selection process steps

[1584] Step 1:

[1585] Server: Extracts registered model information from the database and provides an API in a format that can be accessed by the device.

[1586] Step 2:

[1587] Terminal: Calls the API to obtain model information and displays it to the user.

[1588] Step 3:

[1589] User: Select the model of their choice from the presented models.

[1590] Step 4:

[1591] Terminal: Sends the selected model information to the server.

[1592] Step 5:

[1593] Server: Stores the received model information in a database.

[1594] Idol Customization Process Steps

[1595] Step 1:

[1596] Server: Stores customization options (hairstyle, clothing, accessories, etc.) in a database and provides an API to provide them.

[1597] Step 2:

[1598] Device: Uses the API to retrieve customization options and displays the customization screen to the user.

[1599] Step 3:

[1600] User: Select your preferred customization options, such as hairstyle, clothing, and accessories.

[1601] Step 4:

[1602] Terminal: Sends the selected customization information to the server.

[1603] Step 5:

[1604] Server: Stores the received customization information in a database.

[1605] Idol production process steps

[1606] Step 1:

[1607] Server: Retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it to the device via API.

[1608] Step 2:

[1609] Terminal: Calls the API to retrieve live event information and displays it to the user.

[1610] Step 3:

[1611] User: Check live event information and enter your customized idol.

[1612] Step 4:

[1613] Terminal: Sends the entered idle information to the server.

[1614] Step 5:

[1615] Server: Stores the received entry information in a database.

[1616] Live event execution and voting processing steps

[1617] Step 1:

[1618] Server: Generates data for the virtual live event based on the entry information and starts the live event according to the event time.

[1619] Step 2:

[1620] Terminal: Uses APIs to retrieve data from virtual live events and display live footage to users.

[1621] Step 3:

[1622] Users: View live events and vote for their favorite idols.

[1623] Step 4:

[1624] Terminal: Sends voting information to the server in real time.

[1625] Step 5:

[1626] Server: Counts the received voting information and stores the results in a database.

[1627] Step 6:

[1628] Server: After the event ends, the results are provided to the terminal and displayed to the user.

[1629] Through these modularized processing steps, users can create their own original idol and experience the entire process of having that idol compete against other users in a virtual live performance.

[1630] Example 1

[1631] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1632] Conventional virtual idol creation and live event systems have difficulty allowing users to create original idols and easily compete with each other in performance. Real-time voting and tallying are also difficult, resulting in a poor user experience. Furthermore, customized idol information cannot be easily reused, resulting in a lack of convenience for users who wish to use the system repeatedly.

[1633] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1634] In this invention, the server includes means for accepting model application information and registering it in a database, means for acquiring the registered model information and providing it to a terminal, means for customizing an idol based on the model information selected by the user, means for entering the customized idol and holding a virtual live event, means for accepting votes for the virtual live event and tallying them in real time, means for storing the tally results in a database and providing them to the user, means for storing the customized idol information in the database and making it available for future live events, and means for providing an interface for users to view the idol's performance in real time and vote during the virtual live event. This allows users to easily create and customize original idols in a virtual space and compete against other users in their performances. Real-time voting and tallying are also performed smoothly, improving the user experience. Furthermore, reusing customized idol information improves convenience during repeated use.

[1635] "Model application information" refers to information submitted by a user to apply to be a model, and specifically includes the user's name, profile, photo, etc.

[1636] A "database" is a system that stores information in an organized manner and allows for efficient searching and updating.

[1637] A "terminal" is an electronic device such as a computer or smartphone used by a user.

[1638] An "API" is an interface that allows different software components to communicate with each other.

[1639] "Idols" are characters created and customized by users in the virtual space, and are participants in live events.

[1640] "Customization" refers to the act of a user setting or changing an idol's appearance, such as hairstyle, clothing, and accessories.

[1641] A "virtual live event" is a live performance event that takes place in a virtual space and can be viewed and participated in by multiple users.

[1642] "Voting" is an evaluation action that a user takes for their favorite idol during or after a live event.

[1643] "Interface" refers to the screen and input means that users use to operate the system.

[1644] "Counting" is the process of calculating and analyzing the collected voting data and calculating the results.

[1645] "Storage" is the act of recording information in a database so that it can be reused at a later time.

[1646] The system of the present invention allows users to create original idols in a virtual space and compete with other users for their performance. The system operates in cooperation with a server, terminals, and users. Here, we will explain a specific embodiment of the system, including what hardware and software are used to process and calculate data.

[1647] 1. Model Recruitment

[1648] The server prepares form data for model recruitment and provides an API that the device can access (e.g., https: / / example.com / api / model-form). This allows the device to call the server's API and display the model recruitment page based on the obtained form data. On this page, the user enters application information such as name, profile, and photo and submits it. The device sends the information entered by the user to the server, and the server registers the application information in a database.

[1649] For example, the prompt text might look like this:

[1650] "Enter your name, profile, and photo and submit."

[1651] 2. Selecting an idol body

[1652] The server extracts registered model information from the database and provides it as an API (e.g., https: / / example.com / api / models) that can be retrieved by the device. The device calls the server's API, retrieves the model information, and displays it as a list to the user. The user can then select their preferred model from the list.

[1653] For example, the prompt text might look like this:

[1654] Choose your favorite model from the list.

[1655] 3. Idol Customization

[1656] The server stores customization option data (hairstyle, clothing, accessories, etc.) in a database and provides an API (e.g., https: / / example.com / api / customize-options) that provides this data. The device uses the server's API to obtain the customization options and displays a customization screen to the user. The user selects customization options such as hairstyle, clothing, and accessories, and sends the selected customization information to the server via the device. The server stores this customization information in a database.

[1657] For example, the prompt text might look like this:

[1658] "Choose your hairstyle, outfit, and accessories."

[1659] 4. Idol Production

[1660] The server retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it to the device via API (e.g. https: / / example.com / api / event-info). The device calls the server's API to retrieve the live event information and displays it to the user. The user checks this information and enters their customized idol. The device sends the entry information to the server, which receives it and stores it in the database.

[1661] For example, the prompt text might look like this:

[1662] "Please select the idol you would like to enter in the live event."

[1663] 5. Live Events and Voting

[1664] The server prepares and runs the virtual live event based on the entry information (for example, using game engines such as Unity or Unreal Engine). The device uses the server's API to obtain data about the virtual live event and displays live footage to the user. Users view the live event and vote for their favorite idol. The device accepts votes in real time on the voting page and sends them to the server. The server receives this voting information in real time and tallys it. After the event ends, the server saves the results in a database and notifies the user.

[1665] For example, the prompt text might look like this:

[1666] "Vote for your idol during the live event"

[1667] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1668] Step 1: Model recruitment

[1669] server:

[1670] Prepare the form data and provide the API (e.g., https: / / example.com / api / model-form), which allows devices to access the model recruitment form.

[1671] Input: Server preferences, form data

[1672] Output: API endpoint containing form data

[1673] Device:

[1674] Call the server API to get the form data.

[1675] The model recruitment page is displayed to the user.

[1676] Input: Form data obtained from the API

[1677] Output: Model recruitment page shown to the user

[1678] User:

[1679] On the model recruitment page, enter your application information such as your name, profile, and photo, and click the submit button.

[1680] Input: User's application information (name, profile, photo)

[1681] Output: Submitted application information

[1682] Device:

[1683] The application information entered by the user is sent to the server.

[1684] Input: Application information provided by the user

[1685] Output: Application information sent to the server

[1686] server:

[1687] The received application information is registered in a database.

[1688] Input: Application information sent from the device

[1689] Output: Application information stored in a database

[1690] Step 2: Select the idol body

[1691] server:

[1692] Extract model information from the database and provide an API in a format that can be retrieved by the device (e.g. https: / / example.com / api / models).

[1693] Input: Model information in the database

[1694] Output: API endpoint containing model information

[1695] Device:

[1696] Call the server's API to obtain model information and display it as a list to the user.

[1697] Input: Model information obtained from the API

[1698] Output: A list of models displayed to the user

[1699] User:

[1700] Select the model you like from the ones presented.

[1701] Input: List of models

[1702] Output: Selected model information

[1703] Step 3: Customize your idol

[1704] server:

[1705] Store customization options (hairstyles, clothing, accessories, etc.) in a database and provide an API to serve them (e.g., https: / / example.com / api / customize-options).

[1706] Input: Customization options data in the database

[1707] Output: API endpoint with customization options

[1708] Device:

[1709] Use the server's API to get the customization options and display the customization screen to the user.

[1710] Input: Customization options obtained from the API

[1711] Output: The customized screen that is displayed to the user

[1712] User:

[1713] Choose customization options like hairstyle, clothing, accessories, etc.

[1714] Input: Customization options

[1715] Output: Selected customization information

[1716] Device:

[1717] The customization information selected by the user is sent to the server.

[1718] Input: Customization information selected by the user

[1719] Output: Customization information sent to the server

[1720] server:

[1721] The received customization information is stored in a database.

[1722] Input: Customization information sent from the device

[1723] Output: Customization information stored in the database

[1724] Step 4: Producing idols

[1725] server:

[1726] Retrieve live event information (date and time, participation method, voting system, etc.) from the database and provide it to the device via an API (e.g. https: / / example.com / api / event-info).

[1727] Input: Live event information in the database

[1728] Output: API endpoint containing live event information

[1729] Device:

[1730] Call the server's API to get live event information and display it to the user.

[1731] Input: Live event information obtained from the API

[1732] Output: Live event information displayed to the user

[1733] User:

[1734] Check the live event information and enter your customized idol.

[1735] Input: Live Event Information

[1736] Output: Entry information

[1737] Device:

[1738] The idle information entered by the user is sent to the server.

[1739] Input: User entry information

[1740] Output: Entry information sent to the server

[1741] server:

[1742] The received entry information is stored in a database.

[1743] Input: Entry information sent from the terminal

[1744] Output: Entry information stored in the database

[1745] Step 5: Live Event and Voting

[1746] server:

[1747] Prepare and conduct a virtual live event based on the entry information (e.g., using game engines such as Unity or Unreal Engine).

[1748] Input: Entry information stored in the database

[1749] Output: Virtual live event

[1750] Device:

[1751] Use the server's API to retrieve data from the virtual live event and display live footage to the user.

[1752] Input: Live event data retrieved from the API

[1753] Output: Live video displayed to the user

[1754] User:

[1755] Watch live events and vote for your favorite idol.

[1756] Input: Live video

[1757] Output: Voting information

[1758] Device:

[1759] The voting page accepts votes in real time and sends them to the server.

[1760] Input: Voting information from users

[1761] Output: Voting information sent to the server

[1762] server:

[1763] Voting information from the terminals is received in real time and tallied.

[1764] After the event is over, the results are stored in a database and notified to the user.

[1765] Input: Voting information sent from the device

[1766] Output: Aggregation results, saving to database, notifying user

[1767] (Application example 1)

[1768] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1769] Previously, systems existed that allowed users to compete using the personalities and customization elements of idols created in a virtual space. However, they lacked functionality, particularly functionality for linking with virtual stores to try on fashion items and hold fashion shows. Therefore, there is a need for a system that allows users to enjoy a more comprehensive experience, not only by customizing the appearance of their virtual idols, but also by selecting fashion items and participating in fashion show events. There is also a need for a system that allows users to try on fashion items in real time and compete with other users on their evaluations, thereby improving interaction between users.

[1770] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1771] In this invention, the server includes means for accepting application information for models and registering it in a database, means for acquiring the registered model information and providing it to the user, means for customizing an idol based on the model information selected by the user, means for entering the customized idol and holding a virtual live event, means for accepting votes in the virtual live event and tallying the results, means for providing the tallying results to the user, means for providing a virtual store application and displaying customization options for the idol's appearance and clothing, and means for entering the idol customized by the user in a fashion show event and having it compete with other users. This enables users to customize the appearance and fashion of idols created in a virtual space and compete with other users in real time.

[1772] "Model application information" refers to personal information such as name, profile, and photo that a user provides to the system.

[1773] "Model information" refers to all information such as personal information and appearance data of users registered in the database.

[1774] An "idol" is a character that a user designs and customizes in a virtual space.

[1775] "Customization" refers to the process of changing appearance elements such as hairstyle, clothing, and accessories according to the user's preferences.

[1776] A "virtual live event" is a live performance or fashion show held in a virtual space.

[1777] "Voting" is the act of users rating different idols in real time during a live event.

[1778] "Counting" refers to the process of processing the voting results after the live event has ended and calculating the final ratings.

[1779] A "virtual store" is a virtual shop or boutique operated on the Internet.

[1780] An "application" is software that users use on their smartphones or other devices.

[1781] "Customization options" are various options for changing an idol's appearance.

[1782] A "fashion show event" is a virtual event where users can compete with idols they have customized.

[1783] The present invention is a system that allows users to create original idols in a virtual space and have them compete with other users in terms of performance and fashion sense. Specific examples of the system are described below.

[1784] The server first accepts the model's application information and registers it in a database, including information such as name, profile, and photo. It then retrieves the registered model information and provides it to the device via API.

[1785] The device uses the server's API to obtain model information and displays it to the user. The user can then select their preferred model from this list. Based on the selected model information, the user can customize the idol. Customization options include hairstyle, clothing, and accessories.

[1786] After users have completed customization, they can enter their customized idols into virtual live events and fashion show events. In virtual live events, users can view their idols' performances in real time and compete against other users. Voting is also conducted in real time, and the results are tallied on the server. The tallied results are provided to users after the event ends.

[1787] A virtual store application is also provided, giving users access to customization options for the idol's appearance and clothing, allowing them to enjoy the idol's appearance and fashion created in the virtual space and compete with other users.

[1788] The hardware used is a smartphone, and the software used is an application framework such as React Native or Flutter, with Node.js or Python's Flask as the backend.

[1789] For example, a user opens the app, enters their name, profile, and photo on the model recruitment page, and submits them. This information is then stored in a database by the server. Next, the user selects their preferred model from a list of models presented, and selects customization options such as hairstyle, clothing, and accessories. After that, the user enters their customized idol in a fashion show event scheduled on the app, competing against other users. During the live event, users can vote in real time with other users.

[1790] Examples of prompts for generative AI models include:

[1791] "How do I apply as a new model?"

[1792] "I want to change my idol's hairstyle. How can I customize it?"

[1793] "Please let me know about the next live event."

[1794] "How do I vote for my idol against other users' idols?"

[1795] In this way, the system of the present invention provides a forum where users can customize original idols in a virtual space and compete in fashion sense and performance.

[1796] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1797] Step 1:

[1798] The user opens the model recruitment page, enters application information such as name, profile, and photo, and submits it.

[1799] Input: Personal information you enter, such as your name, profile, and photo.

[1800] Data processing: The terminal sends the input information to the server via the API.

[1801] Output: The application information is sent to the server and stored in a database.

[1802] Specific operation: The terminal displays the application information as an input form, the user enters the required information in the form, and clicks the "Submit" button.

[1803] Step 2:

[1804] The server retrieves model information from the database and provides it to the terminal.

[1805] Input: Model information stored on the server.

[1806] Data processing: The server executes database queries to extract model information and provides it to the terminal in API format.

[1807] Output: The model information arrives on the terminal and is displayed in a list to the user.

[1808] Specific operation: The server extracts the necessary model information from the database and sends that information to the terminal as a JSON format API response.

[1809] Step 3:

[1810] The user selects model information and selects customization options on the customization page.

[1811] Input: Model information and customization options selected by the user (hairstyle, clothing, accessories, etc.).

[1812] Data processing: The device sends the selected model information and customization options to the server via API.

[1813] Output: The customized idle information is saved on the server.

[1814] Specific operation: The user selects the model of their choice from the model list, selects customization options, and clicks the "Apply" button.

[1815] Step 4:

[1816] The user enters the customized idol into a fashion show event.

[1817] Input: Customized idle information and selected event information.

[1818] Data processing: The device sends idle information and event information to the server via API.

[1819] Output: The idle is entered into the selected event.

[1820] Specific operation: The user checks the customized idol, selects the event they want to participate in from the list of fashion show events, and clicks the "Enter" button.

[1821] Step 5:

[1822] Users compete against other users in virtual live events.

[1823] Input: Virtual live event configuration information and user votes.

[1824] Data processing: The server tally the voting information in real time and calculate the rankings.

[1825] Output: The server notifies the user of the results of the live event.

[1826] Specific operation: The server tally the voting results in real time and display the rankings and ratings to users after the event ends.

[1827] Step 6:

[1828] A virtual store allows users to access customization options for their idols' appearance and clothing.

[1829] Input: Virtual store customization options information.

[1830] Data processing: The server provides customization option information to the terminal via the API.

[1831] Output: The customization options are displayed on the device and can be selected by the user.

[1832] Specific operation: The server extracts customization option information from the virtual store and sends it to the terminal in API format. The user can then check and select the options on the terminal.

[1833] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1834] The system of the present invention allows users to create original idols in a virtual space and compete with other users in their performances. Furthermore, by combining it with an emotion engine, it has the function of recognizing the user's emotions and reflecting them in the virtual live performance. A specific example of this is described below.

[1835] 1. Model Recruitment

[1836] server:

[1837] 1. Prepare form data for recruiting models and provide an API for devices to access it.

[1838] 2. Receive model application information sent from the device and register it in the database.

[1839] Device:

[1840] 1. Use the server's API to retrieve the form data and display the model recruitment page.

[1841] 2. The model application information entered by the user is sent to the server.

[1842] User:

[1843] 1. Enter the required information such as your name, profile, and photo on the model recruitment page and submit.

[1844] 2. Selecting an idol body

[1845] server:

[1846] 1. Extract registered model information from the database and provide an API in a form that can be accessed by the device.

[1847] Device:

[1848] 1. Call the API to get model information and display it to the user.

[1849] User:

[1850] 1. Select the model you like from the presented models.

[1851] 3. Idol Customization

[1852] server:

[1853] 1. Store customization options (hairstyle, clothing, accessories, etc.) in a database and provide an API to provide them.

[1854] 2. Save the customization information changed by the user to the database.

[1855] Device:

[1856] 1. Use the API to get the customization options and display the customization screen to the user.

[1857] 2. Send the customization information selected by the user to the server.

[1858] User:

[1859] 1. Choose your favorite customization options, such as hairstyle, clothing, and accessories.

[1860] 4. Idol Production

[1861] server:

[1862] 1. Information about the live event (date and time, participation method, voting system, etc.) is retrieved from the database and provided to the device via API.

[1863] 2. Manage the information and results of idols participating in the event and save the results in a database.

[1864] Device:

[1865] 1. Call the server's API to get the live event information and display it to the user.

[1866] 2. The idle information entered by the user is sent to the server.

[1867] User:

[1868] 1. Check the live event information and enter your customized idol.

[1869] 5. Live Events and Voting

[1870] server:

[1871] 1. Prepare and hold a virtual live event based on the entry information.

[1872] 2. Receive and tally voting information from the terminals in real time.

[1873] 3. After the event is over, the results are saved in the database and notified to the user.

[1874] Device:

[1875] 1. Use the server's API to retrieve data from the virtual live event and display the live video to the user.

[1876] 2. The voting page accepts votes in real time and sends them to the server.

[1877] User:

[1878] 1. Browse live events and vote for your favorite idol.

[1879] 2. Check the results after the event is over.

[1880] 6. Implementing the Emotion Engine

[1881] 6.1 Emotion recognition

[1882] server:

[1883] 1. Analyze user emotional data (facial expressions, voice, text) through an emotion engine and provide an API that reflects the results in real time.

[1884] Device:

[1885] 1. Use the API to obtain emotion recognition results and incorporate them into a virtual live performance.

[1886] User:

[1887] 1. Your emotions are recognized during live events.

[1888] 6.2 Performance Tuning

[1889] server:

[1890] 1. Implement an algorithm to adjust the virtual live performance based on the recognition results of the emotion engine.

[1891] 2. The recognized emotion data is stored in a database and used to adjust performance in future applications.

[1892] Device:

[1893] 1. Use the server's API to retrieve the adjusted live performance data and display it to the user.

[1894] Specific examples

[1895] For example, if User A applies to be a new model, he or she first accesses the model recruitment page on his or her device, enters the necessary information such as name, profile, and photo, and submits it. The server saves this information in the database and completes the model registration.

[1896] User B logs in to the system, retrieves a list of models on their device, and selects their favorite model. Based on the selected model information, they can freely customize hairstyle, clothing, accessories, etc.

[1897] A few days later, a live event is held. User B enters the idol he created into the event and competes against other users. During the live event, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions, and the results are reflected in their performance in real time.

[1898] Through this series of steps, users can raise their own original idol in a virtual space and enjoy watching it grow and react emotionally.

[1899] The processing flow will be explained below.

[1900] Program processing steps

[1901] Model recruitment process steps

[1902] Step 1:

[1903] Server: Prepares form data for model recruitment and provides it to the device via API.

[1904] Step 2:

[1905] Device: Calls the API and displays the model recruitment page based on the retrieved form data.

[1906] Step 3:

[1907] User: Enter the required information such as name, profile, photo, etc. on the model recruitment page and submit.

[1908] Step 4:

[1909] Terminal: Sends the entered model application information to the server.

[1910] Step 5:

[1911] Server: Registers the received model application information in a database.

[1912] Idol element selection process steps

[1913] Step 1:

[1914] Server: Extracts registered model information from the database and provides an API in a format that can be accessed by the device.

[1915] Step 2:

[1916] Terminal: Calls the API to obtain model information and displays it to the user.

[1917] Step 3:

[1918] User: Select the model of their choice from the presented models.

[1919] Step 4:

[1920] Terminal: Sends the selected model information to the server.

[1921] Step 5:

[1922] Server: Stores the received model information in a database.

[1923] Idol Customization Process Steps

[1924] Step 1:

[1925] Server: Stores customization options (hairstyle, clothing, accessories, etc.) in a database and provides an API to provide them.

[1926] Step 2:

[1927] Device: Uses the API to retrieve customization options and displays the customization screen to the user.

[1928] Step 3:

[1929] User: Select your preferred customization options, such as hairstyle, clothing, and accessories.

[1930] Step 4:

[1931] Terminal: Sends the selected customization information to the server.

[1932] Step 5:

[1933] Server: Stores the received customization information in a database.

[1934] Idol production process steps

[1935] Step 1:

[1936] Server: Retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it to the device via API.

[1937] Step 2:

[1938] Terminal: Calls the API to retrieve live event information and displays it to the user.

[1939] Step 3:

[1940] User: Check live event information and enter your customized idol.

[1941] Step 4:

[1942] Terminal: Sends the entered idle information to the server.

[1943] Step 5:

[1944] Server: Stores the received entry information in a database.

[1945] Live event execution and voting processing steps

[1946] Step 1:

[1947] Server: Generates data for the virtual live event based on the entry information and starts the live event according to the event time.

[1948] Step 2:

[1949] Terminal: Uses APIs to retrieve data from virtual live events and display live footage to users.

[1950] Step 3:

[1951] Users: View live events and vote for their favorite idols.

[1952] Step 4:

[1953] Terminal: Sends voting information to the server in real time.

[1954] Step 5:

[1955] Server: Counts the received voting information and stores the results in a database.

[1956] Step 6:

[1957] Server: After the event ends, the results are provided to the terminal and displayed to the user.

[1958] Processing steps for implementing an emotion engine

[1959] Step 1:

[1960] Server: Prepares the emotion engine and provides an API for analyzing user emotion data (facial expressions, voice, text).

[1961] Step 2:

[1962] Terminal: Uses API to obtain emotion data analysis results and presents them to the user.

[1963] Step 3:

[1964] Users: Ensure their emotions are recognized while participating in live events.

[1965] Step 4:

[1966] Server: Based on the analyzed emotion recognition data, the virtual live performance is adjusted in real time.

[1967] Step 5:

[1968] Server: The recognition results are stored in a database and used for performance adjustments in the future.

[1969] Step 6:

[1970] Terminal: Uses the server's API to retrieve the adjusted live performance data and display it to the user.

[1971] Specific examples

[1972] For example, if User A applies to be a new model, he or she first accesses the model recruitment page on his or her device, enters the necessary information such as name, profile, and photo, and submits it. The server saves this information in the database and completes the model registration.

[1973] User B logs in to the system, retrieves a list of models on their device, and selects their favorite model. Based on the selected model information, they can freely customize hairstyle, clothing, accessories, etc.

[1974] A few days later, a live event is held. User B enters the idol he created into the event and competes against other users. During the live event, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions, and the results are reflected in their performance in real time.

[1975] Through this series of steps, users can raise their own original idol in a virtual space and enjoy watching it grow and react emotionally.

[1976] Example 2

[1977] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1978] In conventional virtual character development systems, customizing user-created characters and performing live events was cumbersome, and it was not possible to reflect user emotions in real time. Furthermore, it was difficult to efficiently analyze and automate user input data, leaving challenges in improving the user experience.

[1979] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1980] In this invention, the server includes means for accepting application information and registering it in a database, means for acquiring the registered information and providing it to the user, means for customizing a virtual character based on information selected by the user, means for entering the customized virtual character and holding a virtual event, means for accepting votes for the virtual event and tallying the results, and means for providing the tallying results to the user. This allows users to easily customize their virtual characters and participate in events and vote in real time. Furthermore, by including means for analyzing user emotional data and reflecting this in performance during the virtual event in real time, and means for adjusting the performance of the virtual event based on the analyzed emotional data, it is possible to provide a performance that reflects the user's emotions. Furthermore, by including means for analyzing user input data using a generative AI model and automating operations at each step, the user experience can be improved.

[1981] "Application information" refers to information such as name, profile, and photo that a user provides to create a virtual character.

[1982] "Database" refers to an information system for storing application information, registered information, customized virtual character information, emotional data, voting results, and the like.

[1983] A "virtual character" is a digital agent or avatar that a user can create and customize in a digital environment.

[1984] A "virtual event" is an event in a digital environment where virtual characters perform and users can participate and vote in real time.

[1985] A "vote" is an act in which a user expresses their opinion or support for a particular virtual character during a virtual event.

[1986] "Emotional data" refers to information about emotions extracted from a user's facial expressions, voice, and text, and is reflected in their performance during the virtual event.

[1987] "Performance" refers to the movements, actions, and performances made by a virtual character during a virtual event.

[1988] A "generative AI model" is an artificial intelligence model used to analyze input data from a user and automate operations at each step.

[1989] An "interface" is a screen or operating means that allows a user to interact with the system, enabling operations such as voting and customization.

[1990] The system of the present invention allows users to create original virtual characters in a virtual space, customize those characters, and participate in virtual events. It also has the function of recognizing the user's emotions and reflecting those emotions in the performance of the virtual event. A specific embodiment of the system is described below.

[1991] Model Wanted

[1992] server:

[1993] The server uses a web framework (e.g., Django, Flask) to prepare form data for model recruitment, provides an API endpoint for this (e.g., " / api / model recruitment form"), and receives application information (name, profile, photo, etc.) sent from the device and registers it in a database (e.g., MySQL, PostgreSQL).

[1994] Device:

[1995] The device sends an HTTP GET request to the server's API, retrieves form data in JSON format, displays the model recruitment page using an HTML template (e.g., React, Vue.js), captures the model application information entered by the user, and sends it to the server as an HTTP POST request.

[1996] (Example)

[1997] When User A applies to be a new model, he / she accesses the model recruitment page on his / her device, enters information such as the name "John Doe", profile "singer-songwriter", and photo "john_doe.jpg", and submits it. The server receives this information and stores it in a database.

[1998] Idol Body Selection

[1999] server:

[2000] The server extracts registered model information from the database and provides it at the API endpoint " / api / models".

[2001] Device:

[2002] The device sends an HTTP GET request to the API and displays a list of models to the user using the retrieved information. Model cards are dynamically generated using React and Vue.js, allowing users to select their preferred model.

[2003] (Example)

[2004] User B logs in to the system and selects "John Doe" from the list of registered models.

[2005] Idol Customization

[2006] server:

[2007] The server prepares customization option data (hairstyle, clothing, accessories) and provides it at the API endpoint " / api / customize_options". It also saves the customization information changed by the user in the database.

[2008] Device:

[2009] The device sends an HTTP GET request to the API, displays the customization screen using the retrieved option data, captures the customization information selected by the user, and sends it to the server as an HTTP POST request.

[2010] (Example)

[2011] User B changes "John Doe's" hairstyle to "short hair", clothing to "casual", and accessories to "sunglasses", and sends this information to the server, which stores this information in a database.

[2012] Idol production

[2013] server:

[2014] The server retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it via the API endpoint " / api / live_events". It also saves entry information in the database and prepares and executes the virtual event.

[2015] Device:

[2016] The device sends an HTTP GET request to the API to retrieve live event information and display it to the user. The device also sends the idle information entered by the user to the server as an HTTP POST request.

[2017] (Example)

[2018] User B enters his virtual character into a live event and competes against other users.

[2019] Live Events and Voting

[2020] server:

[2021] The server prepares a virtual live event based on the entry information, generates live video using game engines such as Unity or Unreal Engine, and receives and tally vote information in real time.

[2022] Device:

[2023] The device retrieves streaming data from the API, displays live video to the user using WebRTC or HLS, and captures voting information and sends it to the server as an HTTP POST request.

[2024] (Example)

[2025] Users watch a live event and vote for "John Doe." The server tallies the votes and notifies the results after the event ends.

[2026] Implementing the Emotion Engine

[2027] server:

[2028] The server uses an emotion engine (e.g., Microsoft Azure's Emotion API or Google Cloud's Speech-to-Text API) to analyze users' emotional data, provides an API that reflects the results in the virtual event performance, implements an algorithm that adjusts the virtual performance based on the emotional data, and stores the data in a database.

[2029] Device:

[2030] The device obtains emotion recognition results from the API and reflects them in real time in the virtual live performance.

[2031] (Example)

[2032] During the live event, the user's facial expression data is analyzed, and if the emotion is recognized as "joy," that emotion is reflected in the virtual character's performance in real time.

[2033] Example of generative AI model and prompt used

[2034] Generative AI models:

[2035] It uses OpenAI GPT-4 to analyze input data from users and automate operations at each step.

[2036] Example prompt sentence:

[2037] Based on the system's processing steps, please provide a detailed explanation of the steps a user takes to create an original idol in a virtual space and have that idol participate in a live event. Also, please provide a detailed explanation of how the emotion engine reflects the user's emotions in the live performance.

[2038] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2039] Step 1:

[2040] Process: Preparation and provision of form data for model recruitment.

[2041] Subject and action: The server uses a web framework to prepare form data for model recruitment and provides an API endpoint for this purpose: " / api / model recruitment form".

[2042] Input: Template data for model submission on the server.

[2043] Data processing / calculation: The server converts the template data into JSON format.

[2044] Output: The form data in JSON format provided by the API endpoint.

[2045] Specific operation: For example, define a model recruitment form using Django, convert the form data to JSON, and provide it via an API endpoint.

[2046] Step 2:

[2047] Processing content: Display model recruitment page.

[2048] Subject and Action: The device sends an HTTP GET request to the server's API, retrieves form data in JSON format, and displays a model recruitment page using an HTML template.

[2049] Input: JSON formatted form data retrieved from an API endpoint.

[2050] Data processing / calculation: The terminal parses the JSON data and embeds it into an HTML template.

[2051] Output: The model recruitment page shown to the user.

[2052] Specific operation: Generate form elements based on JSON data obtained using React and display them in the browser.

[2053] Step 3:

[2054] What it does: Collect and submit model application information.

[2055] Subject and Action: The user enters the required information such as name, profile, and photo on the model recruitment page, and the device sends this to the server as an HTTP POST request.

[2056] Input: Data entered by the user, such as name, profile, and photo.

[2057] Data processing / calculation: The terminal converts the form data into JSON format.

[2058] Output: JSON formatted application information sent to the server.

[2059] What it does: When a user enters information into a form and clicks the "Submit" button, the device captures the information and sends it to the server.

[2060] Step 4:

[2061] Process: Receiving and registering model application information.

[2062] Subject and Action: The server receives the model application information sent from the terminal and registers it in the database.

[2063] Input: Model application information received from the device in JSON format.

[2064] Data processing / calculation: The server parses the JSON data and inserts it into the database.

[2065] Output: Application information stored in a database.

[2066] Specific behavior: For example, use a Django model to save the received application information to a database.

[2067] Step 5:

[2068] Processing: Extracts and provides registered model information.

[2069] Subject and action: The server extracts registered model information from the database and provides it at the API endpoint " / api / models".

[2070] Input: A request for model information from the database.

[2071] Data processing / computation: The server executes the database query and converts the results into JSON format.

[2072] Output: Model information in JSON format provided by the API endpoint.

[2073] What happens: The server executes a MySQL query to get the model information and provides it to the endpoint in JSON format.

[2074] Step 6:

[2075] Process: Retrieves and displays model information.

[2076] Subject and Action: The device sends an HTTP GET request to the API endpoint to retrieve model information and display it to the user.

[2077] Input: Model information in JSON format obtained from the API endpoint.

[2078] Data processing / calculation: The terminal parses the JSON data and embeds it into an HTML template.

[2079] Output: A page listing the model information that is displayed to the user.

[2080] Specific operation: Generate a list based on the JSON data obtained using React and display it in the browser.

[2081] Step 7:

[2082] What it does: Select a model.

[2083] Subject and Action: The user selects the model of their choice from the list of models presented.

[2084] Input: A user-chosen model ID.

[2085] Data processing / calculation: The device captures the selected model ID.

[2086] Output: The selected model ID.

[2087] Specific behavior: The user clicks on "John Doe" from the list of models.

[2088] Step 8:

[2089] What it does: Provides customization options.

[2090] Subject and Action: The server provides customization option data (hairstyle, clothing, accessories) at the API endpoint " / api / customize_options".

[2091] Input: Customization options request from database.

[2092] Data processing / computation: The server executes the database query and converts the results into JSON format.

[2093] Output: Customization options provided in JSON format at the API endpoint.

[2094] Specific operation: The server obtains the customization data and provides it to the endpoint in JSON format.

[2095] Step 9:

[2096] Processing content: Display the customization screen.

[2097] Subject and Action: The device sends an HTTP GET request to the API endpoint to retrieve the customization options and display the customization screen.

[2098] Input: Customization options in JSON format retrieved from the API endpoint.

[2099] Data processing / calculation: The terminal parses the JSON data and embeds it into an HTML template.

[2100] Output: The customized screen that is displayed to the user.

[2101] What it does: Uses React to display customization options on the screen and allow the user to select them.

[2102] Step 10:

[2103] Process: Sending customization information.

[2104] Subject and Action: The user selects customization options (hairstyle, clothing, accessories), and the device sends that information to the server as an HTTP POST request.

[2105] Input: The customization options selected by the user (e.g. hairstyle ID, outfit ID, accessory ID).

[2106] Data processing / calculation: The terminal converts the customization data into JSON format.

[2107] Output: Customization information in JSON format sent to the server.

[2108] What happens: When a user saves a customization, the device sends that information to the server.

[2109] Step 11:

[2110] What it does: Saves customization information.

[2111] Subject and Action: The server receives the customization information sent from the terminal and registers it in the database.

[2112] Input: Customization information received from the terminal in JSON format.

[2113] Data processing / calculation: The server parses the JSON data and inserts it into the database.

[2114] Output: Customization information stored in a database.

[2115] What it does: Uses Django models to store customization information in the database.

[2116] Step 12:

[2117] What it does: Providing live event information.

[2118] Subject and Action: The server retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it at the API endpoint " / api / live_events".

[2119] Input: A request for live event information from the database.

[2120] Data processing / computation: The server executes the database query and converts the results into JSON format.

[2121] Output: Live event information in JSON format provided by the API endpoint.

[2122] Specific behavior: The server retrieves live event information and provides it to the endpoint in JSON format.

[2123] Step 13:

[2124] What it does: Displays live event information.

[2125] Subject and Action: The device sends an HTTP GET request to the API endpoint to retrieve live event information and display it to the user.

[2126] Input: Live event information in JSON format retrieved from the API endpoint.

[2127] Data processing / calculation: The terminal parses the JSON data and embeds it into an HTML template.

[2128] Output: The live event information page that is displayed to the user.

[2129] Specific operation: Display live event information obtained using React in the browser.

[2130] Step 14:

[2131] Processing: Idle entry.

[2132] Subject and Action: A user enters a customized virtual character into a live event, and the device sends this information to the server as an HTTP POST request.

[2133] Input: The ID of the virtual character the user is entering.

[2134] Data processing / calculation: The terminal converts the entry information into JSON format.

[2135] Output: Entry information in JSON format sent to the server.

[2136] Specific operation: When the user presses the entry button, the terminal sends the information to the server.

[2137] Step 15:

[2138] What it does: Store your entry information and conduct the virtual live event.

[2139] Subject and Action: The server stores the entry information in a database and prepares and executes the virtual live event.

[2140] Input: Entry information received from the device in JSON format.

[2141] Data processing / computation: The server parses the JSON data, inserts it into a database, and runs algorithms to generate the live performance.

[2142] Output: Entry information stored in a database and live performances generated.

[2143] How it works: Live video is generated using Unity or Unreal Engine and distributed via a streaming server.

[2144] Step 16:

[2145] What it does: Show live events and take votes.

[2146] Subject and Action: The device retrieves live streaming data from the API and displays it to the user. It also captures voting information and sends it to the server as an HTTP POST request.

[2147] Input: Streaming data retrieved from API endpoints and user voting information.

[2148] Data processing / calculation: The device plays back the streaming data and converts the voting information into JSON format.

[2149] Output: The live video displayed to the user and the voting information sent to the server.

[2150] Specific operation: Live video is displayed using WebRTC and HLS, and when a user presses the voting button, the information is sent to the server.

[2151] Step 17:

[2152] Process: Counting voting information and notifying results.

[2153] Subject and Action: The server receives and tally vote information from the terminals in real time. After the event ends, it saves the results in a database and notifies the users.

[2154] Input: Voting information received from the device in JSON format.

[2155] Data processing / calculation: The server aggregates the voting data and converts the results into JSON format.

[2156] Output: Voting results stored in a database and notification to the user.

[2157] Specific operation: The server tally the voting information, calculate the rankings, and notify the results after the event ends.

[2158] Step 18:

[2159] What it does: Analyzes emotion data and adjusts performance.

[2160] Subject and Action: The server uses an emotion engine to analyze the user's facial expressions and voice data, and reflects the results in the virtual event performance in real time. It also adjusts the virtual performance based on the analysis results and stores the data in a database.

[2161] Input: User's facial and voice data.

[2162] Data processing / calculation: Analyze data using the emotion engine and run algorithms that reflect the results in performance.

[2163] Output: Emotion data stored in a database and adjusted performance.

[2164] What it does: Analyzes emotion data and adjusts performance using Microsoft Azure's Emotion API and Google Cloud's Speech-to-Text API.

[2165] (Application example 2)

[2166] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2167] Conventional systems for cultivating and performing idols in virtual spaces lacked features such as performances that reflected the user's emotions in real time and promotional activities within virtual stores. This limited the user experience and made it difficult to provide more interactive and engaging content. Furthermore, there was a need for an effective system that could utilize emotional data to adjust idols' performances and realize continuous promotional activities.

[2168] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2169] In this invention, the server includes means for accepting model application information and registering it in a database, means for acquiring the registered model information and providing it to the user, means for customizing an idol based on the model information selected by the user, means for entering the customized idol and holding a virtual live event, means for accepting votes in the virtual live event and tallying the results, means for providing the tallying results to the user, means for users to implement a promotion in which the idol introduces products in a virtual store, and means for recognizing the user's emotions and reflecting the emotion data in the idol's performance. This makes it possible to create, train, and adjust the performance of idols in a virtual space, and furthermore, promotional activities in the virtual store are continuously carried out, making it possible to provide an interactive experience that reflects the user's emotions.

[2170] "Model application information" refers to information such as name, profile, and photo entered by a user who wishes to create an idol in a virtual space.

[2171] "Database" refers to a storage device for storing and managing model application information, customized idol information, and data on virtual live events and promotional activities.

[2172] A "user" is a person who uses the system to create, customize, and enter idols in a virtual space and participate in virtual live events.

[2173] "Customization" is an operation in which the user changes the appearance and attributes of the selected model, such as hairstyle, clothing, and accessories, to create an individual idol.

[2174] "Entry" refers to an operation to allow a customized idol to participate in a virtual live event or promotional activity.

[2175] A "virtual live event" is a live performance competition event held in a virtual space, in which idols entered by users compete in performance against other users.

[2176] "Voting" is an act by which a user shows support for their favorite idol at a virtual live event, and the results are tallied.

[2177] "Counting" is the process of compiling the voting results received at the virtual live event and calculating the final score.

[2178] A "virtual store" is a fictitious store space that exists on the Internet, where virtual idols introduce products and engage in promotional activities.

[2179] "Promotional activities" are the act of idols introducing and advertising products within a virtual store.

[2180] "Emotion recognition" is a technology that analyzes emotions from a user's facial expressions, voice, etc. and incorporates those emotions into the system.

[2181] "Reflecting in performance" means adjusting the virtual idol's movements and words in real time based on the user's emotions obtained through emotion recognition, resulting in a more interactive performance.

[2182] The system for implementing the present invention includes a server, a user terminal, and an emotion engine as its main components. The system operates as follows.

[2183] The server accepts application information for models and registers it in a database. When accessed from a user's device, the server provides an API that provides registered model information. The idol is customized based on the model information obtained by the user via their device. The customized idol is then sent to the server as entry information, and the virtual live event is prepared.

[2184] During the virtual live event, the server processes data in real time, accepts and tally votes, and provides the results to users. The user terminal also displays the virtual live performance in real time and provides an interface for users to vote.

[2185] Furthermore, users can use idols to promote products in virtual stores. The emotion engine analyzes users' emotional data, such as facial expressions and voice, and reflects that data in real time in the performance of the virtual idols, providing users with an interactive and engaging experience.

[2186] The hardware used includes smartphones and head-mounted displays. These devices are equipped with camera modules and microphones to capture the user's facial expressions and voice. The software uses EmotionEngine for emotion recognition, VirtualIdol for managing virtual idols, and the Requests library for API communication.

[2187] As a concrete example, a user accesses a model recruitment page using a smartphone, enters information such as name, profile, and photo, and submits it. The server stores this information in a database, completing the model registration. When the user promotes a new product in a virtual store, the emotion engine detects the user's joy and excitement and reflects this in real time in the idol's words, actions, and performance.

[2188] An example prompt for a generative AI model is:

[2189] "Create a smartphone app for promoting virtual idols. Include a process where users can create a virtual idol and introduce products in a virtual store. Implement a function where the idol adjusts their performance based on the user's emotions. Also, consider a voting system where users can compete with other users, and describe how to synchronize data with the server."

[2190] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2191] Step 1:

[2192] A user accesses the model recruitment page, enters model application information such as name, profile, and photo, and submits it.

[2193] Input: Data entered by users, such as name, profile, and photo.

[2194] Output: The input data is sent to the server.

[2195] Specific behavior:

[2196] Users access the model recruitment form on their smartphone or head-mounted display screen, enter the necessary information, and then press the send button to send the application information to the server.

[2197] Step 2:

[2198] The server accepts model application information and registers it in a database.

[2199] Input: Model application information submitted by the user.

[2200] Output: Application information registered in the database.

[2201] Specific behavior:

[2202] The server receives application information from users via API communication, stores the data in a database, and then returns a registration completion response to the user.

[2203] Step 3:

[2204] The user accesses the virtual idol's customization page using a terminal and customizes the virtual idol by selecting hairstyle, clothes, accessories, etc.

[2205] Input: Model information stored in the database, user customization selections.

[2206] Output: Customized idle information.

[2207] Specific behavior:

[2208] The user adjusts the appearance of the virtual idol using the customization options provided, and once selections are complete, the customization information is sent to the server.

[2209] Step 4:

[2210] The server stores the customized idol in a database and registers the entry information for the virtual live event.

[2211] Input: Customization information submitted by the user.

[2212] Output: Customized idle information stored in the database, and entry information.

[2213] Specific behavior:

[2214] The server receives the customization information and stores it in a database, while also registering the entry information for participating in the virtual live event.

[2215] Step 5:

[2216] During a virtual live event, the emotion engine analyzes the user's facial expressions and voice to generate emotion data.

[2217] Input: User's facial expression data, voice data.

[2218] Output: User emotion data.

[2219] Specific behavior:

[2220] The emotion engine analyzes the user's facial expressions and voice in real time, generating emotional data that is then reflected in the virtual idol's performance.

[2221] Step 6:

[2222] The device adjusts the virtual idol's performance based on emotional data and displays live footage.

[2223] Input: Parsed emotion data.

[2224] Output: Performance footage of the virtual idol after adjustments.

[2225] Specific behavior:

[2226] The device adjusts the movements and behavior of the virtual idol based on the emotional data sent from the emotion engine, and displays live footage in real time to provide the user with a more interactive experience.

[2227] Step 7:

[2228] A server accepts and tallys voting data during the virtual live event.

[2229] Input: Voting information from users.

[2230] Output: The tallied vote results.

[2231] Specific behavior:

[2232] The server receives the voting data sent by users in real time, tallying it up and calculating the final results, and stores the results in a database after the live event ends.

[2233] Step 8:

[2234] A promotion is carried out in which a user introduces products using idols in a virtual store.

[2235] Input: User promotion request information, customized idle information.

[2236] Output: Product introduction video by a virtual idol.

[2237] Specific behavior:

[2238] Users can request product promotions in the virtual store, and customized idols will generate and display videos introducing the products.

[2239] Step 9:

[2240] The terminal reflects the emotional data generated in real time in the promotional performance of the virtual idol.

[2241] Input: Emotion data, promotional video data.

[2242] Output: Adjusted promotional performance footage.

[2243] Specific behavior:

[2244] The terminal reflects the emotional data acquired from the emotion engine in the promotional video and adjusts the virtual idol's movements and speech to correspond to the user's emotions.

[2245] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2246] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2247] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2248] [Fourth embodiment]

[2249] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2250] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2251] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2252] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2253] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2254] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2255] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2256] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2257] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2258] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2259] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2260] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2261] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2262] The system of the present invention allows users to create original idols in a virtual space and have them compete with other users in terms of their performance. The system has a variety of functions, and specific examples thereof will be described below.

[2263] 1. Model Recruitment

[2264] server:

[2265] 1. Prepare form data for recruiting models and provide an API for devices to access it.

[2266] 2. Receive model application information sent from the device and register it in the database.

[2267] Device:

[2268] 1. Use the server's API to retrieve the form data and display the model recruitment page.

[2269] 2. The model application information entered by the user is sent to the server.

[2270] User:

[2271] 1. Enter the required information such as your name, profile, and photo on the model recruitment page and submit.

[2272] 2. Selecting an idol body

[2273] server:

[2274] 1. Extract registered model information from the database and provide an API in a form that can be accessed by the device.

[2275] Device:

[2276] 1. Call the server's API to obtain model information and display it as a list to the user.

[2277] User:

[2278] 1. Select the model you like from the presented models.

[2279] 3. Idol Customization

[2280] server:

[2281] 1. Store customization options (hairstyle, clothing, accessories, etc.) in a database and provide an API to provide them.

[2282] 2. Save the customization information changed by the user to the database.

[2283] Device:

[2284] 1. Use the server's API to get the customization options and display the customization screen to the user.

[2285] 2. Send the customization information selected by the user to the server.

[2286] User:

[2287] 1. Choose your favorite customization options, such as hairstyle, clothing, and accessories.

[2288] 4. Idol Production

[2289] server:

[2290] 1. Information about the live event (date and time, participation method, voting system, etc.) is retrieved from the database and provided to the device via API.

[2291] 2. Manage the information and results of idols participating in the event and save the results in a database.

[2292] Device:

[2293] 1. Call the server's API to get the live event information and display it to the user.

[2294] 2. The idle information entered by the user is sent to the server.

[2295] User:

[2296] 1. Check the live event information and enter your customized idol.

[2297] 5. Live Events and Voting

[2298] server:

[2299] 1. Prepare and hold a virtual live event based on the entry information.

[2300] 2. Receive and tally voting information from the terminals in real time.

[2301] 3. After the event is over, the results are saved in the database and notified to the user.

[2302] Device:

[2303] 1. Use the server's API to retrieve data from the virtual live event and display the live video to the user.

[2304] 2. The voting page accepts votes in real time and sends them to the server.

[2305] User:

[2306] 1. Browse live events and vote for your favorite idol.

[2307] 2. Check the results after the event is over.

[2308] Specific examples

[2309] When User A applies to be a new model, he / she first accesses the model recruitment page on his / her device, enters the necessary information such as name, profile, photo, etc., and submits it. The server saves this information in the database and completes the model registration.

[2310] User B logs in to the system, retrieves a list of models on their device, and selects their favorite model. Based on the selected model information, they can freely customize hairstyle, clothing, accessories, etc.

[2311] A few days later, a live event is held. User B enters the idol he created into the event and competes with other users. During the live event, users vote in real time, and the server tallies the votes.

[2312] Through this series of steps, users can raise their own original idol in a virtual space and enjoy watching it grow.

[2313] The processing flow will be explained below.

[2314] Program processing steps

[2315] Model recruitment process steps

[2316] Step 1:

[2317] Server: Prepares form data for model recruitment and provides it to the device via API.

[2318] Step 2:

[2319] Device: Calls the API and displays the model recruitment page based on the retrieved form data.

[2320] Step 3:

[2321] User: Enter the required information such as name, profile, photo, etc. on the model recruitment page and submit.

[2322] Step 4:

[2323] Terminal: Sends the entered model application information to the server.

[2324] Step 5:

[2325] Server: Registers the received model application information in a database.

[2326] Idol element selection process steps

[2327] Step 1:

[2328] Server: Extracts registered model information from the database and provides an API in a format that can be accessed by the device.

[2329] Step 2:

[2330] Terminal: Calls the API to obtain model information and displays it to the user.

[2331] Step 3:

[2332] User: Select the model of their choice from the presented models.

[2333] Step 4:

[2334] Terminal: Sends the selected model information to the server.

[2335] Step 5:

[2336] Server: Stores the received model information in a database.

[2337] Idol Customization Process Steps

[2338] Step 1:

[2339] Server: Stores customization options (hairstyle, clothing, accessories, etc.) in a database and provides an API to provide them.

[2340] Step 2:

[2341] Device: Uses the API to retrieve customization options and displays the customization screen to the user.

[2342] Step 3:

[2343] User: Select your preferred customization options, such as hairstyle, clothing, and accessories.

[2344] Step 4:

[2345] Terminal: Sends the selected customization information to the server.

[2346] Step 5:

[2347] Server: Stores the received customization information in a database.

[2348] Idol production process steps

[2349] Step 1:

[2350] Server: Retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it to the device via API.

[2351] Step 2:

[2352] Terminal: Calls the API to retrieve live event information and displays it to the user.

[2353] Step 3:

[2354] User: Check live event information and enter your customized idol.

[2355] Step 4:

[2356] Terminal: Sends the entered idle information to the server.

[2357] Step 5:

[2358] Server: Stores the received entry information in a database.

[2359] Live event execution and voting processing steps

[2360] Step 1:

[2361] Server: Generates data for the virtual live event based on the entry information and starts the live event according to the event time.

[2362] Step 2:

[2363] Terminal: Uses APIs to retrieve data from virtual live events and display live footage to users.

[2364] Step 3:

[2365] Users: View live events and vote for their favorite idols.

[2366] Step 4:

[2367] Terminal: Sends voting information to the server in real time.

[2368] Step 5:

[2369] Server: Counts the received voting information and stores the results in a database.

[2370] Step 6:

[2371] Server: After the event ends, the results are provided to the terminal and displayed to the user.

[2372] Through these modularized processing steps, users can create their own original idol and experience the entire process of having that idol compete against other users in a virtual live performance.

[2373] Example 1

[2374] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2375] Conventional virtual idol creation and live event systems have difficulty allowing users to create original idols and easily compete with each other in performance. Real-time voting and tallying are also difficult, resulting in a poor user experience. Furthermore, customized idol information cannot be easily reused, resulting in a lack of convenience for users who wish to use the system repeatedly.

[2376] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2377] In this invention, the server includes means for accepting model application information and registering it in a database, means for acquiring the registered model information and providing it to a terminal, means for customizing an idol based on the model information selected by the user, means for entering the customized idol and holding a virtual live event, means for accepting votes for the virtual live event and tallying them in real time, means for storing the tally results in a database and providing them to the user, means for storing the customized idol information in the database and making it available for future live events, and means for providing an interface for users to view the idol's performance in real time and vote during the virtual live event. This allows users to easily create and customize original idols in a virtual space and compete against other users in their performances. Real-time voting and tallying are also performed smoothly, improving the user experience. Furthermore, reusing customized idol information improves convenience during repeated use.

[2378] "Model application information" refers to information submitted by a user to apply to be a model, and specifically includes the user's name, profile, photo, etc.

[2379] A "database" is a system that stores information in an organized manner and allows for efficient searching and updating.

[2380] A "terminal" is an electronic device such as a computer or smartphone used by a user.

[2381] An "API" is an interface that allows different software components to communicate with each other.

[2382] "Idols" are characters created and customized by users in the virtual space, and are participants in live events.

[2383] "Customization" refers to the act of a user setting or changing an idol's appearance, such as hairstyle, clothing, and accessories.

[2384] A "virtual live event" is a live performance event that takes place in a virtual space and can be viewed and participated in by multiple users.

[2385] "Voting" is an evaluation action that a user takes for their favorite idol during or after a live event.

[2386] "Interface" refers to the screen and input means that users use to operate the system.

[2387] "Counting" is the process of calculating and analyzing the collected voting data and calculating the results.

[2388] "Storage" is the act of recording information in a database so that it can be reused at a later time.

[2389] The system of the present invention allows users to create original idols in a virtual space and compete with other users for their performance. The system operates in cooperation with a server, terminals, and users. Here, we will explain a specific embodiment of the system, including what hardware and software are used to process and calculate data.

[2390] 1. Model Recruitment

[2391] The server prepares form data for model recruitment and provides an API that the device can access (e.g., https: / / example.com / api / model-form). This allows the device to call the server's API and display the model recruitment page based on the obtained form data. On this page, the user enters application information such as name, profile, and photo and submits it. The device sends the information entered by the user to the server, and the server registers the application information in a database.

[2392] For example, the prompt text might look like this:

[2393] "Enter your name, profile, and photo and submit."

[2394] 2. Selecting an idol body

[2395] The server extracts registered model information from the database and provides it as an API (e.g., https: / / example.com / api / models) that can be retrieved by the device. The device calls the server's API, retrieves the model information, and displays it as a list to the user. The user can then select their preferred model from the list.

[2396] For example, the prompt text might look like this:

[2397] Choose your favorite model from the list.

[2398] 3. Idol Customization

[2399] The server stores customization option data (hairstyle, clothing, accessories, etc.) in a database and provides an API (e.g., https: / / example.com / api / customize-options) that provides this data. The device uses the server's API to obtain the customization options and displays a customization screen to the user. The user selects customization options such as hairstyle, clothing, and accessories, and sends the selected customization information to the server via the device. The server stores this customization information in a database.

[2400] For example, the prompt text might look like this:

[2401] "Choose your hairstyle, outfit, and accessories."

[2402] 4. Idol Production

[2403] The server retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it to the device via API (e.g. https: / / example.com / api / event-info). The device calls the server's API to retrieve the live event information and displays it to the user. The user checks this information and enters their customized idol. The device sends the entry information to the server, which receives it and stores it in the database.

[2404] For example, the prompt text might look like this:

[2405] "Please select the idol you would like to enter in the live event."

[2406] 5. Live Events and Voting

[2407] The server prepares and runs the virtual live event based on the entry information (for example, using game engines such as Unity or Unreal Engine). The device uses the server's API to obtain data about the virtual live event and displays live footage to the user. Users view the live event and vote for their favorite idol. The device accepts votes in real time on the voting page and sends them to the server. The server receives this voting information in real time and tallys it. After the event ends, the server saves the results in a database and notifies the user.

[2408] For example, the prompt text might look like this:

[2409] "Vote for your idol during the live event"

[2410] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2411] Step 1: Model recruitment

[2412] server:

[2413] Prepare the form data and provide the API (e.g., https: / / example.com / api / model-form), which allows devices to access the model recruitment form.

[2414] Input: Server preferences, form data

[2415] Output: API endpoint containing form data

[2416] Device:

[2417] Call the server API to get the form data.

[2418] The model recruitment page is displayed to the user.

[2419] Input: Form data obtained from the API

[2420] Output: Model recruitment page shown to the user

[2421] User:

[2422] On the model recruitment page, enter your application information such as your name, profile, and photo, and click the submit button.

[2423] Input: User's application information (name, profile, photo)

[2424] Output: Submitted application information

[2425] Device:

[2426] The application information entered by the user is sent to the server.

[2427] Input: Application information provided by the user

[2428] Output: Application information sent to the server

[2429] server:

[2430] The received application information is registered in a database.

[2431] Input: Application information sent from the device

[2432] Output: Application information stored in a database

[2433] Step 2: Select the idol body

[2434] server:

[2435] Extract model information from the database and provide an API in a format that can be retrieved by the device (e.g. https: / / example.com / api / models).

[2436] Input: Model information in the database

[2437] Output: API endpoint containing model information

[2438] Device:

[2439] Call the server's API to obtain model information and display it as a list to the user.

[2440] Input: Model information obtained from the API

[2441] Output: A list of models displayed to the user

[2442] User:

[2443] Select the model you like from the ones presented.

[2444] Input: List of models

[2445] Output: Selected model information

[2446] Step 3: Customize your idol

[2447] server:

[2448] Store customization options (hairstyles, clothing, accessories, etc.) in a database and provide an API to serve them (e.g., https: / / example.com / api / customize-options).

[2449] Input: Customization options data in the database

[2450] Output: API endpoint with customization options

[2451] Device:

[2452] Use the server's API to get the customization options and display the customization screen to the user.

[2453] Input: Customization options obtained from the API

[2454] Output: The customized screen that is displayed to the user

[2455] User:

[2456] Choose customization options like hairstyle, clothing, accessories, etc.

[2457] Input: Customization options

[2458] Output: Selected customization information

[2459] Device:

[2460] The customization information selected by the user is sent to the server.

[2461] Input: Customization information selected by the user

[2462] Output: Customization information sent to the server

[2463] server:

[2464] The received customization information is stored in a database.

[2465] Input: Customization information sent from the device

[2466] Output: Customization information stored in the database

[2467] Step 4: Producing idols

[2468] server:

[2469] Retrieve live event information (date and time, participation method, voting system, etc.) from the database and provide it to the device via an API (e.g. https: / / example.com / api / event-info).

[2470] Input: Live event information in the database

[2471] Output: API endpoint containing live event information

[2472] Device:

[2473] Call the server's API to get live event information and display it to the user.

[2474] Input: Live event information obtained from the API

[2475] Output: Live event information displayed to the user

[2476] User:

[2477] Check the live event information and enter your customized idol.

[2478] Input: Live Event Information

[2479] Output: Entry information

[2480] Device:

[2481] The idle information entered by the user is sent to the server.

[2482] Input: User entry information

[2483] Output: Entry information sent to the server

[2484] server:

[2485] The received entry information is stored in a database.

[2486] Input: Entry information sent from the terminal

[2487] Output: Entry information stored in the database

[2488] Step 5: Live Event and Voting

[2489] server:

[2490] Prepare and conduct a virtual live event based on the entry information (e.g., using game engines such as Unity or Unreal Engine).

[2491] Input: Entry information stored in the database

[2492] Output: Virtual live event

[2493] Device:

[2494] Use the server's API to retrieve data from the virtual live event and display live footage to the user.

[2495] Input: Live event data retrieved from the API

[2496] Output: Live video displayed to the user

[2497] User:

[2498] Watch live events and vote for your favorite idol.

[2499] Input: Live video

[2500] Output: Voting information

[2501] Device:

[2502] The voting page accepts votes in real time and sends them to the server.

[2503] Input: Voting information from users

[2504] Output: Voting information sent to the server

[2505] server:

[2506] Voting information from the terminals is received in real time and tallied.

[2507] After the event is over, the results are stored in a database and notified to the user.

[2508] Input: Voting information sent from the device

[2509] Output: Aggregation results, saving to database, notifying user

[2510] (Application example 1)

[2511] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2512] Previously, systems existed that allowed users to compete using the personalities and customization elements of idols created in a virtual space. However, they lacked functionality, particularly functionality for linking with virtual stores to try on fashion items and hold fashion shows. Therefore, there is a need for a system that allows users to enjoy a more comprehensive experience, not only by customizing the appearance of their virtual idols, but also by selecting fashion items and participating in fashion show events. There is also a need for a system that allows users to try on fashion items in real time and compete with other users on their evaluations, thereby improving interaction between users.

[2513] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2514] In this invention, the server includes means for accepting application information for models and registering it in a database, means for acquiring the registered model information and providing it to the user, means for customizing an idol based on the model information selected by the user, means for entering the customized idol and holding a virtual live event, means for accepting votes in the virtual live event and tallying the results, means for providing the tallying results to the user, means for providing a virtual store application and displaying customization options for the idol's appearance and clothing, and means for entering the idol customized by the user in a fashion show event and having it compete with other users. This enables users to customize the appearance and fashion of idols created in a virtual space and compete with other users in real time.

[2515] "Model application information" refers to personal information such as name, profile, and photo that a user provides to the system.

[2516] "Model information" refers to all information such as personal information and appearance data of users registered in the database.

[2517] An "idol" is a character that a user designs and customizes in a virtual space.

[2518] "Customization" refers to the process of changing appearance elements such as hairstyle, clothing, and accessories according to the user's preferences.

[2519] A "virtual live event" is a live performance or fashion show held in a virtual space.

[2520] "Voting" is the act of users rating different idols in real time during a live event.

[2521] "Counting" refers to the process of processing the voting results after the live event has ended and calculating the final ratings.

[2522] A "virtual store" is a virtual shop or boutique operated on the Internet.

[2523] An "application" is software that users use on their smartphones or other devices.

[2524] "Customization options" are various options for changing an idol's appearance.

[2525] A "fashion show event" is a virtual event where users can compete with idols they have customized.

[2526] The present invention is a system that allows users to create original idols in a virtual space and have them compete with other users in terms of performance and fashion sense. Specific examples of the system are described below.

[2527] The server first accepts the model's application information and registers it in a database, including information such as name, profile, and photo. It then retrieves the registered model information and provides it to the device via API.

[2528] The device uses the server's API to obtain model information and displays it to the user. The user can then select their preferred model from this list. Based on the selected model information, the user can customize the idol. Customization options include hairstyle, clothing, and accessories.

[2529] After users have completed customization, they can enter their customized idols into virtual live events and fashion show events. In virtual live events, users can view their idols' performances in real time and compete against other users. Voting is also conducted in real time, and the results are tallied on the server. The tallied results are provided to users after the event ends.

[2530] A virtual store application is also provided, giving users access to customization options for the idol's appearance and clothing, allowing them to enjoy the idol's appearance and fashion created in the virtual space and compete with other users.

[2531] The hardware used is a smartphone, and the software used is an application framework such as React Native or Flutter, with Node.js or Python's Flask as the backend.

[2532] For example, a user opens the app, enters their name, profile, and photo on the model recruitment page, and submits them. This information is then stored in a database by the server. Next, the user selects their preferred model from a list of models presented, and selects customization options such as hairstyle, clothing, and accessories. After that, the user enters their customized idol in a fashion show event scheduled on the app, competing against other users. During the live event, users can vote in real time with other users.

[2533] Examples of prompts for generative AI models include:

[2534] "How do I apply as a new model?"

[2535] "I want to change my idol's hairstyle. How can I customize it?"

[2536] "Please let me know about the next live event."

[2537] "How do I vote for my idol against other users' idols?"

[2538] In this way, the system of the present invention provides a forum where users can customize original idols in a virtual space and compete in fashion sense and performance.

[2539] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2540] Step 1:

[2541] The user opens the model recruitment page, enters application information such as name, profile, and photo, and submits it.

[2542] Input: Personal information you enter, such as your name, profile, and photo.

[2543] Data processing: The terminal sends the input information to the server via the API.

[2544] Output: The application information is sent to the server and stored in a database.

[2545] Specific operation: The terminal displays the application information as an input form, the user enters the required information in the form, and clicks the "Submit" button.

[2546] Step 2:

[2547] The server retrieves model information from the database and provides it to the terminal.

[2548] Input: Model information stored on the server.

[2549] Data processing: The server executes database queries to extract model information and provides it to the terminal in API format.

[2550] Output: The model information arrives on the terminal and is displayed in a list to the user.

[2551] Specific operation: The server extracts the necessary model information from the database and sends that information to the terminal as a JSON format API response.

[2552] Step 3:

[2553] The user selects model information and selects customization options on the customization page.

[2554] Input: Model information and customization options selected by the user (hairstyle, clothing, accessories, etc.).

[2555] Data processing: The device sends the selected model information and customization options to the server via API.

[2556] Output: The customized idle information is saved on the server.

[2557] Specific operation: The user selects the model of their choice from the model list, selects customization options, and clicks the "Apply" button.

[2558] Step 4:

[2559] The user enters the customized idol into a fashion show event.

[2560] Input: Customized idle information and selected event information.

[2561] Data processing: The device sends idle information and event information to the server via API.

[2562] Output: The idle is entered into the selected event.

[2563] Specific operation: The user checks the customized idol, selects the event they want to participate in from the list of fashion show events, and clicks the "Enter" button.

[2564] Step 5:

[2565] Users compete against other users in virtual live events.

[2566] Input: Virtual live event configuration information and user votes.

[2567] Data processing: The server tally the voting information in real time and calculate the rankings.

[2568] Output: The server notifies the user of the results of the live event.

[2569] Specific operation: The server tally the voting results in real time and display the rankings and ratings to users after the event ends.

[2570] Step 6:

[2571] A virtual store allows users to access customization options for their idols' appearance and clothing.

[2572] Input: Virtual store customization options information.

[2573] Data processing: The server provides customization option information to the terminal via the API.

[2574] Output: The customization options are displayed on the device and can be selected by the user.

[2575] Specific operation: The server extracts customization option information from the virtual store and sends it to the terminal in API format. The user can then check and select the options on the terminal.

[2576] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2577] The system of the present invention allows users to create original idols in a virtual space and compete with other users in their performances. Furthermore, by combining it with an emotion engine, it has the function of recognizing the user's emotions and reflecting them in the virtual live performance. A specific example of this is described below.

[2578] 1. Model Recruitment

[2579] server:

[2580] 1. Prepare form data for recruiting models and provide an API for devices to access it.

[2581] 2. Receive model application information sent from the device and register it in the database.

[2582] Device:

[2583] 1. Use the server's API to retrieve the form data and display the model recruitment page.

[2584] 2. The model application information entered by the user is sent to the server.

[2585] User:

[2586] 1. Enter the required information such as your name, profile, and photo on the model recruitment page and submit.

[2587] 2. Selecting an idol body

[2588] server:

[2589] 1. Extract registered model information from the database and provide an API in a form that can be accessed by the device.

[2590] Device:

[2591] 1. Call the API to get model information and display it to the user.

[2592] User:

[2593] 1. Select the model you like from the presented models.

[2594] 3. Idol Customization

[2595] server:

[2596] 1. Store customization options (hairstyle, clothing, accessories, etc.) in a database and provide an API to provide them.

[2597] 2. Save the customization information changed by the user to the database.

[2598] Device:

[2599] 1. Use the API to get the customization options and display the customization screen to the user.

[2600] 2. Send the customization information selected by the user to the server.

[2601] User:

[2602] 1. Choose your favorite customization options, such as hairstyle, clothing, and accessories.

[2603] 4. Idol Production

[2604] server:

[2605] 1. Information about the live event (date and time, participation method, voting system, etc.) is retrieved from the database and provided to the device via API.

[2606] 2. Manage the information and results of idols participating in the event and save the results in a database.

[2607] Device:

[2608] 1. Call the server's API to get the live event information and display it to the user.

[2609] 2. The idle information entered by the user is sent to the server.

[2610] User:

[2611] 1. Check the live event information and enter your customized idol.

[2612] 5. Live Events and Voting

[2613] server:

[2614] 1. Prepare and hold a virtual live event based on the entry information.

[2615] 2. Receive and tally voting information from the terminals in real time.

[2616] 3. After the event is over, the results are saved in the database and notified to the user.

[2617] Device:

[2618] 1. Use the server's API to retrieve data from the virtual live event and display the live video to the user.

[2619] 2. The voting page accepts votes in real time and sends them to the server.

[2620] User:

[2621] 1. Browse live events and vote for your favorite idol.

[2622] 2. Check the results after the event is over.

[2623] 6. Implementing the Emotion Engine

[2624] 6.1 Emotion recognition

[2625] server:

[2626] 1. Analyze user emotional data (facial expressions, voice, text) through an emotion engine and provide an API that reflects the results in real time.

[2627] Device:

[2628] 1. Use the API to obtain emotion recognition results and incorporate them into a virtual live performance.

[2629] User:

[2630] 1. Your emotions are recognized during live events.

[2631] 6.2 Performance Tuning

[2632] server:

[2633] 1. Implement an algorithm to adjust the virtual live performance based on the recognition results of the emotion engine.

[2634] 2. The recognized emotion data is stored in a database and used to adjust performance in future applications.

[2635] Device:

[2636] 1. Use the server's API to retrieve the adjusted live performance data and display it to the user.

[2637] Specific examples

[2638] For example, if User A applies to be a new model, he or she first accesses the model recruitment page on his or her device, enters the necessary information such as name, profile, and photo, and submits it. The server saves this information in the database and completes the model registration.

[2639] User B logs in to the system, retrieves a list of models on their device, and selects their favorite model. Based on the selected model information, they can freely customize hairstyle, clothing, accessories, etc.

[2640] A few days later, a live event is held. User B enters the idol he created into the event and competes against other users. During the live event, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions, and the results are reflected in their performance in real time.

[2641] Through this series of steps, users can raise their own original idol in a virtual space and enjoy watching it grow and react emotionally.

[2642] The processing flow will be explained below.

[2643] Program processing steps

[2644] Model recruitment process steps

[2645] Step 1:

[2646] Server: Prepares form data for model recruitment and provides it to the device via API.

[2647] Step 2:

[2648] Device: Calls the API and displays the model recruitment page based on the retrieved form data.

[2649] Step 3:

[2650] User: Enter the required information such as name, profile, photo, etc. on the model recruitment page and submit.

[2651] Step 4:

[2652] Terminal: Sends the entered model application information to the server.

[2653] Step 5:

[2654] Server: Registers the received model application information in a database.

[2655] Idol element selection process steps

[2656] Step 1:

[2657] Server: Extracts registered model information from the database and provides an API in a format that can be accessed by the device.

[2658] Step 2:

[2659] Terminal: Calls the API to obtain model information and displays it to the user.

[2660] Step 3:

[2661] User: Select the model of their choice from the presented models.

[2662] Step 4:

[2663] Terminal: Sends the selected model information to the server.

[2664] Step 5:

[2665] Server: Stores the received model information in a database.

[2666] Idol Customization Process Steps

[2667] Step 1:

[2668] Server: Stores customization options (hairstyle, clothing, accessories, etc.) in a database and provides an API to provide them.

[2669] Step 2:

[2670] Device: Uses the API to retrieve customization options and displays the customization screen to the user.

[2671] Step 3:

[2672] User: Select your preferred customization options, such as hairstyle, clothing, and accessories.

[2673] Step 4:

[2674] Terminal: Sends the selected customization information to the server.

[2675] Step 5:

[2676] Server: Stores the received customization information in a database.

[2677] Idol production process steps

[2678] Step 1:

[2679] Server: Retrieves live event information (date and time, participation method, voting system, etc.) from the database and provides it to the device via API.

[2680] Step 2:

[2681] Terminal: Calls the API to retrieve live event information and displays it to the user.

[2682] Step 3:

[2683] User: Check live event information and enter your customized idol.

[2684] Step 4:

[2685] Terminal: Sends the entered idle information to the server.

[2686] Step 5:

[2687] Server: Stores the received entry information in a database.

[2688] Live event execution and voting processing steps

[2689] Step 1:

[2690] Server: Generates data for the virtual live event based on the entry information and starts the live event according to the event time.

[2691] Step 2:

[2692] Terminal: Uses APIs to retrieve data from virtual live events and display live footage to users.

[2693] Step 3:

[2694] Users: View live events and vote for their favorite idols.

[2695] Step 4:

[2696] Terminal: Sends voting information to the server in real time.

[2697] Step 5:

[2698] Server: Counts the received voting information and stores the results in a database.

[2699] Step 6:

[2700] Server: After the event ends, the results are provided to the terminal and displayed to the user.

[2701] Processing steps for implementing an emotion engine

[2702] Step 1:

[2703] Server: Prepares the emotion engine and provides an API for analyzing user emotion data (facial expressions, voice, text).

[2704] Step 2:

[2705] Terminal: Uses API to obtain emotion data analysis results and presents them to the user.

[2706] Step 3:

[2707] Users: Ensure their emotions are recognized while participating in live events.

[2708] Step 4:

[2709] Server: Based on the analyzed emotion recognition data, the virtual live performance is adjusted in real time.

[2710] Step 5:

[2711] Server: The recognition results are stored in a database and used for performance adjustments in the future.

[2712] Step 6:

[2713] Terminal: Uses the server's API to retrieve the adjusted live performance data and display it to the user.

[2714] Specific examples

[2715] For example, if User A applies to be a new model, he or she first accesses the model recruitment page on his or her device, enters the necessary information such as name, profile, and photo, and submits it. The server saves this information in the database and completes the model registration.

[2716] User B logs in to the system, retrieves a list of models on their device, and selects their favorite model. Based on the selected model information, they can freely customize hairstyle, clothing, accessories, etc.

[2717] A few days later, a live event is held. User B enters the idol he created into the event and competes against other users. During the live event, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions, and the results are reflected in their performance in real time.

[2718] Through this series of steps, users can raise their own original idol in a virtual space and enjoy watching it grow and react emotionally.

[2719] Example 2

[2720] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2721] In conventional virtual character development systems, customizing user-created characters and performing live events was cumbersome, and it was not possible to reflect user emotions in real time. Furthermore, it was difficult to efficiently analyze and automate user input data, leaving challenges in improving the user experience.

[2722] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2723] In this invention, the server includes means for accepting application information and registering it in a database, means for acquiring the registered information and providing it to the user, means for customizing a virtual character based on information selected by the user, means for entering the customized virtual character and holding a virtual event, means for accepting votes for the virtual event and tallying the results, and means for providing the tallying results to the user. This allows users to easily customize their virtual characters and participate in events and vote in real time. Furthermore, by including means for analyzing user emotional data and reflecting this in performance during the virtual event in real time, and means for adjusting the performance of the virtual event based on the analyzed emotional data, it is possible to provide a performance that reflects the user's emotions. Furthermore, by including means for analyzing user input data using a generative AI model and automating operations at each step, the user experience can be improved.

[2724] "Application information" refers to information such as name, profile, and photo that a user provides to create a virtual character.

[2725] "Database" refers to an information system for storing application information, registered information, customized virtual character information, emotional data, voting results, and the like.

[2726] A "virtual character" is a digital agent or avatar that a user can create and customize in a digital environment.

[2727] A "virtual event" is an event in a digital environment where virtual characters perform and users can participate and vote in real time.

[2728] A "vote" is an act in which a user expresses their opinion or support for a particular virtual character during a virtual event.

[2729] "Emotional data" refers to information about emotions extracted from a user's facial expressions, voice, and text, and is reflected in their performance during the virtual event.

[2730] "Performance" refers to the movements, actions, and performances made by a virtual character during a virtual event.

[2731] A "generative AI model" is an artificial intelligence model used to analyze input data from a user and automate operations at each step.

[2732] An "interface" is a screen or operating means that allows a user to interact with the system, enabling operations such as voting and customization.

[2733] The system of the present invention allows users to create original virtual characters in a virtual space, customize those characters, and participate in virtual events. It also has the function of recognizing the user's emotions and reflecting those emotions in the performance of the virtual event. A specific embodiment of the system is described below.

[2734] Model Wanted

[2735] server:

[2736] The server uses a web framework (e.g., Django, Flask) to prepare form data for model recruitment, provides an API endpoint for this (e.g., " / api / model recruitment form"), and receives application information (name, profile, photo, etc.) sent from the device and registers it in a database (e.g., MySQL, PostgreSQL).

[2737] Device:

[2738] The device sends an HTTP GET request to the server's API, retrieves form data in JSON format, displays the model...

Claims

1. A means of accepting model application information and registering it in a database; A means for acquiring registered model information and providing it to a user; means for customizing the idol based on user-selected model information; A means for entering customized idols and conducting virtual live events; a means for accepting votes and tallying the results at the virtual live event; The system includes a means for providing the aggregated results to a user.

2. 2. The system according to claim 1, further comprising means for storing customized idol information in a database and making it available for subsequent live events.

3. 10. The system of claim 1, further comprising means for providing an interface for users to view and vote for idols' performances in real time during the virtual live event.

Images