System

Abstract

A system is provided.SOLUTION: A system comprising: means for a user to provide conversation data-based on a target person or character; means for a generation AI to generate a realistic conversation using the conversation data-based on a target person or character; and means for distributing a reward for the generated conversation according to a charged usage.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, many people have a growing need for social interaction and contact due to factors such as a declining birthrate, an aging population, the trend toward nuclear families, and a sense of isolation in urban life. Mothers raising children and lonely elderly people in particular are seeking opportunities to talk, and support for these people is essential. However, in reality, it is difficult to always provide someone to talk to, and measures are needed to reduce the number of people who feel lonely. [Means for solving the problem]

[0005] The present invention provides a system in which a user provides conversation data based on a target person or character, and a generation AI uses the conversation data to generate realistic conversations. In this system, the user has a means for selecting a target person or character and sending a conversation generation request. Furthermore, the generated conversation has a means for distributing rewards based on paid usage, and has a function for distributing rewards based on the amount paid to the data provider. Such a system can provide realistic conversations to people who feel socially lonely, helping to alleviate their feelings of loneliness.

[0006] "User" refers to an individual who utilizes this system to provide conversation data and enjoy the generated conversations.

[0007] A "target person" refers to a specific individual, celebrity, or past person with whom a user wishes to have a conversation.

[0008] "Character" refers to a fictional person who does not exist in reality, such as an anime or movie character.

[0009] "Conversation data" refers to data that includes audio and text information related to a target person or character.

[0010] "Generative AI" refers to an artificial intelligence engine that generates realistic conversations based on provided conversation data.

[0011] "Realistic conversation" refers to conversation content generated by generative AI that sounds as if the target person or character is actually speaking.

[0012] "Charged usage" refers to the price paid by a user to use a generated conversation.

[0013] "Reward" refers to profits distributed to data providers based on paid usage.

[0014] The "system" refers to the overall mechanism in which users provide conversation data, the AI ​​generates realistic conversations, and rewards are distributed based on paid usage.

[0015] "Data provider" refers to an individual or organization that provides conversation data to the system and receives compensation in return.

[0016] "Database" refers to an information storage system for storing and managing conversation data and generated conversations.

[0017] "Billing information" refers to credit card information and payment information provided by the user for billing purposes.

[0018] A "conversation generation request" refers to a request made by a user to the system to generate a conversation with a target person or character. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention relates to a system in which users provide conversation data based on target people or characters, a generation AI uses the conversation data to generate realistic conversations, and rewards are distributed based on paid usage of the generated conversations. This system allows users to enjoy realistic conversations, and data providers can also earn rewards.

[0041] System configuration

[0042] This system consists of three main components: a user terminal, a server, and a database.

[0043] 1. User terminal: A device that provides an interface for users to provide conversation data, request conversation creation, and perform billing procedures.

[0044] 2. Server: Includes the generation AI and performs key processes such as storing conversation data, generating conversations, processing billing, and distributing rewards.

[0045] 3. Database: Stores conversation data, generated conversations, user information, billing information, etc.

[0046] Program processing explanation

[0047] User Registration / Login

[0048] 1. User Registration

[0049] The user's device connects to the server and displays a registration form. The user enters the required information (name, email address, password) and sends it to the server.

[0050] The server receives the input information, checks its integrity, and saves it in the database. If successful, the server sends a registration completion message to the user's device.

[0051] Providing conversation data

[0052] 2. Uploading conversation data

[0053] The user's device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button.

[0054] The server receives the conversation data, checks its integrity, and then saves it in the database. If successful, the server sends a message to the user device indicating that the upload is complete.

[0055] Conversation generation

[0056] 3. Conversation Generation Request

[0057] The user's device connects to the server and displays the conversation creation screen. The user selects the target person or character and sends a conversation creation request.

[0058] The server's AI analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is then sent to the user's device.

[0059] Conversation Usage and Rewards

[0060] 4. Paid conversation usage

[0061] The user terminal connects to the server and displays the billing screen. The user enters billing information and presses the billing button.

[0062] The server processes the payment based on the billing information, and if successful, unlocks the conversation data. It distributes a reward based on the billing amount to the data provider and notifies the user and the data provider of the processing result.

[0063] Specific examples

[0064] Each step is illustrated concretely through a scenario in which Yamada Taro uses this system.

[0065] 1. User Registration

[0066] Yamada Taro accesses the system using a user terminal and registers by entering "taro@example.com" and a password in the registration form.

[0067] 2. Providing conversation data

[0068] Taro Yamada selects an audio file of an interview with his favorite actor and uploads it to the system.

[0069] 3. Conversation Generation

[0070] Taro Yamada selects his favorite actor on the system and sends a request to generate a conversation with that actor. The server's AI generates realistic conversation based on the relevant voice data and sends it to Taro Yamada's device.

[0071] 4. Conversational Use and Reward

[0072] Taro Yamada pays 500 yen to use the generated conversation, and the server distributes a portion of that amount to the data provider as a reward.

[0073] In this way, the system of the present invention provides users with an opportunity to enjoy real conversations without feeling lonely, and at the same time, by distributing rewards to data providers, it increases overall satisfaction.

[0074] The processing flow will be explained below.

[0075] User Registration / Login

[0076] 1. User Registration

[0077] Step 1:

[0078] The user terminal connects to the server and displays the registration form.

[0079] Step 2:

[0080] The user enters the required information (name, email address, password) and clicks the send button.

[0081] Step 3:

[0082] The user terminal transmits the input information to the server.

[0083] Step 4:

[0084] The server verifies the integrity of the information entered, checking for missing or incorrect information.

[0085] Step 5:

[0086] After the server verifies it, it stores the information in a database.

[0087] Step 6:

[0088] The server generates a registration completion message and sends it to the user terminal.

[0089] Step 7:

[0090] A message indicating registration completion is displayed on the user's device.

[0091] Providing conversation data

[0092] 2. Uploading conversation data

[0093] Step 1:

[0094] The user terminal connects to the server and displays the data upload screen.

[0095] Step 2:

[0096] The user selects the conversation data file and information about the target person and presses the upload button.

[0097] Step 3:

[0098] The user terminal transmits the selected files and information to the server.

[0099] Step 4:

[0100] The server checks the integrity of the files and information received.

[0101] Step 5:

[0102] The server stores the files and information in a database.

[0103] Step 6:

[0104] The server generates an upload completion message and sends it to the user terminal.

[0105] Step 7:

[0106] A message indicating that the upload is complete will be displayed on the user's device.

[0107] Conversation generation

[0108] 3. Conversation Generation Request

[0109] Step 1:

[0110] The user terminal connects to the server and displays a conversation creation screen.

[0111] Step 2:

[0112] The user selects a target person or character and sends a request to generate a conversation.

[0113] Step 3:

[0114] The user terminal transmits request information to the server.

[0115] Step 4:

[0116] The server's generation AI analyzes the request and retrieves the conversation data of the target person or character from the database.

[0117] Step 5:

[0118] The server's generation AI generates realistic conversations based on the data it acquires.

[0119] Step 6:

[0120] The server transmits the generated conversation data to the user terminal.

[0121] Step 7:

[0122] The generated conversation is displayed on the user terminal.

[0123] Conversation Usage and Rewards

[0124] 4. Paid conversation usage

[0125] Step 1:

[0126] The user terminal connects to the server and displays the billing screen.

[0127] Step 2:

[0128] The user inputs the billing information and presses the billing button.

[0129] Step 3:

[0130] The user terminal transmits the billing information to the server.

[0131] Step 4:

[0132] The server processes the payment based on the billing information received. If an external payment service is used, the server sends a request to that service.

[0133] Step 5:

[0134] The server receives confirmation of successful payment.

[0135] Step 6:

[0136] The server unlocks the conversation data and makes it available.

[0137] Step 7:

[0138] The server calculates the reward based on the usage charge and distributes it to the data provider.

[0139] Step 8:

[0140] The server notifies the user and data provider of the results of the billing and reward processing.

[0141] Step 9:

[0142] The processing results are displayed on the user terminal and the provider terminal.

[0143] Example 1

[0144] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0145] In today's world, users often feel lonely, and especially when they spend a lot of time alone, they want to enjoy real conversations with others. However, in reality, there are limited ways to easily generate and use conversations with target people or characters. Another issue is the lack of a system that provides appropriate rewards to providers of conversation data.

[0146] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0147] In this invention, the server includes: a means for a user to provide conversation data based on a target person or character; a means for a generation AI to generate realistic conversations using the conversation data; a means for distributing rewards for the generated conversations according to paid usage; a means for a user terminal to connect to the server, transmit user registration and login information, and perform authentication; a means for a user to upload conversation data and for the server to store it in a database; a means for inputting billing information and processing payments; and a means for unlocking the conversation data and automatically distributing rewards. This allows users to easily enjoy realistic conversations, and data providers to receive fair rewards.

[0148] A "user terminal" is a device that allows a user to access the system, provide conversation data, request conversation creation, and perform billing procedures.

[0149] A "server" is a computer system that includes the generation AI and performs major processes such as storing conversation data, generating conversations, processing billing, and distributing rewards.

[0150] A "database" is an information management system for storing conversation data, generated conversations, user information, billing information, and the like.

[0151] "Generative AI" refers to algorithms and software that use provided conversational data to generate realistic conversations.

[0152] "Conversation data" refers to data such as audio files and text data related to a target person or character provided by the user.

[0153] "Billing information" refers to credit card information or other payment method information required for users to pay for conversation creation and usage.

[0154] "Reward" refers to money or something of equivalent value distributed to a user who provides conversation data based on the fee-based usage.

[0155] "Realistic conversation" refers to dialogue data that resembles natural conversations between humans, generated by the generative AI based on the conversation data provided.

[0156] The present invention is a system in which a user provides conversation data based on a target person or character, a generation AI uses the conversation data to generate realistic conversations, and rewards are distributed based on paid usage of the generated conversations. This system allows users to enjoy realistic conversations, and data providers can also receive rewards.

[0157] System configuration

[0158] This system consists of three main components: a user terminal, a server, and a database.

[0159] User terminal: A device that provides an interface for users to provide conversation data, request conversation creation, and perform billing procedures.

[0160] Server: Contains the generation AI and performs key processes such as storing conversation data, generating conversations, processing billing, and distributing rewards.

[0161] Database: Stores conversation data, generated conversations, user information, billing information, etc.

[0162] Program processing explanation

[0163] User Registration / Login

[0164] 1. User Registration

[0165] The user's device connects to the server and displays a registration form. The user enters the required information, such as name, email address, and password, and submits it.

[0166] The server receives the input information, checks its integrity, and saves it in the database. If the save is successful, it sends a message to the user's device indicating that registration is complete.

[0167] Examples:

[0168] User A accesses the system using a user terminal and registers by entering "userA@example.com" and a password in the registration form.

[0169] Providing conversation data

[0170] 2. Uploading conversation data

[0171] The user's device connects to the server and displays the data upload screen. The user selects the conversation data they wish to provide and uploads it.

[0172] The server receives the conversation data, checks its integrity, and then saves it in the database. If the save is successful, it sends a message to the user device indicating that the upload is complete.

[0173] Examples:

[0174] User A selects an interview audio file of his / her favorite actor and uploads it to the system.

[0175] Conversation generation

[0176] 3. Conversation Generation Request

[0177] The user's device connects to the server and displays the conversation creation screen. The user selects a target person or character and sends a conversation creation request to the server.

[0178] The server's AI analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is then sent to the user's device.

[0179] Examples:

[0180] User A selects a favorite actor on the system and sends a request to the server to generate a conversation with that actor. The server's AI generates a realistic conversation based on the related voice data and sends it to User A's device.

[0181] Example prompt sentence:

[0182] "Create a conversation with your favorite actor, Hiroshi Tanaka."

[0183] Conversation Usage and Rewards

[0184] 4. Paid conversation usage

[0185] The user's device connects to the server and displays the billing screen. The user enters billing information (e.g., credit card information) and the billing process is carried out.

[0186] The server processes the payment based on the billing information, and if successful, unlocks the conversation data. It distributes a reward based on the billing amount to the data provider and notifies the user and the data provider of the processing result.

[0187] Examples:

[0188] User A pays 500 yen to use the generated conversation. The server distributes a portion of that amount to the data provider as a reward.

[0189] summary

[0190] This system allows users to enjoy real conversations without feeling lonely, while also allowing data providers to receive rewards.

[0191] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0192] Step 1: User Registration

[0193] Input: The user enters information such as name, email address, and password into a registration form on the device.

[0194] Specific operation: The user terminal receives input information and sends the information to the server.

[0195] Data processing: The server checks the integrity of the information received (for example, whether the email address is in the correct format or whether the password meets a certain strength).

[0196] Output: The server stores the information whose integrity has been confirmed in the database and sends a message to the user terminal indicating that registration is complete.

[0197] Step 2: Upload your conversation data

[0198] Input: The user selects the conversation data (for example, an interview audio file) that he or she wants to provide on the terminal.

[0199] Specific operation: The user terminal transmits the selected conversation data to the server.

[0200] Data processing: The server checks the format and content of the conversation data received (for example, whether the file format is appropriate and whether there are any problems with the content).

[0201] Output: After verifying the integrity, the server saves the conversation data in the database and sends a message to the user device indicating that the upload is complete.

[0202] Step 3: Conversation creation request

[0203] Input: The user selects the target person or character on the conversation generation screen from their device and sends a request.

[0204] Specific operation: The user terminal sends a conversation creation request to the server.

[0205] Data processing: The server's generation AI analyzes the request and retrieves the target conversation data from the database.

[0206] Output: The server's generation AI generates realistic conversations based on the acquired data, and sends the generated conversation data to the user's device.

[0207] Step 4: Pay for conversations

[0208] Input: The user inputs billing information (such as credit card information) to use the conversation generated on the terminal.

[0209] Specific operation: The user terminal transmits billing information to the server, and the server processes the payment.

[0210] Data processing: Once the server confirms the successful payment, it unlocks the generated conversation data and calculates the reward based on the payment amount.

[0211] Output: The server sends a notification of payment completion and the released conversation data to the user terminal, and automatically distributes the reward to the data provider.

[0212] In this way, the user terminal, server, and database work together to execute a series of processes that allow the user to enjoy a realistic conversation.

[0213] (Application example 1)

[0214] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0215] In conventional virtual stores, it is difficult for users to obtain detailed product information and they lack a real-time interactive shopping experience. As a result, users often feel dissatisfied with virtual shopping and lose their motivation to purchase. Furthermore, in systems that utilize conversation generation technology, rewards to conversation data providers are insufficient, which reduces the motivation of data providers. The challenge is to solve these problems and improve the satisfaction of both users and data providers.

[0216] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0217] In this invention, the server includes a means for a user to provide conversation data based on a target person or character, a means for a generation AI to generate realistic conversations using the conversation data, a means for distributing rewards for the generated conversations according to paid use, a means for a user to visually check products in a virtual store and provide product explanations and advice using the generated conversations, and a means for answering questions about the products in real time. This allows users to have a real-time interactive shopping experience in the virtual store, and at the same time, appropriate rewards are distributed to conversation data providers, thereby improving overall satisfaction.

[0218] "User" refers to any individual or entity that uses the system, provides conversation data, and uses the generated conversations.

[0219] "Target person or character" refers to a specific person or fictional character that is the basis for the conversation data provided by the user.

[0220] "Conversation Data" refers to user-provided voice and text data based on a target person or character.

[0221] "Generative AI" refers to artificial intelligence that uses provided conversation data to generate realistic conversations.

[0222] "Realistic conversation" refers to a conversation generated by generative AI that resembles a natural conversation between humans.

[0223] "Charged usage" refers to the fee paid by a user to use the generated conversation.

[0224] "Reward distribution" refers to the process by which rewards are distributed to data providers based on billed usage.

[0225] "Virtual store" refers to a virtual shopping environment that users can access via the Internet.

[0226] "Product description" refers to providing information about a product to a user using generative AI.

[0227] "Advice" refers to advice provided by the generating AI to users regarding product selection and usage.

[0228] "Real-time" refers to data processing and information provision occurring immediately, without delay.

[0229] "Answering questions" refers to providing immediate and appropriate answers to questions posed by users to the generated AI.

[0230] The present invention is a system that allows users to have a real-time interactive shopping experience in a virtual store. The system consists of three main components: a user terminal, a server, and a database.

[0231] First, let us explain the user terminal. A user terminal is a device such as smart glasses or a head-mounted display. Through this terminal, the user provides conversation data, requests conversation generation, sends product-related questions, and performs billing procedures. Specifically, the user can visually identify a specific product in a virtual store and request a realistic conversation about that product from the generation AI.

[0232] Next, we will explain the server. The server contains a generative AI and performs key processes such as saving conversation data, generating conversations, processing billing, and distributing rewards. When the server receives a conversation generation request from a user, it retrieves the target conversation data from a database and generates a realistic conversation using a generative AI model (e.g., OpenAI's GPT-3). The generated conversation is sent to the user's device, allowing the user to experience an interactive conversation with a virtual store clerk.

[0233] Finally, we will explain the database. The database is a system that stores user information, conversation data, generated conversations, billing information, etc. The database checks the consistency of the conversation data and provides the data necessary for the server to generate conversations using the generation AI.

[0234] As a concrete example, consider a scenario in which a user is selecting cosmetics in a virtual store. The user puts on smart glasses and enters the virtual store. They find the cosmetics on the shelves and send a question about the specific cosmetics to the system. By entering a prompt such as "Please tell me the features of this cosmetics," the generative AI will provide a detailed product description in real time based on the relevant conversation data.

[0235] This allows users to have an interactive shopping experience that feels just like they are talking to a salesperson in a real store. Meanwhile, since rewards are appropriately distributed to data providers, this also contributes to maintaining their motivation. In this way, the present invention is a system that can improve the satisfaction of both users and data providers.

[0236] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0237] Step 1: User Registration

[0238] The user uses a terminal to enter their name, email address, and password into the registration form and send it to the server. The server receives the input information, checks its integrity, and then stores it in a database. If successful, the server sends a message to the user terminal confirming registration. The input is user information (name, email address, password), and the output is a message confirming registration completion.

[0239] Step 2: Upload your conversation data

[0240] The user uses their device to select conversation data and send it to the server from the upload screen. The server receives the conversation data, checks its integrity, and then stores it in a database. If successful, the server sends a message to the user device indicating that the upload is complete. The input is the conversation data, and the output is a message confirming that the upload is complete.

[0241] Step 3: Conversation creation request

[0242] The user uses their device to select a target person or character and send a conversation generation request to the server. The server receives the request and retrieves the target conversation data from a database. A generative AI model (e.g., OpenAI's GPT-3) is used to generate a realistic conversation based on the prompt. The generated conversation is sent to the user's device. The input is the conversation generation request (target person or character, prompt), and the output is the generated conversation.

[0243] Step 4: Product selection and questions

[0244] Users use their devices to visually check products in a virtual store and send questions about the products to the server. The server receives the questions, generates answers in real time using generative AI, and sends them to the user's device. The input is a question about the product (prompt sentence), and the output is the answer (generated conversation).

[0245] Step 5: Billing process and reward distribution

[0246] The user uses the terminal to carry out the billing procedure to use the generated conversation. The server receives the billing information and processes the payment. If successful, it unlocks the conversation data and distributes rewards to the data provider according to the billing amount. The processing results are notified to the user and the data provider. The input is the billing information, and the output is the billing result and a reward distribution notification.

[0247] Step 6: Deliver a conversational experience

[0248] The user uses the device to experience virtual shopping through generated conversations. Based on the generated conversations, the user can receive product explanations and advice in real time. The input is the generated conversation, and the output is the user's purchasing experience.

[0249] Through these steps, users can have an interactive shopping experience in a virtual store, and at the same time, rewards are appropriately distributed to data providers.

[0250] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0251] The present invention combines a system in which a user provides conversation data based on a target person or character, a generation AI uses the conversation data to generate realistic conversations, and distributes rewards based on paid usage of the generated conversations with an emotion engine that recognizes the user's emotions. This system allows users to enjoy conversations that are realistic and reflect their emotions.

[0252] System configuration

[0253] This system consists of four main components: a user terminal, a server, a database, and an emotion engine.

[0254] 1. User terminal: A device that provides an interface for users to provide conversation data, request conversation generation, provide emotion data, and perform billing procedures.

[0255] 2. Server: Contains the generative AI and emotion engine, and performs key processes such as storing conversation data, generating conversations, recognizing emotions, processing billing, and distributing rewards.

[0256] 3. Database: Stores conversation data, generated conversations, user information, billing information, emotional data, etc.

[0257] 4. Emotion Engine: Analyzes the user's facial expressions, voice, or text to recognize emotions and provide feedback to the generative AI.

[0258] Program processing explanation

[0259] User Registration / Login

[0260] 1. User Registration

[0261] The user's device connects to the server and displays a registration form. The user enters the required information (name, email address, password) and sends it to the server.

[0262] The server receives the input information, checks its integrity, and saves it in the database. If successful, the server sends a registration completion message to the user's device.

[0263] Providing conversation data

[0264] 2. Uploading conversation data

[0265] The user's device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button.

[0266] The server receives the conversation data, checks its integrity, and then saves it in the database. If successful, the server sends a message to the user device indicating that the upload is complete.

[0267] Conversation generation

[0268] 3. Conversation Generation Request

[0269] The user's device connects to the server and displays the conversation creation screen. The user selects the target person or character and sends a conversation creation request.

[0270] The server's AI analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is then sent to the user's device.

[0271] Emotion recognition and conversation regulation

[0272] 4. Emotion recognition

[0273] The user terminal transmits the user's facial expression, voice, or text data to the server.

[0274] The server's emotion engine analyzes this data and recognizes the user's emotions.

[0275] 5. Moderate the conversation

[0276] The server's generation AI receives feedback from the emotion engine and adjusts the conversation content according to the user's emotions.

[0277] The adjusted conversation data is transmitted to the user terminal and displayed on the user terminal.

[0278] Conversation Usage and Rewards

[0279] 6. Paid Conversation Usage

[0280] The user terminal connects to the server and displays the billing screen. The user enters billing information and presses the billing button.

[0281] The server processes the payment based on the billing information, and if successful, unlocks the conversation data. It distributes a reward based on the billing amount to the data provider and notifies the user and the data provider of the processing result.

[0282] Specific examples

[0283] Each step is illustrated concretely through a scenario in which Yamada Taro uses this system.

[0284] 1. User Registration

[0285] Yamada Taro accesses the system using a user terminal and registers by entering "taro@example.com" and a password in the registration form.

[0286] 2. Providing conversation data

[0287] Taro Yamada selects an audio file of an interview with his favorite actor and uploads it to the system.

[0288] 3. Conversation Generation

[0289] Taro Yamada selects his favorite actor on the system and sends a request to generate a conversation with that actor. The server's AI generates realistic conversation based on the relevant voice data and sends it to Taro Yamada's device.

[0290] 4. Emotional awareness and conversation regulation

[0291] While Yamada Taro is using the system to converse, the user's device sends his facial expressions and voice data to the server. The server's emotion engine recognizes Yamada Taro's emotions, and the generation AI adjusts the content of the conversation based on that information. For example, if Yamada Taro is feeling down, the system generates a conversation that includes an encouraging message.

[0292] 5. Conversation Use and Reward

[0293] Taro Yamada pays 500 yen to use the generated conversation, and the server distributes a portion of that amount to the data provider as a reward.

[0294] In this way, the system of the present invention provides users with an opportunity to enjoy real, emotionally relevant conversations without feeling lonely, while at the same time increasing overall satisfaction by distributing rewards to data providers.

[0295] The processing flow will be explained below.

[0296] User Registration / Login

[0297] 1. User Registration

[0298] Step 1:

[0299] The user terminal connects to the server and displays the registration form.

[0300] Step 2:

[0301] The user enters the required information (name, email address, password) and clicks the send button.

[0302] Step 3:

[0303] The user terminal transmits the input information to the server.

[0304] Step 4:

[0305] The server verifies the integrity of the information entered, checking for missing or incorrect information.

[0306] Step 5:

[0307] After the server verifies it, it stores the information in a database.

[0308] Step 6:

[0309] The server generates a registration completion message and sends it to the user terminal.

[0310] Step 7:

[0311] A message indicating registration completion is displayed on the user's device.

[0312] Providing conversation data

[0313] 2. Uploading conversation data

[0314] Step 1:

[0315] The user terminal connects to the server and displays the data upload screen.

[0316] Step 2:

[0317] The user selects the conversation data file and information about the target person and presses the upload button.

[0318] Step 3:

[0319] The user terminal transmits the selected files and information to the server.

[0320] Step 4:

[0321] The server checks the integrity of the files and information received.

[0322] Step 5:

[0323] The server stores the files and information in a database.

[0324] Step 6:

[0325] The server generates an upload completion message and sends it to the user terminal.

[0326] Step 7:

[0327] A message indicating that the upload is complete will be displayed on the user's device.

[0328] Conversation generation

[0329] 3. Conversation Generation Request

[0330] Step 1:

[0331] The user terminal connects to the server and displays a conversation creation screen.

[0332] Step 2:

[0333] The user selects a target person or character and sends a request to generate a conversation.

[0334] Step 3:

[0335] The user terminal transmits request information to the server.

[0336] Step 4:

[0337] The server's generation AI analyzes the request and retrieves the conversation data of the target person or character from the database.

[0338] Step 5:

[0339] The server's generation AI generates realistic conversations based on the data it acquires.

[0340] Step 6:

[0341] The server transmits the generated conversation data to the user terminal.

[0342] Step 7:

[0343] The generated conversation is displayed on the user terminal.

[0344] Emotion recognition and conversation regulation

[0345] 4. Emotion recognition

[0346] Step 1:

[0347] The user terminal transmits the user's facial expression, voice, or text data to the server.

[0348] Step 2:

[0349] The server's emotion engine analyzes this data and recognizes the user's emotions.

[0350] Step 3:

[0351] The server's emotion engine provides the recognized emotion data to the generation AI.

[0352] 5. Moderate the conversation

[0353] Step 1:

[0354] The server's generation AI receives feedback from the emotion engine.

[0355] Step 2:

[0356] The server's generation AI adjusts the conversation content according to the user's emotions.

[0357] Step 3:

[0358] The server transmits the adjusted conversation data to the user terminal.

[0359] Step 4:

[0360] The adjusted conversation is displayed on the user terminal.

[0361] Conversation Usage and Rewards

[0362] 6. Paid Conversation Usage

[0363] Step 1:

[0364] The user terminal connects to the server and displays the billing screen.

[0365] Step 2:

[0366] The user inputs the billing information and presses the billing button.

[0367] Step 3:

[0368] The user terminal transmits the billing information to the server.

[0369] Step 4:

[0370] The server processes the payment based on the billing information received. If an external payment service is used, the server sends a request to that service.

[0371] Step 5:

[0372] The server receives confirmation of successful payment.

[0373] Step 6:

[0374] The server unlocks the conversation data and makes it available.

[0375] Step 7:

[0376] The server calculates the reward based on the usage charge and distributes it to the data provider.

[0377] Step 8:

[0378] The server notifies the user and data provider of the results of the billing and reward processing.

[0379] Step 9:

[0380] The processing results are displayed on the user terminal and the provider terminal.

[0381] Example 2

[0382] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0383] Conventional conversation generation systems have difficulty generating and adjusting conversations in real time based on the user's emotions and situation, resulting in a poor user experience. Furthermore, there are insufficient means for distributing appropriate rewards to conversation data providers, making fair reward distribution difficult. To solve these problems, the present invention aims to provide a system that enables conversation generation based on the user's emotions and fair reward distribution.

[0384] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0385] In this invention, the server includes a means for a user to provide conversation data based on a target person or character, a means for a generative AI model to generate realistic conversations using the conversation data, a means for distributing rewards for the generated conversations based on paid usage, an emotion engine that recognizes the user's emotions, and a means for adjusting the content of the conversations based on feedback from the emotion engine. This enables the generation of realistic conversations based on the user's emotions and fair distribution of rewards.

[0386] "User" refers to an individual or organization that uses the system to provide conversation data or enjoy the generated conversations.

[0387] "Target person or character" refers to a real person or fictional character that the user selects as the target for generating conversation data.

[0388] "Conversation data" refers to information such as voice, text, and images based on a target person or character provided by a user.

[0389] A "generative AI model" refers to an artificial intelligence algorithm or system that generates realistic conversations using provided conversation data.

[0390] "Charged usage" refers to the process whereby a user pays the system to use the generated conversations.

[0391] "Reward distribution" refers to the process of distributing appropriate rewards to providers of conversation data based on billed usage.

[0392] An "emotion engine" refers to a system or algorithm that analyzes a user's facial expressions, voice, or text to recognize their emotions at that moment and feeds the analysis results back into a generative AI model.

[0393] "Feedback" refers to the information the emotion engine provides to the generative AI model about the user's emotional state, allowing it to adjust the conversation content accordingly.

[0394] "Conversational content adjustment" refers to the process of modifying or optimizing the generated conversation to respond to the user's emotions based on feedback from the emotion engine.

[0395] The system allows users to provide conversation data based on target people or characters, generates realistic conversations using a generative AI model, and distributes rewards based on paid usage. The system includes four main components: a user terminal, a server, a database, and an emotion engine.

[0396] System configuration

[0397] 1. User Device:

[0398] A user terminal (e.g., a PC, smartphone, tablet, etc.) is a device that provides an interface for users to interact with the system. The user terminal is used to provide conversation data, send conversation generation requests, provide emotion data, and perform billing procedures.

[0399] 2. Server:

[0400] The server is the core of the system, containing the generative AI model and the emotion engine. The server performs the following main functions:

[0401] Saving conversation data

[0402] Conversation Generation

[0403] emotion recognition

[0404] Billing process

[0405] reward distribution

[0406] 3. Database:

[0407] The database stores conversation data, generated conversations, user information, billing information, emotion data, etc. This allows the necessary data to be efficiently retrieved and processed.

[0408] 4. Emotion Engine:

[0409] An emotion engine is software or algorithm that analyzes a user's facial expressions, voice, or text to recognize emotions. The emotion engine provides the necessary feedback to the generative AI model to adjust the conversation content according to the user's emotions.

[0410] Program processing

[0411] User Registration

[0412] The user's device connects to the server and displays a registration form. The user enters the required information (e.g., name, email address, password) and sends it to the server. The server receives the entered information, checks its integrity, and stores it in the database. If successful, the server sends a message to the user's device indicating that registration is complete.

[0413] Providing conversation data

[0414] The user device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button. The server receives the data, checks its integrity, and saves it in the database. If successful, the server sends a message to the user device indicating that the upload is complete.

[0415] Conversation generation

[0416] The user's device connects to the server and displays the conversation generation screen. The user selects the target person or character and sends a conversation generation request. The server's generation AI model analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is sent to the user's device.

[0417] example:

[0418] "Generate a conversation with actor A."

[0419] Emotion recognition and conversation regulation

[0420] The user device sends the user's facial expression, voice, or text data to the server. The server's emotion engine analyzes this data and recognizes the user's emotions. The server's generative AI model receives feedback from the emotion engine and adjusts the conversation content according to the user's emotions, and the adjusted conversation data is sent to the user device.

[0421] example:

[0422] "Generate a conversation to cheer up Taro Yamada who is feeling down."

[0423] Conversation Usage and Rewards

[0424] The user's terminal connects to the server and displays the billing screen. The user enters billing information and presses the billing button. The server processes the payment based on the billing information, and if successful, unlocks the conversation data. Based on the billing amount, the server distributes rewards to the data provider and notifies the user and data provider of the processing results.

[0425] example:

[0426] "Please handle the billing process and distribute the rewards for using the generated conversations."

[0427] This system allows users to enjoy realistic, emotionally relevant conversations, while also providing fair rewards to data providers.

[0428] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0429] Step 1:

[0430] User Registration

[0431] Input: The user enters personal information such as a name, email address, or password.

[0432] Specific operation: The user's device connects to the server and displays the registration form. The user presses the submit button to send the information entered to the server.

[0433] Data processing and calculation: The server checks the integrity of the information received and stores it in the database.

[0434] Output: The server sends a registration completion message to the user terminal.

[0435] Step 2:

[0436] Providing conversation data

[0437] Input: User-provided conversation data (e.g., audio files).

[0438] Specific operation: The user's device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button.

[0439] Data processing and calculation: The server checks the integrity of the data received and stores it in the database.

[0440] Output: The server sends a message to the user device indicating that the upload is complete.

[0441] Step 3:

[0442] Conversation generation request

[0443] Input: A person or character selected by the user.

[0444] Specific operation: The user's device connects to the server and displays the conversation creation screen. The user selects the target person or character and sends a conversation creation request.

[0445] Data processing and calculation: The server's generated AI model receives the request, retrieves the target conversation data from the database, and generates a realistic conversation.

[0446] Output: Send the generated conversation to the user's device.

[0447] Step 4:

[0448] emotion recognition

[0449] Input: User facial, voice, or text data.

[0450] Specific operation: The user's device captures the user's facial expression and voice data and sends it to the server.

[0451] Data processing and calculation: The server's emotion engine analyzes this data and recognizes the user's emotions.

[0452] Output: Recognized emotion data is sent as feedback to the generative AI model.

[0453] Step 5:

[0454] Adjusting the conversation

[0455] Input: Recognized emotion data and generated speech data.

[0456] How it works: The server's generative AI model receives feedback from the emotion engine and adjusts the conversation content.

[0457] Data processing and calculation: Conversation content is optimized according to the user's emotions.

[0458] Output: The adjusted conversation data is sent to the user terminal.

[0459] Step 6:

[0460] Pay-per-use conversations

[0461] Input: Billing information entered by the user.

[0462] Specific operation: The user's device connects to the server and displays the billing screen. The user enters billing information and presses the billing button.

[0463] Data processing and calculation: The server processes the billing information, and if payment is successful, unlocks the conversation data. Based on the billing amount, rewards are distributed to the data provider.

[0464] Output: Notify the user and data provider of the processing results.

[0465] (Application example 2)

[0466] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0467] Conventional virtual salesperson systems have the problem that interactions with users are fixed and one-way, and are unable to reflect the user's emotions and situation in real time. Furthermore, the quality of the generated conversations is poor, making it difficult to improve the user experience. Furthermore, there is no established reward distribution system to effectively utilize the content of these conversations, making it difficult to maintain the motivation of data providers. Technology that solves these issues is needed.

[0468] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0469] In this invention, the server includes: a means for a user to provide conversation data based on a target person or character; a means for a generation AI to generate realistic conversations using the conversation data; a means for an emotion engine to analyze the user's facial expressions, voice, or text to recognize emotions and adjust the generated conversation; and a means for distributing rewards for the generated conversation based on paid usage. This enables adaptive conversations that reflect the user's emotions in real time, allowing for high-quality dialogue by virtual salespeople. Furthermore, the reward distribution system can improve incentives for data providers.

[0470] A "user terminal" is a device that provides an interface for a user to provide conversation data, request conversation generation, provide emotion data, and perform billing procedures.

[0471] The "server" is a system that includes the generative AI and emotion engine and performs key processes such as storing conversation data, generating conversations, recognizing emotions, processing billing, and distributing rewards.

[0472] A "database" is a storage device that stores conversation data, generated conversations, user information, billing information, emotional data, and the like.

[0473] An "emotion engine" is an analysis system that analyzes a user's facial expressions, voice, or text to recognize emotions and provide feedback to the generative AI.

[0474] "Generative AI" is an artificial intelligence model that generates realistic dialogue based on conversation data.

[0475] "Conversation data based on a target person or character" is data of conversation related to a specific person or character provided by the user.

[0476] A "conversation generation request" is a request for the user to generate a conversation with a target person or character.

[0477] "Paid usage" refers to usage based on the amount paid by a user to use a particular service or content.

[0478] "Reward sharing" is a mechanism for distributing compensation to data providers and other participants based on paid usage.

[0479] The present invention provides a system for generating real-time conversations between a user and a salesperson in a virtual store according to their emotions, and for distributing rewards according to the amount of paid use. Detailed embodiments of the present invention will be described below.

[0480] Hardware and software used

[0481] Smart devices: smartphones, smart glasses, head-mounted displays

[0482] Server: Generative AI (GPT-4), emotion engine (Microsoft Azure Emotion API), database

[0483] User terminal: A device that provides an interface for users to provide conversation data, request conversation generation, provide emotion data, and perform billing procedures.

[0484] Program processing explanation

[0485] User Registration / Login

[0486] The server receives the required information (name, email address, password) and stores the submitted information in a database. The user then registers by entering the information into the registration form on their smart device.

[0487] Providing conversation data

[0488] Users upload conversation data and product information using their own devices. The server receives the data, checks its integrity, and then stores it in the database. This process allows the system to obtain a wealth of data.

[0489] Conversation generation

[0490] When a user asks a question about a product or requests purchasing advice, the server's AI analyzes the request and retrieves relevant conversation data from the database. Based on this data, the AI ​​generates a realistic conversation and provides it to the user.

[0491] Emotion Recognition and Speech Modulation

[0492] When a user converses through a smart device, the device transmits the user's facial expressions and voice data to the emotion engine for analysis. The emotion engine then feeds back the analysis results to the generation AI, which then adjusts the content of the conversation according to the user's emotions. This allows for adaptive conversations to be provided according to the user's emotions.

[0493] Pay-per-use conversations

[0494] When a user uses the premium conversation service, the billing procedure is carried out within the system. The server processes the billing information and, if successful, provides the premium features to the user. It also distributes rewards to data providers based on the billed usage.

[0495] Specific examples

[0496] For example, if a user asks a question about a product in a virtual store and the user has a sad expression on their face, the emotion engine will recognize that emotion and the generation AI will generate a conversation that encourages the user.

[0497] Prompt Sentence Examples

[0498] An example prompt for the user to ask a question might look like this:

[0499] User expression: Sad

[0500] User Question: I've been through a lot lately and need some emotional support. Can you offer some words of encouragement?

[0501] The generative AI analyzes this and generates the following dialogue:

[0502] "I know you're going through a difficult time. I'm sorry to trouble you, but please remember that I'm always by your side. It's important to stop sometimes. Even if it's just a little at a time, keep moving forward."

[0503] This embodiment provides high-quality dialogue that responds to the user's emotions, significantly improving the user experience in the virtual store. Furthermore, by distributing rewards to data providers, the system can be expected to continue to develop.

[0504] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0505] Step 1:

[0506] User Registration / Login

[0507] The user enters the required information (name, email address, password) into the registration form on the smart device and submits it.

[0508] The server receives this input data, checks the data integrity, and if there are no problems, stores it in the database.

[0509] The inputs to this process are the name, email address, and password entered by the user, and the output is the user information stored in a database.

[0510] Step 2:

[0511] Providing conversation data

[0512] Users upload conversation data and product information from their own terminals.

[0513] The server receives this data, checks the data format and content for consistency, and then stores it in the database.

[0514] The input for this process is conversation data and product information uploaded by the user, and the output is conversation data stored in the database.

[0515] Step 3:

[0516] Conversation generation request

[0517] The user selects the person or character with whom they want to create a conversation from their smart device and sends a conversation creation request.

[0518] The server's generation AI receives this request and retrieves conversation data related to the target from the database.

[0519] The generative AI generates realistic dialogue based on the acquired data and sends it to the user's device.

[0520] The inputs to this process are a user's conversation generation request and conversation data related to the target, and the output is the generated realistic conversation.

[0521] Step 4:

[0522] Emotion Recognition and Speech Modulation

[0523] While a user is having a conversation through a smart device, the device captures the user's facial expressions and voice data and transmits them to a server.

[0524] The server's emotion engine analyzes this data and recognizes the user's emotions.

[0525] Based on feedback from the emotion engine, the generative AI adjusts the content of the conversation and generates an adaptive conversation, which is then sent back to the user's device.

[0526] The inputs for this process are the user's facial expressions, voice data, and emotion recognition feedback, and the output is speech that has been adjusted according to the user's emotions.

[0527] Step 5:

[0528] Pay-per-use conversations

[0529] When using the premium conversation service, users complete the billing procedure on their smart device.

[0530] The server processes the billing information and verifies the integrity of the payment. If successful, the premium conversation service is unlocked.

[0531] The server performs processing to distribute rewards to data providers based on the amount charged.

[0532] The input of this process is the user's billing information and the billing amount, and the output is a notification of the release of the premium function and the distribution of rewards.

[0533] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0534] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0535] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0536] [Second embodiment]

[0537] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0538] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0539] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0540] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0541] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0542] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0543] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0544] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0545] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0546] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0547] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0548] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0549] The present invention relates to a system in which users provide conversation data based on target people or characters, a generation AI uses the conversation data to generate realistic conversations, and rewards are distributed based on paid usage of the generated conversations. This system allows users to enjoy realistic conversations, and data providers can also earn rewards.

[0550] System configuration

[0551] This system consists of three main components: a user terminal, a server, and a database.

[0552] 1. User terminal: A device that provides an interface for users to provide conversation data, request conversation creation, and perform billing procedures.

[0553] 2. Server: Includes the generation AI and performs key processes such as storing conversation data, generating conversations, processing billing, and distributing rewards.

[0554] 3. Database: Stores conversation data, generated conversations, user information, billing information, etc.

[0555] Program processing explanation

[0556] User Registration / Login

[0557] 1. User Registration

[0558] The user's device connects to the server and displays a registration form. The user enters the required information (name, email address, password) and sends it to the server.

[0559] The server receives the input information, checks its integrity, and saves it in the database. If successful, the server sends a registration completion message to the user's device.

[0560] Providing conversation data

[0561] 2. Uploading conversation data

[0562] The user's device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button.

[0563] The server receives the conversation data, checks its integrity, and then saves it in the database. If successful, the server sends a message to the user device indicating that the upload is complete.

[0564] Conversation generation

[0565] 3. Conversation Generation Request

[0566] The user's device connects to the server and displays the conversation creation screen. The user selects the target person or character and sends a conversation creation request.

[0567] The server's AI analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is then sent to the user's device.

[0568] Conversation Usage and Rewards

[0569] 4. Paid conversation usage

[0570] The user terminal connects to the server and displays the billing screen. The user enters billing information and presses the billing button.

[0571] The server processes the payment based on the billing information, and if successful, unlocks the conversation data. It distributes a reward based on the billing amount to the data provider and notifies the user and the data provider of the processing result.

[0572] Specific examples

[0573] Each step is illustrated concretely through a scenario in which Yamada Taro uses this system.

[0574] 1. User Registration

[0575] Yamada Taro accesses the system using a user terminal and registers by entering "taro@example.com" and a password in the registration form.

[0576] 2. Providing conversation data

[0577] Taro Yamada selects an audio file of an interview with his favorite actor and uploads it to the system.

[0578] 3. Conversation Generation

[0579] Taro Yamada selects his favorite actor on the system and sends a request to generate a conversation with that actor. The server's AI generates realistic conversation based on the relevant voice data and sends it to Taro Yamada's device.

[0580] 4. Conversational Use and Reward

[0581] Taro Yamada pays 500 yen to use the generated conversation, and the server distributes a portion of that amount to the data provider as a reward.

[0582] In this way, the system of the present invention provides users with an opportunity to enjoy real conversations without feeling lonely, and at the same time, by distributing rewards to data providers, it increases overall satisfaction.

[0583] The processing flow will be explained below.

[0584] User Registration / Login

[0585] 1. User Registration

[0586] Step 1:

[0587] The user terminal connects to the server and displays the registration form.

[0588] Step 2:

[0589] The user enters the required information (name, email address, password) and clicks the send button.

[0590] Step 3:

[0591] The user terminal transmits the input information to the server.

[0592] Step 4:

[0593] The server verifies the integrity of the information entered, checking for missing or incorrect information.

[0594] Step 5:

[0595] After the server verifies it, it stores the information in a database.

[0596] Step 6:

[0597] The server generates a registration completion message and sends it to the user terminal.

[0598] Step 7:

[0599] A message indicating registration completion is displayed on the user's device.

[0600] Providing conversation data

[0601] 2. Uploading conversation data

[0602] Step 1:

[0603] The user terminal connects to the server and displays the data upload screen.

[0604] Step 2:

[0605] The user selects the conversation data file and information about the target person and presses the upload button.

[0606] Step 3:

[0607] The user terminal transmits the selected files and information to the server.

[0608] Step 4:

[0609] The server checks the integrity of the files and information received.

[0610] Step 5:

[0611] The server stores the files and information in a database.

[0612] Step 6:

[0613] The server generates an upload completion message and sends it to the user terminal.

[0614] Step 7:

[0615] A message indicating that the upload is complete will be displayed on the user's device.

[0616] Conversation generation

[0617] 3. Conversation Generation Request

[0618] Step 1:

[0619] The user terminal connects to the server and displays a conversation creation screen.

[0620] Step 2:

[0621] The user selects a target person or character and sends a request to generate a conversation.

[0622] Step 3:

[0623] The user terminal transmits request information to the server.

[0624] Step 4:

[0625] The server's generation AI analyzes the request and retrieves the conversation data of the target person or character from the database.

[0626] Step 5:

[0627] The server's generation AI generates realistic conversations based on the data it acquires.

[0628] Step 6:

[0629] The server transmits the generated conversation data to the user terminal.

[0630] Step 7:

[0631] The generated conversation is displayed on the user terminal.

[0632] Conversation Usage and Rewards

[0633] 4. Paid conversation usage

[0634] Step 1:

[0635] The user terminal connects to the server and displays the billing screen.

[0636] Step 2:

[0637] The user inputs the billing information and presses the billing button.

[0638] Step 3:

[0639] The user terminal transmits the billing information to the server.

[0640] Step 4:

[0641] The server processes the payment based on the billing information received. If an external payment service is used, the server sends a request to that service.

[0642] Step 5:

[0643] The server receives confirmation of successful payment.

[0644] Step 6:

[0645] The server unlocks the conversation data and makes it available.

[0646] Step 7:

[0647] The server calculates the reward based on the usage charge and distributes it to the data provider.

[0648] Step 8:

[0649] The server notifies the user and data provider of the results of the billing and reward processing.

[0650] Step 9:

[0651] The processing results are displayed on the user terminal and the provider terminal.

[0652] Example 1

[0653] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0654] In today's world, users often feel lonely, and especially when they spend a lot of time alone, they want to enjoy real conversations with others. However, in reality, there are limited ways to easily generate and use conversations with target people or characters. Another issue is the lack of a system that provides appropriate rewards to providers of conversation data.

[0655] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0656] In this invention, the server includes: a means for a user to provide conversation data based on a target person or character; a means for a generation AI to generate realistic conversations using the conversation data; a means for distributing rewards for the generated conversations according to paid usage; a means for a user terminal to connect to the server, transmit user registration and login information, and perform authentication; a means for a user to upload conversation data and for the server to store it in a database; a means for inputting billing information and processing payments; and a means for unlocking the conversation data and automatically distributing rewards. This allows users to easily enjoy realistic conversations, and data providers to receive fair rewards.

[0657] A "user terminal" is a device that allows a user to access the system, provide conversation data, request conversation creation, and perform billing procedures.

[0658] A "server" is a computer system that includes the generation AI and performs major processes such as storing conversation data, generating conversations, processing billing, and distributing rewards.

[0659] A "database" is an information management system for storing conversation data, generated conversations, user information, billing information, and the like.

[0660] "Generative AI" refers to algorithms and software that use provided conversational data to generate realistic conversations.

[0661] "Conversation data" refers to data such as audio files and text data related to a target person or character provided by the user.

[0662] "Billing information" refers to credit card information or other payment method information required for users to pay for conversation creation and usage.

[0663] "Reward" refers to money or something of equivalent value distributed to a user who provides conversation data based on the fee-based usage.

[0664] "Realistic conversation" refers to dialogue data that resembles natural conversations between humans, generated by the generative AI based on the conversation data provided.

[0665] The present invention is a system in which a user provides conversation data based on a target person or character, a generation AI uses the conversation data to generate realistic conversations, and rewards are distributed based on paid usage of the generated conversations. This system allows users to enjoy realistic conversations, and data providers can also receive rewards.

[0666] System configuration

[0667] This system consists of three main components: a user terminal, a server, and a database.

[0668] User terminal: A device that provides an interface for users to provide conversation data, request conversation creation, and perform billing procedures.

[0669] Server: Contains the generation AI and performs key processes such as storing conversation data, generating conversations, processing billing, and distributing rewards.

[0670] Database: Stores conversation data, generated conversations, user information, billing information, etc.

[0671] Program processing explanation

[0672] User Registration / Login

[0673] 1. User Registration

[0674] The user's device connects to the server and displays a registration form. The user enters the required information, such as name, email address, and password, and submits it.

[0675] The server receives the input information, checks its integrity, and saves it in the database. If the save is successful, it sends a message to the user's device indicating that registration is complete.

[0676] Examples:

[0677] User A accesses the system using a user terminal and registers by entering "userA@example.com" and a password in the registration form.

[0678] Providing conversation data

[0679] 2. Uploading conversation data

[0680] The user's device connects to the server and displays the data upload screen. The user selects the conversation data they wish to provide and uploads it.

[0681] The server receives the conversation data, checks its integrity, and then saves it in the database. If the save is successful, it sends a message to the user device indicating that the upload is complete.

[0682] Examples:

[0683] User A selects an interview audio file of his / her favorite actor and uploads it to the system.

[0684] Conversation generation

[0685] 3. Conversation Generation Request

[0686] The user's device connects to the server and displays the conversation creation screen. The user selects a target person or character and sends a conversation creation request to the server.

[0687] The server's AI analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is then sent to the user's device.

[0688] Examples:

[0689] User A selects a favorite actor on the system and sends a request to the server to generate a conversation with that actor. The server's AI generates a realistic conversation based on the related voice data and sends it to User A's device.

[0690] Example prompt sentence:

[0691] "Create a conversation with your favorite actor, Hiroshi Tanaka."

[0692] Conversation Usage and Rewards

[0693] 4. Paid conversation usage

[0694] The user's device connects to the server and displays the billing screen. The user enters billing information (e.g., credit card information) and the billing process is carried out.

[0695] The server processes the payment based on the billing information, and if successful, unlocks the conversation data. It distributes a reward based on the billing amount to the data provider and notifies the user and the data provider of the processing result.

[0696] Examples:

[0697] User A pays 500 yen to use the generated conversation. The server distributes a portion of that amount to the data provider as a reward.

[0698] summary

[0699] This system allows users to enjoy real conversations without feeling lonely, while also allowing data providers to receive rewards.

[0700] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0701] Step 1: User Registration

[0702] Input: The user enters information such as name, email address, and password into a registration form on the device.

[0703] Specific operation: The user terminal receives input information and sends the information to the server.

[0704] Data processing: The server checks the integrity of the information received (for example, whether the email address is in the correct format or whether the password meets a certain strength).

[0705] Output: The server stores the information whose integrity has been confirmed in the database and sends a message to the user terminal indicating that registration is complete.

[0706] Step 2: Upload your conversation data

[0707] Input: The user selects the conversation data (for example, an interview audio file) that he or she wants to provide on the terminal.

[0708] Specific operation: The user terminal transmits the selected conversation data to the server.

[0709] Data processing: The server checks the format and content of the conversation data received (for example, whether the file format is appropriate and whether there are any problems with the content).

[0710] Output: After verifying the integrity, the server saves the conversation data in the database and sends a message to the user device indicating that the upload is complete.

[0711] Step 3: Conversation creation request

[0712] Input: The user selects the target person or character on the conversation generation screen from their device and sends a request.

[0713] Specific operation: The user terminal sends a conversation creation request to the server.

[0714] Data processing: The server's generation AI analyzes the request and retrieves the target conversation data from the database.

[0715] Output: The server's generation AI generates realistic conversations based on the acquired data, and sends the generated conversation data to the user's device.

[0716] Step 4: Pay for conversations

[0717] Input: The user inputs billing information (such as credit card information) to use the conversation generated on the terminal.

[0718] Specific operation: The user terminal transmits billing information to the server, and the server processes the payment.

[0719] Data processing: Once the server confirms the successful payment, it unlocks the generated conversation data and calculates the reward based on the payment amount.

[0720] Output: The server sends a notification of payment completion and the released conversation data to the user terminal, and automatically distributes the reward to the data provider.

[0721] In this way, the user terminal, server, and database work together to execute a series of processes that allow the user to enjoy a realistic conversation.

[0722] (Application example 1)

[0723] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0724] In conventional virtual stores, it is difficult for users to obtain detailed product information and they lack a real-time interactive shopping experience. As a result, users often feel dissatisfied with virtual shopping and lose their motivation to purchase. Furthermore, in systems that utilize conversation generation technology, rewards to conversation data providers are insufficient, which reduces the motivation of data providers. The challenge is to solve these problems and improve the satisfaction of both users and data providers.

[0725] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0726] In this invention, the server includes a means for a user to provide conversation data based on a target person or character, a means for a generation AI to generate realistic conversations using the conversation data, a means for distributing rewards for the generated conversations according to paid use, a means for a user to visually check products in a virtual store and provide product explanations and advice using the generated conversations, and a means for answering questions about the products in real time. This allows users to have a real-time interactive shopping experience in the virtual store, and at the same time, appropriate rewards are distributed to conversation data providers, thereby improving overall satisfaction.

[0727] "User" refers to any individual or entity that uses the system, provides conversation data, and uses the generated conversations.

[0728] "Target person or character" refers to a specific person or fictional character that is the basis for the conversation data provided by the user.

[0729] "Conversation Data" refers to user-provided voice and text data based on a target person or character.

[0730] "Generative AI" refers to artificial intelligence that uses provided conversation data to generate realistic conversations.

[0731] "Realistic conversation" refers to a conversation generated by generative AI that resembles a natural conversation between humans.

[0732] "Charged usage" refers to the fee paid by a user to use the generated conversation.

[0733] "Reward distribution" refers to the process by which rewards are distributed to data providers based on billed usage.

[0734] "Virtual store" refers to a virtual shopping environment that users can access via the Internet.

[0735] "Product description" refers to providing information about a product to a user using generative AI.

[0736] "Advice" refers to advice provided by the generating AI to users regarding product selection and usage.

[0737] "Real-time" refers to data processing and information provision occurring immediately, without delay.

[0738] "Answering questions" refers to providing immediate and appropriate answers to questions posed by users to the generated AI.

[0739] The present invention is a system that allows users to have a real-time interactive shopping experience in a virtual store. The system consists of three main components: a user terminal, a server, and a database.

[0740] First, let us explain the user terminal. A user terminal is a device such as smart glasses or a head-mounted display. Through this terminal, the user provides conversation data, requests conversation generation, sends product-related questions, and performs billing procedures. Specifically, the user can visually identify a specific product in a virtual store and request a realistic conversation about that product from the generation AI.

[0741] Next, we will explain the server. The server contains a generative AI and performs key processes such as saving conversation data, generating conversations, processing billing, and distributing rewards. When the server receives a conversation generation request from a user, it retrieves the target conversation data from a database and generates a realistic conversation using a generative AI model (e.g., OpenAI's GPT-3). The generated conversation is sent to the user's device, allowing the user to experience an interactive conversation with a virtual store clerk.

[0742] Finally, we will explain the database. The database is a system that stores user information, conversation data, generated conversations, billing information, etc. The database checks the consistency of the conversation data and provides the data necessary for the server to generate conversations using the generation AI.

[0743] As a concrete example, consider a scenario in which a user is selecting cosmetics in a virtual store. The user puts on smart glasses and enters the virtual store. They find the cosmetics on the shelves and send a question about the specific cosmetics to the system. By entering a prompt such as "Please tell me the features of this cosmetics," the generative AI will provide a detailed product description in real time based on the relevant conversation data.

[0744] This allows users to have an interactive shopping experience that feels just like they are talking to a salesperson in a real store. Meanwhile, since rewards are appropriately distributed to data providers, this also contributes to maintaining their motivation. In this way, the present invention is a system that can improve the satisfaction of both users and data providers.

[0745] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0746] Step 1: User Registration

[0747] The user uses a terminal to enter their name, email address, and password into the registration form and send it to the server. The server receives the input information, checks its integrity, and then stores it in a database. If successful, the server sends a message to the user terminal confirming registration. The input is user information (name, email address, password), and the output is a message confirming registration completion.

[0748] Step 2: Upload your conversation data

[0749] The user uses their device to select conversation data and send it to the server from the upload screen. The server receives the conversation data, checks its integrity, and then stores it in a database. If successful, the server sends a message to the user device indicating that the upload is complete. The input is the conversation data, and the output is a message confirming that the upload is complete.

[0750] Step 3: Conversation creation request

[0751] The user uses their device to select a target person or character and send a conversation generation request to the server. The server receives the request and retrieves the target conversation data from a database. A generative AI model (e.g., OpenAI's GPT-3) is used to generate a realistic conversation based on the prompt. The generated conversation is sent to the user's device. The input is the conversation generation request (target person or character, prompt), and the output is the generated conversation.

[0752] Step 4: Product selection and questions

[0753] Users use their devices to visually check products in a virtual store and send questions about the products to the server. The server receives the questions, generates answers in real time using generative AI, and sends them to the user's device. The input is a question about the product (prompt sentence), and the output is the answer (generated conversation).

[0754] Step 5: Billing process and reward distribution

[0755] The user uses the terminal to carry out the billing procedure to use the generated conversation. The server receives the billing information and processes the payment. If successful, it unlocks the conversation data and distributes rewards to the data provider according to the billing amount. The processing results are notified to the user and the data provider. The input is the billing information, and the output is the billing result and a reward distribution notification.

[0756] Step 6: Deliver a conversational experience

[0757] The user uses the device to experience virtual shopping through generated conversations. Based on the generated conversations, the user can receive product explanations and advice in real time. The input is the generated conversation, and the output is the user's purchasing experience.

[0758] Through these steps, users can have an interactive shopping experience in a virtual store, and at the same time, rewards are appropriately distributed to data providers.

[0759] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0760] The present invention combines a system in which a user provides conversation data based on a target person or character, a generation AI uses the conversation data to generate realistic conversations, and distributes rewards based on paid usage of the generated conversations with an emotion engine that recognizes the user's emotions. This system allows users to enjoy conversations that are realistic and reflect their emotions.

[0761] System configuration

[0762] This system consists of four main components: a user terminal, a server, a database, and an emotion engine.

[0763] 1. User terminal: A device that provides an interface for users to provide conversation data, request conversation generation, provide emotion data, and perform billing procedures.

[0764] 2. Server: Contains the generative AI and emotion engine, and performs key processes such as storing conversation data, generating conversations, recognizing emotions, processing billing, and distributing rewards.

[0765] 3. Database: Stores conversation data, generated conversations, user information, billing information, emotional data, etc.

[0766] 4. Emotion Engine: Analyzes the user's facial expressions, voice, or text to recognize emotions and provide feedback to the generative AI.

[0767] Program processing explanation

[0768] User Registration / Login

[0769] 1. User Registration

[0770] The user's device connects to the server and displays a registration form. The user enters the required information (name, email address, password) and sends it to the server.

[0771] The server receives the input information, checks its integrity, and saves it in the database. If successful, the server sends a registration completion message to the user's device.

[0772] Providing conversation data

[0773] 2. Uploading conversation data

[0774] The user's device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button.

[0775] The server receives the conversation data, checks its integrity, and then saves it in the database. If successful, the server sends a message to the user device indicating that the upload is complete.

[0776] Conversation generation

[0777] 3. Conversation Generation Request

[0778] The user's device connects to the server and displays the conversation creation screen. The user selects the target person or character and sends a conversation creation request.

[0779] The server's AI analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is then sent to the user's device.

[0780] Emotion recognition and conversation regulation

[0781] 4. Emotion recognition

[0782] The user terminal transmits the user's facial expression, voice, or text data to the server.

[0783] The server's emotion engine analyzes this data and recognizes the user's emotions.

[0784] 5. Moderate the conversation

[0785] The server's generation AI receives feedback from the emotion engine and adjusts the conversation content according to the user's emotions.

[0786] The adjusted conversation data is transmitted to the user terminal and displayed on the user terminal.

[0787] Conversation Usage and Rewards

[0788] 6. Paid Conversation Usage

[0789] The user terminal connects to the server and displays the billing screen. The user enters billing information and presses the billing button.

[0790] The server processes the payment based on the billing information, and if successful, unlocks the conversation data. It distributes a reward based on the billing amount to the data provider and notifies the user and the data provider of the processing result.

[0791] Specific examples

[0792] Each step is illustrated concretely through a scenario in which Yamada Taro uses this system.

[0793] 1. User Registration

[0794] Yamada Taro accesses the system using a user terminal and registers by entering "taro@example.com" and a password in the registration form.

[0795] 2. Providing conversation data

[0796] Taro Yamada selects an audio file of an interview with his favorite actor and uploads it to the system.

[0797] 3. Conversation Generation

[0798] Taro Yamada selects his favorite actor on the system and sends a request to generate a conversation with that actor. The server's AI generates realistic conversation based on the relevant voice data and sends it to Taro Yamada's device.

[0799] 4. Emotional awareness and conversation regulation

[0800] While Yamada Taro is using the system to converse, the user's device sends his facial expressions and voice data to the server. The server's emotion engine recognizes Yamada Taro's emotions, and the generation AI adjusts the content of the conversation based on that information. For example, if Yamada Taro is feeling down, the system generates a conversation that includes an encouraging message.

[0801] 5. Conversation Use and Reward

[0802] Taro Yamada pays 500 yen to use the generated conversation, and the server distributes a portion of that amount to the data provider as a reward.

[0803] In this way, the system of the present invention provides users with an opportunity to enjoy real, emotionally relevant conversations without feeling lonely, while at the same time increasing overall satisfaction by distributing rewards to data providers.

[0804] The processing flow will be explained below.

[0805] User Registration / Login

[0806] 1. User Registration

[0807] Step 1:

[0808] The user terminal connects to the server and displays the registration form.

[0809] Step 2:

[0810] The user enters the required information (name, email address, password) and clicks the send button.

[0811] Step 3:

[0812] The user terminal transmits the input information to the server.

[0813] Step 4:

[0814] The server verifies the integrity of the information entered, checking for missing or incorrect information.

[0815] Step 5:

[0816] After the server verifies it, it stores the information in a database.

[0817] Step 6:

[0818] The server generates a registration completion message and sends it to the user terminal.

[0819] Step 7:

[0820] A message indicating registration completion is displayed on the user's device.

[0821] Providing conversation data

[0822] 2. Uploading conversation data

[0823] Step 1:

[0824] The user terminal connects to the server and displays the data upload screen.

[0825] Step 2:

[0826] The user selects the conversation data file and information about the target person and presses the upload button.

[0827] Step 3:

[0828] The user terminal transmits the selected files and information to the server.

[0829] Step 4:

[0830] The server checks the integrity of the files and information received.

[0831] Step 5:

[0832] The server stores the files and information in a database.

[0833] Step 6:

[0834] The server generates an upload completion message and sends it to the user terminal.

[0835] Step 7:

[0836] A message indicating that the upload is complete will be displayed on the user's device.

[0837] Conversation generation

[0838] 3. Conversation Generation Request

[0839] Step 1:

[0840] The user terminal connects to the server and displays a conversation creation screen.

[0841] Step 2:

[0842] The user selects a target person or character and sends a request to generate a conversation.

[0843] Step 3:

[0844] The user terminal transmits request information to the server.

[0845] Step 4:

[0846] The server's generation AI analyzes the request and retrieves the conversation data of the target person or character from the database.

[0847] Step 5:

[0848] The server's generation AI generates realistic conversations based on the data it acquires.

[0849] Step 6:

[0850] The server transmits the generated conversation data to the user terminal.

[0851] Step 7:

[0852] The generated conversation is displayed on the user terminal.

[0853] Emotion recognition and conversation regulation

[0854] 4. Emotion recognition

[0855] Step 1:

[0856] The user terminal transmits the user's facial expression, voice, or text data to the server.

[0857] Step 2:

[0858] The server's emotion engine analyzes this data and recognizes the user's emotions.

[0859] Step 3:

[0860] The server's emotion engine provides the recognized emotion data to the generation AI.

[0861] 5. Moderate the conversation

[0862] Step 1:

[0863] The server's generation AI receives feedback from the emotion engine.

[0864] Step 2:

[0865] The server's generation AI adjusts the conversation content according to the user's emotions.

[0866] Step 3:

[0867] The server transmits the adjusted conversation data to the user terminal.

[0868] Step 4:

[0869] The adjusted conversation is displayed on the user terminal.

[0870] Conversation Usage and Rewards

[0871] 6. Paid Conversation Usage

[0872] Step 1:

[0873] The user terminal connects to the server and displays the billing screen.

[0874] Step 2:

[0875] The user inputs the billing information and presses the billing button.

[0876] Step 3:

[0877] The user terminal transmits the billing information to the server.

[0878] Step 4:

[0879] The server processes the payment based on the billing information received. If an external payment service is used, the server sends a request to that service.

[0880] Step 5:

[0881] The server receives confirmation of successful payment.

[0882] Step 6:

[0883] The server unlocks the conversation data and makes it available.

[0884] Step 7:

[0885] The server calculates the reward based on the usage charge and distributes it to the data provider.

[0886] Step 8:

[0887] The server notifies the user and data provider of the results of the billing and reward processing.

[0888] Step 9:

[0889] The processing results are displayed on the user terminal and the provider terminal.

[0890] Example 2

[0891] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0892] Conventional conversation generation systems have difficulty generating and adjusting conversations in real time based on the user's emotions and situation, resulting in a poor user experience. Furthermore, there are insufficient means for distributing appropriate rewards to conversation data providers, making fair reward distribution difficult. To solve these problems, the present invention aims to provide a system that enables conversation generation based on the user's emotions and fair reward distribution.

[0893] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0894] In this invention, the server includes a means for a user to provide conversation data based on a target person or character, a means for a generative AI model to generate realistic conversations using the conversation data, a means for distributing rewards for the generated conversations based on paid usage, an emotion engine that recognizes the user's emotions, and a means for adjusting the content of the conversations based on feedback from the emotion engine. This enables the generation of realistic conversations based on the user's emotions and fair distribution of rewards.

[0895] "User" refers to an individual or organization that uses the system to provide conversation data or enjoy the generated conversations.

[0896] "Target person or character" refers to a real person or fictional character that the user selects as the target for generating conversation data.

[0897] "Conversation data" refers to information such as voice, text, and images based on a target person or character provided by a user.

[0898] A "generative AI model" refers to an artificial intelligence algorithm or system that generates realistic conversations using provided conversation data.

[0899] "Charged usage" refers to the process whereby a user pays the system to use the generated conversations.

[0900] "Reward distribution" refers to the process of distributing appropriate rewards to providers of conversation data based on billed usage.

[0901] An "emotion engine" refers to a system or algorithm that analyzes a user's facial expressions, voice, or text to recognize their emotions at that moment and feeds the analysis results back into a generative AI model.

[0902] "Feedback" refers to the information the emotion engine provides to the generative AI model about the user's emotional state, allowing it to adjust the conversation content accordingly.

[0903] "Conversational content adjustment" refers to the process of modifying or optimizing the generated conversation to respond to the user's emotions based on feedback from the emotion engine.

[0904] The system allows users to provide conversation data based on target people or characters, generates realistic conversations using a generative AI model, and distributes rewards based on paid usage. The system includes four main components: a user terminal, a server, a database, and an emotion engine.

[0905] System configuration

[0906] 1. User Device:

[0907] A user terminal (e.g., a PC, smartphone, tablet, etc.) is a device that provides an interface for users to interact with the system. The user terminal is used to provide conversation data, send conversation generation requests, provide emotion data, and perform billing procedures.

[0908] 2. Server:

[0909] The server is the core of the system, containing the generative AI model and the emotion engine. The server performs the following main functions:

[0910] Saving conversation data

[0911] Conversation Generation

[0912] emotion recognition

[0913] Billing process

[0914] reward distribution

[0915] 3. Database:

[0916] The database stores conversation data, generated conversations, user information, billing information, emotion data, etc. This allows the necessary data to be efficiently retrieved and processed.

[0917] 4. Emotion Engine:

[0918] An emotion engine is software or algorithm that analyzes a user's facial expressions, voice, or text to recognize emotions. The emotion engine provides the necessary feedback to the generative AI model to adjust the conversation content according to the user's emotions.

[0919] Program processing

[0920] User Registration

[0921] The user's device connects to the server and displays a registration form. The user enters the required information (e.g., name, email address, password) and sends it to the server. The server receives the entered information, checks its integrity, and stores it in the database. If successful, the server sends a message to the user's device indicating that registration is complete.

[0922] Providing conversation data

[0923] The user device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button. The server receives the data, checks its integrity, and saves it in the database. If successful, the server sends a message to the user device indicating that the upload is complete.

[0924] Conversation generation

[0925] The user's device connects to the server and displays the conversation generation screen. The user selects the target person or character and sends a conversation generation request. The server's generation AI model analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is sent to the user's device.

[0926] example:

[0927] "Generate a conversation with actor A."

[0928] Emotion recognition and conversation regulation

[0929] The user device sends the user's facial expression, voice, or text data to the server. The server's emotion engine analyzes this data and recognizes the user's emotions. The server's generative AI model receives feedback from the emotion engine and adjusts the conversation content according to the user's emotions, and the adjusted conversation data is sent to the user device.

[0930] example:

[0931] "Generate a conversation to cheer up Taro Yamada who is feeling down."

[0932] Conversation Usage and Rewards

[0933] The user's terminal connects to the server and displays the billing screen. The user enters billing information and presses the billing button. The server processes the payment based on the billing information, and if successful, unlocks the conversation data. Based on the billing amount, the server distributes rewards to the data provider and notifies the user and data provider of the processing results.

[0934] example:

[0935] "Please handle the billing process and distribute the rewards for using the generated conversations."

[0936] This system allows users to enjoy realistic, emotionally relevant conversations, while also providing fair rewards to data providers.

[0937] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0938] Step 1:

[0939] User Registration

[0940] Input: The user enters personal information such as a name, email address, or password.

[0941] Specific operation: The user's device connects to the server and displays the registration form. The user presses the submit button to send the information entered to the server.

[0942] Data processing and calculation: The server checks the integrity of the information received and stores it in the database.

[0943] Output: The server sends a registration completion message to the user terminal.

[0944] Step 2:

[0945] Providing conversation data

[0946] Input: User-provided conversation data (e.g., audio files).

[0947] Specific operation: The user's device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button.

[0948] Data processing and calculation: The server checks the integrity of the data received and stores it in the database.

[0949] Output: The server sends a message to the user device indicating that the upload is complete.

[0950] Step 3:

[0951] Conversation generation request

[0952] Input: A person or character selected by the user.

[0953] Specific operation: The user's device connects to the server and displays the conversation creation screen. The user selects the target person or character and sends a conversation creation request.

[0954] Data processing and calculation: The server's generated AI model receives the request, retrieves the target conversation data from the database, and generates a realistic conversation.

[0955] Output: Send the generated conversation to the user's device.

[0956] Step 4:

[0957] emotion recognition

[0958] Input: User facial, voice, or text data.

[0959] Specific operation: The user's device captures the user's facial expression and voice data and sends it to the server.

[0960] Data processing and calculation: The server's emotion engine analyzes this data and recognizes the user's emotions.

[0961] Output: Recognized emotion data is sent as feedback to the generative AI model.

[0962] Step 5:

[0963] Adjusting the conversation

[0964] Input: Recognized emotion data and generated speech data.

[0965] How it works: The server's generative AI model receives feedback from the emotion engine and adjusts the conversation content.

[0966] Data processing and calculation: Conversation content is optimized according to the user's emotions.

[0967] Output: The adjusted conversation data is sent to the user terminal.

[0968] Step 6:

[0969] Pay-per-use conversations

[0970] Input: Billing information entered by the user.

[0971] Specific operation: The user's device connects to the server and displays the billing screen. The user enters billing information and presses the billing button.

[0972] Data processing and calculation: The server processes the billing information, and if payment is successful, unlocks the conversation data. Based on the billing amount, rewards are distributed to the data provider.

[0973] Output: Notify the user and data provider of the processing results.

[0974] (Application example 2)

[0975] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0976] Conventional virtual salesperson systems have the problem that interactions with users are fixed and one-way, and are unable to reflect the user's emotions and situation in real time. Furthermore, the quality of the generated conversations is poor, making it difficult to improve the user experience. Furthermore, there is no established reward distribution system to effectively utilize the content of these conversations, making it difficult to maintain the motivation of data providers. Technology that solves these issues is needed.

[0977] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0978] In this invention, the server includes: a means for a user to provide conversation data based on a target person or character; a means for a generation AI to generate realistic conversations using the conversation data; a means for an emotion engine to analyze the user's facial expressions, voice, or text to recognize emotions and adjust the generated conversation; and a means for distributing rewards for the generated conversation based on paid usage. This enables adaptive conversations that reflect the user's emotions in real time, allowing for high-quality dialogue by virtual salespeople. Furthermore, the reward distribution system can improve incentives for data providers.

[0979] A "user terminal" is a device that provides an interface for a user to provide conversation data, request conversation generation, provide emotion data, and perform billing procedures.

[0980] The "server" is a system that includes the generative AI and emotion engine and performs key processes such as storing conversation data, generating conversations, recognizing emotions, processing billing, and distributing rewards.

[0981] A "database" is a storage device that stores conversation data, generated conversations, user information, billing information, emotional data, and the like.

[0982] An "emotion engine" is an analysis system that analyzes a user's facial expressions, voice, or text to recognize emotions and provide feedback to the generative AI.

[0983] "Generative AI" is an artificial intelligence model that generates realistic dialogue based on conversation data.

[0984] "Conversation data based on a target person or character" is data of conversation related to a specific person or character provided by the user.

[0985] A "conversation generation request" is a request for the user to generate a conversation with a target person or character.

[0986] "Paid usage" refers to usage based on the amount paid by a user to use a particular service or content.

[0987] "Reward sharing" is a mechanism for distributing compensation to data providers and other participants based on paid usage.

[0988] The present invention provides a system for generating real-time conversations between a user and a salesperson in a virtual store according to their emotions, and for distributing rewards according to the amount of paid use. Detailed embodiments of the present invention will be described below.

[0989] Hardware and software used

[0990] Smart devices: smartphones, smart glasses, head-mounted displays

[0991] Server: Generative AI (GPT-4), emotion engine (Microsoft Azure Emotion API), database

[0992] User terminal: A device that provides an interface for users to provide conversation data, request conversation generation, provide emotion data, and perform billing procedures.

[0993] Program processing explanation

[0994] User Registration / Login

[0995] The server receives the required information (name, email address, password) and stores the submitted information in a database. The user then registers by entering the information into the registration form on their smart device.

[0996] Providing conversation data

[0997] Users upload conversation data and product information using their own devices. The server receives the data, checks its integrity, and then stores it in the database. This process allows the system to obtain a wealth of data.

[0998] Conversation generation

[0999] When a user asks a question about a product or requests purchasing advice, the server's AI analyzes the request and retrieves relevant conversation data from the database. Based on this data, the AI ​​generates a realistic conversation and provides it to the user.

[1000] Emotion Recognition and Speech Modulation

[1001] When a user converses through a smart device, the device transmits the user's facial expressions and voice data to the emotion engine for analysis. The emotion engine then feeds back the analysis results to the generation AI, which then adjusts the content of the conversation according to the user's emotions. This allows for adaptive conversations to be provided according to the user's emotions.

[1002] Pay-per-use conversations

[1003] When a user uses the premium conversation service, the billing procedure is carried out within the system. The server processes the billing information and, if successful, provides the premium features to the user. It also distributes rewards to data providers based on the billed usage.

[1004] Specific examples

[1005] For example, if a user asks a question about a product in a virtual store and the user has a sad expression on their face, the emotion engine will recognize that emotion and the generation AI will generate a conversation that encourages the user.

[1006] Prompt Sentence Examples

[1007] An example prompt for the user to ask a question might look like this:

[1008] User expression: Sad

[1009] User Question: I've been through a lot lately and need some emotional support. Can you offer some words of encouragement?

[1010] The generative AI analyzes this and generates the following dialogue:

[1011] "I know you're going through a difficult time. I'm sorry to trouble you, but please remember that I'm always by your side. It's important to stop sometimes. Even if it's just a little at a time, keep moving forward."

[1012] This embodiment provides high-quality dialogue that responds to the user's emotions, significantly improving the user experience in the virtual store. Furthermore, by distributing rewards to data providers, the system can be expected to continue to develop.

[1013] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1014] Step 1:

[1015] User Registration / Login

[1016] The user enters the required information (name, email address, password) into the registration form on the smart device and submits it.

[1017] The server receives this input data, checks the data integrity, and if there are no problems, stores it in the database.

[1018] The inputs to this process are the name, email address, and password entered by the user, and the output is the user information stored in a database.

[1019] Step 2:

[1020] Providing conversation data

[1021] Users upload conversation data and product information from their own terminals.

[1022] The server receives this data, checks the data format and content for consistency, and then stores it in the database.

[1023] The input for this process is conversation data and product information uploaded by the user, and the output is conversation data stored in the database.

[1024] Step 3:

[1025] Conversation generation request

[1026] The user selects the person or character with whom they want to create a conversation from their smart device and sends a conversation creation request.

[1027] The server's generation AI receives this request and retrieves conversation data related to the target from the database.

[1028] The generative AI generates realistic dialogue based on the acquired data and sends it to the user's device.

[1029] The inputs to this process are a user's conversation generation request and conversation data related to the target, and the output is the generated realistic conversation.

[1030] Step 4:

[1031] Emotion Recognition and Speech Modulation

[1032] While a user is having a conversation through a smart device, the device captures the user's facial expressions and voice data and transmits them to a server.

[1033] The server's emotion engine analyzes this data and recognizes the user's emotions.

[1034] Based on feedback from the emotion engine, the generative AI adjusts the content of the conversation and generates an adaptive conversation, which is then sent back to the user's device.

[1035] The inputs for this process are the user's facial expressions, voice data, and emotion recognition feedback, and the output is speech that has been adjusted according to the user's emotions.

[1036] Step 5:

[1037] Pay-per-use conversations

[1038] When using the premium conversation service, users complete the billing procedure on their smart device.

[1039] The server processes the billing information and verifies the integrity of the payment. If successful, the premium conversation service is unlocked.

[1040] The server performs processing to distribute rewards to data providers based on the amount charged.

[1041] The input of this process is the user's billing information and the billing amount, and the output is a notification of the release of the premium function and the distribution of rewards.

[1042] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1043] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1044] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1045] [Third embodiment]

[1046] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1047] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1048] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1049] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1050] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1051] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1052] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1053] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1054] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1055] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1056] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1057] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1058] The present invention relates to a system in which users provide conversation data based on target people or characters, a generation AI uses the conversation data to generate realistic conversations, and rewards are distributed based on paid usage of the generated conversations. This system allows users to enjoy realistic conversations, and data providers can also earn rewards.

[1059] System configuration

[1060] This system consists of three main components: a user terminal, a server, and a database.

[1061] 1. User terminal: A device that provides an interface for users to provide conversation data, request conversation creation, and perform billing procedures.

[1062] 2. Server: Includes the generation AI and performs key processes such as storing conversation data, generating conversations, processing billing, and distributing rewards.

[1063] 3. Database: Stores conversation data, generated conversations, user information, billing information, etc.

[1064] Program processing explanation

[1065] User Registration / Login

[1066] 1. User Registration

[1067] The user's device connects to the server and displays a registration form. The user enters the required information (name, email address, password) and sends it to the server.

[1068] The server receives the input information, checks its integrity, and saves it in the database. If successful, the server sends a registration completion message to the user's device.

[1069] Providing conversation data

[1070] 2. Uploading conversation data

[1071] The user's device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button.

[1072] The server receives the conversation data, checks its integrity, and then saves it in the database. If successful, the server sends a message to the user device indicating that the upload is complete.

[1073] Conversation generation

[1074] 3. Conversation Generation Request

[1075] The user's device connects to the server and displays the conversation creation screen. The user selects the target person or character and sends a conversation creation request.

[1076] The server's AI analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is then sent to the user's device.

[1077] Conversation Usage and Rewards

[1078] 4. Paid conversation usage

[1079] The user terminal connects to the server and displays the billing screen. The user enters billing information and presses the billing button.

[1080] The server processes the payment based on the billing information, and if successful, unlocks the conversation data. It distributes a reward based on the billing amount to the data provider and notifies the user and the data provider of the processing result.

[1081] Specific examples

[1082] Each step is illustrated concretely through a scenario in which Yamada Taro uses this system.

[1083] 1. User Registration

[1084] Yamada Taro accesses the system using a user terminal and registers by entering "taro@example.com" and a password in the registration form.

[1085] 2. Providing conversation data

[1086] Taro Yamada selects an audio file of an interview with his favorite actor and uploads it to the system.

[1087] 3. Conversation Generation

[1088] Taro Yamada selects his favorite actor on the system and sends a request to generate a conversation with that actor. The server's AI generates realistic conversation based on the relevant voice data and sends it to Taro Yamada's device.

[1089] 4. Conversational Use and Reward

[1090] Taro Yamada pays 500 yen to use the generated conversation, and the server distributes a portion of that amount to the data provider as a reward.

[1091] In this way, the system of the present invention provides users with an opportunity to enjoy real conversations without feeling lonely, and at the same time, by distributing rewards to data providers, it increases overall satisfaction.

[1092] The processing flow will be explained below.

[1093] User Registration / Login

[1094] 1. User Registration

[1095] Step 1:

[1096] The user terminal connects to the server and displays the registration form.

[1097] Step 2:

[1098] The user enters the required information (name, email address, password) and clicks the send button.

[1099] Step 3:

[1100] The user terminal transmits the input information to the server.

[1101] Step 4:

[1102] The server verifies the integrity of the information entered, checking for missing or incorrect information.

[1103] Step 5:

[1104] After the server verifies it, it stores the information in a database.

[1105] Step 6:

[1106] The server generates a registration completion message and sends it to the user terminal.

[1107] Step 7:

[1108] A message indicating registration completion is displayed on the user's device.

[1109] Providing conversation data

[1110] 2. Uploading conversation data

[1111] Step 1:

[1112] The user terminal connects to the server and displays the data upload screen.

[1113] Step 2:

[1114] The user selects the conversation data file and information about the target person and presses the upload button.

[1115] Step 3:

[1116] The user terminal transmits the selected files and information to the server.

[1117] Step 4:

[1118] The server checks the integrity of the files and information received.

[1119] Step 5:

[1120] The server stores the files and information in a database.

[1121] Step 6:

[1122] The server generates an upload completion message and sends it to the user terminal.

[1123] Step 7:

[1124] A message indicating that the upload is complete will be displayed on the user's device.

[1125] Conversation generation

[1126] 3. Conversation Generation Request

[1127] Step 1:

[1128] The user terminal connects to the server and displays a conversation creation screen.

[1129] Step 2:

[1130] The user selects a target person or character and sends a request to generate a conversation.

[1131] Step 3:

[1132] The user terminal transmits request information to the server.

[1133] Step 4:

[1134] The server's generation AI analyzes the request and retrieves the conversation data of the target person or character from the database.

[1135] Step 5:

[1136] The server's generation AI generates realistic conversations based on the data it acquires.

[1137] Step 6:

[1138] The server transmits the generated conversation data to the user terminal.

[1139] Step 7:

[1140] The generated conversation is displayed on the user terminal.

[1141] Conversation Usage and Rewards

[1142] 4. Paid conversation usage

[1143] Step 1:

[1144] The user terminal connects to the server and displays the billing screen.

[1145] Step 2:

[1146] The user inputs the billing information and presses the billing button.

[1147] Step 3:

[1148] The user terminal transmits the billing information to the server.

[1149] Step 4:

[1150] The server processes the payment based on the billing information received. If an external payment service is used, the server sends a request to that service.

[1151] Step 5:

[1152] The server receives confirmation of successful payment.

[1153] Step 6:

[1154] The server unlocks the conversation data and makes it available.

[1155] Step 7:

[1156] The server calculates the reward based on the usage charge and distributes it to the data provider.

[1157] Step 8:

[1158] The server notifies the user and data provider of the results of the billing and reward processing.

[1159] Step 9:

[1160] The processing results are displayed on the user terminal and the provider terminal.

[1161] Example 1

[1162] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1163] In today's world, users often feel lonely, and especially when they spend a lot of time alone, they want to enjoy real conversations with others. However, in reality, there are limited ways to easily generate and use conversations with target people or characters. Another issue is the lack of a system that provides appropriate rewards to providers of conversation data.

[1164] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1165] In this invention, the server includes: a means for a user to provide conversation data based on a target person or character; a means for a generation AI to generate realistic conversations using the conversation data; a means for distributing rewards for the generated conversations according to paid usage; a means for a user terminal to connect to the server, transmit user registration and login information, and perform authentication; a means for a user to upload conversation data and for the server to store it in a database; a means for inputting billing information and processing payments; and a means for unlocking the conversation data and automatically distributing rewards. This allows users to easily enjoy realistic conversations, and data providers to receive fair rewards.

[1166] A "user terminal" is a device that allows a user to access the system, provide conversation data, request conversation creation, and perform billing procedures.

[1167] A "server" is a computer system that includes the generation AI and performs major processes such as storing conversation data, generating conversations, processing billing, and distributing rewards.

[1168] A "database" is an information management system for storing conversation data, generated conversations, user information, billing information, and the like.

[1169] "Generative AI" refers to algorithms and software that use provided conversational data to generate realistic conversations.

[1170] "Conversation data" refers to data such as audio files and text data related to a target person or character provided by the user.

[1171] "Billing information" refers to credit card information or other payment method information required for users to pay for conversation creation and usage.

[1172] "Reward" refers to money or something of equivalent value distributed to a user who provides conversation data based on the fee-based usage.

[1173] "Realistic conversation" refers to dialogue data that resembles natural conversations between humans, generated by the generative AI based on the conversation data provided.

[1174] The present invention is a system in which a user provides conversation data based on a target person or character, a generation AI uses the conversation data to generate realistic conversations, and rewards are distributed based on paid usage of the generated conversations. This system allows users to enjoy realistic conversations, and data providers can also receive rewards.

[1175] System configuration

[1176] This system consists of three main components: a user terminal, a server, and a database.

[1177] User terminal: A device that provides an interface for users to provide conversation data, request conversation creation, and perform billing procedures.

[1178] Server: Contains the generation AI and performs key processes such as storing conversation data, generating conversations, processing billing, and distributing rewards.

[1179] Database: Stores conversation data, generated conversations, user information, billing information, etc.

[1180] Program processing explanation

[1181] User Registration / Login

[1182] 1. User Registration

[1183] The user's device connects to the server and displays a registration form. The user enters the required information, such as name, email address, and password, and submits it.

[1184] The server receives the input information, checks its integrity, and saves it in the database. If the save is successful, it sends a message to the user's device indicating that registration is complete.

[1185] Examples:

[1186] User A accesses the system using a user terminal and registers by entering "userA@example.com" and a password in the registration form.

[1187] Providing conversation data

[1188] 2. Uploading conversation data

[1189] The user's device connects to the server and displays the data upload screen. The user selects the conversation data they wish to provide and uploads it.

[1190] The server receives the conversation data, checks its integrity, and then saves it in the database. If the save is successful, it sends a message to the user device indicating that the upload is complete.

[1191] Examples:

[1192] User A selects an interview audio file of his / her favorite actor and uploads it to the system.

[1193] Conversation generation

[1194] 3. Conversation Generation Request

[1195] The user's device connects to the server and displays the conversation creation screen. The user selects a target person or character and sends a conversation creation request to the server.

[1196] The server's AI analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is then sent to the user's device.

[1197] Examples:

[1198] User A selects a favorite actor on the system and sends a request to the server to generate a conversation with that actor. The server's AI generates a realistic conversation based on the related voice data and sends it to User A's device.

[1199] Example prompt sentence:

[1200] "Create a conversation with your favorite actor, Hiroshi Tanaka."

[1201] Conversation Usage and Rewards

[1202] 4. Paid conversation usage

[1203] The user's device connects to the server and displays the billing screen. The user enters billing information (e.g., credit card information) and the billing process is carried out.

[1204] The server processes the payment based on the billing information, and if successful, unlocks the conversation data. It distributes a reward based on the billing amount to the data provider and notifies the user and the data provider of the processing result.

[1205] Examples:

[1206] User A pays 500 yen to use the generated conversation. The server distributes a portion of that amount to the data provider as a reward.

[1207] summary

[1208] This system allows users to enjoy real conversations without feeling lonely, while also allowing data providers to receive rewards.

[1209] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1210] Step 1: User Registration

[1211] Input: The user enters information such as name, email address, and password into a registration form on the device.

[1212] Specific operation: The user terminal receives input information and sends the information to the server.

[1213] Data processing: The server checks the integrity of the information received (for example, whether the email address is in the correct format or whether the password meets a certain strength).

[1214] Output: The server stores the information whose integrity has been confirmed in the database and sends a message to the user terminal indicating that registration is complete.

[1215] Step 2: Upload your conversation data

[1216] Input: The user selects the conversation data (for example, an interview audio file) that he or she wants to provide on the terminal.

[1217] Specific operation: The user terminal transmits the selected conversation data to the server.

[1218] Data processing: The server checks the format and content of the conversation data received (for example, whether the file format is appropriate and whether there are any problems with the content).

[1219] Output: After verifying the integrity, the server saves the conversation data in the database and sends a message to the user device indicating that the upload is complete.

[1220] Step 3: Conversation creation request

[1221] Input: The user selects the target person or character on the conversation generation screen from their device and sends a request.

[1222] Specific operation: The user terminal sends a conversation creation request to the server.

[1223] Data processing: The server's generation AI analyzes the request and retrieves the target conversation data from the database.

[1224] Output: The server's generation AI generates realistic conversations based on the acquired data, and sends the generated conversation data to the user's device.

[1225] Step 4: Pay for conversations

[1226] Input: The user inputs billing information (such as credit card information) to use the conversation generated on the terminal.

[1227] Specific operation: The user terminal transmits billing information to the server, and the server processes the payment.

[1228] Data processing: Once the server confirms the successful payment, it unlocks the generated conversation data and calculates the reward based on the payment amount.

[1229] Output: The server sends a notification of payment completion and the released conversation data to the user terminal, and automatically distributes the reward to the data provider.

[1230] In this way, the user terminal, server, and database work together to execute a series of processes that allow the user to enjoy a realistic conversation.

[1231] (Application example 1)

[1232] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1233] In conventional virtual stores, it is difficult for users to obtain detailed product information and they lack a real-time interactive shopping experience. As a result, users often feel dissatisfied with virtual shopping and lose their motivation to purchase. Furthermore, in systems that utilize conversation generation technology, rewards to conversation data providers are insufficient, which reduces the motivation of data providers. The challenge is to solve these problems and improve the satisfaction of both users and data providers.

[1234] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1235] In this invention, the server includes a means for a user to provide conversation data based on a target person or character, a means for a generation AI to generate realistic conversations using the conversation data, a means for distributing rewards for the generated conversations according to paid use, a means for a user to visually check products in a virtual store and provide product explanations and advice using the generated conversations, and a means for answering questions about the products in real time. This allows users to have a real-time interactive shopping experience in the virtual store, and at the same time, appropriate rewards are distributed to conversation data providers, thereby improving overall satisfaction.

[1236] "User" refers to any individual or entity that uses the system, provides conversation data, and uses the generated conversations.

[1237] "Target person or character" refers to a specific person or fictional character that is the basis for the conversation data provided by the user.

[1238] "Conversation Data" refers to user-provided voice and text data based on a target person or character.

[1239] "Generative AI" refers to artificial intelligence that uses provided conversation data to generate realistic conversations.

[1240] "Realistic conversation" refers to a conversation generated by generative AI that resembles a natural conversation between humans.

[1241] "Charged usage" refers to the fee paid by a user to use the generated conversation.

[1242] "Reward distribution" refers to the process by which rewards are distributed to data providers based on billed usage.

[1243] "Virtual store" refers to a virtual shopping environment that users can access via the Internet.

[1244] "Product description" refers to providing information about a product to a user using generative AI.

[1245] "Advice" refers to advice provided by the generating AI to users regarding product selection and usage.

[1246] "Real-time" refers to data processing and information provision occurring immediately, without delay.

[1247] "Answering questions" refers to providing immediate and appropriate answers to questions posed by users to the generated AI.

[1248] The present invention is a system that allows users to have a real-time interactive shopping experience in a virtual store. The system consists of three main components: a user terminal, a server, and a database.

[1249] First, let us explain the user terminal. A user terminal is a device such as smart glasses or a head-mounted display. Through this terminal, the user provides conversation data, requests conversation generation, sends product-related questions, and performs billing procedures. Specifically, the user can visually identify a specific product in a virtual store and request a realistic conversation about that product from the generation AI.

[1250] Next, we will explain the server. The server contains a generative AI and performs key processes such as saving conversation data, generating conversations, processing billing, and distributing rewards. When the server receives a conversation generation request from a user, it retrieves the target conversation data from a database and generates a realistic conversation using a generative AI model (e.g., OpenAI's GPT-3). The generated conversation is sent to the user's device, allowing the user to experience an interactive conversation with a virtual store clerk.

[1251] Finally, we will explain the database. The database is a system that stores user information, conversation data, generated conversations, billing information, etc. The database checks the consistency of the conversation data and provides the data necessary for the server to generate conversations using the generation AI.

[1252] As a concrete example, consider a scenario in which a user is selecting cosmetics in a virtual store. The user puts on smart glasses and enters the virtual store. They find the cosmetics on the shelves and send a question about the specific cosmetics to the system. By entering a prompt such as "Please tell me the features of this cosmetics," the generative AI will provide a detailed product description in real time based on the relevant conversation data.

[1253] This allows users to have an interactive shopping experience that feels just like they are talking to a salesperson in a real store. Meanwhile, since rewards are appropriately distributed to data providers, this also contributes to maintaining their motivation. In this way, the present invention is a system that can improve the satisfaction of both users and data providers.

[1254] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1255] Step 1: User Registration

[1256] The user uses a terminal to enter their name, email address, and password into the registration form and send it to the server. The server receives the input information, checks its integrity, and then stores it in a database. If successful, the server sends a message to the user terminal confirming registration. The input is user information (name, email address, password), and the output is a message confirming registration completion.

[1257] Step 2: Upload your conversation data

[1258] The user uses their device to select conversation data and send it to the server from the upload screen. The server receives the conversation data, checks its integrity, and then stores it in a database. If successful, the server sends a message to the user device indicating that the upload is complete. The input is the conversation data, and the output is a message confirming that the upload is complete.

[1259] Step 3: Conversation creation request

[1260] The user uses their device to select a target person or character and send a conversation generation request to the server. The server receives the request and retrieves the target conversation data from a database. A generative AI model (e.g., OpenAI's GPT-3) is used to generate a realistic conversation based on the prompt. The generated conversation is sent to the user's device. The input is the conversation generation request (target person or character, prompt), and the output is the generated conversation.

[1261] Step 4: Product selection and questions

[1262] Users use their devices to visually check products in a virtual store and send questions about the products to the server. The server receives the questions, generates answers in real time using generative AI, and sends them to the user's device. The input is a question about the product (prompt sentence), and the output is the answer (generated conversation).

[1263] Step 5: Billing process and reward distribution

[1264] The user uses the terminal to carry out the billing procedure to use the generated conversation. The server receives the billing information and processes the payment. If successful, it unlocks the conversation data and distributes rewards to the data provider according to the billing amount. The processing results are notified to the user and the data provider. The input is the billing information, and the output is the billing result and a reward distribution notification.

[1265] Step 6: Deliver a conversational experience

[1266] The user uses the device to experience virtual shopping through generated conversations. Based on the generated conversations, the user can receive product explanations and advice in real time. The input is the generated conversation, and the output is the user's purchasing experience.

[1267] Through these steps, users can have an interactive shopping experience in a virtual store, and at the same time, rewards are appropriately distributed to data providers.

[1268] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1269] The present invention combines a system in which a user provides conversation data based on a target person or character, a generation AI uses the conversation data to generate realistic conversations, and distributes rewards based on paid usage of the generated conversations with an emotion engine that recognizes the user's emotions. This system allows users to enjoy conversations that are realistic and reflect their emotions.

[1270] System configuration

[1271] This system consists of four main components: a user terminal, a server, a database, and an emotion engine.

[1272] 1. User terminal: A device that provides an interface for users to provide conversation data, request conversation generation, provide emotion data, and perform billing procedures.

[1273] 2. Server: Contains the generative AI and emotion engine, and performs key processes such as storing conversation data, generating conversations, recognizing emotions, processing billing, and distributing rewards.

[1274] 3. Database: Stores conversation data, generated conversations, user information, billing information, emotional data, etc.

[1275] 4. Emotion Engine: Analyzes the user's facial expressions, voice, or text to recognize emotions and provide feedback to the generative AI.

[1276] Program processing explanation

[1277] User Registration / Login

[1278] 1. User Registration

[1279] The user's device connects to the server and displays a registration form. The user enters the required information (name, email address, password) and sends it to the server.

[1280] The server receives the input information, checks its integrity, and saves it in the database. If successful, the server sends a registration completion message to the user's device.

[1281] Providing conversation data

[1282] 2. Uploading conversation data

[1283] The user's device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button.

[1284] The server receives the conversation data, checks its integrity, and then saves it in the database. If successful, the server sends a message to the user device indicating that the upload is complete.

[1285] Conversation generation

[1286] 3. Conversation Generation Request

[1287] The user's device connects to the server and displays the conversation creation screen. The user selects the target person or character and sends a conversation creation request.

[1288] The server's AI analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is then sent to the user's device.

[1289] Emotion recognition and conversation regulation

[1290] 4. Emotion recognition

[1291] The user terminal transmits the user's facial expression, voice, or text data to the server.

[1292] The server's emotion engine analyzes this data and recognizes the user's emotions.

[1293] 5. Moderate the conversation

[1294] The server's generation AI receives feedback from the emotion engine and adjusts the conversation content according to the user's emotions.

[1295] The adjusted conversation data is transmitted to the user terminal and displayed on the user terminal.

[1296] Conversation Usage and Rewards

[1297] 6. Paid Conversation Usage

[1298] The user terminal connects to the server and displays the billing screen. The user enters billing information and presses the billing button.

[1299] The server processes the payment based on the billing information, and if successful, unlocks the conversation data. It distributes a reward based on the billing amount to the data provider and notifies the user and the data provider of the processing result.

[1300] Specific examples

[1301] Each step is illustrated concretely through a scenario in which Yamada Taro uses this system.

[1302] 1. User Registration

[1303] Yamada Taro accesses the system using a user terminal and registers by entering "taro@example.com" and a password in the registration form.

[1304] 2. Providing conversation data

[1305] Taro Yamada selects an audio file of an interview with his favorite actor and uploads it to the system.

[1306] 3. Conversation Generation

[1307] Taro Yamada selects his favorite actor on the system and sends a request to generate a conversation with that actor. The server's AI generates realistic conversation based on the relevant voice data and sends it to Taro Yamada's device.

[1308] 4. Emotional awareness and conversation regulation

[1309] While Yamada Taro is using the system to converse, the user's device sends his facial expressions and voice data to the server. The server's emotion engine recognizes Yamada Taro's emotions, and the generation AI adjusts the content of the conversation based on that information. For example, if Yamada Taro is feeling down, the system generates a conversation that includes an encouraging message.

[1310] 5. Conversation Use and Reward

[1311] Taro Yamada pays 500 yen to use the generated conversation, and the server distributes a portion of that amount to the data provider as a reward.

[1312] In this way, the system of the present invention provides users with an opportunity to enjoy real, emotionally relevant conversations without feeling lonely, while at the same time increasing overall satisfaction by distributing rewards to data providers.

[1313] The processing flow will be explained below.

[1314] User Registration / Login

[1315] 1. User Registration

[1316] Step 1:

[1317] The user terminal connects to the server and displays the registration form.

[1318] Step 2:

[1319] The user enters the required information (name, email address, password) and clicks the send button.

[1320] Step 3:

[1321] The user terminal transmits the input information to the server.

[1322] Step 4:

[1323] The server verifies the integrity of the information entered, checking for missing or incorrect information.

[1324] Step 5:

[1325] After the server verifies it, it stores the information in a database.

[1326] Step 6:

[1327] The server generates a registration completion message and sends it to the user terminal.

[1328] Step 7:

[1329] A message indicating registration completion is displayed on the user's device.

[1330] Providing conversation data

[1331] 2. Uploading conversation data

[1332] Step 1:

[1333] The user terminal connects to the server and displays the data upload screen.

[1334] Step 2:

[1335] The user selects the conversation data file and information about the target person and presses the upload button.

[1336] Step 3:

[1337] The user terminal transmits the selected files and information to the server.

[1338] Step 4:

[1339] The server checks the integrity of the files and information received.

[1340] Step 5:

[1341] The server stores the files and information in a database.

[1342] Step 6:

[1343] The server generates an upload completion message and sends it to the user terminal.

[1344] Step 7:

[1345] A message indicating that the upload is complete will be displayed on the user's device.

[1346] Conversation generation

[1347] 3. Conversation Generation Request

[1348] Step 1:

[1349] The user terminal connects to the server and displays a conversation creation screen.

[1350] Step 2:

[1351] The user selects a target person or character and sends a request to generate a conversation.

[1352] Step 3:

[1353] The user terminal transmits request information to the server.

[1354] Step 4:

[1355] The server's generation AI analyzes the request and retrieves the conversation data of the target person or character from the database.

[1356] Step 5:

[1357] The server's generation AI generates realistic conversations based on the data it acquires.

[1358] Step 6:

[1359] The server transmits the generated conversation data to the user terminal.

[1360] Step 7:

[1361] The generated conversation is displayed on the user terminal.

[1362] Emotion recognition and conversation regulation

[1363] 4. Emotion recognition

[1364] Step 1:

[1365] The user terminal transmits the user's facial expression, voice, or text data to the server.

[1366] Step 2:

[1367] The server's emotion engine analyzes this data and recognizes the user's emotions.

[1368] Step 3:

[1369] The server's emotion engine provides the recognized emotion data to the generation AI.

[1370] 5. Moderate the conversation

[1371] Step 1:

[1372] The server's generation AI receives feedback from the emotion engine.

[1373] Step 2:

[1374] The server's generation AI adjusts the conversation content according to the user's emotions.

[1375] Step 3:

[1376] The server transmits the adjusted conversation data to the user terminal.

[1377] Step 4:

[1378] The adjusted conversation is displayed on the user terminal.

[1379] Conversation Usage and Rewards

[1380] 6. Paid Conversation Usage

[1381] Step 1:

[1382] The user terminal connects to the server and displays the billing screen.

[1383] Step 2:

[1384] The user inputs the billing information and presses the billing button.

[1385] Step 3:

[1386] The user terminal transmits the billing information to the server.

[1387] Step 4:

[1388] The server processes the payment based on the billing information received. If an external payment service is used, the server sends a request to that service.

[1389] Step 5:

[1390] The server receives confirmation of successful payment.

[1391] Step 6:

[1392] The server unlocks the conversation data and makes it available.

[1393] Step 7:

[1394] The server calculates the reward based on the usage charge and distributes it to the data provider.

[1395] Step 8:

[1396] The server notifies the user and data provider of the results of the billing and reward processing.

[1397] Step 9:

[1398] The processing results are displayed on the user terminal and the provider terminal.

[1399] Example 2

[1400] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1401] Conventional conversation generation systems have difficulty generating and adjusting conversations in real time based on the user's emotions and situation, resulting in a poor user experience. Furthermore, there are insufficient means for distributing appropriate rewards to conversation data providers, making fair reward distribution difficult. To solve these problems, the present invention aims to provide a system that enables conversation generation based on the user's emotions and fair reward distribution.

[1402] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1403] In this invention, the server includes a means for a user to provide conversation data based on a target person or character, a means for a generative AI model to generate realistic conversations using the conversation data, a means for distributing rewards for the generated conversations based on paid usage, an emotion engine that recognizes the user's emotions, and a means for adjusting the content of the conversations based on feedback from the emotion engine. This enables the generation of realistic conversations based on the user's emotions and fair distribution of rewards.

[1404] "User" refers to an individual or organization that uses the system to provide conversation data or enjoy the generated conversations.

[1405] "Target person or character" refers to a real person or fictional character that the user selects as the target for generating conversation data.

[1406] "Conversation data" refers to information such as voice, text, and images based on a target person or character provided by a user.

[1407] A "generative AI model" refers to an artificial intelligence algorithm or system that generates realistic conversations using provided conversation data.

[1408] "Charged usage" refers to the process whereby a user pays the system to use the generated conversations.

[1409] "Reward distribution" refers to the process of distributing appropriate rewards to providers of conversation data based on billed usage.

[1410] An "emotion engine" refers to a system or algorithm that analyzes a user's facial expressions, voice, or text to recognize their emotions at that moment and feeds the analysis results back into a generative AI model.

[1411] "Feedback" refers to the information the emotion engine provides to the generative AI model about the user's emotional state, allowing it to adjust the conversation content accordingly.

[1412] "Conversational content adjustment" refers to the process of modifying or optimizing the generated conversation to respond to the user's emotions based on feedback from the emotion engine.

[1413] The system allows users to provide conversation data based on target people or characters, generates realistic conversations using a generative AI model, and distributes rewards based on paid usage. The system includes four main components: a user terminal, a server, a database, and an emotion engine.

[1414] System configuration

[1415] 1. User Device:

[1416] A user terminal (e.g., a PC, smartphone, tablet, etc.) is a device that provides an interface for users to interact with the system. The user terminal is used to provide conversation data, send conversation generation requests, provide emotion data, and perform billing procedures.

[1417] 2. Server:

[1418] The server is the core of the system, containing the generative AI model and the emotion engine. The server performs the following main functions:

[1419] Saving conversation data

[1420] Conversation Generation

[1421] emotion recognition

[1422] Billing process

[1423] reward distribution

[1424] 3. Database:

[1425] The database stores conversation data, generated conversations, user information, billing information, emotion data, etc. This allows the necessary data to be efficiently retrieved and processed.

[1426] 4. Emotion Engine:

[1427] An emotion engine is software or algorithm that analyzes a user's facial expressions, voice, or text to recognize emotions. The emotion engine provides the necessary feedback to the generative AI model to adjust the conversation content according to the user's emotions.

[1428] Program processing

[1429] User Registration

[1430] The user's device connects to the server and displays a registration form. The user enters the required information (e.g., name, email address, password) and sends it to the server. The server receives the entered information, checks its integrity, and stores it in the database. If successful, the server sends a message to the user's device indicating that registration is complete.

[1431] Providing conversation data

[1432] The user device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button. The server receives the data, checks its integrity, and saves it in the database. If successful, the server sends a message to the user device indicating that the upload is complete.

[1433] Conversation generation

[1434] The user's device connects to the server and displays the conversation generation screen. The user selects the target person or character and sends a conversation generation request. The server's generation AI model analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is sent to the user's device.

[1435] example:

[1436] "Generate a conversation with actor A."

[1437] Emotion recognition and conversation regulation

[1438] The user device sends the user's facial expression, voice, or text data to the server. The server's emotion engine analyzes this data and recognizes the user's emotions. The server's generative AI model receives feedback from the emotion engine and adjusts the conversation content according to the user's emotions, and the adjusted conversation data is sent to the user device.

[1439] example:

[1440] "Generate a conversation to cheer up Taro Yamada who is feeling down."

[1441] Conversation Usage and Rewards

[1442] The user's terminal connects to the server and displays the billing screen. The user enters billing information and presses the billing button. The server processes the payment based on the billing information, and if successful, unlocks the conversation data. Based on the billing amount, the server distributes rewards to the data provider and notifies the user and data provider of the processing results.

[1443] example:

[1444] "Please handle the billing process and distribute the rewards for using the generated conversations."

[1445] This system allows users to enjoy realistic, emotionally relevant conversations, while also providing fair rewards to data providers.

[1446] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1447] Step 1:

[1448] User Registration

[1449] Input: The user enters personal information such as a name, email address, or password.

[1450] Specific operation: The user's device connects to the server and displays the registration form. The user presses the submit button to send the information entered to the server.

[1451] Data processing and calculation: The server checks the integrity of the information received and stores it in the database.

[1452] Output: The server sends a registration completion message to the user terminal.

[1453] Step 2:

[1454] Providing conversation data

[1455] Input: User-provided conversation data (e.g., audio files).

[1456] Specific operation: The user's device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button.

[1457] Data processing and calculation: The server checks the integrity of the data received and stores it in the database.

[1458] Output: The server sends a message to the user device indicating that the upload is complete.

[1459] Step 3:

[1460] Conversation generation request

[1461] Input: A person or character selected by the user.

[1462] Specific operation: The user's device connects to the server and displays the conversation creation screen. The user selects the target person or character and sends a conversation creation request.

[1463] Data processing and calculation: The server's generated AI model receives the request, retrieves the target conversation data from the database, and generates a realistic conversation.

[1464] Output: Send the generated conversation to the user's device.

[1465] Step 4:

[1466] emotion recognition

[1467] Input: User facial, voice, or text data.

[1468] Specific operation: The user's device captures the user's facial expression and voice data and sends it to the server.

[1469] Data processing and calculation: The server's emotion engine analyzes this data and recognizes the user's emotions.

[1470] Output: Recognized emotion data is sent as feedback to the generative AI model.

[1471] Step 5:

[1472] Adjusting the conversation

[1473] Input: Recognized emotion data and generated speech data.

[1474] How it works: The server's generative AI model receives feedback from the emotion engine and adjusts the conversation content.

[1475] Data processing and calculation: Conversation content is optimized according to the user's emotions.

[1476] Output: The adjusted conversation data is sent to the user terminal.

[1477] Step 6:

[1478] Pay-per-use conversations

[1479] Input: Billing information entered by the user.

[1480] Specific operation: The user's device connects to the server and displays the billing screen. The user enters billing information and presses the billing button.

[1481] Data processing and calculation: The server processes the billing information, and if payment is successful, unlocks the conversation data. Based on the billing amount, rewards are distributed to the data provider.

[1482] Output: Notify the user and data provider of the processing results.

[1483] (Application example 2)

[1484] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1485] Conventional virtual salesperson systems have the problem that interactions with users are fixed and one-way, and are unable to reflect the user's emotions and situation in real time. Furthermore, the quality of the generated conversations is poor, making it difficult to improve the user experience. Furthermore, there is no established reward distribution system to effectively utilize the content of these conversations, making it difficult to maintain the motivation of data providers. Technology that solves these issues is needed.

[1486] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1487] In this invention, the server includes: a means for a user to provide conversation data based on a target person or character; a means for a generation AI to generate realistic conversations using the conversation data; a means for an emotion engine to analyze the user's facial expressions, voice, or text to recognize emotions and adjust the generated conversation; and a means for distributing rewards for the generated conversation based on paid usage. This enables adaptive conversations that reflect the user's emotions in real time, allowing for high-quality dialogue by virtual salespeople. Furthermore, the reward distribution system can improve incentives for data providers.

[1488] A "user terminal" is a device that provides an interface for a user to provide conversation data, request conversation generation, provide emotion data, and perform billing procedures.

[1489] The "server" is a system that includes the generative AI and emotion engine and performs key processes such as storing conversation data, generating conversations, recognizing emotions, processing billing, and distributing rewards.

[1490] A "database" is a storage device that stores conversation data, generated conversations, user information, billing information, emotional data, and the like.

[1491] An "emotion engine" is an analysis system that analyzes a user's facial expressions, voice, or text to recognize emotions and provide feedback to the generative AI.

[1492] "Generative AI" is an artificial intelligence model that generates realistic dialogue based on conversation data.

[1493] "Conversation data based on a target person or character" is data of conversation related to a specific person or character provided by the user.

[1494] A "conversation generation request" is a request for the user to generate a conversation with a target person or character.

[1495] "Paid usage" refers to usage based on the amount paid by a user to use a particular service or content.

[1496] "Reward sharing" is a mechanism for distributing compensation to data providers and other participants based on paid usage.

[1497] The present invention provides a system for generating real-time conversations between a user and a salesperson in a virtual store according to their emotions, and for distributing rewards according to the amount of paid use. Detailed embodiments of the present invention will be described below.

[1498] Hardware and software used

[1499] Smart devices: smartphones, smart glasses, head-mounted displays

[1500] Server: Generative AI (GPT-4), emotion engine (Microsoft Azure Emotion API), database

[1501] User terminal: A device that provides an interface for users to provide conversation data, request conversation generation, provide emotion data, and perform billing procedures.

[1502] Program processing explanation

[1503] User Registration / Login

[1504] The server receives the required information (name, email address, password) and stores the submitted information in a database. The user then registers by entering the information into the registration form on their smart device.

[1505] Providing conversation data

[1506] Users upload conversation data and product information using their own devices. The server receives the data, checks its integrity, and then stores it in the database. This process allows the system to obtain a wealth of data.

[1507] Conversation generation

[1508] When a user asks a question about a product or requests purchasing advice, the server's AI analyzes the request and retrieves relevant conversation data from the database. Based on this data, the AI ​​generates a realistic conversation and provides it to the user.

[1509] Emotion Recognition and Speech Modulation

[1510] When a user converses through a smart device, the device transmits the user's facial expressions and voice data to the emotion engine for analysis. The emotion engine then feeds back the analysis results to the generation AI, which then adjusts the content of the conversation according to the user's emotions. This allows for adaptive conversations to be provided according to the user's emotions.

[1511] Pay-per-use conversations

[1512] When a user uses the premium conversation service, the billing procedure is carried out within the system. The server processes the billing information and, if successful, provides the premium features to the user. It also distributes rewards to data providers based on the billed usage.

[1513] Specific examples

[1514] For example, if a user asks a question about a product in a virtual store and the user has a sad expression on their face, the emotion engine will recognize that emotion and the generation AI will generate a conversation that encourages the user.

[1515] Prompt Sentence Examples

[1516] An example prompt for the user to ask a question might look like this:

[1517] User expression: Sad

[1518] User Question: I've been through a lot lately and need some emotional support. Can you offer some words of encouragement?

[1519] The generative AI analyzes this and generates the following dialogue:

[1520] "I know you're going through a difficult time. I'm sorry to trouble you, but please remember that I'm always by your side. It's important to stop sometimes. Even if it's just a little at a time, keep moving forward."

[1521] This embodiment provides high-quality dialogue that responds to the user's emotions, significantly improving the user experience in the virtual store. Furthermore, by distributing rewards to data providers, the system can be expected to continue to develop.

[1522] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1523] Step 1:

[1524] User Registration / Login

[1525] The user enters the required information (name, email address, password) into the registration form on the smart device and submits it.

[1526] The server receives this input data, checks the data integrity, and if there are no problems, stores it in the database.

[1527] The inputs to this process are the name, email address, and password entered by the user, and the output is the user information stored in a database.

[1528] Step 2:

[1529] Providing conversation data

[1530] Users upload conversation data and product information from their own terminals.

[1531] The server receives this data, checks the data format and content for consistency, and then stores it in the database.

[1532] The input for this process is conversation data and product information uploaded by the user, and the output is conversation data stored in the database.

[1533] Step 3:

[1534] Conversation generation request

[1535] The user selects the person or character with whom they want to create a conversation from their smart device and sends a conversation creation request.

[1536] The server's generation AI receives this request and retrieves conversation data related to the target from the database.

[1537] The generative AI generates realistic dialogue based on the acquired data and sends it to the user's device.

[1538] The inputs to this process are a user's conversation generation request and conversation data related to the target, and the output is the generated realistic conversation.

[1539] Step 4:

[1540] Emotion Recognition and Speech Modulation

[1541] While a user is having a conversation through a smart device, the device captures the user's facial expressions and voice data and transmits them to a server.

[1542] The server's emotion engine analyzes this data and recognizes the user's emotions.

[1543] Based on feedback from the emotion engine, the generative AI adjusts the content of the conversation and generates an adaptive conversation, which is then sent back to the user's device.

[1544] The inputs for this process are the user's facial expressions, voice data, and emotion recognition feedback, and the output is speech that has been adjusted according to the user's emotions.

[1545] Step 5:

[1546] Pay-per-use conversations

[1547] When using the premium conversation service, users complete the billing procedure on their smart device.

[1548] The server processes the billing information and verifies the integrity of the payment. If successful, the premium conversation service is unlocked.

[1549] The server performs processing to distribute rewards to data providers based on the amount charged.

[1550] The input of this process is the user's billing information and the billing amount, and the output is a notification of the release of the premium function and the distribution of rewards.

[1551] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1552] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1553] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1554] [Fourth embodiment]

[1555] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1556] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1557] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1558] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1559] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1560] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1561] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1562] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1563] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1564] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1565] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1566] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1567] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1568] The present invention relates to a system in which users provide conversation data based on target people or characters, a generation AI uses the conversation data to generate realistic conversations, and rewards are distributed based on paid usage of the generated conversations. This system allows users to enjoy realistic conversations, and data providers can also earn rewards.

[1569] System configuration

[1570] This system consists of three main components: a user terminal, a server, and a database.

[1571] 1. User terminal: A device that provides an interface for users to provide conversation data, request conversation creation, and perform billing procedures.

[1572] 2. Server: Includes the generation AI and performs key processes such as storing conversation data, generating conversations, processing billing, and distributing rewards.

[1573] 3. Database: Stores conversation data, generated conversations, user information, billing information, etc.

[1574] Program processing explanation

[1575] User Registration / Login

[1576] 1. User Registration

[1577] The user's device connects to the server and displays a registration form. The user enters the required information (name, email address, password) and sends it to the server.

[1578] The server receives the input information, checks its integrity, and saves it in the database. If successful, the server sends a registration completion message to the user's device.

[1579] Providing conversation data

[1580] 2. Uploading conversation data

[1581] The user's device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button.

[1582] The server receives the conversation data, checks its integrity, and then saves it in the database. If successful, the server sends a message to the user device indicating that the upload is complete.

[1583] Conversation generation

[1584] 3. Conversation Generation Request

[1585] The user's device connects to the server and displays the conversation creation screen. The user selects the target person or character and sends a conversation creation request.

[1586] The server's AI analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is then sent to the user's device.

[1587] Conversation Usage and Rewards

[1588] 4. Paid conversation usage

[1589] The user terminal connects to the server and displays the billing screen. The user enters billing information and presses the billing button.

[1590] The server processes the payment based on the billing information, and if successful, unlocks the conversation data. It distributes a reward based on the billing amount to the data provider and notifies the user and the data provider of the processing result.

[1591] Specific examples

[1592] Each step is illustrated concretely through a scenario in which Yamada Taro uses this system.

[1593] 1. User Registration

[1594] Yamada Taro accesses the system using a user terminal and registers by entering "taro@example.com" and a password in the registration form.

[1595] 2. Providing conversation data

[1596] Taro Yamada selects an audio file of an interview with his favorite actor and uploads it to the system.

[1597] 3. Conversation Generation

[1598] Taro Yamada selects his favorite actor on the system and sends a request to generate a conversation with that actor. The server's AI generates realistic conversation based on the relevant voice data and sends it to Taro Yamada's device.

[1599] 4. Conversational Use and Reward

[1600] Taro Yamada pays 500 yen to use the generated conversation, and the server distributes a portion of that amount to the data provider as a reward.

[1601] In this way, the system of the present invention provides users with an opportunity to enjoy real conversations without feeling lonely, and at the same time, by distributing rewards to data providers, it increases overall satisfaction.

[1602] The processing flow will be explained below.

[1603] User Registration / Login

[1604] 1. User Registration

[1605] Step 1:

[1606] The user terminal connects to the server and displays the registration form.

[1607] Step 2:

[1608] The user enters the required information (name, email address, password) and clicks the send button.

[1609] Step 3:

[1610] The user terminal transmits the input information to the server.

[1611] Step 4:

[1612] The server verifies the integrity of the information entered, checking for missing or incorrect information.

[1613] Step 5:

[1614] After the server verifies it, it stores the information in a database.

[1615] Step 6:

[1616] The server generates a registration completion message and sends it to the user terminal.

[1617] Step 7:

[1618] A message indicating registration completion is displayed on the user's device.

[1619] Providing conversation data

[1620] 2. Uploading conversation data

[1621] Step 1:

[1622] The user terminal connects to the server and displays the data upload screen.

[1623] Step 2:

[1624] The user selects the conversation data file and information about the target person and presses the upload button.

[1625] Step 3:

[1626] The user terminal transmits the selected files and information to the server.

[1627] Step 4:

[1628] The server checks the integrity of the files and information received.

[1629] Step 5:

[1630] The server stores the files and information in a database.

[1631] Step 6:

[1632] The server generates an upload completion message and sends it to the user terminal.

[1633] Step 7:

[1634] A message indicating that the upload is complete will be displayed on the user's device.

[1635] Conversation generation

[1636] 3. Conversation Generation Request

[1637] Step 1:

[1638] The user terminal connects to the server and displays a conversation creation screen.

[1639] Step 2:

[1640] The user selects a target person or character and sends a request to generate a conversation.

[1641] Step 3:

[1642] The user terminal transmits request information to the server.

[1643] Step 4:

[1644] The server's generation AI analyzes the request and retrieves the conversation data of the target person or character from the database.

[1645] Step 5:

[1646] The server's generation AI generates realistic conversations based on the data it acquires.

[1647] Step 6:

[1648] The server transmits the generated conversation data to the user terminal.

[1649] Step 7:

[1650] The generated conversation is displayed on the user terminal.

[1651] Conversation Usage and Rewards

[1652] 4. Paid conversation usage

[1653] Step 1:

[1654] The user terminal connects to the server and displays the billing screen.

[1655] Step 2:

[1656] The user inputs the billing information and presses the billing button.

[1657] Step 3:

[1658] The user terminal transmits the billing information to the server.

[1659] Step 4:

[1660] The server processes the payment based on the billing information received. If an external payment service is used, the server sends a request to that service.

[1661] Step 5:

[1662] The server receives confirmation of successful payment.

[1663] Step 6:

[1664] The server unlocks the conversation data and makes it available.

[1665] Step 7:

[1666] The server calculates the reward based on the usage charge and distributes it to the data provider.

[1667] Step 8:

[1668] The server notifies the user and data provider of the results of the billing and reward processing.

[1669] Step 9:

[1670] The processing results are displayed on the user terminal and the provider terminal.

[1671] Example 1

[1672] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1673] In today's world, users often feel lonely, and especially when they spend a lot of time alone, they want to enjoy real conversations with others. However, in reality, there are limited ways to easily generate and use conversations with target people or characters. Another issue is the lack of a system that provides appropriate rewards to providers of conversation data.

[1674] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1675] In this invention, the server includes: a means for a user to provide conversation data based on a target person or character; a means for a generation AI to generate realistic conversations using the conversation data; a means for distributing rewards for the generated conversations according to paid usage; a means for a user terminal to connect to the server, transmit user registration and login information, and perform authentication; a means for a user to upload conversation data and for the server to store it in a database; a means for inputting billing information and processing payments; and a means for unlocking the conversation data and automatically distributing rewards. This allows users to easily enjoy realistic conversations, and data providers to receive fair rewards.

[1676] A "user terminal" is a device that allows a user to access the system, provide conversation data, request conversation creation, and perform billing procedures.

[1677] A "server" is a computer system that includes the generation AI and performs major processes such as storing conversation data, generating conversations, processing billing, and distributing rewards.

[1678] A "database" is an information management system for storing conversation data, generated conversations, user information, billing information, and the like.

[1679] "Generative AI" refers to algorithms and software that use provided conversational data to generate realistic conversations.

[1680] "Conversation data" refers to data such as audio files and text data related to a target person or character provided by the user.

[1681] "Billing information" refers to credit card information or other payment method information required for users to pay for conversation creation and usage.

[1682] "Reward" refers to money or something of equivalent value distributed to a user who provides conversation data based on the fee-based usage.

[1683] "Realistic conversation" refers to dialogue data that resembles natural conversations between humans, generated by the generative AI based on the conversation data provided.

[1684] The present invention is a system in which a user provides conversation data based on a target person or character, a generation AI uses the conversation data to generate realistic conversations, and rewards are distributed based on paid usage of the generated conversations. This system allows users to enjoy realistic conversations, and data providers can also receive rewards.

[1685] System configuration

[1686] This system consists of three main components: a user terminal, a server, and a database.

[1687] User terminal: A device that provides an interface for users to provide conversation data, request conversation creation, and perform billing procedures.

[1688] Server: Contains the generation AI and performs key processes such as storing conversation data, generating conversations, processing billing, and distributing rewards.

[1689] Database: Stores conversation data, generated conversations, user information, billing information, etc.

[1690] Program processing explanation

[1691] User Registration / Login

[1692] 1. User Registration

[1693] The user's device connects to the server and displays a registration form. The user enters the required information, such as name, email address, and password, and submits it.

[1694] The server receives the input information, checks its integrity, and saves it in the database. If the save is successful, it sends a message to the user's device indicating that registration is complete.

[1695] Examples:

[1696] User A accesses the system using a user terminal and registers by entering "userA@example.com" and a password in the registration form.

[1697] Providing conversation data

[1698] 2. Uploading conversation data

[1699] The user's device connects to the server and displays the data upload screen. The user selects the conversation data they wish to provide and uploads it.

[1700] The server receives the conversation data, checks its integrity, and then saves it in the database. If the save is successful, it sends a message to the user device indicating that the upload is complete.

[1701] Examples:

[1702] User A selects an interview audio file of his / her favorite actor and uploads it to the system.

[1703] Conversation generation

[1704] 3. Conversation Generation Request

[1705] The user's device connects to the server and displays the conversation creation screen. The user selects a target person or character and sends a conversation creation request to the server.

[1706] The server's AI analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is then sent to the user's device.

[1707] Examples:

[1708] User A selects a favorite actor on the system and sends a request to the server to generate a conversation with that actor. The server's AI generates a realistic conversation based on the related voice data and sends it to User A's device.

[1709] Example prompt sentence:

[1710] "Create a conversation with your favorite actor, Hiroshi Tanaka."

[1711] Conversation Usage and Rewards

[1712] 4. Paid conversation usage

[1713] The user's device connects to the server and displays the billing screen. The user enters billing information (e.g., credit card information) and the billing process is carried out.

[1714] The server processes the payment based on the billing information, and if successful, unlocks the conversation data. It distributes a reward based on the billing amount to the data provider and notifies the user and the data provider of the processing result.

[1715] Examples:

[1716] User A pays 500 yen to use the generated conversation. The server distributes a portion of that amount to the data provider as a reward.

[1717] summary

[1718] This system allows users to enjoy real conversations without feeling lonely, while also allowing data providers to receive rewards.

[1719] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1720] Step 1: User Registration

[1721] Input: The user enters information such as name, email address, and password into a registration form on the device.

[1722] Specific operation: The user terminal receives input information and sends the information to the server.

[1723] Data processing: The server checks the integrity of the information received (for example, whether the email address is in the correct format or whether the password meets a certain strength).

[1724] Output: The server stores the information whose integrity has been confirmed in the database and sends a message to the user terminal indicating that registration is complete.

[1725] Step 2: Upload your conversation data

[1726] Input: The user selects the conversation data (for example, an interview audio file) that he or she wants to provide on the terminal.

[1727] Specific operation: The user terminal transmits the selected conversation data to the server.

[1728] Data processing: The server checks the format and content of the conversation data received (for example, whether the file format is appropriate and whether there are any problems with the content).

[1729] Output: After verifying the integrity, the server saves the conversation data in the database and sends a message to the user device indicating that the upload is complete.

[1730] Step 3: Conversation creation request

[1731] Input: The user selects the target person or character on the conversation generation screen from their device and sends a request.

[1732] Specific operation: The user terminal sends a conversation creation request to the server.

[1733] Data processing: The server's generation AI analyzes the request and retrieves the target conversation data from the database.

[1734] Output: The server's generation AI generates realistic conversations based on the acquired data, and sends the generated conversation data to the user's device.

[1735] Step 4: Pay for conversations

[1736] Input: The user inputs billing information (such as credit card information) to use the conversation generated on the terminal.

[1737] Specific operation: The user terminal transmits billing information to the server, and the server processes the payment.

[1738] Data processing: Once the server confirms the successful payment, it unlocks the generated conversation data and calculates the reward based on the payment amount.

[1739] Output: The server sends a notification of payment completion and the released conversation data to the user terminal, and automatically distributes the reward to the data provider.

[1740] In this way, the user terminal, server, and database work together to execute a series of processes that allow the user to enjoy a realistic conversation.

[1741] (Application example 1)

[1742] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1743] In conventional virtual stores, it is difficult for users to obtain detailed product information and they lack a real-time interactive shopping experience. As a result, users often feel dissatisfied with virtual shopping and lose their motivation to purchase. Furthermore, in systems that utilize conversation generation technology, rewards to conversation data providers are insufficient, which reduces the motivation of data providers. The challenge is to solve these problems and improve the satisfaction of both users and data providers.

[1744] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1745] In this invention, the server includes a means for a user to provide conversation data based on a target person or character, a means for a generation AI to generate realistic conversations using the conversation data, a means for distributing rewards for the generated conversations according to paid use, a means for a user to visually check products in a virtual store and provide product explanations and advice using the generated conversations, and a means for answering questions about the products in real time. This allows users to have a real-time interactive shopping experience in the virtual store, and at the same time, appropriate rewards are distributed to conversation data providers, thereby improving overall satisfaction.

[1746] "User" refers to any individual or entity that uses the system, provides conversation data, and uses the generated conversations.

[1747] "Target person or character" refers to a specific person or fictional character that is the basis for the conversation data provided by the user.

[1748] "Conversation Data" refers to user-provided voice and text data based on a target person or character.

[1749] "Generative AI" refers to artificial intelligence that uses provided conversation data to generate realistic conversations.

[1750] "Realistic conversation" refers to a conversation generated by generative AI that resembles a natural conversation between humans.

[1751] "Charged usage" refers to the fee paid by a user to use the generated conversation.

[1752] "Reward distribution" refers to the process by which rewards are distributed to data providers based on billed usage.

[1753] "Virtual store" refers to a virtual shopping environment that users can access via the Internet.

[1754] "Product description" refers to providing information about a product to a user using generative AI.

[1755] "Advice" refers to advice provided by the generating AI to users regarding product selection and usage.

[1756] "Real-time" refers to data processing and information provision occurring immediately, without delay.

[1757] "Answering questions" refers to providing immediate and appropriate answers to questions posed by users to the generated AI.

[1758] The present invention is a system that allows users to have a real-time interactive shopping experience in a virtual store. The system consists of three main components: a user terminal, a server, and a database.

[1759] First, let us explain the user terminal. A user terminal is a device such as smart glasses or a head-mounted display. Through this terminal, the user provides conversation data, requests conversation generation, sends product-related questions, and performs billing procedures. Specifically, the user can visually identify a specific product in a virtual store and request a realistic conversation about that product from the generation AI.

[1760] Next, we will explain the server. The server contains a generative AI and performs key processes such as saving conversation data, generating conversations, processing billing, and distributing rewards. When the server receives a conversation generation request from a user, it retrieves the target conversation data from a database and generates a realistic conversation using a generative AI model (e.g., OpenAI's GPT-3). The generated conversation is sent to the user's device, allowing the user to experience an interactive conversation with a virtual store clerk.

[1761] Finally, we will explain the database. The database is a system that stores user information, conversation data, generated conversations, billing information, etc. The database checks the consistency of the conversation data and provides the data necessary for the server to generate conversations using the generation AI.

[1762] As a concrete example, consider a scenario in which a user is selecting cosmetics in a virtual store. The user puts on smart glasses and enters the virtual store. They find the cosmetics on the shelves and send a question about the specific cosmetics to the system. By entering a prompt such as "Please tell me the features of this cosmetics," the generative AI will provide a detailed product description in real time based on the relevant conversation data.

[1763] This allows users to have an interactive shopping experience that feels just like they are talking to a salesperson in a real store. Meanwhile, since rewards are appropriately distributed to data providers, this also contributes to maintaining their motivation. In this way, the present invention is a system that can improve the satisfaction of both users and data providers.

[1764] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1765] Step 1: User Registration

[1766] The user uses a terminal to enter their name, email address, and password into the registration form and send it to the server. The server receives the input information, checks its integrity, and then stores it in a database. If successful, the server sends a message to the user terminal confirming registration. The input is user information (name, email address, password), and the output is a message confirming registration completion.

[1767] Step 2: Upload your conversation data

[1768] The user uses their device to select conversation data and send it to the server from the upload screen. The server receives the conversation data, checks its integrity, and then stores it in a database. If successful, the server sends a message to the user device indicating that the upload is complete. The input is the conversation data, and the output is a message confirming that the upload is complete.

[1769] Step 3: Conversation creation request

[1770] The user uses their device to select a target person or character and send a conversation generation request to the server. The server receives the request and retrieves the target conversation data from a database. A generative AI model (e.g., OpenAI's GPT-3) is used to generate a realistic conversation based on the prompt. The generated conversation is sent to the user's device. The input is the conversation generation request (target person or character, prompt), and the output is the generated conversation.

[1771] Step 4: Product selection and questions

[1772] Users use their devices to visually check products in a virtual store and send questions about the products to the server. The server receives the questions, generates answers in real time using generative AI, and sends them to the user's device. The input is a question about the product (prompt sentence), and the output is the answer (generated conversation).

[1773] Step 5: Billing process and reward distribution

[1774] The user uses the terminal to carry out the billing procedure to use the generated conversation. The server receives the billing information and processes the payment. If successful, it unlocks the conversation data and distributes rewards to the data provider according to the billing amount. The processing results are notified to the user and the data provider. The input is the billing information, and the output is the billing result and a reward distribution notification.

[1775] Step 6: Deliver a conversational experience

[1776] The user uses the device to experience virtual shopping through generated conversations. Based on the generated conversations, the user can receive product explanations and advice in real time. The input is the generated conversation, and the output is the user's purchasing experience.

[1777] Through these steps, users can have an interactive shopping experience in a virtual store, and at the same time, rewards are appropriately distributed to data providers.

[1778] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1779] The present invention combines a system in which a user provides conversation data based on a target person or character, a generation AI uses the conversation data to generate realistic conversations, and distributes rewards based on paid usage of the generated conversations with an emotion engine that recognizes the user's emotions. This system allows users to enjoy conversations that are realistic and reflect their emotions.

[1780] System configuration

[1781] This system consists of four main components: a user terminal, a server, a database, and an emotion engine.

[1782] 1. User terminal: A device that provides an interface for users to provide conversation data, request conversation generation, provide emotion data, and perform billing procedures.

[1783] 2. Server: Contains the generative AI and emotion engine, and performs key processes such as storing conversation data, generating conversations, recognizing emotions, processing billing, and distributing rewards.

[1784] 3. Database: Stores conversation data, generated conversations, user information, billing information, emotional data, etc.

[1785] 4. Emotion Engine: Analyzes the user's facial expressions, voice, or text to recognize emotions and provide feedback to the generative AI.

[1786] Program processing explanation

[1787] User Registration / Login

[1788] 1. User Registration

[1789] The user's device connects to the server and displays a registration form. The user enters the required information (name, email address, password) and sends it to the server.

[1790] The server receives the input information, checks its integrity, and saves it in the database. If successful, the server sends a registration completion message to the user's device.

[1791] Providing conversation data

[1792] 2. Uploading conversation data

[1793] The user's device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button.

[1794] The server receives the conversation data, checks its integrity, and then saves it in the database. If successful, the server sends a message to the user device indicating that the upload is complete.

[1795] Conversation generation

[1796] 3. Conversation Generation Request

[1797] The user's device connects to the server and displays the conversation creation screen. The user selects the target person or character and sends a conversation creation request.

[1798] The server's AI analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is then sent to the user's device.

[1799] Emotion recognition and conversation regulation

[1800] 4. Emotion recognition

[1801] The user terminal transmits the user's facial expression, voice, or text data to the server.

[1802] The server's emotion engine analyzes this data and recognizes the user's emotions.

[1803] 5. Moderate the conversation

[1804] The server's generation AI receives feedback from the emotion engine and adjusts the conversation content according to the user's emotions.

[1805] The adjusted conversation data is transmitted to the user terminal and displayed on the user terminal.

[1806] Conversation Usage and Rewards

[1807] 6. Paid Conversation Usage

[1808] The user terminal connects to the server and displays the billing screen. The user enters billing information and presses the billing button.

[1809] The server processes the payment based on the billing information, and if successful, unlocks the conversation data. It distributes a reward based on the billing amount to the data provider and notifies the user and the data provider of the processing result.

[1810] Specific examples

[1811] Each step is illustrated concretely through a scenario in which Yamada Taro uses this system.

[1812] 1. User Registration

[1813] Yamada Taro accesses the system using a user terminal and registers by entering "taro@example.com" and a password in the registration form.

[1814] 2. Providing conversation data

[1815] Taro Yamada selects an audio file of an interview with his favorite actor and uploads it to the system.

[1816] 3. Conversation Generation

[1817] Taro Yamada selects his favorite actor on the system and sends a request to generate a conversation with that actor. The server's AI generates realistic conversation based on the relevant voice data and sends it to Taro Yamada's device.

[1818] 4. Emotional awareness and conversation regulation

[1819] While Yamada Taro is using the system to converse, the user's device sends his facial expressions and voice data to the server. The server's emotion engine recognizes Yamada Taro's emotions, and the generation AI adjusts the content of the conversation based on that information. For example, if Yamada Taro is feeling down, the system generates a conversation that includes an encouraging message.

[1820] 5. Conversation Use and Reward

[1821] Taro Yamada pays 500 yen to use the generated conversation, and the server distributes a portion of that amount to the data provider as a reward.

[1822] In this way, the system of the present invention provides users with an opportunity to enjoy real, emotionally relevant conversations without feeling lonely, while at the same time increasing overall satisfaction by distributing rewards to data providers.

[1823] The processing flow will be explained below.

[1824] User Registration / Login

[1825] 1. User Registration

[1826] Step 1:

[1827] The user terminal connects to the server and displays the registration form.

[1828] Step 2:

[1829] The user enters the required information (name, email address, password) and clicks the send button.

[1830] Step 3:

[1831] The user terminal transmits the input information to the server.

[1832] Step 4:

[1833] The server verifies the integrity of the information entered, checking for missing or incorrect information.

[1834] Step 5:

[1835] After the server verifies it, it stores the information in a database.

[1836] Step 6:

[1837] The server generates a registration completion message and sends it to the user terminal.

[1838] Step 7:

[1839] A message indicating registration completion is displayed on the user's device.

[1840] Providing conversation data

[1841] 2. Uploading conversation data

[1842] Step 1:

[1843] The user terminal connects to the server and displays the data upload screen.

[1844] Step 2:

[1845] The user selects the conversation data file and information about the target person and presses the upload button.

[1846] Step 3:

[1847] The user terminal transmits the selected files and information to the server.

[1848] Step 4:

[1849] The server checks the integrity of the files and information received.

[1850] Step 5:

[1851] The server stores the files and information in a database.

[1852] Step 6:

[1853] The server generates an upload completion message and sends it to the user terminal.

[1854] Step 7:

[1855] A message indicating that the upload is complete will be displayed on the user's device.

[1856] Conversation generation

[1857] 3. Conversation Generation Request

[1858] Step 1:

[1859] The user terminal connects to the server and displays a conversation creation screen.

[1860] Step 2:

[1861] The user selects a target person or character and sends a request to generate a conversation.

[1862] Step 3:

[1863] The user terminal transmits request information to the server.

[1864] Step 4:

[1865] The server's generation AI analyzes the request and retrieves the conversation data of the target person or character from the database.

[1866] Step 5:

[1867] The server's generation AI generates realistic conversations based on the data it acquires.

[1868] Step 6:

[1869] The server transmits the generated conversation data to the user terminal.

[1870] Step 7:

[1871] The generated conversation is displayed on the user terminal.

[1872] Emotion recognition and conversation regulation

[1873] 4. Emotion recognition

[1874] Step 1:

[1875] The user terminal transmits the user's facial expression, voice, or text data to the server.

[1876] Step 2:

[1877] The server's emotion engine analyzes this data and recognizes the user's emotions.

[1878] Step 3:

[1879] The server's emotion engine provides the recognized emotion data to the generation AI.

[1880] 5. Moderate the conversation

[1881] Step 1:

[1882] The server's generation AI receives feedback from the emotion engine.

[1883] Step 2:

[1884] The server's generation AI adjusts the conversation content according to the user's emotions.

[1885] Step 3:

[1886] The server transmits the adjusted conversation data to the user terminal.

[1887] Step 4:

[1888] The adjusted conversation is displayed on the user terminal.

[1889] Conversation Usage and Rewards

[1890] 6. Paid Conversation Usage

[1891] Step 1:

[1892] The user terminal connects to the server and displays the billing screen.

[1893] Step 2:

[1894] The user inputs the billing information and presses the billing button.

[1895] Step 3:

[1896] The user terminal transmits the billing information to the server.

[1897] Step 4:

[1898] The server processes the payment based on the billing information received. If an external payment service is used, the server sends a request to that service.

[1899] Step 5:

[1900] The server receives confirmation of successful payment.

[1901] Step 6:

[1902] The server unlocks the conversation data and makes it available.

[1903] Step 7:

[1904] The server calculates the reward based on the usage charge and distributes it to the data provider.

[1905] Step 8:

[1906] The server notifies the user and data provider of the results of the billing and reward processing.

[1907] Step 9:

[1908] The processing results are displayed on the user terminal and the provider terminal.

[1909] Example 2

[1910] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1911] Conventional conversation generation systems have difficulty generating and adjusting conversations in real time based on the user's emotions and situation, resulting in a poor user experience. Furthermore, there are insufficient means for distributing appropriate rewards to conversation data providers, making fair reward distribution difficult. To solve these problems, the present invention aims to provide a system that enables conversation generation based on the user's emotions and fair reward distribution.

[1912] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1913] In this invention, the server includes a means for a user to provide conversation data based on a target person or character, a means for a generative AI model to generate realistic conversations using the conversation data, a means for distributing rewards for the generated conversations based on paid usage, an emotion engine that recognizes the user's emotions, and a means for adjusting the content of the conversations based on feedback from the emotion engine. This enables the generation of realistic conversations based on the user's emotions and fair distribution of rewards.

[1914] "User" refers to an individual or organization that uses the system to provide conversation data or enjoy the generated conversations.

[1915] "Target person or character" refers to a real person or fictional character that the user selects as the target for generating conversation data.

[1916] "Conversation data" refers to information such as voice, text, and images based on a target person or character provided by a user.

[1917] A "generative AI model" refers to an artificial intelligence algorithm or system that generates realistic conversations using provided conversation data.

[1918] "Charged usage" refers to the process whereby a user pays the system to use the generated conversations.

[1919] "Reward distribution" refers to the process of distributing appropriate rewards to providers of conversation data based on billed usage.

[1920] An "emotion engine" refers to a system or algorithm that analyzes a user's facial expressions, voice, or text to recognize their emotions at that moment and feeds the analysis results back into a generative AI model.

[1921] "Feedback" refers to the information the emotion engine provides to the generative AI model about the user's emotional state, allowing it to adjust the conversation content accordingly.

[1922] "Conversational content adjustment" refers to the process of modifying or optimizing the generated conversation to respond to the user's emotions based on feedback from the emotion engine.

[1923] The system allows users to provide conversation data based on target people or characters, generates realistic conversations using a generative AI model, and distributes rewards based on paid usage. The system includes four main components: a user terminal, a server, a database, and an emotion engine.

[1924] System configuration

[1925] 1. User Device:

[1926] A user terminal (e.g., a PC, smartphone, tablet, etc.) is a device that provides an interface for users to interact with the system. The user terminal is used to provide conversation data, send conversation generation requests, provide emotion data, and perform billing procedures.

[1927] 2. Server:

[1928] The server is the core of the system, containing the generative AI model and the emotion engine. The server performs the following main functions:

[1929] Saving conversation data

[1930] Conversation Generation

[1931] emotion recognition

[1932] Billing process

[1933] reward distribution

[1934] 3. Database:

[1935] The database stores conversation data, generated conversations, user information, billing information, emotion data, etc. This allows the necessary data to be efficiently retrieved and processed.

[1936] 4. Emotion Engine:

[1937] An emotion engine is software or algorithm that analyzes a user's facial expressions, voice, or text to recognize emotions. The emotion engine provides the necessary feedback to the generative AI model to adjust the conversation content according to the user's emotions.

[1938] Program processing

[1939] User Registration

[1940] The user's device connects to the server and displays a registration form. The user enters the required information (e.g., name, email address, password) and sends it to the server. The server receives the entered information, checks its integrity, and stores it in the database. If successful, the server sends a message to the user's device indicating that registration is complete.

[1941] Providing conversation data

[1942] The user device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button. The server receives the data, checks its integrity, and saves it in the database. If successful, the server sends a message to the user device indicating that the upload is complete.

[1943] Conversation generation

[1944] The user's device connects to the server and displays the conversation generation screen. The user selects the target person or character and sends a conversation generation request. The server's generation AI model analyzes the request, retrieves the target conversation data from the database, and generates a realistic conversation. The generated conversation is sent to the user's device.

[1945] example:

[1946] "Generate a conversation with actor A."

[1947] Emotion recognition and conversation regulation

[1948] The user device sends the user's facial expression, voice, or text data to the server. The server's emotion engine analyzes this data and recognizes the user's emotions. The server's generative AI model receives feedback from the emotion engine and adjusts the conversation content according to the user's emotions, and the adjusted conversation data is sent to the user device.

[1949] example:

[1950] "Generate a conversation to cheer up Taro Yamada who is feeling down."

[1951] Conversation Usage and Rewards

[1952] The user's terminal connects to the server and displays the billing screen. The user enters billing information and presses the billing button. The server processes the payment based on the billing information, and if successful, unlocks the conversation data. Based on the billing amount, the server distributes rewards to the data provider and notifies the user and data provider of the processing results.

[1953] example:

[1954] "Please handle the billing process and distribute the rewards for using the generated conversations."

[1955] This system allows users to enjoy realistic, emotionally relevant conversations, while also providing fair rewards to data providers.

[1956] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1957] Step 1:

[1958] User Registration

[1959] Input: The user enters personal information such as a name, email address, or password.

[1960] Specific operation: The user's device connects to the server and displays the registration form. The user presses the submit button to send the information entered to the server.

[1961] Data processing and calculation: The server checks the integrity of the information received and stores it in the database.

[1962] Output: The server sends a registration completion message to the user terminal.

[1963] Step 2:

[1964] Providing conversation data

[1965] Input: User-provided conversation data (e.g., audio files).

[1966] Specific operation: The user's device connects to the server and displays the data upload screen. The user selects the conversation data and presses the upload button.

[1967] Data processing and calculation: The server checks the integrity of the data received and stores it in the database.

[1968] Output: The server sends a message to the user device indicating that the upload is complete.

[1969] Step 3:

[1970] Conversation generation request

[1971] Input: A person or character selected by the user.

[1972] Specific operation: The user's device connects to the server and displays the conversation creation screen. The user selects the target person or character and sends a conversation creation request.

[1973] Data processing and calculation: The server's generated AI model receives the request, retrieves the target conversation data from the database, and generates a realistic conversation.

[1974] Output: Send the generated conversation to the user's device.

[1975] Step 4:

[1976] emotion recognition

[1977] Input: User facial, voice, or text data.

[1978] Specific operation: The user's device captures the user's facial expression and voice data and sends it to the server.

[1979] Data processing and calculation: The server's emotion engine analyzes this data and recognizes the user's emotions.

[1980] Output: Recognized emotion data is sent as feedback to the generative AI model.

[1981] Step 5:

[1982] Adjusting the conversation

[1983] Input: Recognized emotion data and generated speech data.

[1984] How it works: The server's generative AI model receives feedback from the emotion engine and adjusts the conversation content.

[1985] Data processing and calculation: Conversation content is optimized according to the user's emotions.

[1986] Output: The adjusted conversation data is sent to the user terminal.

[1987] Step 6:

[1988] Pay-per-use conversations

[1989] Input: Billing information entered by the user.

[1990] Specific operation: The user's device connects to the server and displays the billing screen. The user enters billing information and presses the billing button.

[1991] Data processing and calculation: The server processes the billing information, and if payment is successful, unlocks the conversation data. Based on the billing amount, rewards are distributed to the data provider.

[1992] Output: Notify the user and data provider of the processing results.

[1993] (Application example 2)

[1994] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1995] Conventional virtual salesperson systems have the problem that interactions with users are fixed and one-way, and are unable to reflect the user's emotions and situation in real time. Furthermore, the quality of the generated conversations is poor, making it difficult to improve the user experience. Furthermore, there is no established reward distribution system to effectively utilize the content of these conversations, making it difficult to maintain the motivation of data providers. Technology that solves these issues is needed.

[1996] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1997] In this invention, the server includes: a means for a user to provide conversation data based on a target person or character; a means for a generation AI to generate realistic conversations using the conversation data; a means for an emotion engine to analyze the user's facial expressions, voice, or text to recognize emotions and adjust the generated conversation; and a means for distributing rewards for the generated conversation based on paid usage. This enables adaptive conversations that reflect the user's emotions in real time, allowing for high-quality dialogue by virtual salespeople. Furthermore, the reward distribution system can improve incentives for data providers.

[1998] A "user terminal" is a device that provides an interface for a user to provide conversation data, request conversation generation, provide emotion data, and perform billing procedures.

[1999] The "server" is a system that includes the generative AI and emotion engine and performs key processes such as storing conversation data, generating conversations, recognizing emotions, processing billing, and distributing rewards.

[2000] A "database" is a storage device that stores conversation data, generated conversations, user information, billing information, emotional data, and the like.

[2001] An "emotion engine" is an analysis system that analyzes a user's facial expressions, voice, or text to recognize emotions and provide feedback to the generative AI.

[2002] "Generative AI" is an artificial intelligence model that generates realistic dialogue based on conversation data.

[2003] "Conversation data based on a target person or character" is data of conversation related to a specific person or character provided by the user.

[2004] A "conversation generation request" is a request for the user to generate a conversation with a target person or character.

[2005] "Paid usage" refers to usage based on the amount paid by a user to use a particular service or content.

[2006] "Reward sharing" is a mechanism for distributing compensation to data providers and other participants based on paid usage.

[2007] The present invention provides a system for generating real-time conversations between a user and a salesperson in a virtual store according to their emotions, and for distributing rewards according to the amount of paid use. Detailed embodiments of the present invention will be described below.

[2008] Hardware and software used

[2009] Smart devices: smartphones, smart glasses, head-mounted displays

[2010] Server: Generative AI (GPT-4), emotion engine (Microsoft Azure Emotion API), database

[2011] User terminal: A device that provides an interface for users to provide conversation data, request conversation generation, provide emotion data, and perform billing procedures.

[2012] Program processing explanation

[2013] User Registration / Login

[2014] The server receives the required information (name, email address, password) and stores the submitted information in a database. The user then registers by entering the information into the registration form on their smart device.

[2015] Providing conversation data

[2016] Users upload conversation data and product information using their own devices. The server receives the data, checks its integrity, and then stores it in the database. This process allows the system to obtain a wealth of data.

[2017] Conversation generation

[2018] When a user asks a question about a product or requests purchasing advice, the server's AI analyzes the request and retrieves relevant conversation data from the database. Based on this data, the AI ​​generates a realistic conversation and provides it to the user.

[2019] Emotion Recognition and Speech Modulation

[2020] When a user converses through a smart device, the device transmits the user's facial expressions and voice data to the emotion engine for analysis. The emotion engine then feeds back the analysis results to the generation AI, which then adjusts the content of the conversation according to the user's emotions. This allows for adaptive conversations to be provided according to the user's emotions.

[2021] Pay-per-use conversations

[2022] When a user uses the premium conversation service, the billing procedure is carried out within the system. The server processes the billing information and, if successful, provides the premium features to the user. It also distributes rewards to data providers based on the billed usage.

[2023] Specific examples

[2024] For example, if a user asks a question about a product in a virtual store and the user has a sad expression on their face, the emotion engine will recognize that emotion and the generation AI will generate a conversation that encourages the user.

[2025] Prompt Sentence Examples

[2026] An example prompt for the user to ask a question might look like this:

[2027] User expression: Sad

[2028] User Question: I've been through a lot lately and need some emotional support. Can you offer some words of encouragement?

[2029] The generative AI analyzes this and generates the following dialogue:

[2030] "I know you're going through a difficult time. I'm sorry to trouble you, but please remember that I'm always by your side. It's important to stop sometimes. Even if it's just a little at a time, keep moving forward."

[2031] This embodiment provides high-quality dialogue that responds to the user's emotions, significantly improving the user experience in the virtual store. Furthermore, by distributing rewards to data providers, the system can be expected to continue to develop.

[2032] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2033] Step 1:

[2034] User Registration / Login

[2035] The user enters the required information (name, email address, password) into the registration form on the smart device and submits it.

[2036] The server receives this input data, checks the data integrity, and if there are no problems, stores it in the database.

[2037] The inputs to this process are the name, email address, and password entered by the user, and the output is the user information stored in a database.

[2038] Step 2:

[2039] Providing conversation data

[2040] Users upload conversation data and product information from their own terminals.

[2041] The server receives this data, checks the data format and content for consistency, and then stores it in the database.

[2042] The input for this process is conversation data and product information uploaded by the user, and the output is conversation data stored in the database.

[2043] Step 3:

[2044] Conversation generation request

[2045] The user selects the person or character with whom they want to create a conversation from their smart device and sends a conversation creation request.

[2046] The server's generation AI receives this request and retrieves conversation data related to the target from the database.

[2047] The generative AI generates realistic dialogue based on the acquired data and sends it to the user's device.

[2048] The inputs to this process are a user's conversation generation request and conversation data related to the target, and the output is the generated realistic conversation.

[2049] Step 4:

[2050] Emotion Recognition and Speech Modulation

[2051] While a user is having a conversation through a smart device, the device captures the user's facial expressions and voice data and transmits them to a server.

[2052] The server's emotion engine analyzes this data and recognizes the user's emotions.

[2053] Based on feedback from the emotion engine, the generative AI adjusts the content of the conversation and generates an adaptive conversation, which is then sent back to the user's device.

[2054] The inputs for this process are the user's facial expressions, voice data, and emotion recognition feedback, and the output is speech that has been adjusted according to the user's emotions.

[2055] Step 5:

[2056] Pay-per-use conversations

[2057] When using the premium conversation service, users complete the billing procedure on their smart device.

[2058] The server processes the billing information and verifies the integrity of the payment. If successful, the premium conversation service is unlocked.

[2059] The server performs processing to distribute rewards to data providers based on the amount charged.

[2060] The input of this process is the user's billing information and the billing amount, and the output is a notification of the release of the premium function and the distribution of rewards.

[2061] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2062] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2063] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2064] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2065] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2066] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2067] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2068] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2069] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2070] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2071] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2072] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2073] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2074] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2075] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2076] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2077] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2078] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2079] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2080] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2081] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2082] The following is further disclosed regarding the above embodiment.

[2083] (Claim 1)

[2084] a means for a user to provide conversation data based on a target person or character;

[2085] A means for a generation AI to generate realistic conversations using the conversation data;

[2086] The system includes a means for distributing rewards based on the generated conversations in accordance with charged usage.

[2087] (Claim 2)

[2088] The system of claim 1 further comprising means for the user to select a target person or character and send a conversation generation request.

[2089] (Claim 3)

[2090] 10. The system of claim 1, further comprising means for distributing the fee-based usage reward to a data provider.

[2091] "Example 1"

[2092] (Claim 1)

[2093] a means for a user to provide conversation data based on a target person or character;

[2094] A means for a generation AI to generate realistic conversations using the conversation data;

[2095] means for distributing rewards for the generated conversations according to billing usage;

[2096] A means for a user terminal to connect to the server, transmit user registration and login information, and perform authentication;

[2097] A means for users to upload conversation data and for the server to store it in a database;

[2098] means for inputting billing information and processing payments;

[2099] A means to unlock conversation data and automatically distribute rewards;

[2100] A system including:

[2101] (Claim 2)

[2102] 10. The system of claim 1, further comprising means for a user to select a target person or character and send a conversation generation request.

[2103] (Claim 3)

[2104] 10. The system of claim 1, further comprising means for distributing a fee-based usage reward to the data provider.

[2105] "Application Example 1"

[2106] (Claim 1)

[2107] a means for a user to provide conversation data based on a target person or character;

[2108] A means for a generation AI to generate realistic conversations using the conversation data;

[2109] means for distributing rewards for the generated conversations according to billing usage;

[2110] A means for users to visually view products in a virtual store and provide product descriptions and advice using generated dialogue;

[2111] A way to answer product questions in real time, and

[2112] A system including:

[2113] (Claim 2)

[2114] The system of claim 1 further comprising means for the user to select a target person or character and send a conversation generation request.

[2115] (Claim 3)

[2116] 10. The system of claim 1, further comprising means for distributing the fee-based usage reward to a data provider.

[2117] "Example 2: Combining Emotion Engines"

[2118] (Claim 1)

[2119] a means for a user to provide conversation data based on a target person or character;

[2120] A means for a generative AI model to generate realistic conversations using the conversation data;

[2121] means for distributing rewards for the generated conversations according to billing usage;

[2122] an emotion engine that recognizes the user's emotions;

[2123] means for adjusting the conversation content based on feedback from the emotion engine;

[2124] A system including:

[2125] (Claim 2)

[2126] 10. The system of claim 1, further comprising means for a user to select a target person or character and send a conversation generation request.

[2127] (Claim 3)

[2128] 10. The system of claim 1, further comprising means for distributing the fee-based usage reward to a data provider.

[2129] "Application example 2 when combining emotion engines"

[2130] (Claim 1)

[2131] a means for a user to provide conversation data based on a target person or character;

[2132] A means for a generation AI to generate realistic conversations using the conversation data;

[2133] an emotion engine that analyzes a user's facial expression, voice, or text to recognize emotions and adjusts the generated dialogue;

[2134] The system includes a means for distributing rewards based on the generated conversations in accordance with charged usage.

[2135] (Claim 2)

[2136] The system of claim 1 further comprising means for the user to select a target person or character and send a conversation generation request.

[2137] (Claim 3)

[2138] 10. The system of claim 1, further comprising means for distributing the fee-based usage reward to a data provider. [Explanation of symbols]

[2139] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for a user to provide conversation data based on a target person or character; A means for a generation AI to generate realistic conversations using the conversation data; The system includes a means for distributing rewards based on the generated conversations in accordance with charged usage.

2. The system of claim 1 further comprising means for the user to select a target person or character and send a dialogue generation request.

3. 10. The system of claim 1, further comprising means for distributing the fee-based usage reward to a data provider.

Images