system

Abstract

A system is provided.SOLUTION: A system comprising: means for learning the appearance and personality of a particular character or person through artificial intelligence techniques; means for storing the learned data as a non-fungible token (NFT); means for providing an interface for a user to select and converse with a conversation partner; means for conducting a voice conversation in real-time; and means for receiving a usage fee from the user and distributing revenue among the parties involved.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Direct interaction with characters and people is limited by time and geographical constraints, limiting the opportunities for many fans. Furthermore, hosting events and meet-and-greets with celebrities and characters requires high costs and personnel, meaning only a limited number of people can enjoy them. Furthermore, there is a risk of reputational damage due to unauthorized use such as deepfakes and impersonation. For this reason, there is a need for a safe and easy way for fans to interact with their favorite characters and people in real time, 24 hours a day, 365 days a year. [Means for solving the problem]

[0005] The present invention provides a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people and stores that data as a non-fungible token (NFT). Specifically, it provides a means for users to select a conversation partner and provide an interface for real-time voice conversations. It also has a means for collecting usage fees and sharing revenue, and includes a function that uses the learned artificial intelligence data to generate responses based on the user's conversation content and convert them into voice data. This provides an environment where fans can safely and easily interact with their favorite characters or people in real time, 24 hours a day, 365 days a year.

[0006] A "character" is a fictional person, animal, or other fictional entity with a particular appearance and personality that appears in stories and media.

[0007] "Person" refers to a real person, especially a famous individual or idol with a particular appearance or personality.

[0008] "Artificial intelligence technology" refers to algorithms and software used to simulate human intellectual activity.

[0009] "Learning" is the process of using data to adjust algorithms and find specific patterns and rules.

[0010] A non-fungible token (NFT) is a token that uses blockchain technology to guarantee the uniqueness of a digital asset and is a means of establishing ownership and transferability.

[0011] "Interface" refers to the screens and operating tools that allow users to interact with the system.

[0012] "Real-time" refers to a state in which processing and response are carried out immediately without delay.

[0013] "Voice conversation" refers to interactive communication using voice.

[0014] "Usage Fee" means the fee payable for use of the Service.

[0015] "Revenue sharing" refers to the division of earned revenue among parties based on certain criteria.

[0016] "Response" refers to the reaction that a system generates in response to input from a user. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as a non-fungible token (NFT), and enables users to have real-time voice conversations 24 hours a day, 365 days a year. The system is configured as follows:

[0039] System configuration

[0040] 1. Learning with AI technology

[0041] The server collects data on the appearance, personality, and past behavior of specific characters or people, and uses artificial intelligence algorithms to learn this information.

[0042] The learning results include the character's specific behavioral and response patterns and are saved as a highly accurate model.

[0043] 2. Save as an NFT

[0044] The server stores the learned AI data as an NFT, and uses blockchain technology to guarantee its uniqueness and ownership.

[0045] 3. Providing an interface

[0046] The terminal provides the user with an interface for selecting a conversation partner. The user can log in and select a character of their choice.

[0047] 4. Real-time voice conversation

[0048] The user performs an operation to start a conversation with the selected character. The device captures the user's voice, converts it into text data, and sends it to the server.

[0049] The server uses an AI model to generate an appropriate response based on the text data it receives, which is then converted back into audio data and sent to the device.

[0050] The terminal plays the audio data and the conversation takes place in real time.

[0051] 5. Fee collection and revenue sharing

[0052] The server collects the usage fee from the user, and the payment process is carried out through the terminal.

[0053] Revenues are shared among the parties involved, including copyright holders of characters and people, and service providers.

[0054] Specific examples

[0055] 1. User logs in and selects a character

[0056] The user opens the app and logs in by entering "user123" and "password123" on the login screen. The device sends the entered data to the server and authentication is performed. If authentication is successful, the home screen is displayed on the device.

[0057] The user presses the "Character Selection" button on the home screen and selects, for example, "Character A." The device sends the selected character information to the server, and the server retrieves the corresponding AI data and sends it to the device.

[0058] 2. Initiating and conducting actual conversations

[0059] The user presses the "Start conversation" button and speaks "Hello, Character A" into the device's microphone. The device converts the speech into text and sends it to the server.

[0060] The server analyzes the text data and generates an appropriate response based on the AI ​​model, for example, "Hello, I'm Character A. How's your day?"

[0061] The generated response is converted into audio data and sent to the terminal, which plays the audio data so that the user can hear the response.

[0062] 3. Payment of fees and revenue sharing

[0063] When a user pays for a service, the terminal captures the payment information and sends it to the server, which processes the payment through a payment gateway and distributes the revenue among the parties after verification.

[0064] This system allows fans to safely and easily interact with their favorite characters and people 24 hours a day, 365 days a year, and also reduces the high costs associated with hosting events and inviting celebrities to attend.

[0065] The processing flow will be explained below.

[0066] Step 1:

[0067] The user opens the application. The user accesses the login screen and enters their ID and password.

[0068] Step 2:

[0069] The terminal receives the user's input and sends the ID and password information to the server. The communication is encrypted.

[0070] Step 3:

[0071] The server checks the received ID and password against the database and performs user authentication. If the ID and password do not match, an error message is sent to the terminal.

[0072] Step 4:

[0073] If the authentication is successful, the server generates session information and sends it back to the device. The device uses the received session information to display the home screen.

[0074] Step 5:

[0075] The user presses the "Select Character" button on the home screen and selects the character they want to interact with.

[0076] Step 6:

[0077] The device receives the user's selection and sends the character ID to the server, which then retrieves the AI ​​model data for the selected character.

[0078] Step 7:

[0079] The server sends the acquired AI model data to the device, which then displays a conversation interface with the character based on the received data.

[0080] Step 8:

[0081] The user presses the "Start conversation" button to start the conversation, and the device's microphone is turned on to capture the user's voice.

[0082] Step 9:

[0083] The device records the user's voice and converts the voice data into text data in real time. This text data is then sent to the server.

[0084] Step 10:

[0085] The server analyzes the received text data and generates a response using an AI model. For example, if the user says "Hello, Character A," the response is "Hello! How's your day?"

[0086] Step 11:

[0087] The response text generated by the server is converted into voice data and sent to the terminal.

[0088] Step 12:

[0089] The terminal plays back the received voice data and provides a response to the user, who can then ask questions or make comments.

[0090] Step 13:

[0091] If the user needs to pay a usage fee, the terminal displays a payment screen, and the user enters credit card information, etc.

[0092] Step 14:

[0093] The terminal sends the entered payment information to the server, and the server processes the payment through a payment gateway.

[0094] Step 15:

[0095] The server sends information about the success or failure of the payment to the terminal. If the payment is successful, the usage rights are updated.

[0096] Step 16:

[0097] The terminal notifies the user of the payment result, allowing the user to continue using the system.

[0098] Example 1

[0099] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0100] Conventional dialogue systems with characters or people have difficulty in real-time voice conversation, and have been unable to provide a satisfying experience to users. Furthermore, there was a lack of means to guarantee the uniqueness and ownership of trained AI data, creating the risk of data tampering or unauthorized use. Furthermore, there was no established mechanism for collecting usage fees from users and distributing revenue fairly among the parties involved. New technologies are needed to solve these problems.

[0101] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0102] In this invention, the server includes: means for learning the appearance and personality of a specific character or person using artificial intelligence technology; means for storing the learned data as a non-fungible token (NFT); means for providing an interface for a user to select a conversation partner and converse with them; means for acquiring the user's voice and converting it into text data using voice recognition technology; means for generating an appropriate response based on the text data using an AI model and converting the response into voice data; means for transmitting the converted voice data to a user terminal to conduct a real-time voice conversation; and means for collecting usage fees from users and distributing revenue among the parties involved. This enables real-time voice conversation, guarantees the uniqueness and ownership of the learned data, and enables fair revenue distribution.

[0103] A "specific character or person" refers to a fictional or real person that a user selects when interacting with the character, whose appearance, personality, behavior patterns, etc. are predefined.

[0104] "Artificial intelligence technology" refers to technology that uses machine learning and deep learning models to learn from specific data and generate appropriate responses even for unknown data.

[0105] A "non-fungible token (NFT)" is a unique digital asset created using blockchain technology with guaranteed ownership and authenticity.

[0106] "Interface" refers to the user interface provided to users to select and interact with characters or people, and is usually provided as an application.

[0107] "Speech recognition technology" is a technology that converts a user's voice into text data, and examples include Google Cloud Speech-to-Text and Microsoft Azure Speech Recognition Services.

[0108] An "AI model" is a neural network model trained on collected data and used to generate responses for a specific character or person.

[0109] "Converting to voice data" means converting the text response generated by the AI ​​model into voice data using speech synthesis technology, making it audible to the user.

[0110] "Having a real-time voice conversation" refers to a form in which a user can instantly communicate with a character or person using voice.

[0111] "Collecting fees" means collecting fees from users for using the service, often through an online payment system.

[0112] "Distributing revenues among the parties involved" means distributing the collected usage fees to the copyright holders of characters and people, service providers, and other rights holders in a fair proportion.

[0113] MODE FOR CARRYING OUT THE INVENTION

[0114] The present invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as a non-fungible token (NFT), and enables users to have real-time voice conversations 24 hours a day, 365 days a year. The system is configured as follows:

[0115] Learning with AI technology

[0116] The server collects data on the appearance, personality, and past behavior of specific characters or people. For example, it uses scripts to obtain data from YouTube, social media, etc. This data is trained using the TensorFlow or PyTorch machine learning framework. During the data preprocessing process, noise removal, tokenization, normalization, etc. are performed to build a neural network model. The trained model undergoes accuracy evaluation and is saved if it is highly accurate.

[0117] Stored as an NFT

[0118] The server converts the trained AI data into ONNX format to store it as an NFT, and then issues the NFT using a blockchain platform such as Ethereum or Binance Smart Chain, incorporating the model's unique identifier and metadata in the process.

[0119] Providing an interface

[0120] The device provides the user with an interactive user interface. The user logs in using an application developed with React Native or Flutter. The login screen has username and password input fields for the user to authenticate. If authentication is successful, a character selection screen is displayed, where the user selects the character they want to interact with.

[0121] Real-time voice conversation

[0122] The user initiates a conversation with the selected character. The device uses the microphone to capture the user's voice and converts it into text using the Google Cloud Speech-to-Text API or Microsoft Azure Speech Recognition Service. The text is sent to a server, which uses an AI model (e.g., GTP-3 or BERT) to generate an appropriate response. This response is converted into audio data by a speech synthesis engine (e.g., Amazon Polly or Google Text-to-Speech) and sent to the device. The device plays back the audio data, enabling real-time conversation.

[0123] Collection of usage fees and revenue sharing

[0124] The usage fee paid by users when using the service is collected by the terminal acquiring payment information. The payment processing is carried out by the server using the API of Stripe or PayPal. The usage fee collected is distributed fairly to the copyright holders of the characters and people and the service provider.

[0125] Specific examples

[0126] 1. User login and character selection

[0127] The user opens the app and logs in by entering "user123" and "password123" on the login screen. The device sends the input data to the server and authentication is performed. If authentication is successful, the home screen is displayed.

[0128] The user presses the "Character Selection" button on the home screen and selects "Character A." The device sends the selected character information to the server, and the server retrieves the corresponding AI data and sends it to the device.

[0129] 2. Initiating and conducting actual conversations

[0130] The user presses the "Start conversation" button and speaks "Hello, Character A" into the device's microphone. The device converts the speech into text and sends it to the server.

[0131] The server analyzes the text data and generates an appropriate response based on the AI ​​model, for example, "Hello, I'm Character A. How's your day?"

[0132] The generated response is converted into audio data and sent to the terminal, which plays the audio data so that the user can hear the response.

[0133] Prompt Sentence Examples

[0134] "Hi, I'm Character A. How's your day going?"

[0135] This system allows fans to safely and easily interact with their favorite characters and people 24 hours a day, 365 days a year, and also reduces the high costs associated with hosting events and inviting celebrities to attend.

[0136] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0137] Step 1: Data collection

[0138] The server collects data on the appearance, personality, and past behavior of specific characters or people. Inputs include YouTube video links and social media post URLs. The data is obtained using scraping technology and APIs. The output is the retrieved text, images, and audio data.

[0139] Step 2: Data Preprocessing

[0140] The server preprocesses the collected data. The raw data collected in step 1 is provided as input. Specifically, processes such as noise removal, text tokenization, and normalization are performed. For example, unnecessary special characters are removed from the text data and it is converted into a standard format. The output is the preprocessed data.

[0141] Step 3: Training the model

[0142] The server uses the preprocessed data to train an artificial intelligence model (e.g., TensorFlow or PyTorch). The preprocessed data from step 2 is provided as input. Specifically, the server splits the data into batches and optimizes the model parameters using a GPU. The output is a trained, highly accurate model.

[0143] Step 4: Evaluate and save the model

[0144] The server evaluates and saves the trained model. The model trained in step 3 and the validation dataset are used as input. Specifically, the accuracy of the model is evaluated using the validation data, and if the accuracy meets the criteria, the model is saved. The output is the evaluation result and the saved model.

[0145] Step 5: Save as an NFT

[0146] The server saves the trained AI model as an NFT. The model data saved in step 4 is used as input. Specifically, the server converts the model to ONNX format and issues an NFT on a blockchain platform (e.g., Ethereum or Binance Smart Chain). The output is the model data saved as an NFT.

[0147] Step 6: User authentication and character selection

[0148] The terminal displays a login screen for the user and performs authentication. A username and password are provided as input. The authentication data is sent to the server and checked against a database. If authentication is successful, a character selection screen is displayed. The output is the selected character information.

[0149] Step 7: Capturing and Converting Audio Input

[0150] The device uses a microphone to capture the user's voice. The user's voice data is provided as input. Specifically, the voice is converted into text data using the Google Cloud Speech-to-Text API. The output is text data.

[0151] Step 8: Response Generation

[0152] The server receives the text data and generates a response using an AI model. The text data converted in step 7 is provided as input. Specifically, the AI ​​model (e.g., GPT-3 or BERT) analyzes the text data and generates an appropriate response. The output is the text data of the generated response.

[0153] Step 9: Convert to audio and send

[0154] The server converts the generated text response into audio data and sends it to the device. The text data generated in step 8 is provided as input. Specifically, it converts it into audio data using Amazon Polly or Google Text-to-Speech. The output is audio data.

[0155] Step 10: Playing back real-time voice conversations

[0156] The device receives the audio data and plays it on the speaker. The input is the audio data sent in step 9. Specifically, it decodes the audio data and plays it on the speaker so that it can be heard by the user. The output is audio played in real time.

[0157] Step 11: Fee collection and distribution

[0158] The terminal obtains payment information to collect usage fees and sends it to the server. Payment information is provided as input. The server processes the payment using Stripe or PayPal APIs and distributes revenue among the parties involved. Specifically, after confirming the payment process, the distribution is made via bank transfer or cryptocurrency. The output is the distributed revenue.

[0159] (Application example 1)

[0160] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0161] Currently, providing high-quality customer service to customers visiting physical stores requires a large amount of labor and resources. It is also difficult to provide detailed information about specific products in the store, making it difficult to increase customer satisfaction. Furthermore, while there is a demand for personalized guidance that allows customers to select products based on their preferences and interests, achieving this is complex. To solve these challenges, a system that can interact with customers in real time and provide personalized guidance is needed.

[0162] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0163] In this invention, the server includes: means for learning the characteristics of a specific virtual character using machine learning technology; means for saving the learned information as a non-fungible character (NFT); means for providing an operation screen for a user to select a conversation partner and conduct a conversation; means for real-time voice communication; means for receiving a service fee from the user and distributing the revenue among the relevant parties; means for providing guidance using a display device that supports conversations between customers and virtual characters in physical stores; and means for users to obtain product information in the store and receive support through the character. This makes it possible to improve the customer experience in physical stores and provide personalized, high-quality customer service.

[0164] "Characteristics of a specific virtual person" refers to data that includes information such as the appearance, personality, and past words and actions of a character or celebrity.

[0165] "Machine learning technology" is a technology that allows computers to automatically learn from data and make predictions and classifications.

[0166] A "non-fungible token (NFT)" is a digital asset whose uniqueness and ownership are guaranteed using blockchain technology.

[0167] The "operation screen" is a display screen that provides an interface for a user to select a conversation partner and carry out a conversation.

[0168] "Real-time voice communication" refers to a technology that allows users to instantly communicate with selected characters via voice.

[0169] "Service fee" means the amount of money paid by a user as consideration for using a service.

[0170] "Related parties" refers to people or organizations that are subject to revenue sharing, such as character or system providers and developers.

[0171] A "display device" refers to hardware for displaying information, such as displays installed in physical stores, smart glasses, and head-mounted displays (HMDs).

[0172] "Supporting conversations between virtual characters" refers to a specific virtual character communicating with a user via voice to provide guidance and information.

[0173] "Get product information" means viewing or obtaining detailed information about products in the store.

[0174] "Receiving support through a character" means that a virtual person guides or explains products or services to the user.

[0175] To implement the present invention, the following system configuration is required: The system is composed of a server, a terminal, and a user as its main components.

[0176] 1. System Program

[0177] The server builds a database that uses machine learning technology to learn the characteristics of a specific virtual person. The terminal provides an operation screen that allows the user to select a conversation partner and engage in real-time voice communication with that virtual person. The server also stores the learned information as a non-fungible token (NFT).

[0178] The terminal displays an operation screen for the user to select a virtual person to talk to and transmits the selection information to the server. When the user communicates with the terminal in real time, the terminal converts the user's voice into text information and transmits the text information to the server.

[0179] The server then feeds the received text information to a machine learning model to generate a conversation-based response. The generated response is then converted into voice data by the server and sent to the device. The device then plays the voice data and provides the user with a response from a virtual character.

[0180] 2. Processing Description

[0181] The server is implemented using programming languages ​​such as Python, and uses machine learning technologies such as the NLP (natural language processing) model GPT-3. It also uses libraries such as Google Speech Recognition for speech recognition and pyttsx3 for speech synthesis.

[0182] The device includes hardware such as smart glasses, a head-mounted display (HMD), and a microphone. The device recognizes the user's speech and sends it to the server. The server converts the speech to text and generates an appropriate response using an NLP model. The server then converts the response back into audio data and sends it back to the device.

[0183] As a specific example, a user enters a physical store and puts on smart glasses. When the user says, "What products do you recommend?", the device's microphone picks up the user's voice and sends it to the server. The server analyzes the text data and generates a response using GPT-3. The response might be something like, "Currently, we recommend this smartwatch. It's the latest model with a long battery life and comprehensive health management features."

[0184] The generated response is converted into voice data and sent to the terminal, which then plays the voice data so that the user can hear the voice of the virtual character.

[0185] 3. Examples of concrete examples and prompts

[0186] As a practical example, consider a situation where a user wears smart glasses and asks for assistance in a store. For example, the user says, "Hello, what products do you recommend?"

[0187] Example prompt sentence:

[0188] Customer Question: "Hi, what products do you recommend?"

[0189] Prompt for generative AI model: "A customer asks you for a product recommendation. How would you respond?"

[0190] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0191] Program processing steps

[0192] Step 1:

[0193] A user puts on the smart glasses and launches an application. The input is the initialization data for the smart glasses, and the output is the display of a login screen. Specifically, the terminal displays the login screen to the user, and the user enters login information.

[0194] Step 2:

[0195] The user enters login information and presses the login button. The input is the login information, and the output is an authentication request to the server. The terminal sends the user's input information to the server, and the server performs authentication.

[0196] Step 3:

[0197] The server authenticates the user's login information and returns the result to the device. The input is the user's login information, and the output is the authentication result. Specifically, the server searches the database and verifies the login information. If successful, the home screen is displayed on the device.

[0198] Step 4:

[0199] The user selects a character to interact with from the home screen. The input is character selection information, and the output is transmission of the selection information to the server. The device sends the selected character information to the server, and the server obtains the corresponding learning data.

[0200] Step 5:

[0201] The server retrieves the learning data for the selected character and sends it to the terminal. The input is the character selection information and the output is the learning data. The server retrieves the relevant data from the learning database and sends it to the terminal.

[0202] Step 6:

[0203] The user presses the "Start conversation" button and speaks into the device's microphone. The input is the user's voice, and the output is text data. The device uses voice recognition to convert the user's voice into text, and sends the text data to the server.

[0204] Step 7:

[0205] The server receives the user's text data and generates an appropriate response using a generative AI model. The input is the user's text data and the output is the generated response. The server generates the response using an NLP model (e.g., GPT-3).

[0206] Step 8:

[0207] The response generated by the server is converted into audio data and sent to the device. The input is the generated response and the output is audio data. The server uses a speech synthesis library (e.g., pyttsx3) to generate the audio data and send it to the device.

[0208] Step 9:

[0209] The terminal plays back the voice data received from the server and lets the user listen. The input is voice data, and the output is voice playback. The terminal plays back the voice data, allowing the user to hear the virtual character's response.

[0210] Step 10:

[0211] When the user wants to end the service, he / she performs an operation to end the application. The input is the end operation and the output is the end of the application. The terminal ends the application and stops the display on the smart glasses.

[0212] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0213] This invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as a non-fungible token (NFT), and enables real-time voice conversations with users 24 hours a day, 365 days a year. In addition, by combining this invention with an emotion engine, it can recognize the user's emotions and provide adaptive responses based on those emotions.

[0214] System configuration

[0215] 1. Learning with AI technology

[0216] The server collects data on the appearance, personality, and past behavior of specific characters and people, and uses an artificial intelligence algorithm to learn this information. The learning results, including the character's specific behavior and response patterns, are saved as a highly accurate model.

[0217] 2. Save as an NFT

[0218] The server stores the learned AI data as an NFT, and uses blockchain technology to guarantee its uniqueness and ownership.

[0219] 3. Providing an interface

[0220] The terminal provides the user with an interface for selecting a conversation partner. The user can log in and select a character of their choice.

[0221] 4. Real-time voice conversation

[0222] The user performs an operation to start a conversation with the selected character. The device captures the user's voice, converts it into text data, and sends it to the server.

[0223] The server uses an AI model to generate an appropriate response based on the text data it receives, which is then converted back into audio data and sent to the device.

[0224] The terminal plays the audio data and the conversation takes place in real time.

[0225] 5. Emotion engine integration

[0226] The server is equipped with an emotion engine that analyzes the user's emotional state (e.g., joy, sadness, anger, etc.) from the user's voice data.

[0227] Based on the results of the emotion analysis, the AI ​​model adaptively changes the response content. For example, if the user speaks in a sad voice, the model generates a response such as "Cheer up!"

[0228] 6. Fee collection and revenue sharing

[0229] The server collects usage fees from users. Payments are processed through the terminal. Revenues are distributed among the parties involved, including the copyright holders of characters and people, and service providers.

[0230] Specific examples

[0231] 1. User logs in and selects a character

[0232] The user opens the app and logs in by entering "user123" and "password123" on the login screen. The device sends the entered data to the server and authentication is performed. If authentication is successful, the home screen is displayed on the device.

[0233] The user presses the "Character Selection" button on the home screen and selects, for example, "Character A." The device sends the selected character information to the server, and the server retrieves the corresponding AI data and sends it to the device.

[0234] 2. Initiating and conducting actual conversations

[0235] The user presses the "Start conversation" button and speaks "Hello, Character A" into the device's microphone. The device converts the speech into text and sends it to the server.

[0236] The server analyzes the text data and generates an appropriate response based on the AI ​​model, such as "Hello! How's your day?"

[0237] The server simultaneously transmits the voice data to the emotion engine to analyze the user's emotional state.

[0238] The emotion engine analyzes the user's emotional state and feeds the results back to the AI ​​model. For example, if the user sounds sad, the response will be changed to an encouraging message like "Cheer up!"

[0239] The generated response is converted into voice data and sent to the terminal, which plays the voice data and provides the response to the user.

[0240] 3. Payment of fees and revenue sharing

[0241] If the user needs to pay a usage fee, the terminal displays a payment screen where the user enters credit card information, etc.

[0242] The terminal sends the entered payment information to the server, which then processes the payment through a payment gateway.

[0243] The server sends information about the success or failure of the payment to the terminal. If successful, the usage rights are updated.

[0244] The terminal will notify the user of the payment result, allowing them to continue using the system.

[0245] This system allows fans to safely and easily interact with their favorite characters and people 24 hours a day, 365 days a year, and by providing appropriate responses based on the user's emotions, it enables more personal and intimate communication.

[0246] The processing flow will be explained below.

[0247] Step 1:

[0248] The user opens the application. The user accesses the login screen and enters their ID and password.

[0249] Step 2:

[0250] The terminal receives the user's input and sends the ID and password information to the server. The communication is encrypted.

[0251] Step 3:

[0252] The server checks the received ID and password against the database and performs user authentication. If the ID and password do not match, an error message is sent to the terminal.

[0253] Step 4:

[0254] If the authentication is successful, the server generates session information and sends it back to the device. The device uses the received session information to display the home screen.

[0255] Step 5:

[0256] The user presses the "Select Character" button on the home screen and selects the character they want to interact with.

[0257] Step 6:

[0258] The device receives the user's selection and sends the character ID to the server, which then retrieves the AI ​​model data for the selected character.

[0259] Step 7:

[0260] The server sends the acquired AI model data to the device, which then displays a conversation interface with the character based on the received data.

[0261] Step 8:

[0262] The user presses the "Start conversation" button to start the conversation, and the device's microphone is turned on to capture the user's voice.

[0263] Step 9:

[0264] The device records the user's voice and converts the voice data into text data in real time. This text data is then sent to the server.

[0265] Step 10:

[0266] The server analyzes the received text data and generates a response using an AI model. For example, if the user says "Hello, Character A," the response is "Hello! How's your day?"

[0267] Step 11:

[0268] At the same time, the server sends the user's voice data to the emotion engine to analyze the user's emotional state. The emotion engine analyzes the voice data and identifies the user's emotional state (such as joy, sadness, anger, etc.).

[0269] Step 12:

[0270] The emotion engine sends the analysis results back to the server, which then adaptively changes the response based on the results. For example, if the user speaks in a sad voice, the response might be "Cheer up!"

[0271] Step 13:

[0272] The server converts the response text into voice data and sends it to the terminal.

[0273] Step 14:

[0274] The terminal plays back the received voice data and provides a response to the user, who can then ask questions or make comments.

[0275] Step 15:

[0276] If the user needs to pay a usage fee, the terminal displays a payment screen, and the user enters credit card information, etc.

[0277] Step 16:

[0278] The terminal sends the entered payment information to the server, and the server processes the payment through a payment gateway.

[0279] Step 17:

[0280] The server sends information about the success or failure of the payment to the terminal. If the payment is successful, the usage rights are updated.

[0281] Step 18:

[0282] The terminal notifies the user of the payment result, allowing the user to continue using the system.

[0283] Example 2

[0284] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0285] Conventional AI-based voice conversation systems have struggled to efficiently learn the appearance and personality of characters and people and conduct real-time voice conversations with users. Furthermore, the technology for providing adaptive responses based on the user's emotions has not been sufficiently developed, making it impossible to achieve natural, personal communication with users.

[0286] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0287] In this invention, the server includes means for learning the appearance and personality of a specific character or person using artificial intelligence technology, means for saving the learned data as a non-fungible token (NFT), and means for analyzing emotions from the user's voice and adaptively changing responses based on the analysis results. This makes it possible to learn the characteristics of characters and people with high accuracy, safely save that data, and provide flexible responses according to the user's emotions.

[0288] "Specific character or person" refers to a fictional or real person or character that a user can select as a conversation partner.

[0289] "Artificial intelligence technology" refers to the use of techniques such as machine learning and deep learning to extract patterns from data and make predictions and classifications.

[0290] "Means of learning" refers to the process of collecting data and generating an AI model based on that data.

[0291] A "non-fungible token (NFT)" refers to a token that uses blockchain technology to guarantee the uniqueness and ownership of digital data.

[0292] "Means for providing an interface" refers to the method of providing screens and menus that allow users to access and operate the system.

[0293] "Means for conducting voice conversation in real time" refers to technology that allows a user and a character to have an instantaneous voice conversation.

[0294] "Means for analyzing emotions from a user's voice" refers to technology that analyzes a user's emotional state based on their voice data.

[0295] "Means for adaptively changing responses" refers to a method for changing the content of AI responses based on the results of emotion analysis.

[0296] "Means for collecting usage fees and distributing revenue among the parties involved" refers to a mechanism for collecting service usage fees from users and distributing the revenue obtained among the parties involved.

[0297] MODE FOR CARRYING OUT THE INVENTION

[0298] This invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as non-fungible tokens (NFTs), and provides users with real-time voice conversations 24 hours a day, 365 days a year. Furthermore, by combining it with an emotion engine, it can recognize the user's emotions and provide adaptive responses based on those emotions.

[0299] System Overview

[0300] The system consists of the following main components:

[0301] 1. Server

[0302] 2. Terminal

[0303] 3. Users

[0304] Server Roles

[0305] 1. Collecting training data and generating AI models

[0306] The server collects data on the appearance, personality, and past behavior of specific characters or people, including video, audio, and text data.

[0307] The collected data is trained using an AI algorithm (e.g., TensorFlow or PyTorch). At the initial training stage, data preprocessing (noise removal, text normalization, etc.) is performed.

[0308] Save the completed model to the database.

[0309] 2. Data storage as NFTs

[0310] The trained AI data is stored as a non-fungible token (NFT), and this process uses blockchain technology (e.g., Ethereum).

[0311] The server registers the generated NFT on the blockchain and maintains the owner information.

[0312] 3. Sentiment analysis and response adaptation

[0313] The server uses an emotion engine (for example, IBM Watson's emotion recognition API) to analyze the user's voice data. Based on the analysis results, the AI ​​model adaptively changes the response content.

[0314] For example, if a user says, "I'm feeling a bit down today," emotion recognition can generate a response such as, "Cheer up!"

[0315] 4. Usage Fee Collection and Revenue Sharing

[0316] The server collects the usage fee and manages the payment process, which is carried out using credit card information entered by the user on the terminal.

[0317] Revenues are distributed among the parties involved (character copyright holders, service providers, etc.).

[0318] Device Role

[0319] 1. Providing an interface

[0320] The device provides the user with an interface to select a conversation partner and conduct a conversation. The user launches the app, enters their authentication information on the login screen, and logs in.

[0321] After logging in, users can press the "Select Character" button to choose their preferred character and begin a conversation with that character.

[0322] 2. Audio Data Processing

[0323] The device captures the user's voice and converts it into text data using a speech recognition engine such as Google Speech-to-Text.

[0324] The converted text data is sent to the server, and after an appropriate response is generated, it is converted back into voice data and sent to the terminal.

[0325] The terminal plays the audio data, enabling real-time conversation.

[0326] User Roles

[0327] 1. Log in and select a character

[0328] The user launches the app and enters authentication information such as "user123" and "password123" on the login screen.

[0329] Once authentication is successful, select a character from the home screen and begin a conversation with the selected character.

[0330] 2. Actual conversation progression

[0331] The user presses the "Start conversation" button and speaks into the device's microphone, saying "Hello, Character A."

[0332] The device converts this speech into text data and sends it to the server, which uses the text data to generate a response using an AI model and sends an adaptive reply.

[0333] For example, you can receive responses such as "Hello! How's your day?" or adaptive messages such as "Cheer up!"

[0334] Specific examples and prompt sentence examples

[0335] "Hello, Character A. How was your day?"

[0336] "I've been feeling a bit down lately. Can you cheer me up?"

[0337] "What do you think about today's news?"

[0338] By using these prompts, users can enjoy more personal and natural conversations with the characters.

[0339] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0340] Program processing flow

[0341] Step 1:

[0342] Learning data collection and AI model generation

[0343] The server collects data on the appearance, personality, and past behavior of specific characters or people. The data used for this collection includes text, audio files, images, and videos. Specifically, the data is retrieved from a database via web scraping or API and stored on the server.

[0344] Input: Data about the character's appearance, personality, and past behavior.

[0345] Data processing: noise removal, text normalization, data cleaning.

[0346] Output: A clean dataset after preprocessing.

[0347] The server uses AI algorithms (such as TensorFlow or PyTorch) to train the model based on this data. The trained model then reflects the character's behavior and response patterns.

[0348] Input: Clean dataset after preprocessing.

[0349] Data computation: Applying machine learning algorithms and training models.

[0350] Output: A trained AI model.

[0351] Step 2:

[0352] Storing data as NFTs

[0353] The server stores the learned AI data as a non-fungible token (NFT).

[0354] Input: A trained AI model.

[0355] Data processing: Blockchain technology (e.g., Ethereum) is used to guarantee the uniqueness and ownership of the model for NFT conversion.

[0356] Output: An NFT registered on the blockchain.

[0357] The server registers the generated NFT on the blockchain and stores the owner information. Specifically, the server registers the NFT through the blockchain API.

[0358] Step 3:

[0359] User login and character selection

[0360] The user launches the app and enters their authentication information (e.g., "user123" and "password123") on the login screen.

[0361] The terminal transmits the input authentication information to the server and performs authentication.

[0362] Input: User credentials.

[0363] Data calculation: Check against the database to determine whether authentication is successful.

[0364] Output: Authentication success or failure status.

[0365] If authentication is successful, the home screen will be displayed on the device, and the user can press the "Select Character" button to choose the character they want to interact with.

[0366] Input: Character information selected by the user.

[0367] Data processing: Search and obtain the AI ​​model corresponding to the selected character.

[0368] Output: AI data of the selected character.

[0369] Step 4:

[0370] Real-time voice conversation processing

[0371] The user presses the "Start conversation" button and speaks into the device's microphone.

[0372] The device captures the user's voice and converts it into text data using Google Speech-to-Text.

[0373] Input: User's voice data.

[0374] Data processing: speech-to-text conversion.

[0375] Output: Text data.

[0376] The converted text data is sent to the server, which generates an appropriate response.

[0377] Input: User's text data.

[0378] Data Computation: Using AI models to generate appropriate responses.

[0379] Output: The generated text response.

[0380] The response is converted back into voice data and sent to the terminal.

[0381] Input: The generated text response.

[0382] Data processing: Text to speech conversion.

[0383] Output: The generated audio data.

[0384] The terminal plays the audio data and the conversation takes place in real time.

[0385] Step 5:

[0386] Sentiment analysis and response adaptation

[0387] The server analyzes the user's voice data using an emotion engine (e.g., IBM Watson's emotion recognition API).

[0388] Input: User's voice data.

[0389] Data calculation: Analyze the user's emotional state from voice data.

[0390] Output: User's emotional state data.

[0391] Based on the results of the sentiment analysis, the AI ​​model adaptively changes the response content.

[0392] Input: User emotional state data and generated text responses.

[0393] Data calculation: Modifying the content of responses to match emotional states.

[0394] Output: The final adapted response text.

[0395] For example, if the user sounds sad, the response may be changed to something like "Cheer up!"

[0396] Step 6:

[0397] Collection of usage fees and payment processing

[0398] If the user needs to pay a usage fee, the terminal displays a payment screen where the user enters credit card information, etc.

[0399] Input: User's payment information.

[0400] The terminal sends payment information to the server, and the server processes the payment through a payment gateway (e.g., Stripe).

[0401] Data calculation: Executes payment processing based on payment information.

[0402] Output: Payment success or failure status.

[0403] The server notifies the terminal of the success or failure of the payment, and if successful, the usage rights are updated.

[0404] The terminal notifies the user of the payment result, allowing the user to continue using the system.

[0405] (Application example 2)

[0406] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0407] The objective of this invention is to provide a system that can reproduce the appearance and personality of a specific character or person in real time, while providing appropriate responses according to the user's emotions through dialogue with the character. Furthermore, by having the character act as a shopping assistant in a virtual store, explaining products and recommending items, we aim to realize a more personalized and sympathetic shopping experience.

[0408] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0409] In this invention, the server includes: means for learning the appearance and personality of a specific character or person using artificial intelligence technology; means for storing the learned data as a non-fungible token (NFT); means for providing an interface for a user to select a conversation partner and converse with them; means for conducting real-time voice conversations; means for collecting usage fees from users and distributing revenue among the parties; means for analyzing the emotional state from the user's voice data using an emotion engine and generating an adaptive response based on the analysis results; means for converting the generated response into voice data and providing it to the user as voice; and means for the character to act as a shopping assistant in a virtual store, explaining products and recommending items. This enables real-time, emotion-based conversations with a specific character and a personalized, sympathetic shopping assistant experience in a virtual store.

[0410] "Specific characters or people" are virtual beings whose appearances and personalities are reproduced using artificial intelligence technology.

[0411] "Artificial intelligence technology" is a technology that learns the appearance and personality of a specific character or person and generates responses and actions based on that information.

[0412] A non-fungible token (NFT) is a means of uniquely identifying and proving ownership of a specific digital asset using blockchain technology.

[0413] An "interface" is a device or software that provides the input and display means for a user to select a conversation partner and initiate a conversation.

[0414] "Means for conducting voice conversations in real time" refers to technology that enables a user to have an instant voice conversation with a specific character or person.

[0415] An "emotion engine" is an algorithm and system that analyzes a user's emotional state from their voice data and generates a response based on the analysis results.

[0416] An "adaptive response" is a response that changes depending on the user's utterance content and emotional state, and is generated individually and dynamically.

[0417] A "virtual store" is a virtual shopping environment that exists on the Internet, and is a platform that allows users to make purchases in a virtual space.

[0418] A "shopping assistant" is a virtual character or person who explains products and recommends items to users in a virtual store.

[0419] "Collection" means receiving usage fees from users.

[0420] "Means for distributing revenue among parties involved" refers to a mechanism for appropriately distributing revenue obtained through system operation among parties involved, such as character copyright holders and service providers.

[0421] System Overview

[0422] This invention is a system that reproduces specific characters or people in real time and engages in voice conversations with users. It mainly functions as a shopping assistant in virtual stores. This system consists of three elements: a server, a terminal, and a user.

[0423] Program structure and various technologies

[0424] Speech Recognition Engine

[0425] The server receives the user's voice input and converts the voice into text data using a speech recognition engine (Google Speech-to-Text API).

[0426] Sentiment Analysis Engine

[0427] The server performs emotion analysis on the acquired text data using IBM Watson Tone Analyzer, thereby determining the user's emotional state.

[0428] Dialogue Management System

[0429] The server uses Rasa to generate responses based on the user's conversation content and sentiment analysis results.

[0430] Speech synthesis engine

[0431] The server converts the generated response into audio data using the Google Text-to-Speech API and sends it to the device.

[0432] Program processing

[0433] 1. Voice to Text

[0434] When a user speaks into the smartphone's microphone, the audio is picked up by the device and converted into text data using the Google Speech-to-Text API.

[0435] 2. Emotion analysis

[0436] The transferred text data undergoes emotion analysis using IBM Watson Tone Analyzer to analyze the user's emotional state (e.g., joy, sadness, anger, etc.).

[0437] 3. Response Generation

[0438] It uses Rasa to generate appropriate responses based on the analysis and conversational context, which are adapted to the user's emotions.

[0439] 4. Conversion to audio data and transmission

[0440] The generated response is converted into audio data using the Google Text-to-Speech API, and this audio data is sent to the device, enabling real-time conversation.

[0441] Specific examples

[0442] 1. User login and character selection

[0443] The user starts the app and logs in by entering "user123" and "password123." After logging in, the user selects "Character A" on the character selection screen.

[0444] Example prompt sentence:

[0445] User: Hello, I'm looking for some clothes today.

[0446] ---

[0447] Assistant: Hello, what kind of clothes are you looking for?

[0448] ---

[0449] User: I'd like a casual shirt.

[0450] ---

[0451] 2. Voice input and analysis

[0452] When a user says, "Hello, I'm looking for clothes today," the speech is captured and converted into text data, which is sent to IBM Watson Tone Analyzer for sentiment analysis.

[0453] 3. Response Generation and Serving

[0454] Based on sentiment analysis and the conversation, Rasa generates a response such as "Hello, what kind of clothes are you looking for?" This response is converted to audio using the Google Text-to-Speech API and sent to the device.

[0455] 4. Shopping Assistant

[0456] If a user says, "I want a casual shirt," the assistant will suggest, "How about this shirt? It has a trendy design and is very popular these days," and explain the product. It is also possible to provide more detailed product descriptions and suggestions based on the results of user sentiment analysis.

[0457] This allows the invention to provide emotion-based real-time interaction with specific characters and a personalized and sympathetic shopping assistant experience in virtual stores.

[0458] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0459] Step 1:

[0460] The user launches the app and logs in. The user enters their username and password, and the device sends that information to the server. The server performs authentication, and if successful, displays the character selection screen on the device.

[0461] Input: User's username and password

[0462] Process: The server authenticates the user

[0463] Output: Character selection screen will be displayed upon successful authentication.

[0464] Step 2:

[0465] The user selects a character. The user chooses a specific character on the character selection screen, and the device sends the selection information to the server. The server loads the AI ​​model of the selected character and sends the related data to the device.

[0466] Input: User's character selection information

[0467] Processing: The server loads the AI ​​model and retrieves the relevant data.

[0468] Output: Character information is displayed on the terminal.

[0469] Step 3:

[0470] The user starts a conversation. The user speaks into the smartphone's microphone, and the device picks up the audio. The speech is converted into text data using a speech recognition engine (Google Speech-to-Text API), and the text data is sent to the server.

[0471] Input: User's voice

[0472] Processing: The device receives the voice and converts it into text data.

[0473] Output: The converted text data is sent to the server.

[0474] Step 4:

[0475] The server performs emotion analysis. The server sends the acquired text data to an emotion analysis engine (IBM Watson Tone Analyzer) to analyze the user's emotional state. The analysis results are added to the text data.

[0476] Input: Text data

[0477] Processing: The server analyzes the emotional state using an emotion analysis engine.

[0478] Output: Text data with emotional states added

[0479] Step 5:

[0480] The server generates a response. The server uses a dialogue management system (Rasa) to generate a response based on the sentiment analysis results and the conversation content. The response is in text format.

[0481] Input: Text data with sentiment analysis results added

[0482] Processing: The server generates a response using a dialogue management system

[0483] Output: The generated response text

[0484] Step 6:

[0485] The server converts the response text into speech. The generated response text is converted into audio data using a speech synthesis engine (Google Text-to-Speech API). The converted audio data is then sent back to the device.

[0486] Input: Generated response text

[0487] Processing: The server uses a speech synthesis engine to convert the data into voice data.

[0488] Output: Converted audio data

[0489] Step 7:

[0490] The device plays the audio data. The audio data sent to the device is played back through the speaker and can be heard by the user. This allows for real-time audio conversation between the user and the character.

[0491] Input: Audio data

[0492] Processing: The device plays the audio data.

[0493] Output: The user hears the response

[0494] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0495] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0496] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0497] [Second embodiment]

[0498] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0499] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0500] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0501] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0502] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0503] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0504] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0505] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0506] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0507] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0508] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0509] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0510] The present invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as a non-fungible token (NFT), and enables users to have real-time voice conversations 24 hours a day, 365 days a year. The system is configured as follows:

[0511] System configuration

[0512] 1. Learning with AI technology

[0513] The server collects data on the appearance, personality, and past behavior of specific characters or people, and uses artificial intelligence algorithms to learn this information.

[0514] The learning results include the character's specific behavioral and response patterns and are saved as a highly accurate model.

[0515] 2. Save as an NFT

[0516] The server stores the learned AI data as an NFT, and uses blockchain technology to guarantee its uniqueness and ownership.

[0517] 3. Providing an interface

[0518] The terminal provides the user with an interface for selecting a conversation partner. The user can log in and select a character of their choice.

[0519] 4. Real-time voice conversation

[0520] The user performs an operation to start a conversation with the selected character. The device captures the user's voice, converts it into text data, and sends it to the server.

[0521] The server uses an AI model to generate an appropriate response based on the text data it receives, which is then converted back into audio data and sent to the device.

[0522] The terminal plays the audio data and the conversation takes place in real time.

[0523] 5. Fee collection and revenue sharing

[0524] The server collects the usage fee from the user, and the payment process is carried out through the terminal.

[0525] Revenues are shared among the parties involved, including copyright holders of characters and people, and service providers.

[0526] Specific examples

[0527] 1. User logs in and selects a character

[0528] The user opens the app and logs in by entering "user123" and "password123" on the login screen. The device sends the entered data to the server and authentication is performed. If authentication is successful, the home screen is displayed on the device.

[0529] The user presses the "Character Selection" button on the home screen and selects, for example, "Character A." The device sends the selected character information to the server, and the server retrieves the corresponding AI data and sends it to the device.

[0530] 2. Initiating and conducting actual conversations

[0531] The user presses the "Start conversation" button and speaks "Hello, Character A" into the device's microphone. The device converts the speech into text and sends it to the server.

[0532] The server analyzes the text data and generates an appropriate response based on the AI ​​model, for example, "Hello, I'm Character A. How's your day?"

[0533] The generated response is converted into audio data and sent to the terminal, which plays the audio data so that the user can hear the response.

[0534] 3. Payment of fees and revenue sharing

[0535] When a user pays for a service, the terminal captures the payment information and sends it to the server, which processes the payment through a payment gateway and distributes the revenue among the parties after verification.

[0536] This system allows fans to safely and easily interact with their favorite characters and people 24 hours a day, 365 days a year, and also reduces the high costs associated with hosting events and inviting celebrities to attend.

[0537] The processing flow will be explained below.

[0538] Step 1:

[0539] The user opens the application. The user accesses the login screen and enters their ID and password.

[0540] Step 2:

[0541] The terminal receives the user's input and sends the ID and password information to the server. The communication is encrypted.

[0542] Step 3:

[0543] The server checks the received ID and password against the database and performs user authentication. If the ID and password do not match, an error message is sent to the terminal.

[0544] Step 4:

[0545] If the authentication is successful, the server generates session information and sends it back to the device. The device uses the received session information to display the home screen.

[0546] Step 5:

[0547] The user presses the "Select Character" button on the home screen and selects the character they want to interact with.

[0548] Step 6:

[0549] The device receives the user's selection and sends the character ID to the server, which then retrieves the AI ​​model data for the selected character.

[0550] Step 7:

[0551] The server sends the acquired AI model data to the device, which then displays a conversation interface with the character based on the received data.

[0552] Step 8:

[0553] The user presses the "Start conversation" button to start the conversation, and the device's microphone is turned on to capture the user's voice.

[0554] Step 9:

[0555] The device records the user's voice and converts the voice data into text data in real time. This text data is then sent to the server.

[0556] Step 10:

[0557] The server analyzes the received text data and generates a response using an AI model. For example, if the user says "Hello, Character A," the response is "Hello! How's your day?"

[0558] Step 11:

[0559] The response text generated by the server is converted into voice data and sent to the terminal.

[0560] Step 12:

[0561] The terminal plays back the received voice data and provides a response to the user, who can then ask questions or make comments.

[0562] Step 13:

[0563] If the user needs to pay a usage fee, the terminal displays a payment screen, and the user enters credit card information, etc.

[0564] Step 14:

[0565] The terminal sends the entered payment information to the server, and the server processes the payment through a payment gateway.

[0566] Step 15:

[0567] The server sends information about the success or failure of the payment to the terminal. If the payment is successful, the usage rights are updated.

[0568] Step 16:

[0569] The terminal notifies the user of the payment result, allowing the user to continue using the system.

[0570] Example 1

[0571] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0572] Conventional dialogue systems with characters or people have difficulty in real-time voice conversation, and have been unable to provide a satisfying experience to users. Furthermore, there was a lack of means to guarantee the uniqueness and ownership of trained AI data, creating the risk of data tampering or unauthorized use. Furthermore, there was no established mechanism for collecting usage fees from users and distributing revenue fairly among the parties involved. New technologies are needed to solve these problems.

[0573] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0574] In this invention, the server includes: means for learning the appearance and personality of a specific character or person using artificial intelligence technology; means for storing the learned data as a non-fungible token (NFT); means for providing an interface for a user to select a conversation partner and converse with them; means for acquiring the user's voice and converting it into text data using voice recognition technology; means for generating an appropriate response based on the text data using an AI model and converting the response into voice data; means for transmitting the converted voice data to a user terminal to conduct a real-time voice conversation; and means for collecting usage fees from users and distributing revenue among the parties involved. This enables real-time voice conversation, guarantees the uniqueness and ownership of the learned data, and enables fair revenue distribution.

[0575] A "specific character or person" refers to a fictional or real person that a user selects when interacting with the character, whose appearance, personality, behavior patterns, etc. are predefined.

[0576] "Artificial intelligence technology" refers to technology that uses machine learning and deep learning models to learn from specific data and generate appropriate responses even for unknown data.

[0577] A "non-fungible token (NFT)" is a unique digital asset created using blockchain technology with guaranteed ownership and authenticity.

[0578] "Interface" refers to the user interface provided to users to select and interact with characters or people, and is usually provided as an application.

[0579] "Speech recognition technology" is a technology that converts a user's voice into text data, and examples include Google Cloud Speech-to-Text and Microsoft Azure Speech Recognition Services.

[0580] An "AI model" is a neural network model trained on collected data and used to generate responses for a specific character or person.

[0581] "Converting to voice data" means converting the text response generated by the AI ​​model into voice data using speech synthesis technology, making it audible to the user.

[0582] "Having a real-time voice conversation" refers to a form in which a user can instantly communicate with a character or person using voice.

[0583] "Collecting fees" means collecting fees from users for using the service, often through an online payment system.

[0584] "Distributing revenues among the parties involved" means distributing the collected usage fees to the copyright holders of characters and people, service providers, and other rights holders in a fair proportion.

[0585] MODE FOR CARRYING OUT THE INVENTION

[0586] The present invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as a non-fungible token (NFT), and enables users to have real-time voice conversations 24 hours a day, 365 days a year. The system is configured as follows:

[0587] Learning with AI technology

[0588] The server collects data on the appearance, personality, and past behavior of specific characters or people. For example, it uses scripts to obtain data from YouTube, social media, etc. This data is trained using the TensorFlow or PyTorch machine learning framework. During the data preprocessing process, noise removal, tokenization, normalization, etc. are performed to build a neural network model. The trained model undergoes accuracy evaluation and is saved if it is highly accurate.

[0589] Stored as an NFT

[0590] The server converts the trained AI data into ONNX format to store it as an NFT, and then issues the NFT using a blockchain platform such as Ethereum or Binance Smart Chain, incorporating the model's unique identifier and metadata in the process.

[0591] Providing an interface

[0592] The device provides the user with an interactive user interface. The user logs in using an application developed with React Native or Flutter. The login screen has username and password input fields for the user to authenticate. If authentication is successful, a character selection screen is displayed, where the user selects the character they want to interact with.

[0593] Real-time voice conversation

[0594] The user initiates a conversation with the selected character. The device uses the microphone to capture the user's voice and converts it into text using the Google Cloud Speech-to-Text API or Microsoft Azure Speech Recognition Service. The text is sent to a server, which uses an AI model (e.g., GTP-3 or BERT) to generate an appropriate response. This response is converted into audio data by a speech synthesis engine (e.g., Amazon Polly or Google Text-to-Speech) and sent to the device. The device plays back the audio data, enabling real-time conversation.

[0595] Collection of usage fees and revenue sharing

[0596] The usage fee paid by users when using the service is collected by the terminal acquiring payment information. The payment processing is carried out by the server using the API of Stripe or PayPal. The usage fee collected is distributed fairly to the copyright holders of the characters and people and the service provider.

[0597] Specific examples

[0598] 1. User login and character selection

[0599] The user opens the app and logs in by entering "user123" and "password123" on the login screen. The device sends the input data to the server and authentication is performed. If authentication is successful, the home screen is displayed.

[0600] The user presses the "Character Selection" button on the home screen and selects "Character A." The device sends the selected character information to the server, and the server retrieves the corresponding AI data and sends it to the device.

[0601] 2. Initiating and conducting actual conversations

[0602] The user presses the "Start conversation" button and speaks "Hello, Character A" into the device's microphone. The device converts the speech into text and sends it to the server.

[0603] The server analyzes the text data and generates an appropriate response based on the AI ​​model, for example, "Hello, I'm Character A. How's your day?"

[0604] The generated response is converted into audio data and sent to the terminal, which plays the audio data so that the user can hear the response.

[0605] Prompt Sentence Examples

[0606] "Hi, I'm Character A. How's your day going?"

[0607] This system allows fans to safely and easily interact with their favorite characters and people 24 hours a day, 365 days a year, and also reduces the high costs associated with hosting events and inviting celebrities to attend.

[0608] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0609] Step 1: Data collection

[0610] The server collects data on the appearance, personality, and past behavior of specific characters or people. Inputs include YouTube video links and social media post URLs. The data is obtained using scraping technology and APIs. The output is the retrieved text, images, and audio data.

[0611] Step 2: Data Preprocessing

[0612] The server preprocesses the collected data. The raw data collected in step 1 is provided as input. Specifically, processes such as noise removal, text tokenization, and normalization are performed. For example, unnecessary special characters are removed from the text data and it is converted into a standard format. The output is the preprocessed data.

[0613] Step 3: Training the model

[0614] The server uses the preprocessed data to train an artificial intelligence model (e.g., TensorFlow or PyTorch). The preprocessed data from step 2 is provided as input. Specifically, the server splits the data into batches and optimizes the model parameters using a GPU. The output is a trained, highly accurate model.

[0615] Step 4: Evaluate and save the model

[0616] The server evaluates and saves the trained model. The model trained in step 3 and the validation dataset are used as input. Specifically, the accuracy of the model is evaluated using the validation data, and if the accuracy meets the criteria, the model is saved. The output is the evaluation result and the saved model.

[0617] Step 5: Save as an NFT

[0618] The server saves the trained AI model as an NFT. The model data saved in step 4 is used as input. Specifically, the server converts the model to ONNX format and issues an NFT on a blockchain platform (e.g., Ethereum or Binance Smart Chain). The output is the model data saved as an NFT.

[0619] Step 6: User authentication and character selection

[0620] The terminal displays a login screen for the user and performs authentication. A username and password are provided as input. The authentication data is sent to the server and checked against a database. If authentication is successful, a character selection screen is displayed. The output is the selected character information.

[0621] Step 7: Capturing and Converting Audio Input

[0622] The device uses a microphone to capture the user's voice. The user's voice data is provided as input. Specifically, the voice is converted into text data using the Google Cloud Speech-to-Text API. The output is text data.

[0623] Step 8: Response Generation

[0624] The server receives the text data and generates a response using an AI model. The text data converted in step 7 is provided as input. Specifically, the AI ​​model (e.g., GPT-3 or BERT) analyzes the text data and generates an appropriate response. The output is the text data of the generated response.

[0625] Step 9: Convert to audio and send

[0626] The server converts the generated text response into audio data and sends it to the device. The text data generated in step 8 is provided as input. Specifically, it converts it into audio data using Amazon Polly or Google Text-to-Speech. The output is audio data.

[0627] Step 10: Playing back real-time voice conversations

[0628] The device receives the audio data and plays it on the speaker. The input is the audio data sent in step 9. Specifically, it decodes the audio data and plays it on the speaker so that it can be heard by the user. The output is audio played in real time.

[0629] Step 11: Fee collection and distribution

[0630] The terminal obtains payment information to collect usage fees and sends it to the server. Payment information is provided as input. The server processes the payment using Stripe or PayPal APIs and distributes revenue among the parties involved. Specifically, after confirming the payment process, the distribution is made via bank transfer or cryptocurrency. The output is the distributed revenue.

[0631] (Application example 1)

[0632] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0633] Currently, providing high-quality customer service to customers visiting physical stores requires a large amount of labor and resources. It is also difficult to provide detailed information about specific products in the store, making it difficult to increase customer satisfaction. Furthermore, while there is a demand for personalized guidance that allows customers to select products based on their preferences and interests, achieving this is complex. To solve these challenges, a system that can interact with customers in real time and provide personalized guidance is needed.

[0634] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0635] In this invention, the server includes: means for learning the characteristics of a specific virtual character using machine learning technology; means for saving the learned information as a non-fungible character (NFT); means for providing an operation screen for a user to select a conversation partner and conduct a conversation; means for real-time voice communication; means for receiving a service fee from the user and distributing the revenue among the relevant parties; means for providing guidance using a display device that supports conversations between customers and virtual characters in physical stores; and means for users to obtain product information in the store and receive support through the character. This makes it possible to improve the customer experience in physical stores and provide personalized, high-quality customer service.

[0636] "Characteristics of a specific virtual person" refers to data that includes information such as the appearance, personality, and past words and actions of a character or celebrity.

[0637] "Machine learning technology" is a technology that allows computers to automatically learn from data and make predictions and classifications.

[0638] A "non-fungible token (NFT)" is a digital asset whose uniqueness and ownership are guaranteed using blockchain technology.

[0639] The "operation screen" is a display screen that provides an interface for a user to select a conversation partner and carry out a conversation.

[0640] "Real-time voice communication" refers to a technology that allows users to instantly communicate with selected characters via voice.

[0641] "Service fee" means the amount of money paid by a user as consideration for using a service.

[0642] "Related parties" refers to people or organizations that are subject to revenue sharing, such as character or system providers and developers.

[0643] A "display device" refers to hardware for displaying information, such as displays installed in physical stores, smart glasses, and head-mounted displays (HMDs).

[0644] "Supporting conversations between virtual characters" refers to a specific virtual character communicating with a user via voice to provide guidance and information.

[0645] "Get product information" means viewing or obtaining detailed information about products in the store.

[0646] "Receiving support through a character" means that a virtual person guides or explains products or services to the user.

[0647] To implement the present invention, the following system configuration is required: The system is composed of a server, a terminal, and a user as its main components.

[0648] 1. System Program

[0649] The server builds a database that uses machine learning technology to learn the characteristics of a specific virtual person. The terminal provides an operation screen that allows the user to select a conversation partner and engage in real-time voice communication with that virtual person. The server also stores the learned information as a non-fungible token (NFT).

[0650] The terminal displays an operation screen for the user to select a virtual person to talk to and transmits the selection information to the server. When the user communicates with the terminal in real time, the terminal converts the user's voice into text information and transmits the text information to the server.

[0651] The server then feeds the received text information to a machine learning model to generate a conversation-based response. The generated response is then converted into voice data by the server and sent to the device. The device then plays the voice data and provides the user with a response from a virtual character.

[0652] 2. Processing Description

[0653] The server is implemented using programming languages ​​such as Python, and uses machine learning technologies such as the NLP (natural language processing) model GPT-3. It also uses libraries such as Google Speech Recognition for speech recognition and pyttsx3 for speech synthesis.

[0654] The device includes hardware such as smart glasses, a head-mounted display (HMD), and a microphone. The device recognizes the user's speech and sends it to the server. The server converts the speech to text and generates an appropriate response using an NLP model. The server then converts the response back into audio data and sends it back to the device.

[0655] As a specific example, a user enters a physical store and puts on smart glasses. When the user says, "What products do you recommend?", the device's microphone picks up the user's voice and sends it to the server. The server analyzes the text data and generates a response using GPT-3. The response might be something like, "Currently, we recommend this smartwatch. It's the latest model with a long battery life and comprehensive health management features."

[0656] The generated response is converted into voice data and sent to the terminal, which then plays the voice data so that the user can hear the voice of the virtual character.

[0657] 3. Examples of concrete examples and prompts

[0658] As a practical example, consider a situation where a user wears smart glasses and asks for assistance in a store. For example, the user says, "Hello, what products do you recommend?"

[0659] Example prompt sentence:

[0660] Customer Question: "Hi, what products do you recommend?"

[0661] Prompt for generative AI model: "A customer asks you for a product recommendation. How would you respond?"

[0662] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0663] Program processing steps

[0664] Step 1:

[0665] A user puts on the smart glasses and launches an application. The input is the initialization data for the smart glasses, and the output is the display of a login screen. Specifically, the terminal displays the login screen to the user, and the user enters login information.

[0666] Step 2:

[0667] The user enters login information and presses the login button. The input is the login information, and the output is an authentication request to the server. The terminal sends the user's input information to the server, and the server performs authentication.

[0668] Step 3:

[0669] The server authenticates the user's login information and returns the result to the device. The input is the user's login information, and the output is the authentication result. Specifically, the server searches the database and verifies the login information. If successful, the home screen is displayed on the device.

[0670] Step 4:

[0671] The user selects a character to interact with from the home screen. The input is character selection information, and the output is transmission of the selection information to the server. The device sends the selected character information to the server, and the server obtains the corresponding learning data.

[0672] Step 5:

[0673] The server retrieves the learning data for the selected character and sends it to the terminal. The input is the character selection information and the output is the learning data. The server retrieves the relevant data from the learning database and sends it to the terminal.

[0674] Step 6:

[0675] The user presses the "Start conversation" button and speaks into the device's microphone. The input is the user's voice, and the output is text data. The device uses voice recognition to convert the user's voice into text, and sends the text data to the server.

[0676] Step 7:

[0677] The server receives the user's text data and generates an appropriate response using a generative AI model. The input is the user's text data and the output is the generated response. The server generates the response using an NLP model (e.g., GPT-3).

[0678] Step 8:

[0679] The response generated by the server is converted into audio data and sent to the device. The input is the generated response and the output is audio data. The server uses a speech synthesis library (e.g., pyttsx3) to generate the audio data and send it to the device.

[0680] Step 9:

[0681] The terminal plays back the voice data received from the server and lets the user listen. The input is voice data, and the output is voice playback. The terminal plays back the voice data, allowing the user to hear the virtual character's response.

[0682] Step 10:

[0683] When the user wants to end the service, he / she performs an operation to end the application. The input is the end operation and the output is the end of the application. The terminal ends the application and stops the display on the smart glasses.

[0684] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0685] This invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as a non-fungible token (NFT), and enables real-time voice conversations with users 24 hours a day, 365 days a year. In addition, by combining this invention with an emotion engine, it can recognize the user's emotions and provide adaptive responses based on those emotions.

[0686] System configuration

[0687] 1. Learning with AI technology

[0688] The server collects data on the appearance, personality, and past behavior of specific characters and people, and uses an artificial intelligence algorithm to learn this information. The learning results, including the character's specific behavior and response patterns, are saved as a highly accurate model.

[0689] 2. Save as an NFT

[0690] The server stores the learned AI data as an NFT, and uses blockchain technology to guarantee its uniqueness and ownership.

[0691] 3. Providing an interface

[0692] The terminal provides the user with an interface for selecting a conversation partner. The user can log in and select a character of their choice.

[0693] 4. Real-time voice conversation

[0694] The user performs an operation to start a conversation with the selected character. The device captures the user's voice, converts it into text data, and sends it to the server.

[0695] The server uses an AI model to generate an appropriate response based on the text data it receives, which is then converted back into audio data and sent to the device.

[0696] The terminal plays the audio data and the conversation takes place in real time.

[0697] 5. Emotion engine integration

[0698] The server is equipped with an emotion engine that analyzes the user's emotional state (e.g., joy, sadness, anger, etc.) from the user's voice data.

[0699] Based on the results of the emotion analysis, the AI ​​model adaptively changes the response content. For example, if the user speaks in a sad voice, the model generates a response such as "Cheer up!"

[0700] 6. Fee collection and revenue sharing

[0701] The server collects usage fees from users. Payments are processed through the terminal. Revenues are distributed among the parties involved, including the copyright holders of characters and people, and service providers.

[0702] Specific examples

[0703] 1. User logs in and selects a character

[0704] The user opens the app and logs in by entering "user123" and "password123" on the login screen. The device sends the entered data to the server and authentication is performed. If authentication is successful, the home screen is displayed on the device.

[0705] The user presses the "Character Selection" button on the home screen and selects, for example, "Character A." The device sends the selected character information to the server, and the server retrieves the corresponding AI data and sends it to the device.

[0706] 2. Initiating and conducting actual conversations

[0707] The user presses the "Start conversation" button and speaks "Hello, Character A" into the device's microphone. The device converts the speech into text and sends it to the server.

[0708] The server analyzes the text data and generates an appropriate response based on the AI ​​model, such as "Hello! How's your day?"

[0709] The server simultaneously transmits the voice data to the emotion engine to analyze the user's emotional state.

[0710] The emotion engine analyzes the user's emotional state and feeds the results back to the AI ​​model. For example, if the user sounds sad, the response will be changed to an encouraging message like "Cheer up!"

[0711] The generated response is converted into voice data and sent to the terminal, which plays the voice data and provides the response to the user.

[0712] 3. Payment of fees and revenue sharing

[0713] If the user needs to pay a usage fee, the terminal displays a payment screen where the user enters credit card information, etc.

[0714] The terminal sends the entered payment information to the server, which then processes the payment through a payment gateway.

[0715] The server sends information about the success or failure of the payment to the terminal. If successful, the usage rights are updated.

[0716] The terminal will notify the user of the payment result, allowing them to continue using the system.

[0717] This system allows fans to safely and easily interact with their favorite characters and people 24 hours a day, 365 days a year, and by providing appropriate responses based on the user's emotions, it enables more personal and intimate communication.

[0718] The processing flow will be explained below.

[0719] Step 1:

[0720] The user opens the application. The user accesses the login screen and enters their ID and password.

[0721] Step 2:

[0722] The terminal receives the user's input and sends the ID and password information to the server. The communication is encrypted.

[0723] Step 3:

[0724] The server checks the received ID and password against the database and performs user authentication. If the ID and password do not match, an error message is sent to the terminal.

[0725] Step 4:

[0726] If the authentication is successful, the server generates session information and sends it back to the device. The device uses the received session information to display the home screen.

[0727] Step 5:

[0728] The user presses the "Select Character" button on the home screen and selects the character they want to interact with.

[0729] Step 6:

[0730] The device receives the user's selection and sends the character ID to the server, which then retrieves the AI ​​model data for the selected character.

[0731] Step 7:

[0732] The server sends the acquired AI model data to the device, which then displays a conversation interface with the character based on the received data.

[0733] Step 8:

[0734] The user presses the "Start conversation" button to start the conversation, and the device's microphone is turned on to capture the user's voice.

[0735] Step 9:

[0736] The device records the user's voice and converts the voice data into text data in real time. This text data is then sent to the server.

[0737] Step 10:

[0738] The server analyzes the received text data and generates a response using an AI model. For example, if the user says "Hello, Character A," the response is "Hello! How's your day?"

[0739] Step 11:

[0740] At the same time, the server sends the user's voice data to the emotion engine to analyze the user's emotional state. The emotion engine analyzes the voice data and identifies the user's emotional state (such as joy, sadness, anger, etc.).

[0741] Step 12:

[0742] The emotion engine sends the analysis results back to the server, which then adaptively changes the response based on the results. For example, if the user speaks in a sad voice, the response might be "Cheer up!"

[0743] Step 13:

[0744] The server converts the response text into voice data and sends it to the terminal.

[0745] Step 14:

[0746] The terminal plays back the received voice data and provides a response to the user, who can then ask questions or make comments.

[0747] Step 15:

[0748] If the user needs to pay a usage fee, the terminal displays a payment screen, and the user enters credit card information, etc.

[0749] Step 16:

[0750] The terminal sends the entered payment information to the server, and the server processes the payment through a payment gateway.

[0751] Step 17:

[0752] The server sends information about the success or failure of the payment to the terminal. If the payment is successful, the usage rights are updated.

[0753] Step 18:

[0754] The terminal notifies the user of the payment result, allowing the user to continue using the system.

[0755] Example 2

[0756] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0757] Conventional AI-based voice conversation systems have struggled to efficiently learn the appearance and personality of characters and people and conduct real-time voice conversations with users. Furthermore, the technology for providing adaptive responses based on the user's emotions has not been sufficiently developed, making it impossible to achieve natural, personal communication with users.

[0758] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0759] In this invention, the server includes means for learning the appearance and personality of a specific character or person using artificial intelligence technology, means for saving the learned data as a non-fungible token (NFT), and means for analyzing emotions from the user's voice and adaptively changing responses based on the analysis results. This makes it possible to learn the characteristics of characters and people with high accuracy, safely save that data, and provide flexible responses according to the user's emotions.

[0760] "Specific character or person" refers to a fictional or real person or character that a user can select as a conversation partner.

[0761] "Artificial intelligence technology" refers to the use of techniques such as machine learning and deep learning to extract patterns from data and make predictions and classifications.

[0762] "Means of learning" refers to the process of collecting data and generating an AI model based on that data.

[0763] A "non-fungible token (NFT)" refers to a token that uses blockchain technology to guarantee the uniqueness and ownership of digital data.

[0764] "Means for providing an interface" refers to the method of providing screens and menus that allow users to access and operate the system.

[0765] "Means for conducting voice conversation in real time" refers to technology that allows a user and a character to have an instantaneous voice conversation.

[0766] "Means for analyzing emotions from a user's voice" refers to technology that analyzes a user's emotional state based on their voice data.

[0767] "Means for adaptively changing responses" refers to a method for changing the content of AI responses based on the results of emotion analysis.

[0768] "Means for collecting usage fees and distributing revenue among the parties involved" refers to a mechanism for collecting service usage fees from users and distributing the revenue obtained among the parties involved.

[0769] MODE FOR CARRYING OUT THE INVENTION

[0770] This invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as non-fungible tokens (NFTs), and provides users with real-time voice conversations 24 hours a day, 365 days a year. Furthermore, by combining it with an emotion engine, it can recognize the user's emotions and provide adaptive responses based on those emotions.

[0771] System Overview

[0772] The system consists of the following main components:

[0773] 1. Server

[0774] 2. Terminal

[0775] 3. Users

[0776] Server Roles

[0777] 1. Collecting training data and generating AI models

[0778] The server collects data on the appearance, personality, and past behavior of specific characters or people, including video, audio, and text data.

[0779] The collected data is trained using an AI algorithm (e.g., TensorFlow or PyTorch). At the initial training stage, data preprocessing (noise removal, text normalization, etc.) is performed.

[0780] Save the completed model to the database.

[0781] 2. Data storage as NFTs

[0782] The trained AI data is stored as a non-fungible token (NFT), and this process uses blockchain technology (e.g., Ethereum).

[0783] The server registers the generated NFT on the blockchain and maintains the owner information.

[0784] 3. Sentiment analysis and response adaptation

[0785] The server uses an emotion engine (for example, IBM Watson's emotion recognition API) to analyze the user's voice data. Based on the analysis results, the AI ​​model adaptively changes the response content.

[0786] For example, if a user says, "I'm feeling a bit down today," emotion recognition can generate a response such as, "Cheer up!"

[0787] 4. Usage Fee Collection and Revenue Sharing

[0788] The server collects the usage fee and manages the payment process, which is carried out using credit card information entered by the user on the terminal.

[0789] Revenues are distributed among the parties involved (character copyright holders, service providers, etc.).

[0790] Device Role

[0791] 1. Providing an interface

[0792] The device provides the user with an interface to select a conversation partner and conduct a conversation. The user launches the app, enters their authentication information on the login screen, and logs in.

[0793] After logging in, users can press the "Select Character" button to choose their preferred character and begin a conversation with that character.

[0794] 2. Audio Data Processing

[0795] The device captures the user's voice and converts it into text data using a speech recognition engine such as Google Speech-to-Text.

[0796] The converted text data is sent to the server, and after an appropriate response is generated, it is converted back into voice data and sent to the terminal.

[0797] The terminal plays the audio data, enabling real-time conversation.

[0798] User Roles

[0799] 1. Log in and select a character

[0800] The user launches the app and enters authentication information such as "user123" and "password123" on the login screen.

[0801] Once authentication is successful, select a character from the home screen and begin a conversation with the selected character.

[0802] 2. Actual conversation progression

[0803] The user presses the "Start conversation" button and speaks into the device's microphone, saying "Hello, Character A."

[0804] The device converts this speech into text data and sends it to the server, which uses the text data to generate a response using an AI model and sends an adaptive reply.

[0805] For example, you can receive responses such as "Hello! How's your day?" or adaptive messages such as "Cheer up!"

[0806] Specific examples and prompt sentence examples

[0807] "Hello, Character A. How was your day?"

[0808] "I've been feeling a bit down lately. Can you cheer me up?"

[0809] "What do you think about today's news?"

[0810] By using these prompts, users can enjoy more personal and natural conversations with the characters.

[0811] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0812] Program processing flow

[0813] Step 1:

[0814] Learning data collection and AI model generation

[0815] The server collects data on the appearance, personality, and past behavior of specific characters or people. The data used for this collection includes text, audio files, images, and videos. Specifically, the data is retrieved from a database via web scraping or API and stored on the server.

[0816] Input: Data about the character's appearance, personality, and past behavior.

[0817] Data processing: noise removal, text normalization, data cleaning.

[0818] Output: A clean dataset after preprocessing.

[0819] The server uses AI algorithms (such as TensorFlow or PyTorch) to train the model based on this data. The trained model then reflects the character's behavior and response patterns.

[0820] Input: Clean dataset after preprocessing.

[0821] Data computation: Applying machine learning algorithms and training models.

[0822] Output: A trained AI model.

[0823] Step 2:

[0824] Storing data as NFTs

[0825] The server stores the learned AI data as a non-fungible token (NFT).

[0826] Input: A trained AI model.

[0827] Data processing: Blockchain technology (e.g., Ethereum) is used to guarantee the uniqueness and ownership of the model for NFT conversion.

[0828] Output: An NFT registered on the blockchain.

[0829] The server registers the generated NFT on the blockchain and stores the owner information. Specifically, the server registers the NFT through the blockchain API.

[0830] Step 3:

[0831] User login and character selection

[0832] The user launches the app and enters their authentication information (e.g., "user123" and "password123") on the login screen.

[0833] The terminal transmits the input authentication information to the server and performs authentication.

[0834] Input: User credentials.

[0835] Data calculation: Check against the database to determine whether authentication is successful.

[0836] Output: Authentication success or failure status.

[0837] If authentication is successful, the home screen will be displayed on the device, and the user can press the "Select Character" button to choose the character they want to interact with.

[0838] Input: Character information selected by the user.

[0839] Data processing: Search and obtain the AI ​​model corresponding to the selected character.

[0840] Output: AI data of the selected character.

[0841] Step 4:

[0842] Real-time voice conversation processing

[0843] The user presses the "Start conversation" button and speaks into the device's microphone.

[0844] The device captures the user's voice and converts it into text data using Google Speech-to-Text.

[0845] Input: User's voice data.

[0846] Data processing: speech-to-text conversion.

[0847] Output: Text data.

[0848] The converted text data is sent to the server, which generates an appropriate response.

[0849] Input: User's text data.

[0850] Data Computation: Using AI models to generate appropriate responses.

[0851] Output: The generated text response.

[0852] The response is converted back into voice data and sent to the terminal.

[0853] Input: The generated text response.

[0854] Data processing: Text to speech conversion.

[0855] Output: The generated audio data.

[0856] The terminal plays the audio data and the conversation takes place in real time.

[0857] Step 5:

[0858] Sentiment analysis and response adaptation

[0859] The server analyzes the user's voice data using an emotion engine (e.g., IBM Watson's emotion recognition API).

[0860] Input: User's voice data.

[0861] Data calculation: Analyze the user's emotional state from voice data.

[0862] Output: User's emotional state data.

[0863] Based on the results of the sentiment analysis, the AI ​​model adaptively changes the response content.

[0864] Input: User emotional state data and generated text responses.

[0865] Data calculation: Modifying the content of responses to match emotional states.

[0866] Output: The final adapted response text.

[0867] For example, if the user sounds sad, the response may be changed to something like "Cheer up!"

[0868] Step 6:

[0869] Collection of usage fees and payment processing

[0870] If the user needs to pay a usage fee, the terminal displays a payment screen where the user enters credit card information, etc.

[0871] Input: User's payment information.

[0872] The terminal sends payment information to the server, and the server processes the payment through a payment gateway (e.g., Stripe).

[0873] Data calculation: Executes payment processing based on payment information.

[0874] Output: Payment success or failure status.

[0875] The server notifies the terminal of the success or failure of the payment, and if successful, the usage rights are updated.

[0876] The terminal notifies the user of the payment result, allowing the user to continue using the system.

[0877] (Application example 2)

[0878] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0879] The objective of this invention is to provide a system that can reproduce the appearance and personality of a specific character or person in real time, while providing appropriate responses according to the user's emotions through dialogue with the character. Furthermore, by having the character act as a shopping assistant in a virtual store, explaining products and recommending items, we aim to realize a more personalized and sympathetic shopping experience.

[0880] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0881] In this invention, the server includes: means for learning the appearance and personality of a specific character or person using artificial intelligence technology; means for storing the learned data as a non-fungible token (NFT); means for providing an interface for a user to select a conversation partner and converse with them; means for conducting real-time voice conversations; means for collecting usage fees from users and distributing revenue among the parties; means for analyzing the emotional state from the user's voice data using an emotion engine and generating an adaptive response based on the analysis results; means for converting the generated response into voice data and providing it to the user as voice; and means for the character to act as a shopping assistant in a virtual store, explaining products and recommending items. This enables real-time, emotion-based conversations with a specific character and a personalized, sympathetic shopping assistant experience in a virtual store.

[0882] "Specific characters or people" are virtual beings whose appearances and personalities are reproduced using artificial intelligence technology.

[0883] "Artificial intelligence technology" is a technology that learns the appearance and personality of a specific character or person and generates responses and actions based on that information.

[0884] A non-fungible token (NFT) is a means of uniquely identifying and proving ownership of a specific digital asset using blockchain technology.

[0885] An "interface" is a device or software that provides the input and display means for a user to select a conversation partner and initiate a conversation.

[0886] "Means for conducting voice conversations in real time" refers to technology that enables a user to have an instant voice conversation with a specific character or person.

[0887] An "emotion engine" is an algorithm and system that analyzes a user's emotional state from their voice data and generates a response based on the analysis results.

[0888] An "adaptive response" is a response that changes depending on the user's utterance content and emotional state, and is generated individually and dynamically.

[0889] A "virtual store" is a virtual shopping environment that exists on the Internet, and is a platform that allows users to make purchases in a virtual space.

[0890] A "shopping assistant" is a virtual character or person who explains products and recommends items to users in a virtual store.

[0891] "Collection" means receiving usage fees from users.

[0892] "Means for distributing revenue among parties involved" refers to a mechanism for appropriately distributing revenue obtained through system operation among parties involved, such as character copyright holders and service providers.

[0893] System Overview

[0894] This invention is a system that reproduces specific characters or people in real time and engages in voice conversations with users. It mainly functions as a shopping assistant in virtual stores. This system consists of three elements: a server, a terminal, and a user.

[0895] Program structure and various technologies

[0896] Speech Recognition Engine

[0897] The server receives the user's voice input and converts the voice into text data using a speech recognition engine (Google Speech-to-Text API).

[0898] Sentiment Analysis Engine

[0899] The server performs emotion analysis on the acquired text data using IBM Watson Tone Analyzer, thereby determining the user's emotional state.

[0900] Dialogue Management System

[0901] The server uses Rasa to generate responses based on the user's conversation content and sentiment analysis results.

[0902] Speech synthesis engine

[0903] The server converts the generated response into audio data using the Google Text-to-Speech API and sends it to the device.

[0904] Program processing

[0905] 1. Voice to Text

[0906] When a user speaks into the smartphone's microphone, the audio is picked up by the device and converted into text data using the Google Speech-to-Text API.

[0907] 2. Emotion analysis

[0908] The transferred text data undergoes emotion analysis using IBM Watson Tone Analyzer to analyze the user's emotional state (e.g., joy, sadness, anger, etc.).

[0909] 3. Response Generation

[0910] It uses Rasa to generate appropriate responses based on the analysis and conversational context, which are adapted to the user's emotions.

[0911] 4. Conversion to audio data and transmission

[0912] The generated response is converted into audio data using the Google Text-to-Speech API, and this audio data is sent to the device, enabling real-time conversation.

[0913] Specific examples

[0914] 1. User login and character selection

[0915] The user starts the app and logs in by entering "user123" and "password123." After logging in, the user selects "Character A" on the character selection screen.

[0916] Example prompt sentence:

[0917] User: Hello, I'm looking for some clothes today.

[0918] ---

[0919] Assistant: Hello, what kind of clothes are you looking for?

[0920] ---

[0921] User: I'd like a casual shirt.

[0922] ---

[0923] 2. Voice input and analysis

[0924] When a user says, "Hello, I'm looking for clothes today," the speech is captured and converted into text data, which is sent to IBM Watson Tone Analyzer for sentiment analysis.

[0925] 3. Response Generation and Serving

[0926] Based on sentiment analysis and the conversation, Rasa generates a response such as "Hello, what kind of clothes are you looking for?" This response is converted to audio using the Google Text-to-Speech API and sent to the device.

[0927] 4. Shopping Assistant

[0928] If a user says, "I want a casual shirt," the assistant will suggest, "How about this shirt? It has a trendy design and is very popular these days," and explain the product. It is also possible to provide more detailed product descriptions and suggestions based on the results of user sentiment analysis.

[0929] This allows the invention to provide emotion-based real-time interaction with specific characters and a personalized and sympathetic shopping assistant experience in virtual stores.

[0930] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0931] Step 1:

[0932] The user launches the app and logs in. The user enters their username and password, and the device sends that information to the server. The server performs authentication, and if successful, displays the character selection screen on the device.

[0933] Input: User's username and password

[0934] Process: The server authenticates the user

[0935] Output: Character selection screen will be displayed upon successful authentication.

[0936] Step 2:

[0937] The user selects a character. The user chooses a specific character on the character selection screen, and the device sends the selection information to the server. The server loads the AI ​​model of the selected character and sends the related data to the device.

[0938] Input: User's character selection information

[0939] Processing: The server loads the AI ​​model and retrieves the relevant data.

[0940] Output: Character information is displayed on the terminal.

[0941] Step 3:

[0942] The user starts a conversation. The user speaks into the smartphone's microphone, and the device picks up the audio. The speech is converted into text data using a speech recognition engine (Google Speech-to-Text API), and the text data is sent to the server.

[0943] Input: User's voice

[0944] Processing: The device receives the voice and converts it into text data.

[0945] Output: The converted text data is sent to the server.

[0946] Step 4:

[0947] The server performs emotion analysis. The server sends the acquired text data to an emotion analysis engine (IBM Watson Tone Analyzer) to analyze the user's emotional state. The analysis results are added to the text data.

[0948] Input: Text data

[0949] Processing: The server analyzes the emotional state using an emotion analysis engine.

[0950] Output: Text data with emotional states added

[0951] Step 5:

[0952] The server generates a response. The server uses a dialogue management system (Rasa) to generate a response based on the sentiment analysis results and the conversation content. The response is in text format.

[0953] Input: Text data with sentiment analysis results added

[0954] Processing: The server generates a response using a dialogue management system

[0955] Output: The generated response text

[0956] Step 6:

[0957] The server converts the response text into speech. The generated response text is converted into audio data using a speech synthesis engine (Google Text-to-Speech API). The converted audio data is then sent back to the device.

[0958] Input: Generated response text

[0959] Processing: The server uses a speech synthesis engine to convert the data into voice data.

[0960] Output: Converted audio data

[0961] Step 7:

[0962] The device plays the audio data. The audio data sent to the device is played back through the speaker and can be heard by the user. This allows for real-time audio conversation between the user and the character.

[0963] Input: Audio data

[0964] Processing: The device plays the audio data.

[0965] Output: The user hears the response

[0966] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0967] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0968] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0969] [Third embodiment]

[0970] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0971] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0972] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0973] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0974] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0975] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0976] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0977] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0978] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0979] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0980] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0981] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0982] The present invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as a non-fungible token (NFT), and enables users to have real-time voice conversations 24 hours a day, 365 days a year. The system is configured as follows:

[0983] System configuration

[0984] 1. Learning with AI technology

[0985] The server collects data on the appearance, personality, and past behavior of specific characters or people, and uses artificial intelligence algorithms to learn this information.

[0986] The learning results include the character's specific behavioral and response patterns and are saved as a highly accurate model.

[0987] 2. Save as an NFT

[0988] The server stores the learned AI data as an NFT, and uses blockchain technology to guarantee its uniqueness and ownership.

[0989] 3. Providing an interface

[0990] The terminal provides the user with an interface for selecting a conversation partner. The user can log in and select a character of their choice.

[0991] 4. Real-time voice conversation

[0992] The user performs an operation to start a conversation with the selected character. The device captures the user's voice, converts it into text data, and sends it to the server.

[0993] The server uses an AI model to generate an appropriate response based on the text data it receives, which is then converted back into audio data and sent to the device.

[0994] The terminal plays the audio data and the conversation takes place in real time.

[0995] 5. Fee collection and revenue sharing

[0996] The server collects the usage fee from the user, and the payment process is carried out through the terminal.

[0997] Revenues are shared among the parties involved, including copyright holders of characters and people, and service providers.

[0998] Specific examples

[0999] 1. User logs in and selects a character

[1000] The user opens the app and logs in by entering "user123" and "password123" on the login screen. The device sends the entered data to the server and authentication is performed. If authentication is successful, the home screen is displayed on the device.

[1001] The user presses the "Character Selection" button on the home screen and selects, for example, "Character A." The device sends the selected character information to the server, and the server retrieves the corresponding AI data and sends it to the device.

[1002] 2. Initiating and conducting actual conversations

[1003] The user presses the "Start conversation" button and speaks "Hello, Character A" into the device's microphone. The device converts the speech into text and sends it to the server.

[1004] The server analyzes the text data and generates an appropriate response based on the AI ​​model, for example, "Hello, I'm Character A. How's your day?"

[1005] The generated response is converted into audio data and sent to the terminal, which plays the audio data so that the user can hear the response.

[1006] 3. Payment of fees and revenue sharing

[1007] When a user pays for a service, the terminal captures the payment information and sends it to the server, which processes the payment through a payment gateway and distributes the revenue among the parties after verification.

[1008] This system allows fans to safely and easily interact with their favorite characters and people 24 hours a day, 365 days a year, and also reduces the high costs associated with hosting events and inviting celebrities to attend.

[1009] The processing flow will be explained below.

[1010] Step 1:

[1011] The user opens the application. The user accesses the login screen and enters their ID and password.

[1012] Step 2:

[1013] The terminal receives the user's input and sends the ID and password information to the server. The communication is encrypted.

[1014] Step 3:

[1015] The server checks the received ID and password against the database and performs user authentication. If the ID and password do not match, an error message is sent to the terminal.

[1016] Step 4:

[1017] If the authentication is successful, the server generates session information and sends it back to the device. The device uses the received session information to display the home screen.

[1018] Step 5:

[1019] The user presses the "Select Character" button on the home screen and selects the character they want to interact with.

[1020] Step 6:

[1021] The device receives the user's selection and sends the character ID to the server, which then retrieves the AI ​​model data for the selected character.

[1022] Step 7:

[1023] The server sends the acquired AI model data to the device, which then displays a conversation interface with the character based on the received data.

[1024] Step 8:

[1025] The user presses the "Start conversation" button to start the conversation, and the device's microphone is turned on to capture the user's voice.

[1026] Step 9:

[1027] The device records the user's voice and converts the voice data into text data in real time. This text data is then sent to the server.

[1028] Step 10:

[1029] The server analyzes the received text data and generates a response using an AI model. For example, if the user says "Hello, Character A," the response is "Hello! How's your day?"

[1030] Step 11:

[1031] The response text generated by the server is converted into voice data and sent to the terminal.

[1032] Step 12:

[1033] The terminal plays back the received voice data and provides a response to the user, who can then ask questions or make comments.

[1034] Step 13:

[1035] If the user needs to pay a usage fee, the terminal displays a payment screen, and the user enters credit card information, etc.

[1036] Step 14:

[1037] The terminal sends the entered payment information to the server, and the server processes the payment through a payment gateway.

[1038] Step 15:

[1039] The server sends information about the success or failure of the payment to the terminal. If the payment is successful, the usage rights are updated.

[1040] Step 16:

[1041] The terminal notifies the user of the payment result, allowing the user to continue using the system.

[1042] Example 1

[1043] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1044] Conventional dialogue systems with characters or people have difficulty in real-time voice conversation, and have been unable to provide a satisfying experience to users. Furthermore, there was a lack of means to guarantee the uniqueness and ownership of trained AI data, creating the risk of data tampering or unauthorized use. Furthermore, there was no established mechanism for collecting usage fees from users and distributing revenue fairly among the parties involved. New technologies are needed to solve these problems.

[1045] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1046] In this invention, the server includes: means for learning the appearance and personality of a specific character or person using artificial intelligence technology; means for storing the learned data as a non-fungible token (NFT); means for providing an interface for a user to select a conversation partner and converse with them; means for acquiring the user's voice and converting it into text data using voice recognition technology; means for generating an appropriate response based on the text data using an AI model and converting the response into voice data; means for transmitting the converted voice data to a user terminal to conduct a real-time voice conversation; and means for collecting usage fees from users and distributing revenue among the parties involved. This enables real-time voice conversation, guarantees the uniqueness and ownership of the learned data, and enables fair revenue distribution.

[1047] A "specific character or person" refers to a fictional or real person that a user selects when interacting with the character, whose appearance, personality, behavior patterns, etc. are predefined.

[1048] "Artificial intelligence technology" refers to technology that uses machine learning and deep learning models to learn from specific data and generate appropriate responses even for unknown data.

[1049] A "non-fungible token (NFT)" is a unique digital asset created using blockchain technology with guaranteed ownership and authenticity.

[1050] "Interface" refers to the user interface provided to users to select and interact with characters or people, and is usually provided as an application.

[1051] "Speech recognition technology" is a technology that converts a user's voice into text data, and examples include Google Cloud Speech-to-Text and Microsoft Azure Speech Recognition Services.

[1052] An "AI model" is a neural network model trained on collected data and used to generate responses for a specific character or person.

[1053] "Converting to voice data" means converting the text response generated by the AI ​​model into voice data using speech synthesis technology, making it audible to the user.

[1054] "Having a real-time voice conversation" refers to a form in which a user can instantly communicate with a character or person using voice.

[1055] "Collecting fees" means collecting fees from users for using the service, often through an online payment system.

[1056] "Distributing revenues among the parties involved" means distributing the collected usage fees to the copyright holders of characters and people, service providers, and other rights holders in a fair proportion.

[1057] MODE FOR CARRYING OUT THE INVENTION

[1058] The present invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as a non-fungible token (NFT), and enables users to have real-time voice conversations 24 hours a day, 365 days a year. The system is configured as follows:

[1059] Learning with AI technology

[1060] The server collects data on the appearance, personality, and past behavior of specific characters or people. For example, it uses scripts to obtain data from YouTube, social media, etc. This data is trained using the TensorFlow or PyTorch machine learning framework. During the data preprocessing process, noise removal, tokenization, normalization, etc. are performed to build a neural network model. The trained model undergoes accuracy evaluation and is saved if it is highly accurate.

[1061] Stored as an NFT

[1062] The server converts the trained AI data into ONNX format to store it as an NFT, and then issues the NFT using a blockchain platform such as Ethereum or Binance Smart Chain, incorporating the model's unique identifier and metadata in the process.

[1063] Providing an interface

[1064] The device provides the user with an interactive user interface. The user logs in using an application developed with React Native or Flutter. The login screen has username and password input fields for the user to authenticate. If authentication is successful, a character selection screen is displayed, where the user selects the character they want to interact with.

[1065] Real-time voice conversation

[1066] The user initiates a conversation with the selected character. The device uses the microphone to capture the user's voice and converts it into text using the Google Cloud Speech-to-Text API or Microsoft Azure Speech Recognition Service. The text is sent to a server, which uses an AI model (e.g., GTP-3 or BERT) to generate an appropriate response. This response is converted into audio data by a speech synthesis engine (e.g., Amazon Polly or Google Text-to-Speech) and sent to the device. The device plays back the audio data, enabling real-time conversation.

[1067] Collection of usage fees and revenue sharing

[1068] The usage fee paid by users when using the service is collected by the terminal acquiring payment information. The payment processing is carried out by the server using the API of Stripe or PayPal. The usage fee collected is distributed fairly to the copyright holders of the characters and people and the service provider.

[1069] Specific examples

[1070] 1. User login and character selection

[1071] The user opens the app and logs in by entering "user123" and "password123" on the login screen. The device sends the input data to the server and authentication is performed. If authentication is successful, the home screen is displayed.

[1072] The user presses the "Character Selection" button on the home screen and selects "Character A." The device sends the selected character information to the server, and the server retrieves the corresponding AI data and sends it to the device.

[1073] 2. Initiating and conducting actual conversations

[1074] The user presses the "Start conversation" button and speaks "Hello, Character A" into the device's microphone. The device converts the speech into text and sends it to the server.

[1075] The server analyzes the text data and generates an appropriate response based on the AI ​​model, for example, "Hello, I'm Character A. How's your day?"

[1076] The generated response is converted into audio data and sent to the terminal, which plays the audio data so that the user can hear the response.

[1077] Prompt Sentence Examples

[1078] "Hi, I'm Character A. How's your day going?"

[1079] This system allows fans to safely and easily interact with their favorite characters and people 24 hours a day, 365 days a year, and also reduces the high costs associated with hosting events and inviting celebrities to attend.

[1080] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1081] Step 1: Data collection

[1082] The server collects data on the appearance, personality, and past behavior of specific characters or people. Inputs include YouTube video links and social media post URLs. The data is obtained using scraping technology and APIs. The output is the retrieved text, images, and audio data.

[1083] Step 2: Data Preprocessing

[1084] The server preprocesses the collected data. The raw data collected in step 1 is provided as input. Specifically, processes such as noise removal, text tokenization, and normalization are performed. For example, unnecessary special characters are removed from the text data and it is converted into a standard format. The output is the preprocessed data.

[1085] Step 3: Training the model

[1086] The server uses the preprocessed data to train an artificial intelligence model (e.g., TensorFlow or PyTorch). The preprocessed data from step 2 is provided as input. Specifically, the server splits the data into batches and optimizes the model parameters using a GPU. The output is a trained, highly accurate model.

[1087] Step 4: Evaluate and save the model

[1088] The server evaluates and saves the trained model. The model trained in step 3 and the validation dataset are used as input. Specifically, the accuracy of the model is evaluated using the validation data, and if the accuracy meets the criteria, the model is saved. The output is the evaluation result and the saved model.

[1089] Step 5: Save as an NFT

[1090] The server saves the trained AI model as an NFT. The model data saved in step 4 is used as input. Specifically, the server converts the model to ONNX format and issues an NFT on a blockchain platform (e.g., Ethereum or Binance Smart Chain). The output is the model data saved as an NFT.

[1091] Step 6: User authentication and character selection

[1092] The terminal displays a login screen for the user and performs authentication. A username and password are provided as input. The authentication data is sent to the server and checked against a database. If authentication is successful, a character selection screen is displayed. The output is the selected character information.

[1093] Step 7: Capturing and Converting Audio Input

[1094] The device uses a microphone to capture the user's voice. The user's voice data is provided as input. Specifically, the voice is converted into text data using the Google Cloud Speech-to-Text API. The output is text data.

[1095] Step 8: Response Generation

[1096] The server receives the text data and generates a response using an AI model. The text data converted in step 7 is provided as input. Specifically, the AI ​​model (e.g., GPT-3 or BERT) analyzes the text data and generates an appropriate response. The output is the text data of the generated response.

[1097] Step 9: Convert to audio and send

[1098] The server converts the generated text response into audio data and sends it to the device. The text data generated in step 8 is provided as input. Specifically, it converts it into audio data using Amazon Polly or Google Text-to-Speech. The output is audio data.

[1099] Step 10: Playing back real-time voice conversations

[1100] The device receives the audio data and plays it on the speaker. The input is the audio data sent in step 9. Specifically, it decodes the audio data and plays it on the speaker so that it can be heard by the user. The output is audio played in real time.

[1101] Step 11: Fee collection and distribution

[1102] The terminal obtains payment information to collect usage fees and sends it to the server. Payment information is provided as input. The server processes the payment using Stripe or PayPal APIs and distributes revenue among the parties involved. Specifically, after confirming the payment process, the distribution is made via bank transfer or cryptocurrency. The output is the distributed revenue.

[1103] (Application example 1)

[1104] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1105] Currently, providing high-quality customer service to customers visiting physical stores requires a large amount of labor and resources. It is also difficult to provide detailed information about specific products in the store, making it difficult to increase customer satisfaction. Furthermore, while there is a demand for personalized guidance that allows customers to select products based on their preferences and interests, achieving this is complex. To solve these challenges, a system that can interact with customers in real time and provide personalized guidance is needed.

[1106] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1107] In this invention, the server includes: means for learning the characteristics of a specific virtual character using machine learning technology; means for saving the learned information as a non-fungible character (NFT); means for providing an operation screen for a user to select a conversation partner and conduct a conversation; means for real-time voice communication; means for receiving a service fee from the user and distributing the revenue among the relevant parties; means for providing guidance using a display device that supports conversations between customers and virtual characters in physical stores; and means for users to obtain product information in the store and receive support through the character. This makes it possible to improve the customer experience in physical stores and provide personalized, high-quality customer service.

[1108] "Characteristics of a specific virtual person" refers to data that includes information such as the appearance, personality, and past words and actions of a character or celebrity.

[1109] "Machine learning technology" is a technology that allows computers to automatically learn from data and make predictions and classifications.

[1110] A "non-fungible token (NFT)" is a digital asset whose uniqueness and ownership are guaranteed using blockchain technology.

[1111] The "operation screen" is a display screen that provides an interface for a user to select a conversation partner and carry out a conversation.

[1112] "Real-time voice communication" refers to a technology that allows users to instantly communicate with selected characters via voice.

[1113] "Service fee" means the amount of money paid by a user as consideration for using a service.

[1114] "Related parties" refers to people or organizations that are subject to revenue sharing, such as character or system providers and developers.

[1115] A "display device" refers to hardware for displaying information, such as displays installed in physical stores, smart glasses, and head-mounted displays (HMDs).

[1116] "Supporting conversations between virtual characters" refers to a specific virtual character communicating with a user via voice to provide guidance and information.

[1117] "Get product information" means viewing or obtaining detailed information about products in the store.

[1118] "Receiving support through a character" means that a virtual person guides or explains products or services to the user.

[1119] To implement the present invention, the following system configuration is required: The system is composed of a server, a terminal, and a user as its main components.

[1120] 1. System Program

[1121] The server builds a database that uses machine learning technology to learn the characteristics of a specific virtual person. The terminal provides an operation screen that allows the user to select a conversation partner and engage in real-time voice communication with that virtual person. The server also stores the learned information as a non-fungible token (NFT).

[1122] The terminal displays an operation screen for the user to select a virtual person to talk to and transmits the selection information to the server. When the user communicates with the terminal in real time, the terminal converts the user's voice into text information and transmits the text information to the server.

[1123] The server then feeds the received text information to a machine learning model to generate a conversation-based response. The generated response is then converted into voice data by the server and sent to the device. The device then plays the voice data and provides the user with a response from a virtual character.

[1124] 2. Processing Description

[1125] The server is implemented using programming languages ​​such as Python, and uses machine learning technologies such as the NLP (natural language processing) model GPT-3. It also uses libraries such as Google Speech Recognition for speech recognition and pyttsx3 for speech synthesis.

[1126] The device includes hardware such as smart glasses, a head-mounted display (HMD), and a microphone. The device recognizes the user's speech and sends it to the server. The server converts the speech to text and generates an appropriate response using an NLP model. The server then converts the response back into audio data and sends it back to the device.

[1127] As a specific example, a user enters a physical store and puts on smart glasses. When the user says, "What products do you recommend?", the device's microphone picks up the user's voice and sends it to the server. The server analyzes the text data and generates a response using GPT-3. The response might be something like, "Currently, we recommend this smartwatch. It's the latest model with a long battery life and comprehensive health management features."

[1128] The generated response is converted into voice data and sent to the terminal, which then plays the voice data so that the user can hear the voice of the virtual character.

[1129] 3. Examples of concrete examples and prompts

[1130] As a practical example, consider a situation where a user wears smart glasses and asks for assistance in a store. For example, the user says, "Hello, what products do you recommend?"

[1131] Example prompt sentence:

[1132] Customer Question: "Hi, what products do you recommend?"

[1133] Prompt for generative AI model: "A customer asks you for a product recommendation. How would you respond?"

[1134] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1135] Program processing steps

[1136] Step 1:

[1137] A user puts on the smart glasses and launches an application. The input is the initialization data for the smart glasses, and the output is the display of a login screen. Specifically, the terminal displays the login screen to the user, and the user enters login information.

[1138] Step 2:

[1139] The user enters login information and presses the login button. The input is the login information, and the output is an authentication request to the server. The terminal sends the user's input information to the server, and the server performs authentication.

[1140] Step 3:

[1141] The server authenticates the user's login information and returns the result to the device. The input is the user's login information, and the output is the authentication result. Specifically, the server searches the database and verifies the login information. If successful, the home screen is displayed on the device.

[1142] Step 4:

[1143] The user selects a character to interact with from the home screen. The input is character selection information, and the output is transmission of the selection information to the server. The device sends the selected character information to the server, and the server obtains the corresponding learning data.

[1144] Step 5:

[1145] The server retrieves the learning data for the selected character and sends it to the terminal. The input is the character selection information and the output is the learning data. The server retrieves the relevant data from the learning database and sends it to the terminal.

[1146] Step 6:

[1147] The user presses the "Start conversation" button and speaks into the device's microphone. The input is the user's voice, and the output is text data. The device uses voice recognition to convert the user's voice into text, and sends the text data to the server.

[1148] Step 7:

[1149] The server receives the user's text data and generates an appropriate response using a generative AI model. The input is the user's text data and the output is the generated response. The server generates the response using an NLP model (e.g., GPT-3).

[1150] Step 8:

[1151] The response generated by the server is converted into audio data and sent to the device. The input is the generated response and the output is audio data. The server uses a speech synthesis library (e.g., pyttsx3) to generate the audio data and send it to the device.

[1152] Step 9:

[1153] The terminal plays back the voice data received from the server and lets the user listen. The input is voice data, and the output is voice playback. The terminal plays back the voice data, allowing the user to hear the virtual character's response.

[1154] Step 10:

[1155] When the user wants to end the service, he / she performs an operation to end the application. The input is the end operation and the output is the end of the application. The terminal ends the application and stops the display on the smart glasses.

[1156] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1157] This invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as a non-fungible token (NFT), and enables real-time voice conversations with users 24 hours a day, 365 days a year. In addition, by combining this invention with an emotion engine, it can recognize the user's emotions and provide adaptive responses based on those emotions.

[1158] System configuration

[1159] 1. Learning with AI technology

[1160] The server collects data on the appearance, personality, and past behavior of specific characters and people, and uses an artificial intelligence algorithm to learn this information. The learning results, including the character's specific behavior and response patterns, are saved as a highly accurate model.

[1161] 2. Save as an NFT

[1162] The server stores the learned AI data as an NFT, and uses blockchain technology to guarantee its uniqueness and ownership.

[1163] 3. Providing an interface

[1164] The terminal provides the user with an interface for selecting a conversation partner. The user can log in and select a character of their choice.

[1165] 4. Real-time voice conversation

[1166] The user performs an operation to start a conversation with the selected character. The device captures the user's voice, converts it into text data, and sends it to the server.

[1167] The server uses an AI model to generate an appropriate response based on the text data it receives, which is then converted back into audio data and sent to the device.

[1168] The terminal plays the audio data and the conversation takes place in real time.

[1169] 5. Emotion engine integration

[1170] The server is equipped with an emotion engine that analyzes the user's emotional state (e.g., joy, sadness, anger, etc.) from the user's voice data.

[1171] Based on the results of the emotion analysis, the AI ​​model adaptively changes the response content. For example, if the user speaks in a sad voice, the model generates a response such as "Cheer up!"

[1172] 6. Fee collection and revenue sharing

[1173] The server collects usage fees from users. Payments are processed through the terminal. Revenues are distributed among the parties involved, including the copyright holders of characters and people, and service providers.

[1174] Specific examples

[1175] 1. User logs in and selects a character

[1176] The user opens the app and logs in by entering "user123" and "password123" on the login screen. The device sends the entered data to the server and authentication is performed. If authentication is successful, the home screen is displayed on the device.

[1177] The user presses the "Character Selection" button on the home screen and selects, for example, "Character A." The device sends the selected character information to the server, and the server retrieves the corresponding AI data and sends it to the device.

[1178] 2. Initiating and conducting actual conversations

[1179] The user presses the "Start conversation" button and speaks "Hello, Character A" into the device's microphone. The device converts the speech into text and sends it to the server.

[1180] The server analyzes the text data and generates an appropriate response based on the AI ​​model, such as "Hello! How's your day?"

[1181] The server simultaneously transmits the voice data to the emotion engine to analyze the user's emotional state.

[1182] The emotion engine analyzes the user's emotional state and feeds the results back to the AI ​​model. For example, if the user sounds sad, the response will be changed to an encouraging message like "Cheer up!"

[1183] The generated response is converted into voice data and sent to the terminal, which plays the voice data and provides the response to the user.

[1184] 3. Payment of fees and revenue sharing

[1185] If the user needs to pay a usage fee, the terminal displays a payment screen where the user enters credit card information, etc.

[1186] The terminal sends the entered payment information to the server, which then processes the payment through a payment gateway.

[1187] The server sends information about the success or failure of the payment to the terminal. If successful, the usage rights are updated.

[1188] The terminal will notify the user of the payment result, allowing them to continue using the system.

[1189] This system allows fans to safely and easily interact with their favorite characters and people 24 hours a day, 365 days a year, and by providing appropriate responses based on the user's emotions, it enables more personal and intimate communication.

[1190] The processing flow will be explained below.

[1191] Step 1:

[1192] The user opens the application. The user accesses the login screen and enters their ID and password.

[1193] Step 2:

[1194] The terminal receives the user's input and sends the ID and password information to the server. The communication is encrypted.

[1195] Step 3:

[1196] The server checks the received ID and password against the database and performs user authentication. If the ID and password do not match, an error message is sent to the terminal.

[1197] Step 4:

[1198] If the authentication is successful, the server generates session information and sends it back to the device. The device uses the received session information to display the home screen.

[1199] Step 5:

[1200] The user presses the "Select Character" button on the home screen and selects the character they want to interact with.

[1201] Step 6:

[1202] The device receives the user's selection and sends the character ID to the server, which then retrieves the AI ​​model data for the selected character.

[1203] Step 7:

[1204] The server sends the acquired AI model data to the device, which then displays a conversation interface with the character based on the received data.

[1205] Step 8:

[1206] The user presses the "Start conversation" button to start the conversation, and the device's microphone is turned on to capture the user's voice.

[1207] Step 9:

[1208] The device records the user's voice and converts the voice data into text data in real time. This text data is then sent to the server.

[1209] Step 10:

[1210] The server analyzes the received text data and generates a response using an AI model. For example, if the user says "Hello, Character A," the response is "Hello! How's your day?"

[1211] Step 11:

[1212] At the same time, the server sends the user's voice data to the emotion engine to analyze the user's emotional state. The emotion engine analyzes the voice data and identifies the user's emotional state (such as joy, sadness, anger, etc.).

[1213] Step 12:

[1214] The emotion engine sends the analysis results back to the server, which then adaptively changes the response based on the results. For example, if the user speaks in a sad voice, the response might be "Cheer up!"

[1215] Step 13:

[1216] The server converts the response text into voice data and sends it to the terminal.

[1217] Step 14:

[1218] The terminal plays back the received voice data and provides a response to the user, who can then ask questions or make comments.

[1219] Step 15:

[1220] If the user needs to pay a usage fee, the terminal displays a payment screen, and the user enters credit card information, etc.

[1221] Step 16:

[1222] The terminal sends the entered payment information to the server, and the server processes the payment through a payment gateway.

[1223] Step 17:

[1224] The server sends information about the success or failure of the payment to the terminal. If the payment is successful, the usage rights are updated.

[1225] Step 18:

[1226] The terminal notifies the user of the payment result, allowing the user to continue using the system.

[1227] Example 2

[1228] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1229] Conventional AI-based voice conversation systems have struggled to efficiently learn the appearance and personality of characters and people and conduct real-time voice conversations with users. Furthermore, the technology for providing adaptive responses based on the user's emotions has not been sufficiently developed, making it impossible to achieve natural, personal communication with users.

[1230] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1231] In this invention, the server includes means for learning the appearance and personality of a specific character or person using artificial intelligence technology, means for saving the learned data as a non-fungible token (NFT), and means for analyzing emotions from the user's voice and adaptively changing responses based on the analysis results. This makes it possible to learn the characteristics of characters and people with high accuracy, safely save that data, and provide flexible responses according to the user's emotions.

[1232] "Specific character or person" refers to a fictional or real person or character that a user can select as a conversation partner.

[1233] "Artificial intelligence technology" refers to the use of techniques such as machine learning and deep learning to extract patterns from data and make predictions and classifications.

[1234] "Means of learning" refers to the process of collecting data and generating an AI model based on that data.

[1235] A "non-fungible token (NFT)" refers to a token that uses blockchain technology to guarantee the uniqueness and ownership of digital data.

[1236] "Means for providing an interface" refers to the method of providing screens and menus that allow users to access and operate the system.

[1237] "Means for conducting voice conversation in real time" refers to technology that allows a user and a character to have an instantaneous voice conversation.

[1238] "Means for analyzing emotions from a user's voice" refers to technology that analyzes a user's emotional state based on their voice data.

[1239] "Means for adaptively changing responses" refers to a method for changing the content of AI responses based on the results of emotion analysis.

[1240] "Means for collecting usage fees and distributing revenue among the parties involved" refers to a mechanism for collecting service usage fees from users and distributing the revenue obtained among the parties involved.

[1241] MODE FOR CARRYING OUT THE INVENTION

[1242] This invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as non-fungible tokens (NFTs), and provides users with real-time voice conversations 24 hours a day, 365 days a year. Furthermore, by combining it with an emotion engine, it can recognize the user's emotions and provide adaptive responses based on those emotions.

[1243] System Overview

[1244] The system consists of the following main components:

[1245] 1. Server

[1246] 2. Terminal

[1247] 3. Users

[1248] Server Roles

[1249] 1. Collecting training data and generating AI models

[1250] The server collects data on the appearance, personality, and past behavior of specific characters or people, including video, audio, and text data.

[1251] The collected data is trained using an AI algorithm (e.g., TensorFlow or PyTorch). At the initial training stage, data preprocessing (noise removal, text normalization, etc.) is performed.

[1252] Save the completed model to the database.

[1253] 2. Data storage as NFTs

[1254] The trained AI data is stored as a non-fungible token (NFT), and this process uses blockchain technology (e.g., Ethereum).

[1255] The server registers the generated NFT on the blockchain and maintains the owner information.

[1256] 3. Sentiment analysis and response adaptation

[1257] The server uses an emotion engine (for example, IBM Watson's emotion recognition API) to analyze the user's voice data. Based on the analysis results, the AI ​​model adaptively changes the response content.

[1258] For example, if a user says, "I'm feeling a bit down today," emotion recognition can generate a response such as, "Cheer up!"

[1259] 4. Usage Fee Collection and Revenue Sharing

[1260] The server collects the usage fee and manages the payment process, which is carried out using credit card information entered by the user on the terminal.

[1261] Revenues are distributed among the parties involved (character copyright holders, service providers, etc.).

[1262] Device Role

[1263] 1. Providing an interface

[1264] The device provides the user with an interface to select a conversation partner and conduct a conversation. The user launches the app, enters their authentication information on the login screen, and logs in.

[1265] After logging in, users can press the "Select Character" button to choose their preferred character and begin a conversation with that character.

[1266] 2. Audio Data Processing

[1267] The device captures the user's voice and converts it into text data using a speech recognition engine such as Google Speech-to-Text.

[1268] The converted text data is sent to the server, and after an appropriate response is generated, it is converted back into voice data and sent to the terminal.

[1269] The terminal plays the audio data, enabling real-time conversation.

[1270] User Roles

[1271] 1. Log in and select a character

[1272] The user launches the app and enters authentication information such as "user123" and "password123" on the login screen.

[1273] Once authentication is successful, select a character from the home screen and begin a conversation with the selected character.

[1274] 2. Actual conversation progression

[1275] The user presses the "Start conversation" button and speaks into the device's microphone, saying "Hello, Character A."

[1276] The device converts this speech into text data and sends it to the server, which uses the text data to generate a response using an AI model and sends an adaptive reply.

[1277] For example, you can receive responses such as "Hello! How's your day?" or adaptive messages such as "Cheer up!"

[1278] Specific examples and prompt sentence examples

[1279] "Hello, Character A. How was your day?"

[1280] "I've been feeling a bit down lately. Can you cheer me up?"

[1281] "What do you think about today's news?"

[1282] By using these prompts, users can enjoy more personal and natural conversations with the characters.

[1283] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1284] Program processing flow

[1285] Step 1:

[1286] Learning data collection and AI model generation

[1287] The server collects data on the appearance, personality, and past behavior of specific characters or people. The data used for this collection includes text, audio files, images, and videos. Specifically, the data is retrieved from a database via web scraping or API and stored on the server.

[1288] Input: Data about the character's appearance, personality, and past behavior.

[1289] Data processing: noise removal, text normalization, data cleaning.

[1290] Output: A clean dataset after preprocessing.

[1291] The server uses AI algorithms (such as TensorFlow or PyTorch) to train the model based on this data. The trained model then reflects the character's behavior and response patterns.

[1292] Input: Clean dataset after preprocessing.

[1293] Data computation: Applying machine learning algorithms and training models.

[1294] Output: A trained AI model.

[1295] Step 2:

[1296] Storing data as NFTs

[1297] The server stores the learned AI data as a non-fungible token (NFT).

[1298] Input: A trained AI model.

[1299] Data processing: Blockchain technology (e.g., Ethereum) is used to guarantee the uniqueness and ownership of the model for NFT conversion.

[1300] Output: An NFT registered on the blockchain.

[1301] The server registers the generated NFT on the blockchain and stores the owner information. Specifically, the server registers the NFT through the blockchain API.

[1302] Step 3:

[1303] User login and character selection

[1304] The user launches the app and enters their authentication information (e.g., "user123" and "password123") on the login screen.

[1305] The terminal transmits the input authentication information to the server and performs authentication.

[1306] Input: User credentials.

[1307] Data calculation: Check against the database to determine whether authentication is successful.

[1308] Output: Authentication success or failure status.

[1309] If authentication is successful, the home screen will be displayed on the device, and the user can press the "Select Character" button to choose the character they want to interact with.

[1310] Input: Character information selected by the user.

[1311] Data processing: Search and obtain the AI ​​model corresponding to the selected character.

[1312] Output: AI data of the selected character.

[1313] Step 4:

[1314] Real-time voice conversation processing

[1315] The user presses the "Start conversation" button and speaks into the device's microphone.

[1316] The device captures the user's voice and converts it into text data using Google Speech-to-Text.

[1317] Input: User's voice data.

[1318] Data processing: speech-to-text conversion.

[1319] Output: Text data.

[1320] The converted text data is sent to the server, which generates an appropriate response.

[1321] Input: User's text data.

[1322] Data Computation: Using AI models to generate appropriate responses.

[1323] Output: The generated text response.

[1324] The response is converted back into voice data and sent to the terminal.

[1325] Input: The generated text response.

[1326] Data processing: Text to speech conversion.

[1327] Output: The generated audio data.

[1328] The terminal plays the audio data and the conversation takes place in real time.

[1329] Step 5:

[1330] Sentiment analysis and response adaptation

[1331] The server analyzes the user's voice data using an emotion engine (e.g., IBM Watson's emotion recognition API).

[1332] Input: User's voice data.

[1333] Data calculation: Analyze the user's emotional state from voice data.

[1334] Output: User's emotional state data.

[1335] Based on the results of the sentiment analysis, the AI ​​model adaptively changes the response content.

[1336] Input: User emotional state data and generated text responses.

[1337] Data calculation: Modifying the content of responses to match emotional states.

[1338] Output: The final adapted response text.

[1339] For example, if the user sounds sad, the response may be changed to something like "Cheer up!"

[1340] Step 6:

[1341] Collection of usage fees and payment processing

[1342] If the user needs to pay a usage fee, the terminal displays a payment screen where the user enters credit card information, etc.

[1343] Input: User's payment information.

[1344] The terminal sends payment information to the server, and the server processes the payment through a payment gateway (e.g., Stripe).

[1345] Data calculation: Executes payment processing based on payment information.

[1346] Output: Payment success or failure status.

[1347] The server notifies the terminal of the success or failure of the payment, and if successful, the usage rights are updated.

[1348] The terminal notifies the user of the payment result, allowing the user to continue using the system.

[1349] (Application example 2)

[1350] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1351] The objective of this invention is to provide a system that can reproduce the appearance and personality of a specific character or person in real time, while providing appropriate responses according to the user's emotions through dialogue with the character. Furthermore, by having the character act as a shopping assistant in a virtual store, explaining products and recommending items, we aim to realize a more personalized and sympathetic shopping experience.

[1352] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1353] In this invention, the server includes: means for learning the appearance and personality of a specific character or person using artificial intelligence technology; means for storing the learned data as a non-fungible token (NFT); means for providing an interface for a user to select a conversation partner and converse with them; means for conducting real-time voice conversations; means for collecting usage fees from users and distributing revenue among the parties; means for analyzing the emotional state from the user's voice data using an emotion engine and generating an adaptive response based on the analysis results; means for converting the generated response into voice data and providing it to the user as voice; and means for the character to act as a shopping assistant in a virtual store, explaining products and recommending items. This enables real-time, emotion-based conversations with a specific character and a personalized, sympathetic shopping assistant experience in a virtual store.

[1354] "Specific characters or people" are virtual beings whose appearances and personalities are reproduced using artificial intelligence technology.

[1355] "Artificial intelligence technology" is a technology that learns the appearance and personality of a specific character or person and generates responses and actions based on that information.

[1356] A non-fungible token (NFT) is a means of uniquely identifying and proving ownership of a specific digital asset using blockchain technology.

[1357] An "interface" is a device or software that provides the input and display means for a user to select a conversation partner and initiate a conversation.

[1358] "Means for conducting voice conversations in real time" refers to technology that enables a user to have an instant voice conversation with a specific character or person.

[1359] An "emotion engine" is an algorithm and system that analyzes a user's emotional state from their voice data and generates a response based on the analysis results.

[1360] An "adaptive response" is a response that changes depending on the user's utterance content and emotional state, and is generated individually and dynamically.

[1361] A "virtual store" is a virtual shopping environment that exists on the Internet, and is a platform that allows users to make purchases in a virtual space.

[1362] A "shopping assistant" is a virtual character or person who explains products and recommends items to users in a virtual store.

[1363] "Collection" means receiving usage fees from users.

[1364] "Means for distributing revenue among parties involved" refers to a mechanism for appropriately distributing revenue obtained through system operation among parties involved, such as character copyright holders and service providers.

[1365] System Overview

[1366] This invention is a system that reproduces specific characters or people in real time and engages in voice conversations with users. It mainly functions as a shopping assistant in virtual stores. This system consists of three elements: a server, a terminal, and a user.

[1367] Program structure and various technologies

[1368] Speech Recognition Engine

[1369] The server receives the user's voice input and converts the voice into text data using a speech recognition engine (Google Speech-to-Text API).

[1370] Sentiment Analysis Engine

[1371] The server performs emotion analysis on the acquired text data using IBM Watson Tone Analyzer, thereby determining the user's emotional state.

[1372] Dialogue Management System

[1373] The server uses Rasa to generate responses based on the user's conversation content and sentiment analysis results.

[1374] Speech synthesis engine

[1375] The server converts the generated response into audio data using the Google Text-to-Speech API and sends it to the device.

[1376] Program processing

[1377] 1. Voice to Text

[1378] When a user speaks into the smartphone's microphone, the audio is picked up by the device and converted into text data using the Google Speech-to-Text API.

[1379] 2. Emotion analysis

[1380] The transferred text data undergoes emotion analysis using IBM Watson Tone Analyzer to analyze the user's emotional state (e.g., joy, sadness, anger, etc.).

[1381] 3. Response Generation

[1382] It uses Rasa to generate appropriate responses based on the analysis and conversational context, which are adapted to the user's emotions.

[1383] 4. Conversion to audio data and transmission

[1384] The generated response is converted into audio data using the Google Text-to-Speech API, and this audio data is sent to the device, enabling real-time conversation.

[1385] Specific examples

[1386] 1. User login and character selection

[1387] The user starts the app and logs in by entering "user123" and "password123." After logging in, the user selects "Character A" on the character selection screen.

[1388] Example prompt sentence:

[1389] User: Hello, I'm looking for some clothes today.

[1390] ---

[1391] Assistant: Hello, what kind of clothes are you looking for?

[1392] ---

[1393] User: I'd like a casual shirt.

[1394] ---

[1395] 2. Voice input and analysis

[1396] When a user says, "Hello, I'm looking for clothes today," the speech is captured and converted into text data, which is sent to IBM Watson Tone Analyzer for sentiment analysis.

[1397] 3. Response Generation and Serving

[1398] Based on sentiment analysis and the conversation, Rasa generates a response such as "Hello, what kind of clothes are you looking for?" This response is converted to audio using the Google Text-to-Speech API and sent to the device.

[1399] 4. Shopping Assistant

[1400] If a user says, "I want a casual shirt," the assistant will suggest, "How about this shirt? It has a trendy design and is very popular these days," and explain the product. It is also possible to provide more detailed product descriptions and suggestions based on the results of user sentiment analysis.

[1401] This allows the invention to provide emotion-based real-time interaction with specific characters and a personalized and sympathetic shopping assistant experience in virtual stores.

[1402] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1403] Step 1:

[1404] The user launches the app and logs in. The user enters their username and password, and the device sends that information to the server. The server performs authentication, and if successful, displays the character selection screen on the device.

[1405] Input: User's username and password

[1406] Process: The server authenticates the user

[1407] Output: Character selection screen will be displayed upon successful authentication.

[1408] Step 2:

[1409] The user selects a character. The user chooses a specific character on the character selection screen, and the device sends the selection information to the server. The server loads the AI ​​model of the selected character and sends the related data to the device.

[1410] Input: User's character selection information

[1411] Processing: The server loads the AI ​​model and retrieves the relevant data.

[1412] Output: Character information is displayed on the terminal.

[1413] Step 3:

[1414] The user starts a conversation. The user speaks into the smartphone's microphone, and the device picks up the audio. The speech is converted into text data using a speech recognition engine (Google Speech-to-Text API), and the text data is sent to the server.

[1415] Input: User's voice

[1416] Processing: The device receives the voice and converts it into text data.

[1417] Output: The converted text data is sent to the server.

[1418] Step 4:

[1419] The server performs emotion analysis. The server sends the acquired text data to an emotion analysis engine (IBM Watson Tone Analyzer) to analyze the user's emotional state. The analysis results are added to the text data.

[1420] Input: Text data

[1421] Processing: The server analyzes the emotional state using an emotion analysis engine.

[1422] Output: Text data with emotional states added

[1423] Step 5:

[1424] The server generates a response. The server uses a dialogue management system (Rasa) to generate a response based on the sentiment analysis results and the conversation content. The response is in text format.

[1425] Input: Text data with sentiment analysis results added

[1426] Processing: The server generates a response using a dialogue management system

[1427] Output: The generated response text

[1428] Step 6:

[1429] The server converts the response text into speech. The generated response text is converted into audio data using a speech synthesis engine (Google Text-to-Speech API). The converted audio data is then sent back to the device.

[1430] Input: Generated response text

[1431] Processing: The server uses a speech synthesis engine to convert the data into voice data.

[1432] Output: Converted audio data

[1433] Step 7:

[1434] The device plays the audio data. The audio data sent to the device is played back through the speaker and can be heard by the user. This allows for real-time audio conversation between the user and the character.

[1435] Input: Audio data

[1436] Processing: The device plays the audio data.

[1437] Output: The user hears the response

[1438] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1439] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1440] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1441] [Fourth embodiment]

[1442] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1443] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1444] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1445] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1446] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1447] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1448] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1449] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1450] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1451] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1452] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1453] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1454] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1455] The present invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as a non-fungible token (NFT), and enables users to have real-time voice conversations 24 hours a day, 365 days a year. The system is configured as follows:

[1456] System configuration

[1457] 1. Learning with AI technology

[1458] The server collects data on the appearance, personality, and past behavior of specific characters or people, and uses artificial intelligence algorithms to learn this information.

[1459] The learning results include the character's specific behavioral and response patterns and are saved as a highly accurate model.

[1460] 2. Save as an NFT

[1461] The server stores the learned AI data as an NFT, and uses blockchain technology to guarantee its uniqueness and ownership.

[1462] 3. Providing an interface

[1463] The terminal provides the user with an interface for selecting a conversation partner. The user can log in and select a character of their choice.

[1464] 4. Real-time voice conversation

[1465] The user performs an operation to start a conversation with the selected character. The device captures the user's voice, converts it into text data, and sends it to the server.

[1466] The server uses an AI model to generate an appropriate response based on the text data it receives, which is then converted back into audio data and sent to the device.

[1467] The terminal plays the audio data and the conversation takes place in real time.

[1468] 5. Fee collection and revenue sharing

[1469] The server collects the usage fee from the user, and the payment process is carried out through the terminal.

[1470] Revenues are shared among the parties involved, including copyright holders of characters and people, and service providers.

[1471] Specific examples

[1472] 1. User logs in and selects a character

[1473] The user opens the app and logs in by entering "user123" and "password123" on the login screen. The device sends the entered data to the server and authentication is performed. If authentication is successful, the home screen is displayed on the device.

[1474] The user presses the "Character Selection" button on the home screen and selects, for example, "Character A." The device sends the selected character information to the server, and the server retrieves the corresponding AI data and sends it to the device.

[1475] 2. Initiating and conducting actual conversations

[1476] The user presses the "Start conversation" button and speaks "Hello, Character A" into the device's microphone. The device converts the speech into text and sends it to the server.

[1477] The server analyzes the text data and generates an appropriate response based on the AI ​​model, for example, "Hello, I'm Character A. How's your day?"

[1478] The generated response is converted into audio data and sent to the terminal, which plays the audio data so that the user can hear the response.

[1479] 3. Payment of fees and revenue sharing

[1480] When a user pays for a service, the terminal captures the payment information and sends it to the server, which processes the payment through a payment gateway and distributes the revenue among the parties after verification.

[1481] This system allows fans to safely and easily interact with their favorite characters and people 24 hours a day, 365 days a year, and also reduces the high costs associated with hosting events and inviting celebrities to attend.

[1482] The processing flow will be explained below.

[1483] Step 1:

[1484] The user opens the application. The user accesses the login screen and enters their ID and password.

[1485] Step 2:

[1486] The terminal receives the user's input and sends the ID and password information to the server. The communication is encrypted.

[1487] Step 3:

[1488] The server checks the received ID and password against the database and performs user authentication. If the ID and password do not match, an error message is sent to the terminal.

[1489] Step 4:

[1490] If the authentication is successful, the server generates session information and sends it back to the device. The device uses the received session information to display the home screen.

[1491] Step 5:

[1492] The user presses the "Select Character" button on the home screen and selects the character they want to interact with.

[1493] Step 6:

[1494] The device receives the user's selection and sends the character ID to the server, which then retrieves the AI ​​model data for the selected character.

[1495] Step 7:

[1496] The server sends the acquired AI model data to the device, which then displays a conversation interface with the character based on the received data.

[1497] Step 8:

[1498] The user presses the "Start conversation" button to start the conversation, and the device's microphone is turned on to capture the user's voice.

[1499] Step 9:

[1500] The device records the user's voice and converts the voice data into text data in real time. This text data is then sent to the server.

[1501] Step 10:

[1502] The server analyzes the received text data and generates a response using an AI model. For example, if the user says "Hello, Character A," the response is "Hello! How's your day?"

[1503] Step 11:

[1504] The response text generated by the server is converted into voice data and sent to the terminal.

[1505] Step 12:

[1506] The terminal plays back the received voice data and provides a response to the user, who can then ask questions or make comments.

[1507] Step 13:

[1508] If the user needs to pay a usage fee, the terminal displays a payment screen, and the user enters credit card information, etc.

[1509] Step 14:

[1510] The terminal sends the entered payment information to the server, and the server processes the payment through a payment gateway.

[1511] Step 15:

[1512] The server sends information about the success or failure of the payment to the terminal. If the payment is successful, the usage rights are updated.

[1513] Step 16:

[1514] The terminal notifies the user of the payment result, allowing the user to continue using the system.

[1515] Example 1

[1516] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1517] Conventional dialogue systems with characters or people have difficulty in real-time voice conversation, and have been unable to provide a satisfying experience to users. Furthermore, there was a lack of means to guarantee the uniqueness and ownership of trained AI data, creating the risk of data tampering or unauthorized use. Furthermore, there was no established mechanism for collecting usage fees from users and distributing revenue fairly among the parties involved. New technologies are needed to solve these problems.

[1518] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1519] In this invention, the server includes: means for learning the appearance and personality of a specific character or person using artificial intelligence technology; means for storing the learned data as a non-fungible token (NFT); means for providing an interface for a user to select a conversation partner and converse with them; means for acquiring the user's voice and converting it into text data using voice recognition technology; means for generating an appropriate response based on the text data using an AI model and converting the response into voice data; means for transmitting the converted voice data to a user terminal to conduct a real-time voice conversation; and means for collecting usage fees from users and distributing revenue among the parties involved. This enables real-time voice conversation, guarantees the uniqueness and ownership of the learned data, and enables fair revenue distribution.

[1520] A "specific character or person" refers to a fictional or real person that a user selects when interacting with the character, whose appearance, personality, behavior patterns, etc. are predefined.

[1521] "Artificial intelligence technology" refers to technology that uses machine learning and deep learning models to learn from specific data and generate appropriate responses even for unknown data.

[1522] A "non-fungible token (NFT)" is a unique digital asset created using blockchain technology with guaranteed ownership and authenticity.

[1523] "Interface" refers to the user interface provided to users to select and interact with characters or people, and is usually provided as an application.

[1524] "Speech recognition technology" is a technology that converts a user's voice into text data, and examples include Google Cloud Speech-to-Text and Microsoft Azure Speech Recognition Services.

[1525] An "AI model" is a neural network model trained on collected data and used to generate responses for a specific character or person.

[1526] "Converting to voice data" means converting the text response generated by the AI ​​model into voice data using speech synthesis technology, making it audible to the user.

[1527] "Having a real-time voice conversation" refers to a form in which a user can instantly communicate with a character or person using voice.

[1528] "Collecting fees" means collecting fees from users for using the service, often through an online payment system.

[1529] "Distributing revenues among the parties involved" means distributing the collected usage fees to the copyright holders of characters and people, service providers, and other rights holders in a fair proportion.

[1530] MODE FOR CARRYING OUT THE INVENTION

[1531] The present invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as a non-fungible token (NFT), and enables users to have real-time voice conversations 24 hours a day, 365 days a year. The system is configured as follows:

[1532] Learning with AI technology

[1533] The server collects data on the appearance, personality, and past behavior of specific characters or people. For example, it uses scripts to obtain data from YouTube, social media, etc. This data is trained using the TensorFlow or PyTorch machine learning framework. During the data preprocessing process, noise removal, tokenization, normalization, etc. are performed to build a neural network model. The trained model undergoes accuracy evaluation and is saved if it is highly accurate.

[1534] Stored as an NFT

[1535] The server converts the trained AI data into ONNX format to store it as an NFT, and then issues the NFT using a blockchain platform such as Ethereum or Binance Smart Chain, incorporating the model's unique identifier and metadata in the process.

[1536] Providing an interface

[1537] The device provides the user with an interactive user interface. The user logs in using an application developed with React Native or Flutter. The login screen has username and password input fields for the user to authenticate. If authentication is successful, a character selection screen is displayed, where the user selects the character they want to interact with.

[1538] Real-time voice conversation

[1539] The user initiates a conversation with the selected character. The device uses the microphone to capture the user's voice and converts it into text using the Google Cloud Speech-to-Text API or Microsoft Azure Speech Recognition Service. The text is sent to a server, which uses an AI model (e.g., GTP-3 or BERT) to generate an appropriate response. This response is converted into audio data by a speech synthesis engine (e.g., Amazon Polly or Google Text-to-Speech) and sent to the device. The device plays back the audio data, enabling real-time conversation.

[1540] Collection of usage fees and revenue sharing

[1541] The usage fee paid by users when using the service is collected by the terminal acquiring payment information. The payment processing is carried out by the server using the API of Stripe or PayPal. The usage fee collected is distributed fairly to the copyright holders of the characters and people and the service provider.

[1542] Specific examples

[1543] 1. User login and character selection

[1544] The user opens the app and logs in by entering "user123" and "password123" on the login screen. The device sends the input data to the server and authentication is performed. If authentication is successful, the home screen is displayed.

[1545] The user presses the "Character Selection" button on the home screen and selects "Character A." The device sends the selected character information to the server, and the server retrieves the corresponding AI data and sends it to the device.

[1546] 2. Initiating and conducting actual conversations

[1547] The user presses the "Start conversation" button and speaks "Hello, Character A" into the device's microphone. The device converts the speech into text and sends it to the server.

[1548] The server analyzes the text data and generates an appropriate response based on the AI ​​model, for example, "Hello, I'm Character A. How's your day?"

[1549] The generated response is converted into audio data and sent to the terminal, which plays the audio data so that the user can hear the response.

[1550] Prompt Sentence Examples

[1551] "Hi, I'm Character A. How's your day going?"

[1552] This system allows fans to safely and easily interact with their favorite characters and people 24 hours a day, 365 days a year, and also reduces the high costs associated with hosting events and inviting celebrities to attend.

[1553] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1554] Step 1: Data collection

[1555] The server collects data on the appearance, personality, and past behavior of specific characters or people. Inputs include YouTube video links and social media post URLs. The data is obtained using scraping technology and APIs. The output is the retrieved text, images, and audio data.

[1556] Step 2: Data Preprocessing

[1557] The server preprocesses the collected data. The raw data collected in step 1 is provided as input. Specifically, processes such as noise removal, text tokenization, and normalization are performed. For example, unnecessary special characters are removed from the text data and it is converted into a standard format. The output is the preprocessed data.

[1558] Step 3: Training the model

[1559] The server uses the preprocessed data to train an artificial intelligence model (e.g., TensorFlow or PyTorch). The preprocessed data from step 2 is provided as input. Specifically, the server splits the data into batches and optimizes the model parameters using a GPU. The output is a trained, highly accurate model.

[1560] Step 4: Evaluate and save the model

[1561] The server evaluates and saves the trained model. The model trained in step 3 and the validation dataset are used as input. Specifically, the accuracy of the model is evaluated using the validation data, and if the accuracy meets the criteria, the model is saved. The output is the evaluation result and the saved model.

[1562] Step 5: Save as an NFT

[1563] The server saves the trained AI model as an NFT. The model data saved in step 4 is used as input. Specifically, the server converts the model to ONNX format and issues an NFT on a blockchain platform (e.g., Ethereum or Binance Smart Chain). The output is the model data saved as an NFT.

[1564] Step 6: User authentication and character selection

[1565] The terminal displays a login screen for the user and performs authentication. A username and password are provided as input. The authentication data is sent to the server and checked against a database. If authentication is successful, a character selection screen is displayed. The output is the selected character information.

[1566] Step 7: Capturing and Converting Audio Input

[1567] The device uses a microphone to capture the user's voice. The user's voice data is provided as input. Specifically, the voice is converted into text data using the Google Cloud Speech-to-Text API. The output is text data.

[1568] Step 8: Response Generation

[1569] The server receives the text data and generates a response using an AI model. The text data converted in step 7 is provided as input. Specifically, the AI ​​model (e.g., GPT-3 or BERT) analyzes the text data and generates an appropriate response. The output is the text data of the generated response.

[1570] Step 9: Convert to audio and send

[1571] The server converts the generated text response into audio data and sends it to the device. The text data generated in step 8 is provided as input. Specifically, it converts it into audio data using Amazon Polly or Google Text-to-Speech. The output is audio data.

[1572] Step 10: Playing back real-time voice conversations

[1573] The device receives the audio data and plays it on the speaker. The input is the audio data sent in step 9. Specifically, it decodes the audio data and plays it on the speaker so that it can be heard by the user. The output is audio played in real time.

[1574] Step 11: Fee collection and distribution

[1575] The terminal obtains payment information to collect usage fees and sends it to the server. Payment information is provided as input. The server processes the payment using Stripe or PayPal APIs and distributes revenue among the parties involved. Specifically, after confirming the payment process, the distribution is made via bank transfer or cryptocurrency. The output is the distributed revenue.

[1576] (Application example 1)

[1577] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1578] Currently, providing high-quality customer service to customers visiting physical stores requires a large amount of labor and resources. It is also difficult to provide detailed information about specific products in the store, making it difficult to increase customer satisfaction. Furthermore, while there is a demand for personalized guidance that allows customers to select products based on their preferences and interests, achieving this is complex. To solve these challenges, a system that can interact with customers in real time and provide personalized guidance is needed.

[1579] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1580] In this invention, the server includes: means for learning the characteristics of a specific virtual character using machine learning technology; means for saving the learned information as a non-fungible character (NFT); means for providing an operation screen for a user to select a conversation partner and conduct a conversation; means for real-time voice communication; means for receiving a service fee from the user and distributing the revenue among the relevant parties; means for providing guidance using a display device that supports conversations between customers and virtual characters in physical stores; and means for users to obtain product information in the store and receive support through the character. This makes it possible to improve the customer experience in physical stores and provide personalized, high-quality customer service.

[1581] "Characteristics of a specific virtual person" refers to data that includes information such as the appearance, personality, and past words and actions of a character or celebrity.

[1582] "Machine learning technology" is a technology that allows computers to automatically learn from data and make predictions and classifications.

[1583] A "non-fungible token (NFT)" is a digital asset whose uniqueness and ownership are guaranteed using blockchain technology.

[1584] The "operation screen" is a display screen that provides an interface for a user to select a conversation partner and carry out a conversation.

[1585] "Real-time voice communication" refers to a technology that allows users to instantly communicate with selected characters via voice.

[1586] "Service fee" means the amount of money paid by a user as consideration for using a service.

[1587] "Related parties" refers to people or organizations that are subject to revenue sharing, such as character or system providers and developers.

[1588] A "display device" refers to hardware for displaying information, such as displays installed in physical stores, smart glasses, and head-mounted displays (HMDs).

[1589] "Supporting conversations between virtual characters" refers to a specific virtual character communicating with a user via voice to provide guidance and information.

[1590] "Get product information" means viewing or obtaining detailed information about products in the store.

[1591] "Receiving support through a character" means that a virtual person guides or explains products or services to the user.

[1592] To implement the present invention, the following system configuration is required: The system is composed of a server, a terminal, and a user as its main components.

[1593] 1. System Program

[1594] The server builds a database that uses machine learning technology to learn the characteristics of a specific virtual person. The terminal provides an operation screen that allows the user to select a conversation partner and engage in real-time voice communication with that virtual person. The server also stores the learned information as a non-fungible token (NFT).

[1595] The terminal displays an operation screen for the user to select a virtual person to talk to and transmits the selection information to the server. When the user communicates with the terminal in real time, the terminal converts the user's voice into text information and transmits the text information to the server.

[1596] The server then feeds the received text information to a machine learning model to generate a conversation-based response. The generated response is then converted into voice data by the server and sent to the device. The device then plays the voice data and provides the user with a response from a virtual character.

[1597] 2. Processing Description

[1598] The server is implemented using programming languages ​​such as Python, and uses machine learning technologies such as the NLP (natural language processing) model GPT-3. It also uses libraries such as Google Speech Recognition for speech recognition and pyttsx3 for speech synthesis.

[1599] The device includes hardware such as smart glasses, a head-mounted display (HMD), and a microphone. The device recognizes the user's speech and sends it to the server. The server converts the speech to text and generates an appropriate response using an NLP model. The server then converts the response back into audio data and sends it back to the device.

[1600] As a specific example, a user enters a physical store and puts on smart glasses. When the user says, "What products do you recommend?", the device's microphone picks up the user's voice and sends it to the server. The server analyzes the text data and generates a response using GPT-3. The response might be something like, "Currently, we recommend this smartwatch. It's the latest model with a long battery life and comprehensive health management features."

[1601] The generated response is converted into voice data and sent to the terminal, which then plays the voice data so that the user can hear the voice of the virtual character.

[1602] 3. Examples of concrete examples and prompts

[1603] As a practical example, consider a situation where a user wears smart glasses and asks for assistance in a store. For example, the user says, "Hello, what products do you recommend?"

[1604] Example prompt sentence:

[1605] Customer Question: "Hi, what products do you recommend?"

[1606] Prompt for generative AI model: "A customer asks you for a product recommendation. How would you respond?"

[1607] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1608] Program processing steps

[1609] Step 1:

[1610] A user puts on the smart glasses and launches an application. The input is the initialization data for the smart glasses, and the output is the display of a login screen. Specifically, the terminal displays the login screen to the user, and the user enters login information.

[1611] Step 2:

[1612] The user enters login information and presses the login button. The input is the login information, and the output is an authentication request to the server. The terminal sends the user's input information to the server, and the server performs authentication.

[1613] Step 3:

[1614] The server authenticates the user's login information and returns the result to the device. The input is the user's login information, and the output is the authentication result. Specifically, the server searches the database and verifies the login information. If successful, the home screen is displayed on the device.

[1615] Step 4:

[1616] The user selects a character to interact with from the home screen. The input is character selection information, and the output is transmission of the selection information to the server. The device sends the selected character information to the server, and the server obtains the corresponding learning data.

[1617] Step 5:

[1618] The server retrieves the learning data for the selected character and sends it to the terminal. The input is the character selection information and the output is the learning data. The server retrieves the relevant data from the learning database and sends it to the terminal.

[1619] Step 6:

[1620] The user presses the "Start conversation" button and speaks into the device's microphone. The input is the user's voice, and the output is text data. The device uses voice recognition to convert the user's voice into text, and sends the text data to the server.

[1621] Step 7:

[1622] The server receives the user's text data and generates an appropriate response using a generative AI model. The input is the user's text data and the output is the generated response. The server generates the response using an NLP model (e.g., GPT-3).

[1623] Step 8:

[1624] The response generated by the server is converted into audio data and sent to the device. The input is the generated response and the output is audio data. The server uses a speech synthesis library (e.g., pyttsx3) to generate the audio data and send it to the device.

[1625] Step 9:

[1626] The terminal plays back the voice data received from the server and lets the user listen. The input is voice data, and the output is voice playback. The terminal plays back the voice data, allowing the user to hear the virtual character's response.

[1627] Step 10:

[1628] When the user wants to end the service, he / she performs an operation to end the application. The input is the end operation and the output is the end of the application. The terminal ends the application and stops the display on the smart glasses.

[1629] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1630] This invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as a non-fungible token (NFT), and enables real-time voice conversations with users 24 hours a day, 365 days a year. In addition, by combining this invention with an emotion engine, it can recognize the user's emotions and provide adaptive responses based on those emotions.

[1631] System configuration

[1632] 1. Learning with AI technology

[1633] The server collects data on the appearance, personality, and past behavior of specific characters and people, and uses an artificial intelligence algorithm to learn this information. The learning results, including the character's specific behavior and response patterns, are saved as a highly accurate model.

[1634] 2. Save as an NFT

[1635] The server stores the learned AI data as an NFT, and uses blockchain technology to guarantee its uniqueness and ownership.

[1636] 3. Providing an interface

[1637] The terminal provides the user with an interface for selecting a conversation partner. The user can log in and select a character of their choice.

[1638] 4. Real-time voice conversation

[1639] The user performs an operation to start a conversation with the selected character. The device captures the user's voice, converts it into text data, and sends it to the server.

[1640] The server uses an AI model to generate an appropriate response based on the text data it receives, which is then converted back into audio data and sent to the device.

[1641] The terminal plays the audio data and the conversation takes place in real time.

[1642] 5. Emotion engine integration

[1643] The server is equipped with an emotion engine that analyzes the user's emotional state (e.g., joy, sadness, anger, etc.) from the user's voice data.

[1644] Based on the results of the emotion analysis, the AI ​​model adaptively changes the response content. For example, if the user speaks in a sad voice, the model generates a response such as "Cheer up!"

[1645] 6. Fee collection and revenue sharing

[1646] The server collects usage fees from users. Payments are processed through the terminal. Revenues are distributed among the parties involved, including the copyright holders of characters and people, and service providers.

[1647] Specific examples

[1648] 1. User logs in and selects a character

[1649] The user opens the app and logs in by entering "user123" and "password123" on the login screen. The device sends the entered data to the server and authentication is performed. If authentication is successful, the home screen is displayed on the device.

[1650] The user presses the "Character Selection" button on the home screen and selects, for example, "Character A." The device sends the selected character information to the server, and the server retrieves the corresponding AI data and sends it to the device.

[1651] 2. Initiating and conducting actual conversations

[1652] The user presses the "Start conversation" button and speaks "Hello, Character A" into the device's microphone. The device converts the speech into text and sends it to the server.

[1653] The server analyzes the text data and generates an appropriate response based on the AI ​​model, such as "Hello! How's your day?"

[1654] The server simultaneously transmits the voice data to the emotion engine to analyze the user's emotional state.

[1655] The emotion engine analyzes the user's emotional state and feeds the results back to the AI ​​model. For example, if the user sounds sad, the response will be changed to an encouraging message like "Cheer up!"

[1656] The generated response is converted into voice data and sent to the terminal, which plays the voice data and provides the response to the user.

[1657] 3. Payment of fees and revenue sharing

[1658] If the user needs to pay a usage fee, the terminal displays a payment screen where the user enters credit card information, etc.

[1659] The terminal sends the entered payment information to the server, which then processes the payment through a payment gateway.

[1660] The server sends information about the success or failure of the payment to the terminal. If successful, the usage rights are updated.

[1661] The terminal will notify the user of the payment result, allowing them to continue using the system.

[1662] This system allows fans to safely and easily interact with their favorite characters and people 24 hours a day, 365 days a year, and by providing appropriate responses based on the user's emotions, it enables more personal and intimate communication.

[1663] The processing flow will be explained below.

[1664] Step 1:

[1665] The user opens the application. The user accesses the login screen and enters their ID and password.

[1666] Step 2:

[1667] The terminal receives the user's input and sends the ID and password information to the server. The communication is encrypted.

[1668] Step 3:

[1669] The server checks the received ID and password against the database and performs user authentication. If the ID and password do not match, an error message is sent to the terminal.

[1670] Step 4:

[1671] If the authentication is successful, the server generates session information and sends it back to the device. The device uses the received session information to display the home screen.

[1672] Step 5:

[1673] The user presses the "Select Character" button on the home screen and selects the character they want to interact with.

[1674] Step 6:

[1675] The device receives the user's selection and sends the character ID to the server, which then retrieves the AI ​​model data for the selected character.

[1676] Step 7:

[1677] The server sends the acquired AI model data to the device, which then displays a conversation interface with the character based on the received data.

[1678] Step 8:

[1679] The user presses the "Start conversation" button to start the conversation, and the device's microphone is turned on to capture the user's voice.

[1680] Step 9:

[1681] The device records the user's voice and converts the voice data into text data in real time. This text data is then sent to the server.

[1682] Step 10:

[1683] The server analyzes the received text data and generates a response using an AI model. For example, if the user says "Hello, Character A," the response is "Hello! How's your day?"

[1684] Step 11:

[1685] At the same time, the server sends the user's voice data to the emotion engine to analyze the user's emotional state. The emotion engine analyzes the voice data and identifies the user's emotional state (such as joy, sadness, anger, etc.).

[1686] Step 12:

[1687] The emotion engine sends the analysis results back to the server, which then adaptively changes the response based on the results. For example, if the user speaks in a sad voice, the response might be "Cheer up!"

[1688] Step 13:

[1689] The server converts the response text into voice data and sends it to the terminal.

[1690] Step 14:

[1691] The terminal plays back the received voice data and provides a response to the user, who can then ask questions or make comments.

[1692] Step 15:

[1693] If the user needs to pay a usage fee, the terminal displays a payment screen, and the user enters credit card information, etc.

[1694] Step 16:

[1695] The terminal sends the entered payment information to the server, and the server processes the payment through a payment gateway.

[1696] Step 17:

[1697] The server sends information about the success or failure of the payment to the terminal. If the payment is successful, the usage rights are updated.

[1698] Step 18:

[1699] The terminal notifies the user of the payment result, allowing the user to continue using the system.

[1700] Example 2

[1701] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1702] Conventional AI-based voice conversation systems have struggled to efficiently learn the appearance and personality of characters and people and conduct real-time voice conversations with users. Furthermore, the technology for providing adaptive responses based on the user's emotions has not been sufficiently developed, making it impossible to achieve natural, personal communication with users.

[1703] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1704] In this invention, the server includes means for learning the appearance and personality of a specific character or person using artificial intelligence technology, means for saving the learned data as a non-fungible token (NFT), and means for analyzing emotions from the user's voice and adaptively changing responses based on the analysis results. This makes it possible to learn the characteristics of characters and people with high accuracy, safely save that data, and provide flexible responses according to the user's emotions.

[1705] "Specific character or person" refers to a fictional or real person or character that a user can select as a conversation partner.

[1706] "Artificial intelligence technology" refers to the use of techniques such as machine learning and deep learning to extract patterns from data and make predictions and classifications.

[1707] "Means of learning" refers to the process of collecting data and generating an AI model based on that data.

[1708] A "non-fungible token (NFT)" refers to a token that uses blockchain technology to guarantee the uniqueness and ownership of digital data.

[1709] "Means for providing an interface" refers to the method of providing screens and menus that allow users to access and operate the system.

[1710] "Means for conducting voice conversation in real time" refers to technology that allows a user and a character to have an instantaneous voice conversation.

[1711] "Means for analyzing emotions from a user's voice" refers to technology that analyzes a user's emotional state based on their voice data.

[1712] "Means for adaptively changing responses" refers to a method for changing the content of AI responses based on the results of emotion analysis.

[1713] "Means for collecting usage fees and distributing revenue among the parties involved" refers to a mechanism for collecting service usage fees from users and distributing the revenue obtained among the parties involved.

[1714] MODE FOR CARRYING OUT THE INVENTION

[1715] This invention is a system that uses artificial intelligence technology to learn the appearance and personality of specific characters or people, stores the learned data as non-fungible tokens (NFTs), and provides users with real-time voice conversations 24 hours a day, 365 days a year. Furthermore, by combining it with an emotion engine, it can recognize the user's emotions and provide adaptive responses based on those emotions.

[1716] System Overview

[1717] The system consists of the following main components:

[1718] 1. Server

[1719] 2. Terminal

[1720] 3. Users

[1721] Server Roles

[1722] 1. Collecting training data and generating AI models

[1723] The server collects data on the appearance, personality, and past behavior of specific characters or people, including video, audio, and text data.

[1724] The collected data is trained using an AI algorithm (e.g., TensorFlow or PyTorch). At the initial training stage, data preprocessing (noise removal, text normalization, etc.) is performed.

[1725] Save the completed model to the database.

[1726] 2. Data storage as NFTs

[1727] The trained AI data is stored as a non-fungible token (NFT), and this process uses blockchain technology (e.g., Ethereum).

[1728] The server registers the generated NFT on the blockchain and maintains the owner information.

[1729] 3. Sentiment analysis and response adaptation

[1730] The server uses an emotion engine (for example, IBM Watson's emotion recognition API) to analyze the user's voice data. Based on the analysis results, the AI ​​model adaptively changes the response content.

[1731] For example, if a user says, "I'm feeling a bit down today," emotion recognition can generate a response such as, "Cheer up!"

[1732] 4. Usage Fee Collection and Revenue Sharing

[1733] The server collects the usage fee and manages the payment process, which is carried out using credit card information entered by the user on the terminal.

[1734] Revenues are distributed among the parties involved (character copyright holders, service providers, etc.).

[1735] Device Role

[1736] 1. Providing an interface

[1737] The device provides the user with an interface to select a conversation partner and conduct a conversation. The user launches the app, enters their authentication information on the login screen, and logs in.

[1738] After logging in, users can press the "Select Character" button to choose their preferred character and begin a conversation with that character.

[1739] 2. Audio Data Processing

[1740] The device captures the user's voice and converts it into text data using a speech recognition engine such as Google Speech-to-Text.

[1741] The converted text data is sent to the server, and after an appropriate response is generated, it is converted back into voice data and sent to the terminal.

[1742] The terminal plays the audio data, enabling real-time conversation.

[1743] User Roles

[1744] 1. Log in and select a character

[1745] The user launches the app and enters authentication information such as "user123" and "password123" on the login screen.

[1746] Once authentication is successful, select a character from the home screen and begin a conversation with the selected character.

[1747] 2. Actual conversation progression

[1748] The user presses the "Start conversation" button and speaks into the device's microphone, saying "Hello, Character A."

[1749] The device converts this speech into text data and sends it to the server, which uses the text data to generate a response using an AI model and sends an adaptive reply.

[1750] For example, you can receive responses such as "Hello! How's your day?" or adaptive messages such as "Cheer up!"

[1751] Specific examples and prompt sentence examples

[1752] "Hello, Character A. How was your day?"

[1753] "I've been feeling a bit down lately. Can you cheer me up?"

[1754] "What do you think about today's news?"

[1755] By using these prompts, users can enjoy more personal and natural conversations with the characters.

[1756] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1757] Program processing flow

[1758] Step 1:

[1759] Learning data collection and AI model generation

[1760] The server collects data on the appearance, personality, and past behavior of specific characters or people. The data used for this collection includes text, audio files, images, and videos. Specifically, the data is retrieved from a database via web scraping or API and stored on the server.

[1761] Input: Data about the character's appearance, personality, and past behavior.

[1762] Data processing: noise removal, text normalization, data cleaning.

[1763] Output: A clean dataset after preprocessing.

[1764] The server uses AI algorithms (such as TensorFlow or PyTorch) to train the model based on this data. The trained model then reflects the character's behavior and response patterns.

[1765] Input: Clean dataset after preprocessing.

[1766] Data computation: Applying machine learning algorithms and training models.

[1767] Output: A trained AI model.

[1768] Step 2:

[1769] Storing data as NFTs

[1770] The server stores the learned AI data as a non-fungible token (NFT).

[1771] Input: A trained AI model.

[1772] Data processing: Blockchain technology (e.g., Ethereum) is used to guarantee the uniqueness and ownership of the model for NFT conversion.

[1773] Output: An NFT registered on the blockchain.

[1774] The server registers the generated NFT on the blockchain and stores the owner information. Specifically, the server registers the NFT through the blockchain API.

[1775] Step 3:

[1776] User login and character selection

[1777] The user launches the app and enters their authentication information (e.g., "user123" and "password123") on the login screen.

[1778] The terminal transmits the input authentication information to the server and performs authentication.

[1779] Input: User credentials.

[1780] Data calculation: Check against the database to determine whether authentication is successful.

[1781] Output: Authentication success or failure status.

[1782] If authentication is successful, the home screen will be displayed on the device, and the user can press the "Select Character" button to choose the character they want to interact with.

[1783] Input: Character information selected by the user.

[1784] Data processing: Search and obtain the AI ​​model corresponding to the selected character.

[1785] Output: AI data of the selected character.

[1786] Step 4:

[1787] Real-time voice conversation processing

[1788] The user presses the "Start conversation" button and speaks into the device's microphone.

[1789] The device captures the user's voice and converts it into text data using Google Speech-to-Text.

[1790] Input: User's voice data.

[1791] Data processing: speech-to-text conversion.

[1792] Output: Text data.

[1793] The converted text data is sent to the server, which generates an appropriate response.

[1794] Input: User's text data.

[1795] Data Computation: Using AI models to generate appropriate responses.

[1796] Output: The generated text response.

[1797] The response is converted back into voice data and sent to the terminal.

[1798] Input: The generated text response.

[1799] Data processing: Text to speech conversion.

[1800] Output: The generated audio data.

[1801] The terminal plays the audio data and the conversation takes place in real time.

[1802] Step 5:

[1803] Sentiment analysis and response adaptation

[1804] The server analyzes the user's voice data using an emotion engine (e.g., IBM Watson's emotion recognition API).

[1805] Input: User's voice data.

[1806] Data calculation: Analyze the user's emotional state from voice data.

[1807] Output: User's emotional state data.

[1808] Based on the results of the sentiment analysis, the AI ​​model adaptively changes the response content.

[1809] Input: User emotional state data and generated text responses.

[1810] Data calculation: Modifying the content of responses to match emotional states.

[1811] Output: The final adapted response text.

[1812] For example, if the user sounds sad, the response may be changed to something like "Cheer up!"

[1813] Step 6:

[1814] Collection of usage fees and payment processing

[1815] If the user needs to pay a usage fee, the terminal displays a payment screen where the user enters credit card information, etc.

[1816] Input: User's payment information.

[1817] The terminal sends payment information to the server, and the server processes the payment through a payment gateway (e.g., Stripe).

[1818] Data calculation: Executes payment processing based on payment information.

[1819] Output: Payment success or failure status.

[1820] The server notifies the terminal of the success or failure of the payment, and if successful, the usage rights are updated.

[1821] The terminal notifies the user of the payment result, allowing the user to continue using the system.

[1822] (Application example 2)

[1823] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1824] The objective of this invention is to provide a system that can reproduce the appearance and personality of a specific character or person in real time, while providing appropriate responses according to the user's emotions through dialogue with the character. Furthermore, by having the character act as a shopping assistant in a virtual store, explaining products and recommending items, we aim to realize a more personalized and sympathetic shopping experience.

[1825] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1826] In this invention, the server includes: means for learning the appearance and personality of a specific character or person using artificial intelligence technology; means for storing the learned data as a non-fungible token (NFT); means for providing an interface for a user to select a conversation partner and converse with them; means for conducting real-time voice conversations; means for collecting usage fees from users and distributing revenue among the parties; means for analyzing the emotional state from the user's voice data using an emotion engine and generating an adaptive response based on the analysis results; means for converting the generated response into voice data and providing it to the user as voice; and means for the character to act as a shopping assistant in a virtual store, explaining products and recommending items. This enables real-time, emotion-based conversations with a specific character and a personalized, sympathetic shopping assistant experience in a virtual store.

[1827] "Specific characters or people" are virtual beings whose appearances and personalities are reproduced using artificial intelligence technology.

[1828] "Artificial intelligence technology" is a technology that learns the appearance and personality of a specific character or person and generates responses and actions based on that information.

[1829] A non-fungible token (NFT) is a means of uniquely identifying and proving ownership of a specific digital asset using blockchain technology.

[1830] An "interface" is a device or software that provides the input and display means for a user to select a conversation partner and initiate a conversation.

[1831] "Means for conducting voice conversations in real time" refers to technology that enables a user to have an instant voice conversation with a specific character or person.

[1832] An "emotion engine" is an algorithm and system that analyzes a user's emotional state from their voice data and generates a response based on the analysis results.

[1833] An "adaptive response" is a response that changes depending on the user's utterance content and emotional state, and is generated individually and dynamically.

[1834] A "virtual store" is a virtual shopping environment that exists on the Internet, and is a platform that allows users to make purchases in a virtual space.

[1835] A "shopping assistant" is a virtual character or person who explains products and recommends items to users in a virtual store.

[1836] "Collection" means receiving usage fees from users.

[1837] "Means for distributing revenue among parties involved" refers to a mechanism for appropriately distributing revenue obtained through system operation among parties involved, such as character copyright holders and service providers.

[1838] System Overview

[1839] This invention is a system that reproduces specific characters or people in real time and engages in voice conversations with users. It mainly functions as a shopping assistant in virtual stores. This system consists of three elements: a server, a terminal, and a user.

[1840] Program structure and various technologies

[1841] Speech Recognition Engine

[1842] The server receives the user's voice input and converts the voice into text data using a speech recognition engine (Google Speech-to-Text API).

[1843] Sentiment Analysis Engine

[1844] The server performs emotion analysis on the acquired text data using IBM Watson Tone Analyzer, thereby determining the user's emotional state.

[1845] Dialogue Management System

[1846] The server uses Rasa to generate responses based on the user's conversation content and sentiment analysis results.

[1847] Speech synthesis engine

[1848] The server converts the generated response into audio data using the Google Text-to-Speech API and sends it to the device.

[1849] Program processing

[1850] 1. Voice to Text

[1851] When a user speaks into the smartphone's microphone, the audio is picked up by the device and converted into text data using the Google Speech-to-Text API.

[1852] 2. Emotion analysis

[1853] The transferred text data undergoes emotion analysis using IBM Watson Tone Analyzer to analyze the user's emotional state (e.g., joy, sadness, anger, etc.).

[1854] 3. Response Generation

[1855] It uses Rasa to generate appropriate responses based on the analysis and conversational context, which are adapted to the user's emotions.

[1856] 4. Conversion to audio data and transmission

[1857] The generated response is converted into audio data using the Google Text-to-Speech API, and this audio data is sent to the device, enabling real-time conversation.

[1858] Specific examples

[1859] 1. User login and character selection

[1860] The user starts the app and logs in by entering "user123" and "password123." After logging in, the user selects "Character A" on the character selection screen.

[1861] Example prompt sentence:

[1862] User: Hello, I'm looking for some clothes today.

[1863] ---

[1864] Assistant: Hello, what kind of clothes are you looking for?

[1865] ---

[1866] User: I'd like a casual shirt.

[1867] ---

[1868] 2. Voice input and analysis

[1869] When a user says, "Hello, I'm looking for clothes today," the speech is captured and converted into text data, which is sent to IBM Watson Tone Analyzer for sentiment analysis.

[1870] 3. Response Generation and Serving

[1871] Based on sentiment analysis and the conversation, Rasa generates a response such as "Hello, what kind of clothes are you looking for?" This response is converted to audio using the Google Text-to-Speech API and sent to the device.

[1872] 4. Shopping Assistant

[1873] If a user says, "I want a casual shirt," the assistant will suggest, "How about this shirt? It has a trendy design and is very popular these days," and explain the product. It is also possible to provide more detailed product descriptions and suggestions based on the results of user sentiment analysis.

[1874] This allows the invention to provide emotion-based real-time interaction with specific characters and a personalized and sympathetic shopping assistant experience in virtual stores.

[1875] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1876] Step 1:

[1877] The user launches the app and logs in. The user enters their username and password, and the device sends that information to the server. The server performs authentication, and if successful, displays the character selection screen on the device.

[1878] Input: User's username and password

[1879] Process: The server authenticates the user

[1880] Output: Character selection screen will be displayed upon successful authentication.

[1881] Step 2:

[1882] The user selects a character. The user chooses a specific character on the character selection screen, and the device sends the selection information to the server. The server loads the AI ​​model of the selected character and sends the related data to the device.

[1883] Input: User's character selection information

[1884] Processing: The server loads the AI ​​model and retrieves the relevant data.

[1885] Output: Character information is displayed on the terminal.

[1886] Step 3:

[1887] The user starts a conversation. The user speaks into the smartphone's microphone, and the device picks up the audio. The speech is converted into text data using a speech recognition engine (Google Speech-to-Text API), and the text data is sent to the server.

[1888] Input: User's voice

[1889] Processing: The device receives the voice and converts it into text data.

[1890] Output: The converted text data is sent to the server.

[1891] Step 4:

[1892] The server performs emotion analysis. The server sends the acquired text data to an emotion analysis engine (IBM Watson Tone Analyzer) to analyze the user's emotional state. The analysis results are added to the text data.

[1893] Input: Text data

[1894] Processing: The server analyzes the emotional state using an emotion analysis engine.

[1895] Output: Text data with emotional states added

[1896] Step 5:

[1897] The server generates a response. The server uses a dialogue management system (Rasa) to generate a response based on the sentiment analysis results and the conversation content. The response is in text format.

[1898] Input: Text data with sentiment analysis results added

[1899] Processing: The server generates a response using a dialogue management system

[1900] Output: The generated response text

[1901] Step 6:

[1902] The server converts the response text into speech. The generated response text is converted into audio data using a speech synthesis engine (Google Text-to-Speech API). The converted audio data is then sent back to the device.

[1903] Input: Generated response text

[1904] Processing: The server uses a speech synthesis engine to convert the data into voice data.

[1905] Output: Converted audio data

[1906] Step 7:

[1907] The device plays the audio data. The audio data sent to the device is played back through the speaker and can be heard by the user. This allows for real-time audio conversation between the user and the character.

[1908] Input: Audio data

[1909] Processing: The device plays the audio data.

[1910] Output: The user hears the response

[1911] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1912] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1913] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1914] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1915] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1916] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1917] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1918] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1919] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1920] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1921] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1922] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1923] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1924] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1925] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1926] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1927] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1928] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1929] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1930] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1931] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1932] The following is further disclosed regarding the above embodiment.

[1933] (Claim 1)

[1934] A means for learning the appearance and personality of a specific character or person using artificial intelligence technology;

[1935] A means of storing the learned data as a non-fungible token (NFT);

[1936] means for providing an interface for a user to select a conversation partner and converse with;

[1937] a means for conducting real-time voice conversations;

[1938] A means of collecting fees from users and distributing the revenue among the parties involved;

[1939] A system including:

[1940] (Claim 2)

[1941] 10. The system of claim 1, further comprising means for using the learned artificial intelligence data to generate a response based on the user's conversation content and converting the response into voice data.

[1942] (Claim 3)

[1943] 10. The system of claim 1, further comprising means for converting user input speech into text data and providing the text data to the artificial intelligence.

[1944] "Example 1"

[1945] (Claim 1)

[1946] A means for learning the appearance and personality of a specific character or person using artificial intelligence technology;

[1947] A means of storing the learned data as a non-fungible token (NFT);

[1948] means for providing an interface for a user to select a conversation partner and converse with;

[1949] A means for acquiring a user's voice and converting it into text data using voice recognition technology;

[1950] A means for generating an appropriate response using an AI model based on the text data and converting the response into voice data;

[1951] means for transmitting the converted voice data to a user terminal and conducting a voice conversation in real time;

[1952] A means of collecting fees from users and distributing the revenue among the parties involved;

[1953] A system including:

[1954] (Claim 2)

[1955] 10. The system of claim 1, further comprising means for using the learned artificial intelligence data to generate a response based on the user's conversation content and converting the response into voice data.

[1956] (Claim 3)

[1957] 10. The system of claim 1, further comprising means for converting user input speech into text data and providing the text data to the artificial intelligence.

[1958] "Application Example 1"

[1959] (Claim 1)

[1960] A means for learning the characteristics of a specific virtual person using machine learning technology;

[1961] a means for storing the learned information as a non-fungible signature (NFT);

[1962] means for providing an operation screen for a user to select a conversation partner and carry out a conversation;

[1963] a means for real-time voice communication;

[1964] a means for collecting service fees from users and distributing the revenue among the relevant parties;

[1965] a means for providing guidance using a display device that supports conversations between customers and virtual characters in a real store;

[1966] A means for users to obtain product information in stores and receive support through characters,

[1967] A system including:

[1968] (Claim 2)

[1969] 10. The system of claim 1, further comprising means for using the learned machine learning data to generate a response based on a user's voice input and converting the response into voice data.

[1970] (Claim 3)

[1971] 10. The system of claim 1, further comprising means for converting a user's voice input into text information and feeding the text information into machine learning.

[1972] "Example 2: Combining Emotion Engines"

[1973] (Claim 1)

[1974] A means for learning the appearance and personality of a specific character or person using artificial intelligence technology;

[1975] A means of storing the learned data as a non-fungible token (NFT);

[1976] means for providing an interface for a user to select a conversation partner and converse with;

[1977] a means for conducting real-time voice conversations;

[1978] means for analyzing emotions from the user's voice and adaptively changing a response based on the analysis result;

[1979] A means of collecting fees from users and distributing the revenue among the parties involved;

[1980] A system including:

[1981] (Claim 2)

[1982] 10. The system of claim 1, further comprising means for using the learned artificial intelligence data to generate a response based on the content of the user's conversation and converting the response into voice data.

[1983] (Claim 3)

[1984] 2. The system according to claim 1, further comprising means for converting a user's voice input into text data and supplying the text data to the artificial intelligence.

[1985] "Application example 2 when combining emotion engines"

[1986] (Claim 1)

[1987] A means for learning the appearance and personality of a specific character or person using artificial intelligence technology;

[1988] A means of storing the learned data as a non-fungible token (NFT);

[1989] means for providing an interface for a user to select a conversation partner and converse with;

[1990] a means for conducting real-time voice conversations;

[1991] A means of collecting fees from users and distributing the revenue among the parties involved;

[1992] means for analyzing an emotional state of a user from voice data using an emotion engine and generating an adaptive response based on the analysis result;

[1993] means for converting the generated response into voice data and providing it to the user as voice;

[1994] As a shopping assistant in a virtual store, the characters will explain products and introduce recommended items,

[1995] A system including:

[1996] (Claim 2)

[1997] 10. The system of claim 1, further comprising means for using the learned artificial intelligence data to generate a response based on the user's conversation content and sentiment analysis results, and converting the response into voice data.

[1998] (Claim 3)

[1999] 10. The system of claim 1, further comprising means for converting a user's input speech into text data and providing the text data and a result of sentiment analysis to the artificial intelligence. [Explanation of symbols]

[2000] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for learning the appearance and personality of a specific character or person using artificial intelligence technology; A means of storing the learned data as a non-fungible token (NFT); means for providing an interface for a user to select a conversation partner and converse with; a means for conducting real-time voice conversations; A means of collecting fees from users and distributing the revenue among the parties involved; A system including:

2. 10. The system of claim 1, further comprising means for using the learned artificial intelligence data to generate a response based on the content of the user's conversation and converting the response into voice data.

3. 2. The system according to claim 1, further comprising means for converting speech input by a user into text data and supplying the text data to the artificial intelligence.

Images