system

Abstract

We provide the system. [Solution] A means of accepting access from the user's device, A means for authenticating the user's login information, means for receiving and analyzing messages sent from the aforementioned terminal, Means for generating a response based on the analysis results, Means for transmitting the generated response to the terminal, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is a problem that the opportunity for direct interaction between an idol and fans is limited, and it is difficult for fans to enjoy conversations with the idol. Therefore, in order to improve fan satisfaction, a system that can interact with a character similar to an idol is necessary. Also, in order to realize natural conversations, a method for analyzing messages and generating appropriate responses is important.

Means for Solving the Problems

[0005] In order to solve the above problems, the present invention provides the following means.

[0006] The system includes means for receiving access from a user's terminal, means for authenticating the user's login information, means for receiving and analyzing messages sent from the terminal, means for generating a response based on the analysis results, and means for sending the generated response to the terminal. Furthermore, the analysis means is means for analyzing messages using a natural language processing algorithm, and the response generation means is means for selecting and generating a response based on the personality of an idol character. This allows fans to enjoy natural conversations with characters that resemble idols.

[0007] "User's device" refers to a device such as a smartphone or personal computer used by a fan, and is the means by which they access the idol chatbot system.

[0008] "Means of accepting access" refers to the interface or process that allows users to log in to the system or send messages through their devices.

[0009] "Login information" refers to authentication information such as usernames and passwords that users need to access the system.

[0010] "Authentication methods" refer to the process of verifying the entered login information and confirming that the user has legitimate access rights.

[0011] "Means for receiving and analyzing messages" refers to algorithms and processes that receive text messages sent by users and analyze their content in order to understand it.

[0012] "Natural language processing algorithms" refer to technologies and programs used to analyze human language and understand its meaning.

[0013] "Means of generating a response" refers to the process or algorithm that creates an appropriate response based on the content of the analyzed message.

[0014] A "response generation engine" refers to software or a module that generates a response based on analysis results.

[0015] "Idol character personality" refers to the unique characteristics and behavioral style of a particular idol, and is the information referenced when generating responses.

[0016] "Means for transmitting a response to the terminal" refers to the process or communication means for delivering the generated response to the user's terminal. [Brief explanation of the drawing]

[0017] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), etc.

[0021] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. An embodiment of this system is described below.

[0039] System Configuration

[0040] This system consists of a server, a user terminal, and an idle chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission. The user terminal is used by the user to access the system and interact with the chatbot. The chatbot runs on the server and generates responses based on the idle character.

[0041] Program processing

[0042] The following explains in detail how the system works.

[0043] 1. The user accesses from their device.

[0044] The user launches the chat app on their device and accesses the system. Here, the user is redirected to a login screen where they enter their username and password.

[0045] 2. The server authenticates the login information.

[0046] The server receives the user's login information, compares it with the database, and performs authentication. If authentication is successful, the user can proceed to the chat screen.

[0047] 3. The user enters a message.

[0048] The user enters a message in the input field displayed on the chat screen and clicks the send button.

[0049] 4. The server receives and parses the message.

[0050] The server receives messages sent by users. Next, it uses natural language processing algorithms to analyze the messages and understand the user's intentions and emotions.

[0051] 5. The server generates a response.

[0052] Based on the analysis results, the server uses a response generation engine to generate an appropriate response. This response is selected and generated based on the personality of the idol character.

[0053] 6. The server sends a response to the user's terminal.

[0054] The server sends the generated response to the user's terminal and displays it on the chat screen.

[0055] Specific example

[0056] For example, consider the case where a user says to the chatbot, "The weather's nice today."

[0057] 1. The user types the message "The weather is nice today" from their device and presses the send button.

[0058] 2. The server receives this message and parses it using a natural language processing algorithm.

[0059] 3. The server identifies the topic "weather" from the analysis results and further recognizes the associated emotion as "positive."

[0060] 4. The server uses its response generation engine to generate the response, "It's such lovely weather! Shall we go for a walk?"

[0061] 5. The server sends this response to the user's device and displays it on the user's chat screen.

[0062] In this way, users can enjoy natural conversations with idol characters. This system can greatly improve user satisfaction even in situations where opportunities to directly interact with idols are limited.

[0063] The following describes the processing flow.

[0064] Step 1:

[0065] The user launches the "Idle Chatbot" app on their device and accesses the login screen. The user enters their username and password as login information.

[0066] Step 2:

[0067] The terminal sends the entered login information to the server.

[0068] Step 3:

[0069] The server compares the received login information with the database and performs authentication. If authentication is successful, the user is redirected to the chat screen.

[0070] Step 4:

[0071] The user enters a message in the input field displayed on the chat screen and clicks the send button. For example, they might type "Hello".

[0072] Step 5:

[0073] The terminal sends the entered message to the server.

[0074] Step 6:

[0075] The server analyzes the messages it receives using natural language processing algorithms. For example, it recognizes the message "Hello" as a "greeting."

[0076] Step 7:

[0077] Based on the analysis results, the server uses a response generation engine to generate an appropriate reply. For example, it might generate a reply such as, "Hi! How are you doing today?"

[0078] Step 8:

[0079] The server sends the generated response to the user's terminal.

[0080] Step 9:

[0081] The device displays the response it received on the chat screen. The user's device displays the message, "Hi! How are you doing today?"

[0082] Step 10:

[0083] The user types a new message and clicks the send button. For example, they might type, "I'm really busy today."

[0084] Step 11:

[0085] The device sends a new message to the server.

[0086] Step 12:

[0087] The server then analyzes the new message again using a natural language processing algorithm. For example, it analyzes the message "I'm really busy today" and recognizes the state of being "busy."

[0088] Step 13:

[0089] The server uses the response generation engine again to generate a response based on the new analysis results. For example, it might generate a response like, "Thank you so much for your hard work! Is there anything I can help you with?"

[0090] Step 14:

[0091] The server sends a new response to the user's device, which then displays it on the chat screen.

[0092] (Example 1)

[0093] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0094] Traditional chat systems made it difficult for users to enjoy natural conversations with artificial intelligence, and in particular, it was challenging to create conversations that reflected the unique personality of idol characters. Furthermore, achieving natural conversations required advanced processing, highlighting the need for improved user experience.

[0095] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0096] In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, and means for receiving and analyzing messages sent from the terminal. This allows the user to enjoy natural conversations with an idol character by using natural language processing algorithms and generative AI models in the message analysis and response generation means. Furthermore, by including means for adjusting responses based on the idol character's personality, responses that reflect the character's characteristics can be provided, resulting in a more satisfying user experience.

[0097] "Means of accepting access" refers to devices or software that provide the functionality for users to connect to the system from their terminals and initiate communication.

[0098] "Means for authenticating login information" refers to devices or software used to verify the username and password entered by the user and to confirm their identity.

[0099] "Means for receiving and analyzing messages" refers to a device or software that receives text messages sent from a terminal and processes them in order to understand their content.

[0100] "Means for generating a response" refers to a device or software for creating an appropriate response based on an analyzed message.

[0101] "Means for sending a response" refers to the device or software necessary to send the generated response to the user's terminal and continue the conversation.

[0102] A "natural language processing algorithm" is a set of computational methods and models used to understand and generate human language.

[0103] A "generative AI model" is a type of artificial intelligence that generates new data or text based on input, and operates based on specific rules or training data.

[0104] A "prompt" is input text provided to a generative AI model to elicit a specific response.

[0105] An "idol character" is a character that imitates a fictional entertainer or celebrity, possessing specific personalities and characteristics.

[0106] "Personality" refers to the unique character, speaking style, and behavior of an idol character.

[0107] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. Specific embodiments of this system are described below.

[0108] System Configuration

[0109] This system consists of a server, a user terminal, and an idle chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission, while the user terminal is used by the user to access the system and interact with the chatbot. The chatbot runs on the server and generates responses based on the idle character.

[0110] Hardware and software to be used

[0111] server:

[0112] Database server (e.g., MySQL®)

[0113] Natural language processing algorithms (e.g., Google® Cloud Natural Language API)

[0114] Response generation engine (e.g., OpenAI® GPT-3®)

[0115] User's device:

[0116] Chat application installed on a smartphone or computer

[0117] Detailed Operation Description

[0118] 1. The user accesses from a device:

[0119] The user launches a chat application on their smartphone or computer and is redirected to the login screen.

[0120] The user enters their username and password and clicks the "Login" button.

[0121] 2. The server authenticates the login information:

[0122] The server receives the login information entered by the user, compares it with the registered information in the database, and performs authentication.

[0123] If authentication is successful, the server creates a session for the user, and the user proceeds to the main chat screen.

[0124] 3. The user enters a message:

[0125] The user types a message on the main chat screen and clicks the "Send" button.

[0126] 4. The server receives and parses the message:

[0127] The server receives messages sent by users and parses them using the Google Cloud Natural Language API.

[0128] The server understands the user's intentions and emotions from the analysis results.

[0129] 5. The server generates a response:

[0130] Based on the analysis results, the server generates an appropriate response using OpenAI GPT-3.

[0131] The generated responses are tailored based on the personality of the idol character.

[0132] 6. The server sends a response to the user's terminal:

[0133] The server sends the generated response to the user's terminal and displays it on the chat screen.

[0134] Specific example

[0135] For example, if a user says to the chatbot, "The weather is nice today," the following process will occur.

[0136] The user types the message "The weather's nice today" into the chat app on their device and presses the send button.

[0137] The server receives this message and parses it using the Google Cloud Natural Language API.

[0138] The server identifies the topic "weather" from the analysis results and further recognizes the associated emotion as "positive."

[0139] The server uses OpenAI GPT-3 to generate the response, "It's such nice weather! Shall we go for a walk?"

[0140] The server sends this response to the user's device and displays it on the chat screen.

[0141] Example of a prompt

[0142] The following are examples of prompt statements for a generative AI model.

[0143] The user typed "The weather's nice today." Generate a response from the idol character to this message.

[0144] This system allows users to enjoy natural conversations with idol characters, providing a high level of satisfaction even in situations where opportunities to directly interact with idols are limited.

[0145] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0146] Step 1:

[0147] Users access from their devices

[0148] Input: The user launches the chat app on their smartphone or computer and enters their username and password on the login screen.

[0149] Data processing: The terminal sends the user's login information to the server.

[0150] Output: The user's login information (username and password) is sent to the server.

[0151] Specific actions:

[0152] The user launches the chat app by tapping it from the device's home screen.

[0153] Enter your username and password on the login screen and click the "Login" button.

[0154] Step 2:

[0155] The server authenticates the login information.

[0156] Input: The user's login information (username and password) is sent to the server.

[0157] Data processing: The server receives login information, compares it with the database, and performs authentication.

[0158] Output: An authentication result (success or failure) is generated.

[0159] Specific actions:

[0160] The server verifies the username and password against the database.

[0161] Once authentication is successful, the server creates a session and sends an instruction to the user to display the main chat screen.

[0162] Step 3:

[0163] The user enters a message.

[0164] Input: The user enters a message in the input field displayed on the main chat screen and clicks the "Send" button.

[0165] Data processing: The terminal sends the message entered by the user to the server.

[0166] Output: The user's message is sent to the server.

[0167] Specific actions:

[0168] The user types "The weather is nice today" into the input field on the chat screen.

[0169] The user clicks the "Send" button.

[0170] Step 4:

[0171] The server receives and parses the message.

[0172] Input: The message sent by the user is sent to the server.

[0173] Data processing: The server receives the message and analyzes its content using the Google Cloud Natural Language API. The analysis extracts the message's intent and sentiment.

[0174] Output: Analysis results (such as "weather" and "positive emotions") are generated.

[0175] Specific actions:

[0176] The server receives the user's message.

[0177] Using the Google Cloud Natural Language API, the message content is analyzed to identify "weather" and "positive emotions."

[0178] Step 5:

[0179] The server generates a response.

[0180] Input: Analysis results (such as "weather" and "positive emotions") are sent to the server.

[0181] Data processing: The server uses OpenAI GPT-3 to generate an appropriate response based on the analysis results. The generated response is adjusted based on the personality of the idol character.

[0182] Output: The generated response message is obtained.

[0183] Specific actions:

[0184] The server sends the analysis results to the response generation engine.

[0185] The response generation engine (OpenAI GPT-3) generates the response message, "It's really nice weather! Shall we go for a walk?"

[0186] Step 6:

[0187] The server sends a response to the user's terminal.

[0188] Input: The generated response message is on the server.

[0189] Data processing: The server sends the generated response message to the user's terminal.

[0190] Output: A response message is displayed on the user's terminal.

[0191] Specific actions:

[0192] The server sends the generated response message to the user's terminal.

[0193] The user's device receives this response message and displays it on the chat screen.

[0194] (Application Example 1)

[0195] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0196] Traditional chat systems are limited to text messaging, which makes it difficult for users to enjoy a more natural conversational experience. Furthermore, especially when conversing with entertainment characters, simply exchanging text messages lacks immersion and real-time interaction, resulting in lower user satisfaction. There is a need to address these issues and provide a more realistic conversational experience.

[0197] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0198] In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, means for receiving messages sent from the terminal as voice input and converting them into text using a speech recognition algorithm, means for analyzing the converted text, means for generating a response based on the analysis results, and means for converting the generated response into speech and sending it to the terminal. This enables the user to enjoy natural and real-time voice conversations with entertainment characters.

[0199] "User's device" refers to the electronic device that the user uses to access the system, and includes smartphones, tablets, personal computers, head-mounted displays, etc.

[0200] "Login information" refers to information such as usernames and passwords used to authenticate access rights to the system.

[0201] "Voice input" refers to a method of inputting messages to the system using voice.

[0202] A "speech recognition algorithm" refers to a technology that analyzes a user's voice and converts it into text data.

[0203] "Means of converting to text" refers to the processes and technologies for converting voice input into text-based data.

[0204] "Natural language processing algorithms" refer to technologies used to analyze text data and understand its meaning and context.

[0205] "Means of analysis" refers to the processes and technologies used to analyze input text data and understand the user's intentions and emotions.

[0206] "Means for generating a response" refers to the processes and techniques for generating an appropriate response based on the analysis results.

[0207] "Means of converting to speech" refers to the processes and technologies for converting generated text responses into audio data.

[0208] An "entertainment character" refers to a virtual personality or character created for interaction with users, possessing specific personality traits and characteristics.

[0209] This invention relates to a system that allows users to enjoy natural conversations with entertainment characters. The system consists of a user's terminal, a server, and a chatbot that supports entertainment characters.

[0210] System Configuration

[0211] The system works as follows:

[0212] 1. Means for accepting access from the user's terminal

[0213] Users access the system using a device (such as a smartphone, tablet, or head-mounted display). A login screen is displayed on the device, and the user enters their login information (username and password).

[0214] 2. Means by which the server authenticates login information

[0215] The server receives the user's login information, compares it with the database, and performs authentication. If authentication is successful, the user can proceed to the chat screen or virtual environment.

[0216] 3. Means by which the server receives voice input and performs speech recognition.

[0217] The user inputs a message by voice into their device. The server receives this voice message and converts the speech into text using a speech recognition algorithm (e.g., Google Speech-to-Text API).

[0218] 4. Means by which the server parses text data

[0219] The server analyzes the converted text data using natural language processing algorithms (e.g., OpenAI GPT-4®) to understand the user's intent and emotions.

[0220] 5. Means by which the server generates a response

[0221] Based on the analysis results, the server uses a response generation engine (e.g., OpenAI GPT-4) to generate an appropriate response. This response is selected and generated based on the personality of the entertainment character.

[0222] 6. Means for converting the response generated by the server into speech and retransmitting it.

[0223] The generated text response is converted into speech using a speech synthesis system (e.g., Amazon Polly) and sent to the user's device. The device then plays the received audio and provides it to the user.

[0224] Specific example

[0225] For example, consider a scenario where a user wears a head-mounted display and uses voice input to say, "The weather's nice today."

[0226] 1. The user enters a voice message from their device saying, "The weather is nice today."

[0227] 2. The server receives this voice message and converts it to text using the Google Speech-to-Text API.

[0228] 3. The converted text data is analyzed using OpenAI GPT-4 to identify the topic "weather" and the emotion "positive".

[0229] 4. Based on the analysis results, the server generates a response such as, "It's really nice weather! Shall we go for a walk?"

[0230] 5. Convert this response into audio data using Amazon Polly and send it to the user's device.

[0231] 6. The user's device will play audio data, allowing the user to enjoy natural conversations with entertainment characters.

[0232] Example of a prompt

[0233] As an idol character, generate a response to the following user message: The weather is nice today.

[0234] Such a system allows users to enjoy natural, real-time conversations with entertainment characters, significantly improving the quality of the experience.

[0235] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0236] Step 1:

[0237] The user accesses the device.

[0238] Input: Users access the system using a head-mounted display (HMD) or smartphone.

[0239] Operation: The user launches the application, goes to the login screen, and enters their username and password.

[0240] Output: User login information is sent to the server.

[0241] Step 2:

[0242] The server authenticates the login information.

[0243] Enter: Username and password.

[0244] Operation: The server verifies the login information against the database and performs authentication. Authentication is successful if the information is correct, and fails if it is incorrect.

[0245] Output: If login is successful, the user proceeds to the chat screen or virtual environment. If login fails, an error message is displayed.

[0246] Step 3:

[0247] The user sends a message using voice input.

[0248] Input: The user says "The weather is nice today" using their voice.

[0249] Operation: The device records a voice message and sends it to the server.

[0250] Output: Audio data is sent to the server.

[0251] Step 4:

[0252] The server converts the audio data into text.

[0253] Input: User's voice data.

[0254] Operation: The server uses a speech recognition algorithm (e.g., Google Speech-to-Text API) to convert the audio data into text data.

[0255] Output: The text data "The weather is nice today" is generated.

[0256] Step 5:

[0257] The server parses the text data.

[0258] Input: Text data "The weather is nice today."

[0259] Operation: The server analyzes text data using natural language processing algorithms (e.g., OpenAI GPT-4) to understand the user's intent and emotions.

[0260] Output: The analysis results yield the topic "weather" and the emotion "positive".

[0261] Step 6:

[0262] The server generates a response.

[0263] Input: Analysis result - Topic: "Weather" and emotion: "Positive".

[0264] Operation: The server uses a response generation engine (e.g., OpenAI GPT-4) to generate a response such as, "It's such nice weather! Shall we go for a walk?"

[0265] Output: The response text "It's really nice weather! Shall we go for a walk?" is generated.

[0266] Step 7:

[0267] The server converts the response text into speech and resends it.

[0268] Input: Response text "It's such lovely weather! Shall we go for a walk?"

[0269] Operation: The server uses a text-to-speech system (e.g., Amazon Polly) to convert text into audio data. It then sends the generated audio data to the user's device.

[0270] Output: Audio data is sent to the user's device.

[0271] Step 8:

[0272] The user's device plays the audio data.

[0273] Input: Audio data sent from the server.

[0274] Operation: The device plays audio data and provides it to the user.

[0275] Output: Users can enjoy natural conversations with entertainment characters in audio format.

[0276] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0277] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. In particular, it describes a system that incorporates an emotion engine that recognizes the user's emotions and generates responses.

[0278] System Configuration

[0279] This system consists of a server, a user terminal, and an idle chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission. The user terminal is used by the user to access the system and interact with the chatbot. The chatbot runs on the server and generates responses based on the idle character. Furthermore, the system includes an emotion engine that recognizes the user's emotions.

[0280] Program processing

[0281] The following explains in detail how the system works.

[0282] 1. The user accesses from the terminal

[0283] The user starts the chat app on the terminal and accesses the system. The user transitions to the login screen and enters the username and password.

[0284] 2. The server authenticates the login information

[0285] The server receives the user's login information, verifies it against the database, and performs authentication. If the authentication is successful, the user can proceed to the chat screen.

[0286] 3. The user enters a message

[0287] The user enters a message in the input field displayed on the chat screen and clicks the send button. For example, enter "I'm a bit tired today."

[0288] 4. The server receives and analyzes the message

[0289] The server receives the message sent by the user. Next, it analyzes the message using a natural language processing algorithm to understand the user's intention and emotion.

[0290] 5. The server recognizes the emotion

[0291] The emotion engine built into the server recognizes the user's emotion from the analyzed message. For example, it recognizes "fatigue" from the message "I'm a bit tired today."

[0292] 6. The server generates a response

[0293] Based on the analysis results and recognized emotions, the server uses a response generation engine to generate an appropriate response. This response is tailored based on the user's emotions and the personality of the idol character. For example, it might generate a response such as, "Good work! Get some rest."

[0294] 7. The server sends a response to the user's terminal.

[0295] The server sends the generated response to the user's terminal and displays it on the chat screen.

[0296] Specific example

[0297] For example, consider a scenario where a user says to the chatbot, "Today was a very busy day at work."

[0298] 1. The user types the message "Today was a very busy day at work" from their device and presses the send button.

[0299] 2. The server receives this message and parses it using a natural language processing algorithm.

[0300] 3. The server's emotion engine recognizes the emotions of "busyness" and "fatigue."

[0301] 4. Based on the analysis results and recognized emotions, the server uses a response generation engine to generate a response such as, "You seem to have been very busy. You must be exhausted! Try to do something to relax."

[0302] 5. The server sends this response to the user's device and displays it on the user's chat screen.

[0303] In this way, users can not only enjoy natural conversations with idol characters but also receive responses that resonate with their emotions. This system can greatly improve user satisfaction even in situations where opportunities to directly interact with idols are limited.

[0304] The following describes the process flow.

[0305] Step 1:

[0306] The user launches the "Idle Chatbot" app on the terminal and accesses the login screen. The user enters the username and password as login information.

[0307] Step 2:

[0308] The terminal sends the entered login information to the server.

[0309] Step 3:

[0310] The server compares the received login information with the database for authentication. If the authentication is successful, the user is redirected to the chat screen.

[0311] Step 4:

[0312] The user enters a message in the input field displayed on the chat screen and clicks the send button. For example, enter "I'm a little tired today".

[0313] Step 5:

[0314] The terminal sends the entered message to the server.

[0315] Step 6:

[0316] The server analyzes the received message using a natural language processing algorithm. For example, recognize the message "I'm a little tired today" as "feeling of fatigue".

[0317] Step 7:

[0318] An emotion engine embedded within the server recognizes the user's emotions from the analyzed messages. For example, it recognizes the emotion "fatigue" from the message "I'm a little tired today."

[0319] Step 8:

[0320] Based on the analysis results and recognized emotions, the server uses a response generation engine to generate an appropriate response. This response is tailored based on the user's emotions and the personality of the idol character. For example, it might generate a response such as, "Good work! Get some rest."

[0321] Step 9:

[0322] The server sends the generated response to the user's terminal.

[0323] Step 10:

[0324] The device displays the response it received on the chat screen. The user's device displays the message, "Good work! Get some rest."

[0325] Step 11:

[0326] The user types a new message and clicks the send button. For example, they might type, "Thank you, I'm going to rest for a bit."

[0327] Step 12:

[0328] The device sends a new message to the server.

[0329] Step 13:

[0330] The server then analyzes the new message again using a natural language processing algorithm. For example, it recognizes the intentions of "gratitude" and "rest" from the message "Thank you, I'll take a short rest."

[0331] Step 14:

[0332] The server then uses the emotion engine again to recognize the user's emotions. For example, it recognizes the emotion of "gratitude" from the message "Thank you, I'll take a break."

[0333] Step 15:

[0334] Based on the new analysis results and recognized emotions, the server uses the response generation engine again to generate a response. For example, it might generate a response like, "Okay, please continue to relax."

[0335] Step 16:

[0336] The server sends a new response to the user's device, which then displays it on the chat screen.

[0337] (Example 2)

[0338] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal".

[0339] Traditional chat systems have struggled to accurately understand users' emotions and intentions and generate appropriate responses. As a result, users were not satisfied, and the systems failed to provide a natural conversational experience. Furthermore, generating responses based on the personality of specific characters was difficult, resulting in a lack of technology to achieve realistic dialogue. Solving these challenges is highly desirable.

[0340] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0341] In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, means for receiving and analyzing messages sent from the terminal, means for recognizing the user's emotions from the analyzed messages, means for generating a response based on the analysis results and the recognized emotions, and means for sending the generated response to the terminal. This makes it possible to accurately recognize the user's emotions and provide a natural conversation experience that takes into account the character's personality.

[0342] A "user's terminal" is an electronic device used by a user to access and interact with the system.

[0343] "Login information" refers to information such as usernames and passwords that are necessary for user authentication.

[0344] A "message" is text or written information that a user sends to a system through their device.

[0345] A "natural language processing algorithm" is a computational method and process for analyzing text information and understanding its context and meaning.

[0346] An "emotion engine" is a software component that recognizes a user's emotions from analyzed messages.

[0347] A "response generation engine" is a software component that generates appropriate responses based on analysis results and recognized emotions.

[0348] "Character personality" refers to the characteristics and behavioral traits assigned to a specific fictional person or character.

[0349] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. The system consists of a server, a user terminal, and an idol chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission. The user terminal is used by the user to access the system and interact with the chatbot. The chatbot operates on the server and generates responses based on the idol character. Furthermore, the system includes an emotion engine that recognizes the user's emotions.

[0350] Hardware and software to be used

[0351] The system's hardware includes smartphones and personal computers as user terminals, and high-performance computing devices as servers. The software uses chat applications, natural language processing algorithms (e.g., spaCy, NLTK), sentiment engines (e.g., IBM Watson® NLU, Google Cloud Natural Language API), and response generation engines (e.g., OpenAI's GPT-4).

[0352] Program processing

[0353] The following describes the specific processing details of this system's program.

[0354] The user accesses the system from their device, launches the chat application, and is directed to the login screen. The user enters their username and password and sends them to the server. The server authenticates the user, and if successful, the user proceeds to the chat screen. Next, the user types and sends a message. The server receives the message and analyzes it using a natural language processing algorithm. Subsequently, the emotion engine recognizes the user's emotions from the analysis results, and the response generation engine generates a response based on the recognized emotions. Finally, the server sends the generated response to the user's device and displays it on the chat screen.

[0355] Specific example

[0356] For example, if a user says to the chatbot, "I was very busy at work today," the process will proceed as follows:

[0357] 1. The user types the message "Today was a very busy day at work" from their device and presses the send button.

[0358] 2. The server receives this message and parses its content using a natural language processing algorithm.

[0359] 3. The emotion engine within the server recognizes "busyness" and "fatigue."

[0360] 4. Based on the analysis results and recognized emotions, the server uses a response generation engine to generate a response such as, "You seem to have been very busy. You must be exhausted! Try to do something to relax."

[0361] 5. The server sends the generated response to the user's device and displays it on the chat screen.

[0362] In this way, users can not only enjoy natural conversations with idol characters, but also receive responses that resonate with their emotions.

[0363] Example of a prompt

[0364] Examples of prompts for a generative AI model are as follows:

[0365] Please create a system where, when a user says "Today was a very busy day at work," the emotion engine recognizes "busyness" and "fatigue," and the character then generates a response such as "Great job! Get some rest."

[0366] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0367] Step 1:

[0368] The user accesses it from their device.

[0369] Enter: Username and password.

[0370] The server displays a login screen and provides input fields for username and password.

[0371] The user enters their username and password and clicks the login button.

[0372] Output: Login information is sent from the terminal to the server.

[0373] Step 2:

[0374] The server authenticates the login information.

[0375] Input: Username and password sent from the device.

[0376] The server compares the user information stored in the database with the entered information.

[0377] If authentication is successful, the server sends an instruction to redirect the user to the chat screen.

[0378] Output: Authentication result and instructions to transition to the chat screen.

[0379] Step 3:

[0380] The user enters a message.

[0381] Input: User's message (e.g., "Today was a very busy day at work").

[0382] The user enters text into the message input field on the chat screen and clicks the send button.

[0383] Output: A message is sent from the terminal to the server.

[0384] Step 4:

[0385] The server receives and parses the message.

[0386] Input: Message sent by the user.

[0387] The server receives the message and parses it using a natural language processing algorithm (e.g., spaCy, NLTK).

[0388] As part of data processing, the text is split and keywords and context are extracted.

[0389] Output: Analysis results (prediction of user intent and emotions).

[0390] Step 5:

[0391] The server recognizes emotions.

[0392] Input: Analysis results.

[0393] The system uses an emotion engine on the server (e.g., IBM Watson NLU, Google Cloud Natural Language API) to recognize the user's emotions from the analysis results.

[0394] As a data calculation, emotions are identified based on extracted keywords and context.

[0395] Output: Recognized emotions (e.g., "busy," "fatigue").

[0396] Step 6:

[0397] The server generates a response.

[0398] Input: Recognized emotions and analysis results.

[0399] The server uses a response generation engine (e.g., OpenAI's GPT-4) to generate an appropriate response.

[0400] As part of data processing, text is generated that matches the user's emotions and the character's personality.

[0401] For example, a response like, "You seem to have been very busy. You must be exhausted! Try to do something to relax," might be generated.

[0402] Output: The generated response.

[0403] Step 7:

[0404] The server sends a response to the user's terminal.

[0405] Input: Generated response.

[0406] The server sends the generated response to the user's terminal.

[0407] The device receives the response and displays it on the chat screen.

[0408] Output: The response displayed on the user's chat screen. The message displayed is: "You seem to have been very busy. You must be exhausted! Try to do something to relax."

[0409] As described above, appropriate data processing and calculations are performed at each step based on the input data, and finally, a response that resonates with the user's emotions is generated and displayed.

[0410] (Application Example 2)

[0411] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0412] In modern content delivery services, users demand personalized responses that address their individual emotions. However, traditional systems struggle to understand user emotions and generate appropriate responses. Furthermore, there is a lack of methods to provide interactive experiences that allow fans to feel satisfied through natural conversations with idol characters.

[0413] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, means for receiving and analyzing messages sent from the terminal, means for generating a response based on the analysis results, means for transmitting the generated response to the terminal, means for recognizing the user's emotions, means for adjusting the generated response based on the user's emotions, means for providing individual responses to the user in conjunction with video content, and means for including messages to alleviate the user's emotions. As a result, the user can enjoy an interactive experience with an idol character through natural dialogue that responds to their own emotions.

[0414] "User's device" refers to the device that a user uses to access the internet and utilize various services.

[0415] "Means of accepting access" refers to a function that receives connection requests from users' terminals and provides an interface for accessing the system.

[0416] "Means of authenticating login information" refers to a function that verifies authentication information such as the username and password entered by the user to confirm whether they are a legitimate user.

[0417] "Means for receiving and analyzing messages" refers to a function in which the system receives messages sent by users and analyzes their content.

[0418] "Means for generating a response based on analysis results" refers to a function that creates an appropriate response based on the analysis results of the message.

[0419] "Means for transmitting the generated response to the terminal" refers to a function that transmits the system-generated response to the user's terminal for display.

[0420] "Means of recognizing user emotions" refers to a function that analyzes and recognizes user emotions from analyzed messages.

[0421] "Means for adjusting generated responses based on the user's emotions" refers to a function that adjusts the generated responses to match the user's emotions and provides them in a more appropriate form.

[0422] "A means of providing individualized responses to users in conjunction with video content" refers to a function in which the system references video content and provides individualized and appropriate responses to users.

[0423] "Means of including messages to alleviate emotions" refers to features that include encouraging or comforting messages to reduce the emotions of the recognized user.

[0424] This invention relates to a chat system that allows users to enjoy natural conversations with idol characters. In particular, this system incorporates an emotion engine that recognizes the user's emotions and generates and adjusts responses accordingly. Detailed embodiments for carrying out this invention are described below.

[0425] This system includes the following components:

[0426] 1. Server:

[0427] Access Acceptance: Accepts access from the user's device. Specifically, it provides an interface for receiving HTTP requests.

[0428] Login Authentication: Authenticates the user's login information (username, password). Authentication is performed by comparing it with the database.

[0429] Message analysis: Receives messages sent by users and analyzes them using natural language processing algorithms.

[0430] Emotion Recognition: Recognizes the user's emotions from the analyzed message. Use an emotion engine (e.g., Transformers' emotion analysis model).

[0431] Response Generation: Generates appropriate responses based on analysis results and emotion recognition. Adjusts responses based on the personality and emotional state of the idol character.

[0432] Send response: Send the generated response to the user's device.

[0433] 2. User's device:

[0434] Access: Access the system through the chat app and enter your username and password on the login screen.

[0435] Chat interface: Provides a user interface for entering and sending messages.

[0436] Response Display: Displays the response received from the server.

[0437] 3. Emotional Engine:

[0438] Natural language processing algorithms: For example, use the Transformers library from Hugging Face.

[0439] Emotion analysis models: In particular, generative AI models such as BERT and RoBERTa are used to recognize emotions.

[0440] Specific example

[0441] For example, if a user sends a message saying "I'm a little tired today," the server will perform the following actions:

[0442] 1. The server receives the user's message and analyzes it using a natural language processing algorithm.

[0443] 2. The emotional engine recognizes "fatigue."

[0444] 3. The response generation engine generates a response based on the "feeling of fatigue," such as, "That must have been tough. Take a break and relax."

[0445] 4. The server sends this response to the user's device and displays it in the chat interface.

[0446] Examples of prompt messages include the following:

[0447] "Analyze the following user message, recognize and analyze the user's emotions, and generate a personalized response based on that: 'I'm a little tired today.'"

[0448] This invention allows users to enjoy an interactive experience with idol characters through natural dialogue that responds to their own emotions.

[0449] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0450] Step 1:

[0451] The server accepts access from the user's device. Specifically, it receives HTTP requests and grants access. At this stage, the input is a connection request from the user's device, and the output is a response granting access permission.

[0452] Step 2:

[0453] The server performs login authentication. The user enters their username and password and submits them from their terminal. The server compares this authentication information with the database to verify that the user is legitimate. At this stage, the input is the user's login information, and the output is the result of successful or unsuccessful authentication.

[0454] Step 3:

[0455] The user types a message into the chat interface and sends it. The input content is the message entered by the user. The message is sent from the terminal to the server.

[0456] Step 4:

[0457] The server receives messages sent from the user's terminal and analyzes them using natural language processing algorithms. The input here is the user's message, and the analyzed information (such as intent and emotion) is output.

[0458] Step 5:

[0459] The emotion engine within the server recognizes the user's emotions from the analyzed messages. The input is the analyzed information, and the output is the recognized emotion (e.g., "fatigue").

[0460] Step 6:

[0461] The server's response generation engine generates an appropriate response based on the analysis results and recognized emotions. The idol character's personality is also taken into consideration during response generation. At this stage, the input is the analysis results and recognized emotions, and the output is the generated response.

[0462] Step 7:

[0463] The server sends the generated response to the user's terminal. The input is the generated response, and the output is the transmission of the response to the terminal.

[0464] Step 8:

[0465] The terminal displays the received response in the chat interface. The input is the response received from the server, and the output is the displayed message.

[0466] In this way, users can enjoy natural conversations with idol characters.

[0467] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0468] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0469] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0470] [Second Embodiment]

[0471] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0472] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0473] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0474] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0475] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0476] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0477] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0478] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0479] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0480] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0481] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0482] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0483] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. An embodiment of this system is described below.

[0484] System Configuration

[0485] This system consists of a server, a user terminal, and an idle chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission. The user terminal is used by the user to access the system and interact with the chatbot. The chatbot runs on the server and generates responses based on the idle character.

[0486] Program processing

[0487] The following explains in detail how the system works.

[0488] 1. The user accesses from their device.

[0489] The user launches the chat app on their device and accesses the system. Here, the user is redirected to a login screen where they enter their username and password.

[0490] 2. The server authenticates the login information.

[0491] The server receives the user's login information, compares it with the database, and performs authentication. If authentication is successful, the user can proceed to the chat screen.

[0492] 3. The user enters a message.

[0493] The user enters a message in the input field displayed on the chat screen and clicks the send button.

[0494] 4. The server receives and parses the message.

[0495] The server receives messages sent by users. Next, it uses natural language processing algorithms to analyze the messages and understand the user's intentions and emotions.

[0496] 5. The server generates a response.

[0497] Based on the analysis results, the server uses a response generation engine to generate an appropriate response. This response is selected and generated based on the personality of the idol character.

[0498] 6. The server sends a response to the user's terminal.

[0499] The server sends the generated response to the user's terminal and displays it on the chat screen.

[0500] Specific example

[0501] For example, consider the case where a user says to the chatbot, "The weather's nice today."

[0502] 1. The user types the message "The weather is nice today" from their device and presses the send button.

[0503] 2. The server receives this message and parses it using a natural language processing algorithm.

[0504] 3. The server identifies the topic "weather" from the analysis results and further recognizes the associated emotion as "positive."

[0505] 4. The server uses its response generation engine to generate the response, "It's such lovely weather! Shall we go for a walk?"

[0506] 5. The server sends this response to the user's device and displays it on the user's chat screen.

[0507] In this way, users can enjoy natural conversations with idol characters. This system can greatly improve user satisfaction even in situations where opportunities to directly interact with idols are limited.

[0508] The following describes the processing flow.

[0509] Step 1:

[0510] The user launches the "Idle Chatbot" app on their device and accesses the login screen. The user enters their username and password as login information.

[0511] Step 2:

[0512] The terminal sends the entered login information to the server.

[0513] Step 3:

[0514] The server compares the received login information with the database and performs authentication. If authentication is successful, the user is redirected to the chat screen.

[0515] Step 4:

[0516] The user enters a message in the input field displayed on the chat screen and clicks the send button. For example, they might type "Hello".

[0517] Step 5:

[0518] The terminal sends the entered message to the server.

[0519] Step 6:

[0520] The server analyzes the messages it receives using natural language processing algorithms. For example, it recognizes the message "Hello" as a "greeting."

[0521] Step 7:

[0522] Based on the analysis results, the server uses a response generation engine to generate an appropriate reply. For example, it might generate a reply such as, "Hi! How are you doing today?"

[0523] Step 8:

[0524] The server sends the generated response to the user's terminal.

[0525] Step 9:

[0526] The device displays the response it received on the chat screen. The user's device displays the message, "Hi! How are you doing today?"

[0527] Step 10:

[0528] The user types a new message and clicks the send button. For example, they might type, "I'm really busy today."

[0529] Step 11:

[0530] The device sends a new message to the server.

[0531] Step 12:

[0532] The server then analyzes the new message again using a natural language processing algorithm. For example, it analyzes the message "I'm really busy today" and recognizes the state of being "busy."

[0533] Step 13:

[0534] The server uses the response generation engine again to generate a response based on the new analysis results. For example, it might generate a response like, "Thank you so much for your hard work! Is there anything I can help you with?"

[0535] Step 14:

[0536] The server sends a new response to the user's device, which then displays it on the chat screen.

[0537] (Example 1)

[0538] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0539] Traditional chat systems made it difficult for users to enjoy natural conversations with artificial intelligence, and in particular, it was challenging to create conversations that reflected the unique personality of idol characters. Furthermore, achieving natural conversations required advanced processing, highlighting the need for improved user experience.

[0540] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0541] In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, and means for receiving and analyzing messages sent from the terminal. This allows the user to enjoy natural conversations with an idol character by using natural language processing algorithms and generative AI models in the message analysis and response generation means. Furthermore, by including means for adjusting responses based on the idol character's personality, responses that reflect the character's characteristics can be provided, resulting in a more satisfying user experience.

[0542] "Means of accepting access" refers to devices or software that provide the functionality for users to connect to the system from their terminals and initiate communication.

[0543] "Means for authenticating login information" refers to devices or software used to verify the username and password entered by the user and to confirm their identity.

[0544] "Means for receiving and analyzing messages" refers to a device or software that receives text messages sent from a terminal and processes them in order to understand their content.

[0545] "Means for generating a response" refers to a device or software for creating an appropriate response based on an analyzed message.

[0546] "Means for sending a response" refers to the device or software necessary to send the generated response to the user's terminal and continue the conversation.

[0547] A "natural language processing algorithm" is a set of computational methods and models used to understand and generate human language.

[0548] A "generative AI model" is a type of artificial intelligence that generates new data or text based on input, and operates based on specific rules or training data.

[0549] A "prompt" is input text provided to a generative AI model to elicit a specific response.

[0550] An "idol character" is a character that imitates a fictional entertainer or celebrity, possessing specific personalities and characteristics.

[0551] "Personality" refers to the unique character, speaking style, and behavior of an idol character.

[0552] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. Specific embodiments of this system are described below.

[0553] System Configuration

[0554] This system consists of a server, a user terminal, and an idle chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission, while the user terminal is used by the user to access the system and interact with the chatbot. The chatbot runs on the server and generates responses based on the idle character.

[0555] Hardware and software to be used

[0556] server:

[0557] Database server (e.g., MySQL)

[0558] Natural language processing algorithms (e.g., Google Cloud Natural Language API)

[0559] Response generation engine (e.g., OpenAI GPT-3)

[0560] User's device:

[0561] Chat application installed on a smartphone or computer

[0562] Detailed Operation Description

[0563] 1. The user accesses from a device:

[0564] The user launches a chat application on their smartphone or computer and is redirected to the login screen.

[0565] The user enters their username and password and clicks the "Login" button.

[0566] 2. The server authenticates the login information:

[0567] The server receives the login information entered by the user, compares it with the registered information in the database, and performs authentication.

[0568] If authentication is successful, the server creates a session for the user, and the user proceeds to the main chat screen.

[0569] 3. The user enters a message:

[0570] The user types a message on the main chat screen and clicks the "Send" button.

[0571] 4. The server receives and parses the message:

[0572] The server receives messages sent by users and parses them using the Google Cloud Natural Language API.

[0573] The server understands the user's intentions and emotions from the analysis results.

[0574] 5. The server generates a response:

[0575] Based on the analysis results, the server generates an appropriate response using OpenAI GPT-3.

[0576] The generated responses are tailored based on the personality of the idol character.

[0577] 6. The server sends a response to the user's terminal:

[0578] The server sends the generated response to the user's terminal and displays it on the chat screen.

[0579] Specific example

[0580] For example, if a user says to the chatbot, "The weather is nice today," the following process will occur.

[0581] The user types the message "The weather's nice today" into the chat app on their device and presses the send button.

[0582] The server receives this message and parses it using the Google Cloud Natural Language API.

[0583] The server identifies the topic "weather" from the analysis results and further recognizes the associated emotion as "positive."

[0584] The server uses OpenAI GPT-3 to generate the response, "It's such nice weather! Shall we go for a walk?"

[0585] The server sends this response to the user's device and displays it on the chat screen.

[0586] Example of a prompt

[0587] The following are examples of prompt statements for a generative AI model.

[0588] The user typed "The weather's nice today." Generate a response from the idol character to this message.

[0589] This system allows users to enjoy natural conversations with idol characters, providing a high level of satisfaction even in situations where opportunities to directly interact with idols are limited.

[0590] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0591] Step 1:

[0592] Users access from their devices

[0593] Input: The user launches the chat app on their smartphone or computer and enters their username and password on the login screen.

[0594] Data processing: The terminal sends the user's login information to the server.

[0595] Output: The user's login information (username and password) is sent to the server.

[0596] Specific actions:

[0597] The user launches the chat app by tapping it from the device's home screen.

[0598] Enter your username and password on the login screen and click the "Login" button.

[0599] Step 2:

[0600] The server authenticates the login information.

[0601] Input: The user's login information (username and password) is sent to the server.

[0602] Data processing: The server receives login information, compares it with the database, and performs authentication.

[0603] Output: An authentication result (success or failure) is generated.

[0604] Specific actions:

[0605] The server verifies the username and password against the database.

[0606] Once authentication is successful, the server creates a session and sends an instruction to the user to display the main chat screen.

[0607] Step 3:

[0608] The user enters a message.

[0609] Input: The user enters a message in the input field displayed on the main chat screen and clicks the "Send" button.

[0610] Data processing: The terminal sends the message entered by the user to the server.

[0611] Output: The user's message is sent to the server.

[0612] Specific actions:

[0613] The user types "The weather is nice today" into the input field on the chat screen.

[0614] The user clicks the "Send" button.

[0615] Step 4:

[0616] The server receives and parses the message.

[0617] Input: The message sent by the user is sent to the server.

[0618] Data processing: The server receives the message and analyzes its content using the Google Cloud Natural Language API. The analysis extracts the message's intent and sentiment.

[0619] Output: Analysis results (such as "weather" and "positive emotions") are generated.

[0620] Specific actions:

[0621] The server receives the user's message.

[0622] Using the Google Cloud Natural Language API, the message content is analyzed to identify "weather" and "positive emotions."

[0623] Step 5:

[0624] The server generates a response.

[0625] Input: Analysis results (such as "weather" and "positive emotions") are sent to the server.

[0626] Data processing: The server uses OpenAI GPT-3 to generate an appropriate response based on the analysis results. The generated response is adjusted based on the personality of the idol character.

[0627] Output: The generated response message is obtained.

[0628] Specific actions:

[0629] The server sends the analysis results to the response generation engine.

[0630] The response generation engine (OpenAI GPT-3) generates the response message, "It's really nice weather! Shall we go for a walk?"

[0631] Step 6:

[0632] The server sends a response to the user's terminal.

[0633] Input: The generated response message is on the server.

[0634] Data processing: The server sends the generated response message to the user's terminal.

[0635] Output: A response message is displayed on the user's terminal.

[0636] Specific actions:

[0637] The server sends the generated response message to the user's terminal.

[0638] The user's device receives this response message and displays it on the chat screen.

[0639] (Application Example 1)

[0640] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0641] Traditional chat systems are limited to text messaging, which makes it difficult for users to enjoy a more natural conversational experience. Furthermore, especially when conversing with entertainment characters, simply exchanging text messages lacks immersion and real-time interaction, resulting in lower user satisfaction. There is a need to address these issues and provide a more realistic conversational experience.

[0642] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0643] In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, means for receiving messages sent from the terminal as voice input and converting them into text using a speech recognition algorithm, means for analyzing the converted text, means for generating a response based on the analysis results, and means for converting the generated response into speech and sending it to the terminal. This enables the user to enjoy natural and real-time voice conversations with entertainment characters.

[0644] "User's device" refers to the electronic device that the user uses to access the system, and includes smartphones, tablets, personal computers, head-mounted displays, etc.

[0645] "Login information" refers to information such as usernames and passwords used to authenticate access rights to the system.

[0646] "Voice input" refers to a method of inputting messages to the system using voice.

[0647] A "speech recognition algorithm" refers to a technology that analyzes a user's voice and converts it into text data.

[0648] "Means of converting to text" refers to the processes and technologies for converting voice input into text-based data.

[0649] "Natural language processing algorithms" refer to technologies used to analyze text data and understand its meaning and context.

[0650] "Means of analysis" refers to the processes and technologies used to analyze input text data and understand the user's intentions and emotions.

[0651] "Means for generating a response" refers to the processes and techniques for generating an appropriate response based on the analysis results.

[0652] "Means of converting to speech" refers to the processes and technologies for converting generated text responses into audio data.

[0653] An "entertainment character" refers to a virtual personality or character created for interaction with users, possessing specific personality traits and characteristics.

[0654] This invention relates to a system that allows users to enjoy natural conversations with entertainment characters. The system consists of a user's terminal, a server, and a chatbot that supports entertainment characters.

[0655] System Configuration

[0656] The system works as follows:

[0657] 1. Means for accepting access from the user's terminal

[0658] Users access the system using a device (such as a smartphone, tablet, or head-mounted display). A login screen is displayed on the device, and the user enters their login information (username and password).

[0659] 2. Means by which the server authenticates login information

[0660] The server receives the user's login information, compares it with the database, and performs authentication. If authentication is successful, the user can proceed to the chat screen or virtual environment.

[0661] 3. Means by which the server receives voice input and performs speech recognition.

[0662] The user inputs a message by voice into their device. The server receives this voice message and converts the speech into text using a speech recognition algorithm (e.g., Google Speech-to-Text API).

[0663] 4. Means by which the server parses text data

[0664] The server analyzes the converted text data using natural language processing algorithms (e.g., OpenAI GPT-4) to understand the user's intent and emotions.

[0665] 5. Means by which the server generates a response

[0666] Based on the analysis results, the server uses a response generation engine (e.g., OpenAI GPT-4) to generate an appropriate response. This response is selected and generated based on the personality of the entertainment character.

[0667] 6. Means for converting the response generated by the server into speech and retransmitting it.

[0668] The generated text response is converted into speech using a speech synthesis system (e.g., Amazon Polly) and sent to the user's device. The device then plays the received audio and provides it to the user.

[0669] Specific example

[0670] For example, consider a scenario where a user wears a head-mounted display and uses voice input to say, "The weather's nice today."

[0671] 1. The user enters a voice message from their device saying, "The weather is nice today."

[0672] 2. The server receives this voice message and converts it to text using the Google Speech-to-Text API.

[0673] 3. The converted text data is analyzed using OpenAI GPT-4 to identify the topic "weather" and the emotion "positive".

[0674] 4. Based on the analysis results, the server generates a response such as, "It's really nice weather! Shall we go for a walk?"

[0675] 5. Convert this response into audio data using Amazon Polly and send it to the user's device.

[0676] 6. The user's device will play audio data, allowing the user to enjoy natural conversations with entertainment characters.

[0677] Example of a prompt

[0678] As an idol character, generate a response to the following user message: The weather is nice today.

[0679] Such a system allows users to enjoy natural, real-time conversations with entertainment characters, significantly improving the quality of the experience.

[0680] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0681] Step 1:

[0682] The user accesses the device.

[0683] Input: Users access the system using a head-mounted display (HMD) or smartphone.

[0684] Operation: The user launches the application, goes to the login screen, and enters their username and password.

[0685] Output: User login information is sent to the server.

[0686] Step 2:

[0687] The server authenticates the login information.

[0688] Enter: Username and password.

[0689] Operation: The server verifies the login information against the database and performs authentication. Authentication is successful if the information is correct, and failed if it is incorrect.

[0690] Output: If login is successful, the user proceeds to the chat screen or virtual environment. If login fails, an error message is displayed.

[0691] Step 3:

[0692] The user sends a message using voice input.

[0693] Input: The user says "The weather is nice today" in their voice.

[0694] Operation: The device records a voice message and sends it to the server.

[0695] Output: Audio data is sent to the server.

[0696] Step 4:

[0697] The server converts the audio data into text.

[0698] Input: User's voice data.

[0699] Operation: The server uses a speech recognition algorithm (e.g., Google Speech-to-Text API) to convert the audio data into text data.

[0700] Output: The text data "The weather is nice today" is generated.

[0701] Step 5:

[0702] The server parses the text data.

[0703] Input: Text data "The weather is nice today."

[0704] Operation: The server analyzes text data using natural language processing algorithms (e.g., OpenAI GPT-4) to understand the user's intent and emotions.

[0705] Output: The analysis results yield the topic "weather" and the emotion "positive".

[0706] Step 6:

[0707] The server generates a response.

[0708] Input: Analysis result - Topic "weather" and emotion "positive".

[0709] Operation: The server uses a response generation engine (e.g., OpenAI GPT-4) to generate a response such as, "It's such nice weather! Shall we go for a walk?"

[0710] Output: The response text "It's really nice weather! Shall we go for a walk?" is generated.

[0711] Step 7:

[0712] The server converts the response text into speech and resends it.

[0713] Input: Response text "It's such lovely weather! Shall we go for a walk?"

[0714] Operation: The server uses a text-to-speech system (e.g., Amazon Polly) to convert text into audio data. It then sends the generated audio data to the user's device.

[0715] Output: Audio data is sent to the user's device.

[0716] Step 8:

[0717] The user's device plays the audio data.

[0718] Input: Audio data sent from the server.

[0719] Operation: The device plays audio data and provides it to the user.

[0720] Output: Users can enjoy natural conversations with entertainment characters in audio format.

[0721] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0722] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. In particular, it describes a system that incorporates an emotion engine that recognizes the user's emotions and generates responses.

[0723] System Configuration

[0724] This system consists of a server, a user terminal, and an idle chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission. The user terminal is used by the user to access the system and interact with the chatbot. The chatbot runs on the server and generates responses based on the idle character. Furthermore, the system includes an emotion engine that recognizes the user's emotions.

[0725] Program processing

[0726] The following explains in detail how the system works.

[0727] 1. The user accesses from their device.

[0728] The user launches the chat app on their device and accesses the system. The user is then directed to the login screen and enters their username and password.

[0729] 2. The server authenticates the login information.

[0730] The server receives the user's login information, compares it with the database, and performs authentication. If authentication is successful, the user can proceed to the chat screen.

[0731] 3. The user enters a message.

[0732] The user enters a message in the input field displayed on the chat screen and clicks the send button. For example, they might type, "I'm a little tired today."

[0733] 4. The server receives and parses the message.

[0734] The server receives messages sent by users. Next, it uses natural language processing algorithms to analyze the messages and understand the user's intentions and emotions.

[0735] 5. The server recognizes emotions.

[0736] An emotion engine embedded within the server recognizes the user's emotions from the analyzed messages. For example, it recognizes "fatigue" from the message "I'm a little tired today."

[0737] 6. The server generates a response.

[0738] Based on the analysis results and recognized emotions, the server uses a response generation engine to generate an appropriate response. This response is tailored based on the user's emotions and the personality of the idol character. For example, it might generate a response such as, "Good work! Get some rest."

[0739] 7. The server sends a response to the user's terminal.

[0740] The server sends the generated response to the user's terminal and displays it on the chat screen.

[0741] Specific example

[0742] For example, consider a scenario where a user says to the chatbot, "Today was a very busy day at work."

[0743] 1. The user types the message "Today was a very busy day at work" from their device and presses the send button.

[0744] 2. The server receives this message and parses it using a natural language processing algorithm.

[0745] 3. The server's emotion engine recognizes the emotions of "busyness" and "fatigue."

[0746] 4. Based on the analysis results and recognized emotions, the server uses a response generation engine to generate a response such as, "You seem to have been very busy. You must be exhausted! Try to do something to relax."

[0747] 5. The server sends this response to the user's device and displays it on the user's chat screen.

[0748] In this way, users can not only enjoy natural conversations with idol characters but also receive responses that resonate with their emotions. This system can greatly improve user satisfaction even in situations where opportunities to directly interact with idols are limited.

[0749] The following describes the processing flow.

[0750] Step 1:

[0751] The user launches the "Idle Chatbot" app on their device and accesses the login screen. The user enters their username and password as login information.

[0752] Step 2:

[0753] The terminal sends the entered login information to the server.

[0754] Step 3:

[0755] The server compares the received login information with the database and performs authentication. If authentication is successful, the user is redirected to the chat screen.

[0756] Step 4:

[0757] The user enters a message in the input field displayed on the chat screen and clicks the send button. For example, they might type, "I'm a little tired today."

[0758] Step 5:

[0759] The terminal sends the entered message to the server.

[0760] Step 6:

[0761] The server analyzes the received messages using natural language processing algorithms. For example, it recognizes the message "I'm a little tired today" as "a feeling of fatigue."

[0762] Step 7:

[0763] An emotion engine embedded within the server recognizes the user's emotions from the analyzed messages. For example, it recognizes the emotion "fatigue" from the message "I'm a little tired today."

[0764] Step 8:

[0765] Based on the analysis results and recognized emotions, the server uses a response generation engine to generate an appropriate response. This response is tailored based on the user's emotions and the personality of the idol character. For example, it might generate a response such as, "Good work! Get some rest."

[0766] Step 9:

[0767] The server sends the generated response to the user's terminal.

[0768] Step 10:

[0769] The device displays the response it received on the chat screen. The user's device displays the message, "Good work! Get some rest."

[0770] Step 11:

[0771] The user types a new message and clicks the send button. For example, they might type, "Thank you, I'm going to rest for a bit."

[0772] Step 12:

[0773] The device sends a new message to the server.

[0774] Step 13:

[0775] The server then analyzes the new message again using a natural language processing algorithm. For example, it recognizes the intentions of "gratitude" and "rest" from the message "Thank you, I'll take a short rest."

[0776] Step 14:

[0777] The server then uses the emotion engine again to recognize the user's emotions. For example, it recognizes the emotion of "gratitude" from the message "Thank you, I'll take a break."

[0778] Step 15:

[0779] Based on the new analysis results and recognized emotions, the server uses the response generation engine again to generate a response. For example, it might generate a response like, "Okay, please continue to relax."

[0780] Step 16:

[0781] The server sends a new response to the user's device, which then displays it on the chat screen.

[0782] (Example 2)

[0783] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0784] Traditional chat systems have struggled to accurately understand users' emotions and intentions and generate appropriate responses. As a result, users were not satisfied, and the systems failed to provide a natural conversational experience. Furthermore, generating responses based on the personality of specific characters was difficult, resulting in a lack of technology to achieve realistic dialogue. Solving these challenges is highly desirable.

[0785] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0786] In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, means for receiving and analyzing messages sent from the terminal, means for recognizing the user's emotions from the analyzed messages, means for generating a response based on the analysis results and the recognized emotions, and means for sending the generated response to the terminal. This makes it possible to accurately recognize the user's emotions and provide a natural conversation experience that takes into account the character's personality.

[0787] A "user's terminal" is an electronic device used by a user to access and interact with the system.

[0788] "Login information" refers to information such as usernames and passwords that are necessary for user authentication.

[0789] A "message" is text or written information that a user sends to a system through their device.

[0790] A "natural language processing algorithm" is a computational method and process for analyzing text information and understanding its context and meaning.

[0791] An "emotion engine" is a software component that recognizes a user's emotions from analyzed messages.

[0792] A "response generation engine" is a software component that generates appropriate responses based on analysis results and recognized emotions.

[0793] "Character personality" refers to the characteristics and behavioral traits assigned to a specific fictional person or character.

[0794] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. The system consists of a server, a user terminal, and an idol chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission. The user terminal is used by the user to access the system and interact with the chatbot. The chatbot operates on the server and generates responses based on the idol character. Furthermore, the system includes an emotion engine that recognizes the user's emotions.

[0795] Hardware and software to be used

[0796] The system's hardware includes smartphones and personal computers as user terminals, and high-performance computing devices as servers. The software uses chat applications, natural language processing algorithms (e.g., spaCy, NLTK), sentiment engines (e.g., IBM Watson NLU, Google Cloud Natural Language API), and response generation engines (e.g., OpenAI's GPT-4).

[0797] Program processing

[0798] The following describes the specific processing details of this system's program.

[0799] The user accesses the system from their device, launches the chat application, and is directed to the login screen. The user enters their username and password and sends them to the server. The server authenticates the user, and if successful, the user proceeds to the chat screen. Next, the user types and sends a message. The server receives the message and analyzes it using a natural language processing algorithm. Subsequently, the emotion engine recognizes the user's emotions from the analysis results, and the response generation engine generates a response based on the recognized emotions. Finally, the server sends the generated response to the user's device and displays it on the chat screen.

[0800] Specific example

[0801] For example, if a user says to the chatbot, "I was very busy at work today," the process will proceed as follows:

[0802] 1. The user types the message "Today was a very busy day at work" from their device and presses the send button.

[0803] 2. The server receives this message and parses its content using a natural language processing algorithm.

[0804] 3. The emotion engine within the server recognizes "busyness" and "fatigue."

[0805] 4. Based on the analysis results and recognized emotions, the server uses a response generation engine to generate a response such as, "You seem to have been very busy. You must be exhausted! Try to do something to relax."

[0806] 5. The server sends the generated response to the user's device and displays it on the chat screen.

[0807] In this way, users can not only enjoy natural conversations with idol characters, but also receive responses that resonate with their emotions.

[0808] Example of a prompt

[0809] Examples of prompts for a generative AI model are as follows:

[0810] Please create a system where, when a user says "Today was a very busy day at work," the emotion engine recognizes "busyness" and "fatigue," and the character then generates a response such as "Great job! Get some rest."

[0811] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0812] Step 1:

[0813] The user accesses it from their device.

[0814] Enter: Username and password.

[0815] The server displays a login screen and provides input fields for username and password.

[0816] The user enters their username and password and clicks the login button.

[0817] Output: Login information is sent from the terminal to the server.

[0818] Step 2:

[0819] The server authenticates the login information.

[0820] Input: Username and password sent from the device.

[0821] The server compares the user information stored in the database with the entered information.

[0822] If authentication is successful, the server sends an instruction to redirect the user to the chat screen.

[0823] Output: Authentication result and instructions to transition to the chat screen.

[0824] Step 3:

[0825] The user enters a message.

[0826] Input: User's message (e.g., "Today was a very busy day at work").

[0827] The user enters text into the message input field on the chat screen and clicks the send button.

[0828] Output: A message is sent from the terminal to the server.

[0829] Step 4:

[0830] The server receives and parses the message.

[0831] Input: Message sent by the user.

[0832] The server receives the message and parses it using a natural language processing algorithm (e.g., spaCy, NLTK).

[0833] As part of data processing, the text is split and keywords and context are extracted.

[0834] Output: Analysis results (prediction of user intent and emotions).

[0835] Step 5:

[0836] The server recognizes emotions.

[0837] Input: Analysis results.

[0838] The system uses an emotion engine on the server (e.g., IBM Watson NLU, Google Cloud Natural Language API) to recognize the user's emotions from the analysis results.

[0839] As a data processing technique, emotions are identified based on extracted keywords and context.

[0840] Output: Recognized emotions (e.g., "busyness," "fatigue").

[0841] Step 6:

[0842] The server generates a response.

[0843] Input: Recognized emotions and analysis results.

[0844] The server uses a response generation engine (e.g., OpenAI's GPT-4) to generate an appropriate response.

[0845] As part of data processing, text is generated that matches the user's emotions and the character's personality.

[0846] For example, a response like, "You seem to have been very busy. You must be exhausted! Try to do something to relax," might be generated.

[0847] Output: The generated response.

[0848] Step 7:

[0849] The server sends a response to the user's terminal.

[0850] Input: Generated response.

[0851] The server sends the generated response to the user's terminal.

[0852] The device receives the response and displays it on the chat screen.

[0853] Output: The response displayed on the user's chat screen. The message displayed is: "You seem to have been very busy. You must be exhausted! Try to do something to relax."

[0854] As described above, appropriate data processing and calculations are performed at each step based on the input data, and finally, a response that resonates with the user's emotions is generated and displayed.

[0855] (Application Example 2)

[0856] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0857] In modern content delivery services, users demand personalized responses that address their individual emotions. However, traditional systems struggle to understand user emotions and generate appropriate responses. Furthermore, there is a lack of methods to provide interactive experiences that allow fans to feel satisfied through natural conversations with idol characters.

[0858] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, means for receiving and analyzing messages sent from the terminal, means for generating a response based on the analysis results, means for transmitting the generated response to the terminal, means for recognizing the user's emotions, means for adjusting the generated response based on the user's emotions, means for providing individual responses to the user in conjunction with video content, and means for including messages to alleviate the user's emotions. As a result, the user can enjoy an interactive experience with an idol character through natural dialogue that responds to their own emotions.

[0859] "User's device" refers to the device that a user uses to access the internet and utilize various services.

[0860] "Means of accepting access" refers to a function that receives connection requests from users' terminals and provides an interface for accessing the system.

[0861] "Means of authenticating login information" refers to a function that verifies authentication information such as the username and password entered by the user to confirm whether they are a legitimate user.

[0862] "Means for receiving and analyzing messages" refers to a function in which the system receives messages sent by users and analyzes their content.

[0863] "Means for generating a response based on analysis results" refers to a function that creates an appropriate response based on the analysis results of the message.

[0864] "Means for transmitting the generated response to the terminal" refers to a function that transmits the system-generated response to the user's terminal for display.

[0865] "Means of recognizing user emotions" refers to a function that analyzes and recognizes user emotions from analyzed messages.

[0866] "Means for adjusting generated responses based on the user's emotions" refers to a function that adjusts the generated responses to match the user's emotions and provides them in a more appropriate form.

[0867] "A means of providing individualized responses to users in conjunction with video content" refers to a function in which the system references video content and provides individualized and appropriate responses to users.

[0868] "Means of including messages to alleviate emotions" refers to features that include encouraging or comforting messages to reduce the emotions of the recognized user.

[0869] This invention relates to a chat system that allows users to enjoy natural conversations with idol characters. In particular, this system incorporates an emotion engine that recognizes the user's emotions and generates and adjusts responses accordingly. Detailed embodiments for carrying out this invention are described below.

[0870] This system includes the following components:

[0871] 1. Server:

[0872] Access Acceptance: Accepts access from the user's device. Specifically, it provides an interface for receiving HTTP requests.

[0873] Login Authentication: Authenticates the user's login information (username, password). Authentication is performed by comparing it with the database.

[0874] Message analysis: Receives messages sent by users and analyzes them using natural language processing algorithms.

[0875] Emotion Recognition: Recognizes the user's emotions from the analyzed message. Use an emotion engine (e.g., Transformers' emotion analysis model).

[0876] Response Generation: Generates appropriate responses based on analysis results and emotion recognition. Adjusts responses based on the personality and emotional state of the idol character.

[0877] Send response: Send the generated response to the user's device.

[0878] 2. User's device:

[0879] Access: Access the system through the chat app and enter your username and password on the login screen.

[0880] Chat interface: Provides a user interface for entering and sending messages.

[0881] Response Display: Displays the response received from the server.

[0882] 3. Emotional Engine:

[0883] Natural language processing algorithms: For example, use the Transformers library from Hugging Face.

[0884] Emotion analysis models: In particular, generative AI models such as BERT and RoBERTa are used to recognize emotions.

[0885] Specific example

[0886] For example, if a user sends a message saying "I'm a little tired today," the server will perform the following actions:

[0887] 1. The server receives the user's message and analyzes it using a natural language processing algorithm.

[0888] 2. The emotional engine recognizes "fatigue."

[0889] 3. The response generation engine generates a response based on the "feeling of fatigue," such as, "That must have been tough. Take a break and relax."

[0890] 4. The server sends this response to the user's device and displays it in the chat interface.

[0891] Examples of prompt messages include the following:

[0892] "Analyze the following user message, recognize and analyze the user's emotions, and generate a personalized response based on that: 'I'm a little tired today.'"

[0893] This invention allows users to enjoy an interactive experience with idol characters through natural dialogue that responds to their own emotions.

[0894] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0895] Step 1:

[0896] The server accepts access from the user's device. Specifically, it receives HTTP requests and grants access. At this stage, the input is a connection request from the user's device, and the output is a response granting access permission.

[0897] Step 2:

[0898] The server performs login authentication. The user enters their username and password and submits them from their terminal. The server compares this authentication information with the database to verify that the user is legitimate. At this stage, the input is the user's login information, and the output is the result of successful or unsuccessful authentication.

[0899] Step 3:

[0900] The user types a message into the chat interface and sends it. The input content is the message entered by the user. The message is sent from the terminal to the server.

[0901] Step 4:

[0902] The server receives messages sent from the user's terminal and analyzes them using natural language processing algorithms. The input here is the user's message, and the analyzed information (such as intent and emotion) is output.

[0903] Step 5:

[0904] The emotion engine within the server recognizes the user's emotions from the analyzed messages. The input is the analyzed information, and the output is the recognized emotion (e.g., "fatigue").

[0905] Step 6:

[0906] The server's response generation engine generates an appropriate response based on the analysis results and recognized emotions. The personality of the idol character is also taken into consideration during response generation. At this stage, the input is the analysis results and recognized emotions, and the output is the generated response.

[0907] Step 7:

[0908] The server sends the generated response to the user's terminal. The input is the generated response, and the output is the transmission of the response to the terminal.

[0909] Step 8:

[0910] The terminal displays the received response in the chat interface. The input is the response received from the server, and the output is the displayed message.

[0911] In this way, users can enjoy natural conversations with idol characters.

[0912] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0913] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0914] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0915] [Third Embodiment]

[0916] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0917] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0918] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0919] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0920] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0921] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0922] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0923] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0924] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0925] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0926] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0927] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0928] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. An embodiment of this system is described below.

[0929] System Configuration

[0930] This system consists of a server, a user terminal, and an idle chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission. The user terminal is used by the user to access the system and interact with the chatbot. The chatbot runs on the server and generates responses based on the idle character.

[0931] Program processing

[0932] The following explains in detail how the system works.

[0933] 1. The user accesses from their device.

[0934] The user launches the chat app on their device and accesses the system. Here, the user is redirected to a login screen where they enter their username and password.

[0935] 2. The server authenticates the login information.

[0936] The server receives the user's login information, compares it with the database, and performs authentication. If authentication is successful, the user can proceed to the chat screen.

[0937] 3. The user enters a message.

[0938] The user enters a message in the input field displayed on the chat screen and clicks the send button.

[0939] 4. The server receives and parses the message.

[0940] The server receives messages sent by users. Next, it uses natural language processing algorithms to analyze the messages and understand the user's intentions and emotions.

[0941] 5. The server generates a response.

[0942] Based on the analysis results, the server uses a response generation engine to generate an appropriate response. This response is selected and generated based on the personality of the idol character.

[0943] 6. The server sends a response to the user's terminal.

[0944] The server sends the generated response to the user's terminal and displays it on the chat screen.

[0945] Specific example

[0946] For example, consider the case where a user says to the chatbot, "The weather's nice today."

[0947] 1. The user types the message "The weather is nice today" from their device and presses the send button.

[0948] 2. The server receives this message and parses it using a natural language processing algorithm.

[0949] 3. The server identifies the topic "weather" from the analysis results and further recognizes the associated emotion as "positive."

[0950] 4. The server uses its response generation engine to generate the response, "It's such lovely weather! Shall we go for a walk?"

[0951] 5. The server sends this response to the user's device and displays it on the user's chat screen.

[0952] In this way, users can enjoy natural conversations with idol characters. This system can greatly improve user satisfaction even in situations where opportunities to directly interact with idols are limited.

[0953] The following describes the processing flow.

[0954] Step 1:

[0955] The user launches the "Idle Chatbot" app on their device and accesses the login screen. The user enters their username and password as login information.

[0956] Step 2:

[0957] The terminal sends the entered login information to the server.

[0958] Step 3:

[0959] The server compares the received login information with the database and performs authentication. If authentication is successful, the user is redirected to the chat screen.

[0960] Step 4:

[0961] The user enters a message in the input field displayed on the chat screen and clicks the send button. For example, they might type "Hello".

[0962] Step 5:

[0963] The terminal sends the entered message to the server.

[0964] Step 6:

[0965] The server analyzes the messages it receives using natural language processing algorithms. For example, it recognizes the message "Hello" as a "greeting."

[0966] Step 7:

[0967] Based on the analysis results, the server uses a response generation engine to generate an appropriate reply. For example, it might generate a reply such as, "Hi! How are you doing today?"

[0968] Step 8:

[0969] The server sends the generated response to the user's terminal.

[0970] Step 9:

[0971] The device displays the response it received on the chat screen. The user's device displays the message, "Hi! How are you doing today?"

[0972] Step 10:

[0973] The user types a new message and clicks the send button. For example, they might type, "I'm really busy today."

[0974] Step 11:

[0975] The device sends a new message to the server.

[0976] Step 12:

[0977] The server then analyzes the new message again using a natural language processing algorithm. For example, it analyzes the message "I'm really busy today" and recognizes the state of being "busy."

[0978] Step 13:

[0979] The server uses the response generation engine again to generate a response based on the new analysis results. For example, it might generate a response like, "Thank you so much for your hard work! Is there anything I can help you with?"

[0980] Step 14:

[0981] The server sends a new response to the user's device, which then displays it on the chat screen.

[0982] (Example 1)

[0983] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0984] Traditional chat systems made it difficult for users to enjoy natural conversations with artificial intelligence, and in particular, it was challenging to create conversations that reflected the unique personality of idol characters. Furthermore, achieving natural conversations required advanced processing, highlighting the need for improved user experience.

[0985] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0986] In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, and means for receiving and analyzing messages sent from the terminal. This allows the user to enjoy natural conversations with an idol character by using natural language processing algorithms and generative AI models in the message analysis and response generation means. Furthermore, by including means for adjusting responses based on the idol character's personality, responses that reflect the character's characteristics can be provided, resulting in a more satisfying user experience.

[0987] "Means of accepting access" refers to devices or software that provide the functionality for users to connect to the system from their terminals and initiate communication.

[0988] "Means for authenticating login information" refers to devices or software used to verify the username and password entered by the user and to confirm their identity.

[0989] "Means for receiving and analyzing messages" refers to a device or software that receives text messages sent from a terminal and processes them in order to understand their content.

[0990] "Means for generating a response" refers to a device or software for creating an appropriate response based on an analyzed message.

[0991] "Means for sending a response" refers to the device or software necessary to send the generated response to the user's terminal and continue the conversation.

[0992] A "natural language processing algorithm" is a set of computational methods and models used to understand and generate human language.

[0993] A "generative AI model" is a type of artificial intelligence that generates new data or text based on input, and operates based on specific rules or training data.

[0994] A "prompt" is input text provided to a generative AI model to elicit a specific response.

[0995] An "idol character" is a character that imitates a fictional entertainer or celebrity, possessing specific personalities and characteristics.

[0996] "Personality" refers to the unique character, speaking style, and behavior of an idol character.

[0997] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. Specific embodiments of this system are described below.

[0998] System Configuration

[0999] This system consists of a server, a user terminal, and an idle chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission, while the user terminal is used by the user to access the system and interact with the chatbot. The chatbot runs on the server and generates responses based on the idle character.

[1000] Hardware and software to be used

[1001] server:

[1002] Database server (e.g., MySQL)

[1003] Natural language processing algorithms (e.g., Google Cloud Natural Language API)

[1004] Response generation engine (e.g., OpenAI GPT-3)

[1005] User's device:

[1006] Chat application installed on a smartphone or computer

[1007] Detailed Operation Description

[1008] 1. The user accesses from a device:

[1009] The user launches a chat application on their smartphone or computer and is redirected to the login screen.

[1010] The user enters their username and password and clicks the "Login" button.

[1011] 2. The server authenticates the login information:

[1012] The server receives the login information entered by the user, compares it with the registered information in the database, and performs authentication.

[1013] If authentication is successful, the server creates a session for the user, and the user proceeds to the main chat screen.

[1014] 3. The user enters a message:

[1015] The user types a message on the main chat screen and clicks the "Send" button.

[1016] 4. The server receives and parses the message:

[1017] The server receives messages sent by users and parses them using the Google Cloud Natural Language API.

[1018] The server understands the user's intentions and emotions from the analysis results.

[1019] 5. The server generates a response:

[1020] Based on the analysis results, the server generates an appropriate response using OpenAI GPT-3.

[1021] The generated responses are tailored based on the personality of the idol character.

[1022] 6. The server sends a response to the user's terminal:

[1023] The server sends the generated response to the user's terminal and displays it on the chat screen.

[1024] Specific example

[1025] For example, if a user says to the chatbot, "The weather is nice today," the following process will occur.

[1026] The user types the message "The weather's nice today" into the chat app on their device and presses the send button.

[1027] The server receives this message and parses it using the Google Cloud Natural Language API.

[1028] The server identifies the topic "weather" from the analysis results and further recognizes the associated emotion as "positive."

[1029] The server uses OpenAI GPT-3 to generate the response, "It's such nice weather! Shall we go for a walk?"

[1030] The server sends this response to the user's device and displays it on the chat screen.

[1031] Example of a prompt

[1032] The following are examples of prompt statements for a generative AI model.

[1033] The user typed "The weather's nice today." Generate a response from the idol character to this message.

[1034] This system allows users to enjoy natural conversations with idol characters, providing a high level of satisfaction even in situations where opportunities to directly interact with idols are limited.

[1035] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1036] Step 1:

[1037] Users access from their devices

[1038] Input: The user launches the chat app on their smartphone or computer and enters their username and password on the login screen.

[1039] Data processing: The terminal sends the user's login information to the server.

[1040] Output: The user's login information (username and password) is sent to the server.

[1041] Specific actions:

[1042] The user launches the chat app by tapping it from the device's home screen.

[1043] Enter your username and password on the login screen and click the "Login" button.

[1044] Step 2:

[1045] The server authenticates the login information.

[1046] Input: The user's login information (username and password) is sent to the server.

[1047] Data processing: The server receives login information, compares it with the database, and performs authentication.

[1048] Output: An authentication result (success or failure) is generated.

[1049] Specific actions:

[1050] The server verifies the username and password against the database.

[1051] Once authentication is successful, the server creates a session and sends an instruction to the user to display the main chat screen.

[1052] Step 3:

[1053] The user enters a message.

[1054] Input: The user enters a message in the input field displayed on the main chat screen and clicks the "Send" button.

[1055] Data processing: The terminal sends the message entered by the user to the server.

[1056] Output: The user's message is sent to the server.

[1057] Specific actions:

[1058] The user types "The weather is nice today" into the input field on the chat screen.

[1059] The user clicks the "Send" button.

[1060] Step 4:

[1061] The server receives and parses the message.

[1062] Input: The message sent by the user is sent to the server.

[1063] Data processing: The server receives the message and analyzes its content using the Google Cloud Natural Language API. The analysis extracts the message's intent and sentiment.

[1064] Output: Analysis results (such as "weather" and "positive emotions") are generated.

[1065] Specific actions:

[1066] The server receives the user's message.

[1067] Using the Google Cloud Natural Language API, the message content is analyzed to identify "weather" and "positive emotions."

[1068] Step 5:

[1069] The server generates a response.

[1070] Input: Analysis results (such as "weather" and "positive emotions") are sent to the server.

[1071] Data processing: The server uses OpenAI GPT-3 to generate an appropriate response based on the analysis results. The generated response is adjusted based on the personality of the idol character.

[1072] Output: The generated response message is obtained.

[1073] Specific actions:

[1074] The server sends the analysis results to the response generation engine.

[1075] The response generation engine (OpenAI GPT-3) generates the response message, "It's really nice weather! Shall we go for a walk?"

[1076] Step 6:

[1077] The server sends a response to the user's terminal.

[1078] Input: The generated response message is on the server.

[1079] Data processing: The server sends the generated response message to the user's terminal.

[1080] Output: A response message is displayed on the user's terminal.

[1081] Specific actions:

[1082] The server sends the generated response message to the user's terminal.

[1083] The user's device receives this response message and displays it on the chat screen.

[1084] (Application Example 1)

[1085] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1086] Traditional chat systems are limited to text messaging, which makes it difficult for users to enjoy a more natural conversational experience. Furthermore, especially when conversing with entertainment characters, simply exchanging text messages lacks immersion and real-time interaction, resulting in lower user satisfaction. There is a need to address these issues and provide a more realistic conversational experience.

[1087] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1088] In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, means for receiving messages sent from the terminal as voice input and converting them into text using a speech recognition algorithm, means for analyzing the converted text, means for generating a response based on the analysis results, and means for converting the generated response into speech and sending it to the terminal. This enables the user to enjoy natural and real-time voice conversations with entertainment characters.

[1089] "User's device" refers to the electronic device that the user uses to access the system, and includes smartphones, tablets, personal computers, head-mounted displays, etc.

[1090] "Login information" refers to information such as usernames and passwords used to authenticate access rights to the system.

[1091] "Voice input" refers to a method of inputting messages to the system using voice.

[1092] A "speech recognition algorithm" refers to a technology that analyzes a user's voice and converts it into text data.

[1093] "Means of converting to text" refers to the processes and technologies for converting voice input into text-based data.

[1094] "Natural language processing algorithms" refer to technologies used to analyze text data and understand its meaning and context.

[1095] "Means of analysis" refers to the processes and technologies used to analyze input text data and understand the user's intentions and emotions.

[1096] "Means for generating a response" refers to the processes and techniques for generating an appropriate response based on the analysis results.

[1097] "Means of converting to speech" refers to the processes and technologies for converting generated text responses into audio data.

[1098] An "entertainment character" refers to a virtual personality or character created for interaction with users, possessing specific personality traits and characteristics.

[1099] This invention relates to a system that allows users to enjoy natural conversations with entertainment characters. The system consists of a user's terminal, a server, and a chatbot that supports entertainment characters.

[1100] System Configuration

[1101] The system works as follows:

[1102] 1. Means for accepting access from the user's terminal

[1103] Users access the system using a device (such as a smartphone, tablet, or head-mounted display). A login screen is displayed on the device, and the user enters their login information (username and password).

[1104] 2. Means by which the server authenticates login information

[1105] The server receives the user's login information, compares it with the database, and performs authentication. If authentication is successful, the user can proceed to the chat screen or virtual environment.

[1106] 3. Means by which the server receives voice input and performs speech recognition.

[1107] The user inputs a message by voice into their device. The server receives this voice message and converts the speech into text using a speech recognition algorithm (e.g., Google Speech-to-Text API).

[1108] 4. Means by which the server parses text data

[1109] The server analyzes the converted text data using natural language processing algorithms (e.g., OpenAI GPT-4) to understand the user's intent and emotions.

[1110] 5. Means by which the server generates a response

[1111] Based on the analysis results, the server uses a response generation engine (e.g., OpenAI GPT-4) to generate an appropriate response. This response is selected and generated based on the personality of the entertainment character.

[1112] 6. Means for converting the response generated by the server into speech and retransmitting it.

[1113] The generated text response is converted into speech using a speech synthesis system (e.g., Amazon Polly) and sent to the user's device. The device then plays the received audio and provides it to the user.

[1114] Specific example

[1115] For example, consider a scenario where a user wears a head-mounted display and uses voice input to say, "The weather's nice today."

[1116] 1. The user enters a voice message from their device saying, "The weather is nice today."

[1117] 2. The server receives this voice message and converts it to text using the Google Speech-to-Text API.

[1118] 3. The converted text data is analyzed using OpenAI GPT-4 to identify the topic "weather" and the emotion "positive".

[1119] 4. Based on the analysis results, the server generates a response such as, "It's really nice weather! Shall we go for a walk?"

[1120] 5. Convert this response into audio data using Amazon Polly and send it to the user's device.

[1121] 6. The user's device will play audio data, allowing the user to enjoy natural conversations with entertainment characters.

[1122] Example of a prompt

[1123] As an idol character, generate a response to the following user message: The weather is nice today.

[1124] Such a system allows users to enjoy natural, real-time conversations with entertainment characters, significantly improving the quality of the experience.

[1125] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1126] Step 1:

[1127] The user accesses the device.

[1128] Input: Users access the system using a head-mounted display (HMD) or smartphone.

[1129] Operation: The user launches the application, goes to the login screen, and enters their username and password.

[1130] Output: User login information is sent to the server.

[1131] Step 2:

[1132] The server authenticates the login information.

[1133] Enter: Username and password.

[1134] Operation: The server verifies the login information against the database and performs authentication. Authentication is successful if the information is correct, and failed if it is incorrect.

[1135] Output: If login is successful, the user proceeds to the chat screen or virtual environment. If login fails, an error message is displayed.

[1136] Step 3:

[1137] The user sends a message using voice input.

[1138] Input: The user says "The weather is nice today" in their voice.

[1139] Operation: The device records a voice message and sends it to the server.

[1140] Output: Audio data is sent to the server.

[1141] Step 4:

[1142] The server converts the audio data into text.

[1143] Input: User's voice data.

[1144] Operation: The server uses a speech recognition algorithm (e.g., Google Speech-to-Text API) to convert the audio data into text data.

[1145] Output: The text data "The weather is nice today" is generated.

[1146] Step 5:

[1147] The server parses the text data.

[1148] Input: Text data "The weather is nice today."

[1149] Operation: The server analyzes text data using natural language processing algorithms (e.g., OpenAI GPT-4) to understand the user's intent and emotions.

[1150] Output: The analysis results yield the topic "weather" and the emotion "positive".

[1151] Step 6:

[1152] The server generates a response.

[1153] Input: Analysis result - Topic "weather" and emotion "positive".

[1154] Operation: The server uses a response generation engine (e.g., OpenAI GPT-4) to generate a response such as, "It's such nice weather! Shall we go for a walk?"

[1155] Output: The response text "It's really nice weather! Shall we go for a walk?" is generated.

[1156] Step 7:

[1157] The server converts the response text into speech and resends it.

[1158] Input: Response text "It's such lovely weather! Shall we go for a walk?"

[1159] Operation: The server uses a text-to-speech system (e.g., Amazon Polly) to convert text into audio data. It then sends the generated audio data to the user's device.

[1160] Output: Audio data is sent to the user's device.

[1161] Step 8:

[1162] The user's device plays the audio data.

[1163] Input: Audio data sent from the server.

[1164] Operation: The device plays audio data and provides it to the user.

[1165] Output: Users can enjoy natural conversations with entertainment characters in audio format.

[1166] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1167] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. In particular, it describes a system that incorporates an emotion engine that recognizes the user's emotions and generates responses.

[1168] System Configuration

[1169] This system consists of a server, a user terminal, and an idle chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission. The user terminal is used by the user to access the system and interact with the chatbot. The chatbot runs on the server and generates responses based on the idle character. Furthermore, the system includes an emotion engine that recognizes the user's emotions.

[1170] Program processing

[1171] The following explains in detail how the system works.

[1172] 1. The user accesses from their device.

[1173] The user launches the chat app on their device and accesses the system. The user is then directed to the login screen and enters their username and password.

[1174] 2. The server authenticates the login information.

[1175] The server receives the user's login information, compares it with the database, and performs authentication. If authentication is successful, the user can proceed to the chat screen.

[1176] 3. The user enters a message.

[1177] The user enters a message in the input field displayed on the chat screen and clicks the send button. For example, they might type, "I'm a little tired today."

[1178] 4. The server receives and parses the message.

[1179] The server receives messages sent by users. Next, it uses natural language processing algorithms to analyze the messages and understand the user's intentions and emotions.

[1180] 5. The server recognizes emotions.

[1181] An emotion engine embedded within the server recognizes the user's emotions from the analyzed messages. For example, it recognizes "fatigue" from the message "I'm a little tired today."

[1182] 6. The server generates a response.

[1183] Based on the analysis results and recognized emotions, the server uses a response generation engine to generate an appropriate response. This response is tailored based on the user's emotions and the personality of the idol character. For example, it might generate a response such as, "Good work! Get some rest."

[1184] 7. The server sends a response to the user's terminal.

[1185] The server sends the generated response to the user's terminal and displays it on the chat screen.

[1186] Specific example

[1187] For example, consider a scenario where a user says to the chatbot, "Today was a very busy day at work."

[1188] 1. The user types the message "Today was a very busy day at work" from their device and presses the send button.

[1189] 2. The server receives this message and parses it using a natural language processing algorithm.

[1190] 3. The server's emotion engine recognizes the emotions of "busyness" and "fatigue."

[1191] 4. Based on the analysis results and recognized emotions, the server uses a response generation engine to generate a response such as, "You seem to have been very busy. You must be exhausted! Try to do something to relax."

[1192] 5. The server sends this response to the user's device and displays it on the user's chat screen.

[1193] In this way, users can not only enjoy natural conversations with idol characters but also receive responses that resonate with their emotions. This system can greatly improve user satisfaction even in situations where opportunities to directly interact with idols are limited.

[1194] The following describes the processing flow.

[1195] Step 1:

[1196] The user launches the "Idle Chatbot" app on their device and accesses the login screen. The user enters their username and password as login information.

[1197] Step 2:

[1198] The terminal sends the entered login information to the server.

[1199] Step 3:

[1200] The server compares the received login information with the database and performs authentication. If authentication is successful, the user is redirected to the chat screen.

[1201] Step 4:

[1202] The user enters a message in the input field displayed on the chat screen and clicks the send button. For example, they might type, "I'm a little tired today."

[1203] Step 5:

[1204] The terminal sends the entered message to the server.

[1205] Step 6:

[1206] The server analyzes the received messages using natural language processing algorithms. For example, it recognizes the message "I'm a little tired today" as "a feeling of fatigue."

[1207] Step 7:

[1208] An emotion engine embedded within the server recognizes the user's emotions from the analyzed messages. For example, it recognizes the emotion "fatigue" from the message "I'm a little tired today."

[1209] Step 8:

[1210] Based on the analysis results and recognized emotions, the server uses a response generation engine to generate an appropriate response. This response is tailored based on the user's emotions and the personality of the idol character. For example, it might generate a response such as, "Good work! Get some rest."

[1211] Step 9:

[1212] The server sends the generated response to the user's terminal.

[1213] Step 10:

[1214] The device displays the response it received on the chat screen. The user's device displays the message, "Good work! Get some rest."

[1215] Step 11:

[1216] The user types a new message and clicks the send button. For example, they might type, "Thank you, I'm going to rest for a bit."

[1217] Step 12:

[1218] The device sends a new message to the server.

[1219] Step 13:

[1220] The server then analyzes the new message again using a natural language processing algorithm. For example, it recognizes the intentions of "gratitude" and "rest" from the message "Thank you, I'll take a short rest."

[1221] Step 14:

[1222] The server then uses the emotion engine again to recognize the user's emotions. For example, it recognizes the emotion of "gratitude" from the message "Thank you, I'll take a break."

[1223] Step 15:

[1224] Based on the new analysis results and recognized emotions, the server uses the response generation engine again to generate a response. For example, it might generate a response like, "Okay, please continue to relax."

[1225] Step 16:

[1226] The server sends a new response to the user's device, which then displays it on the chat screen.

[1227] (Example 2)

[1228] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1229] Traditional chat systems have struggled to accurately understand users' emotions and intentions and generate appropriate responses. As a result, users were not satisfied, and the systems failed to provide a natural conversational experience. Furthermore, generating responses based on the personality of specific characters was difficult, resulting in a lack of technology to achieve realistic dialogue. Solving these challenges is highly desirable.

[1230] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1231] In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, means for receiving and analyzing messages sent from the terminal, means for recognizing the user's emotions from the analyzed messages, means for generating a response based on the analysis results and the recognized emotions, and means for sending the generated response to the terminal. This makes it possible to accurately recognize the user's emotions and provide a natural conversation experience that takes into account the character's personality.

[1232] A "user's terminal" is an electronic device used by a user to access and interact with the system.

[1233] "Login information" refers to information such as usernames and passwords that are necessary for user authentication.

[1234] A "message" is text or written information that a user sends to a system through their device.

[1235] A "natural language processing algorithm" is a computational method and process for analyzing text information and understanding its context and meaning.

[1236] An "emotion engine" is a software component that recognizes a user's emotions from analyzed messages.

[1237] A "response generation engine" is a software component that generates appropriate responses based on analysis results and recognized emotions.

[1238] "Character personality" refers to the characteristics and behavioral traits assigned to a specific fictional person or character.

[1239] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. The system consists of a server, a user terminal, and an idol chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission. The user terminal is used by the user to access the system and interact with the chatbot. The chatbot operates on the server and generates responses based on the idol character. Furthermore, the system includes an emotion engine that recognizes the user's emotions.

[1240] Hardware and software to be used

[1241] The system's hardware includes smartphones and personal computers as user terminals, and high-performance computing devices as servers. The software uses chat applications, natural language processing algorithms (e.g., spaCy, NLTK), sentiment engines (e.g., IBM Watson NLU, Google Cloud Natural Language API), and response generation engines (e.g., OpenAI's GPT-4).

[1242] Program processing

[1243] The following describes the specific processing details of this system's program.

[1244] The user accesses the system from their device, launches the chat application, and is directed to the login screen. The user enters their username and password and sends them to the server. The server authenticates the user, and if successful, the user proceeds to the chat screen. Next, the user types and sends a message. The server receives the message and analyzes it using a natural language processing algorithm. Subsequently, the emotion engine recognizes the user's emotions from the analysis results, and the response generation engine generates a response based on the recognized emotions. Finally, the server sends the generated response to the user's device and displays it on the chat screen.

[1245] Specific example

[1246] For example, if a user says to the chatbot, "I was very busy at work today," the process will proceed as follows:

[1247] 1. The user types the message "Today was a very busy day at work" from their device and presses the send button.

[1248] 2. The server receives this message and parses its content using a natural language processing algorithm.

[1249] 3. The emotion engine within the server recognizes "busyness" and "fatigue."

[1250] 4. Based on the analysis results and recognized emotions, the server uses a response generation engine to generate a response such as, "You seem to have been very busy. You must be exhausted! Try to do something to relax."

[1251] 5. The server sends the generated response to the user's device and displays it on the chat screen.

[1252] In this way, users can not only enjoy natural conversations with idol characters, but also receive responses that resonate with their emotions.

[1253] Example of a prompt

[1254] Examples of prompts for a generative AI model are as follows:

[1255] Please create a system where, when a user says "Today was a very busy day at work," the emotion engine recognizes "busyness" and "fatigue," and the character then generates a response such as "Great job! Get some rest."

[1256] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1257] Step 1:

[1258] The user accesses it from their device.

[1259] Enter: Username and password.

[1260] The server displays a login screen and provides input fields for username and password.

[1261] The user enters their username and password and clicks the login button.

[1262] Output: Login information is sent from the terminal to the server.

[1263] Step 2:

[1264] The server authenticates the login information.

[1265] Input: Username and password sent from the device.

[1266] The server compares the user information stored in the database with the entered information.

[1267] If authentication is successful, the server sends an instruction to redirect the user to the chat screen.

[1268] Output: Authentication result and instructions to transition to the chat screen.

[1269] Step 3:

[1270] The user enters a message.

[1271] Input: User's message (e.g., "Today was a very busy day at work").

[1272] The user enters text into the message input field on the chat screen and clicks the send button.

[1273] Output: A message is sent from the terminal to the server.

[1274] Step 4:

[1275] The server receives and parses the message.

[1276] Input: Message sent by the user.

[1277] The server receives the message and parses it using a natural language processing algorithm (e.g., spaCy, NLTK).

[1278] As part of data processing, the text is split and keywords and context are extracted.

[1279] Output: Analysis results (prediction of user intent and emotions).

[1280] Step 5:

[1281] The server recognizes emotions.

[1282] Input: Analysis results.

[1283] The system uses an emotion engine on the server (e.g., IBM Watson NLU, Google Cloud Natural Language API) to recognize the user's emotions from the analysis results.

[1284] As a data processing technique, emotions are identified based on extracted keywords and context.

[1285] Output: Recognized emotions (e.g., "busyness," "fatigue").

[1286] Step 6:

[1287] The server generates a response.

[1288] Input: Recognized emotions and analysis results.

[1289] The server uses a response generation engine (e.g., OpenAI's GPT-4) to generate an appropriate response.

[1290] As part of data processing, text is generated that matches the user's emotions and the character's personality.

[1291] For example, a response like, "You seem to have been very busy. You must be exhausted! Try to do something to relax," might be generated.

[1292] Output: The generated response.

[1293] Step 7:

[1294] The server sends a response to the user's terminal.

[1295] Input: Generated response.

[1296] The server sends the generated response to the user's terminal.

[1297] The device receives the response and displays it on the chat screen.

[1298] Output: The response displayed on the user's chat screen. The message displayed is: "You seem to have been very busy. You must be exhausted! Try to do something to relax."

[1299] As described above, appropriate data processing and calculations are performed at each step based on the input data, and finally, a response that resonates with the user's emotions is generated and displayed.

[1300] (Application Example 2)

[1301] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1302] In modern content delivery services, users demand personalized responses that address their individual emotions. However, traditional systems struggle to understand user emotions and generate appropriate responses. Furthermore, there is a lack of methods to provide interactive experiences that allow fans to feel satisfied through natural conversations with idol characters.

[1303] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, means for receiving and analyzing messages sent from the terminal, means for generating a response based on the analysis results, means for transmitting the generated response to the terminal, means for recognizing the user's emotions, means for adjusting the generated response based on the user's emotions, means for providing individual responses to the user in conjunction with video content, and means for including messages to alleviate the user's emotions. As a result, the user can enjoy an interactive experience with an idol character through natural dialogue that responds to their own emotions.

[1304] "User's device" refers to the device that a user uses to access the internet and utilize various services.

[1305] "Means of accepting access" refers to a function that receives connection requests from users' terminals and provides an interface for accessing the system.

[1306] "Means of authenticating login information" refers to a function that verifies authentication information such as the username and password entered by the user to confirm whether they are a legitimate user.

[1307] "Means for receiving and analyzing messages" refers to a function in which the system receives messages sent by users and analyzes their content.

[1308] "Means for generating a response based on analysis results" refers to a function that creates an appropriate response based on the analysis results of the message.

[1309] "Means for transmitting the generated response to the terminal" refers to a function that transmits the system-generated response to the user's terminal for display.

[1310] "Means of recognizing user emotions" refers to a function that analyzes and recognizes user emotions from analyzed messages.

[1311] "Means for adjusting generated responses based on the user's emotions" refers to a function that adjusts the generated responses to match the user's emotions and provides them in a more appropriate form.

[1312] "A means of providing individualized responses to users in conjunction with video content" refers to a function in which the system references video content and provides individualized and appropriate responses to users.

[1313] "Means of including messages to alleviate emotions" refers to features that include encouraging or comforting messages to reduce the emotions of the recognized user.

[1314] This invention relates to a chat system that allows users to enjoy natural conversations with idol characters. In particular, this system incorporates an emotion engine that recognizes the user's emotions and generates and adjusts responses accordingly. Detailed embodiments for carrying out this invention are described below.

[1315] This system includes the following components:

[1316] 1. Server:

[1317] Access Acceptance: Accepts access from the user's device. Specifically, it provides an interface for receiving HTTP requests.

[1318] Login Authentication: Authenticates the user's login information (username, password). Authentication is performed by comparing it with the database.

[1319] Message analysis: Receives messages sent by users and analyzes them using natural language processing algorithms.

[1320] Emotion Recognition: Recognizes the user's emotions from the analyzed message. Use an emotion engine (e.g., Transformers' emotion analysis model).

[1321] Response Generation: Generates appropriate responses based on analysis results and emotion recognition. Adjusts responses based on the personality and emotional state of the idol character.

[1322] Send response: Send the generated response to the user's device.

[1323] 2. User's device:

[1324] Access: Access the system through the chat app and enter your username and password on the login screen.

[1325] Chat interface: Provides a user interface for entering and sending messages.

[1326] Response Display: Displays the response received from the server.

[1327] 3. Emotional Engine:

[1328] Natural language processing algorithms: For example, use the Transformers library from Hugging Face.

[1329] Emotion analysis models: In particular, generative AI models such as BERT and RoBERTa are used to recognize emotions.

[1330] Specific example

[1331] For example, if a user sends a message saying "I'm a little tired today," the server will perform the following actions:

[1332] 1. The server receives the user's message and analyzes it using a natural language processing algorithm.

[1333] 2. The emotional engine recognizes "fatigue."

[1334] 3. The response generation engine generates a response based on the "feeling of fatigue," such as, "That must have been tough. Take a break and relax."

[1335] 4. The server sends this response to the user's device and displays it in the chat interface.

[1336] Examples of prompt messages include the following:

[1337] "Analyze the following user message, recognize and analyze the user's emotions, and generate a personalized response based on that: 'I'm a little tired today.'"

[1338] This invention allows users to enjoy an interactive experience with idol characters through natural dialogue that responds to their own emotions.

[1339] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1340] Step 1:

[1341] The server accepts access from the user's device. Specifically, it receives HTTP requests and grants access. At this stage, the input is a connection request from the user's device, and the output is a response granting access permission.

[1342] Step 2:

[1343] The server performs login authentication. The user enters their username and password and submits them from their terminal. The server compares this authentication information with the database to verify that the user is legitimate. At this stage, the input is the user's login information, and the output is the result of successful or unsuccessful authentication.

[1344] Step 3:

[1345] The user types a message into the chat interface and sends it. The input content is the message entered by the user. The message is sent from the terminal to the server.

[1346] Step 4:

[1347] The server receives messages sent from the user's terminal and analyzes them using natural language processing algorithms. The input here is the user's message, and the analyzed information (such as intent and emotion) is output.

[1348] Step 5:

[1349] The emotion engine within the server recognizes the user's emotions from the analyzed messages. The input is the analyzed information, and the output is the recognized emotion (e.g., "fatigue").

[1350] Step 6:

[1351] The server's response generation engine generates an appropriate response based on the analysis results and recognized emotions. The personality of the idol character is also taken into consideration during response generation. At this stage, the input is the analysis results and recognized emotions, and the output is the generated response.

[1352] Step 7:

[1353] The server sends the generated response to the user's terminal. The input is the generated response, and the output is the transmission of the response to the terminal.

[1354] Step 8:

[1355] The terminal displays the received response in the chat interface. The input is the response received from the server, and the output is the displayed message.

[1356] In this way, users can enjoy natural conversations with idol characters.

[1357] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1358] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1359] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1360] [Fourth Embodiment]

[1361] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1362] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1363] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1364] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1365] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1366] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1367] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1368] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1369] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1370] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1371] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1372] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1373] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1374] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. An embodiment of this system is described below.

[1375] System Configuration

[1376] This system consists of a server, a user terminal, and an idle chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission. The user terminal is used by the user to access the system and interact with the chatbot. The chatbot runs on the server and generates responses based on the idle character.

[1377] Program processing

[1378] The following explains in detail how the system works.

[1379] 1. The user accesses from their device.

[1380] The user launches the chat app on their device and accesses the system. Here, the user is redirected to a login screen where they enter their username and password.

[1381] 2. The server authenticates the login information.

[1382] The server receives the user's login information, compares it with the database, and performs authentication. If authentication is successful, the user can proceed to the chat screen.

[1383] 3. The user enters a message.

[1384] The user enters a message in the input field displayed on the chat screen and clicks the send button.

[1385] 4. The server receives and parses the message.

[1386] The server receives messages sent by users. Next, it uses natural language processing algorithms to analyze the messages and understand the user's intentions and emotions.

[1387] 5. The server generates a response.

[1388] Based on the analysis results, the server uses a response generation engine to generate an appropriate response. This response is selected and generated based on the personality of the idol character.

[1389] 6. The server sends a response to the user's terminal.

[1390] The server sends the generated response to the user's terminal and displays it on the chat screen.

[1391] Specific example

[1392] For example, consider the case where a user says to the chatbot, "The weather's nice today."

[1393] 1. The user types the message "The weather is nice today" from their device and presses the send button.

[1394] 2. The server receives this message and parses it using a natural language processing algorithm.

[1395] 3. The server identifies the topic "weather" from the analysis results and further recognizes the associated emotion as "positive."

[1396] 4. The server uses its response generation engine to generate the response, "It's such lovely weather! Shall we go for a walk?"

[1397] 5. The server sends this response to the user's device and displays it on the user's chat screen.

[1398] In this way, users can enjoy natural conversations with idol characters. This system can greatly improve user satisfaction even in situations where opportunities to directly interact with idols are limited.

[1399] The following describes the processing flow.

[1400] Step 1:

[1401] The user launches the "Idle Chatbot" app on their device and accesses the login screen. The user enters their username and password as login information.

[1402] Step 2:

[1403] The terminal sends the entered login information to the server.

[1404] Step 3:

[1405] The server compares the received login information with the database and performs authentication. If authentication is successful, the user is redirected to the chat screen.

[1406] Step 4:

[1407] The user enters a message in the input field displayed on the chat screen and clicks the send button. For example, they might type "Hello".

[1408] Step 5:

[1409] The terminal sends the entered message to the server.

[1410] Step 6:

[1411] The server analyzes the messages it receives using natural language processing algorithms. For example, it recognizes the message "Hello" as a "greeting."

[1412] Step 7:

[1413] Based on the analysis results, the server uses a response generation engine to generate an appropriate reply. For example, it might generate a reply such as, "Hi! How are you doing today?"

[1414] Step 8:

[1415] The server sends the generated response to the user's terminal.

[1416] Step 9:

[1417] The device displays the response it received on the chat screen. The user's device displays the message, "Hi! How are you doing today?"

[1418] Step 10:

[1419] The user types a new message and clicks the send button. For example, they might type, "I'm really busy today."

[1420] Step 11:

[1421] The device sends a new message to the server.

[1422] Step 12:

[1423] The server then analyzes the new message again using a natural language processing algorithm. For example, it analyzes the message "I'm really busy today" and recognizes the state of being "busy."

[1424] Step 13:

[1425] The server uses the response generation engine again to generate a response based on the new analysis results. For example, it might generate a response like, "Thank you so much for your hard work! Is there anything I can help you with?"

[1426] Step 14:

[1427] The server sends a new response to the user's device, which then displays it on the chat screen.

[1428] (Example 1)

[1429] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1430] Traditional chat systems made it difficult for users to enjoy natural conversations with artificial intelligence, and in particular, it was challenging to create conversations that reflected the unique personality of idol characters. Furthermore, achieving natural conversations required advanced processing, highlighting the need for improved user experience.

[1431] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1432] In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, and means for receiving and analyzing messages sent from the terminal. This allows the user to enjoy natural conversations with an idol character by using natural language processing algorithms and generative AI models in the message analysis and response generation means. Furthermore, by including means for adjusting responses based on the idol character's personality, responses that reflect the character's characteristics can be provided, resulting in a more satisfying user experience.

[1433] "Means of accepting access" refers to devices or software that provide the functionality for users to connect to the system from their terminals and initiate communication.

[1434] "Means for authenticating login information" refers to devices or software used to verify the username and password entered by the user and to confirm their identity.

[1435] "Means for receiving and analyzing messages" refers to a device or software that receives text messages sent from a terminal and processes them in order to understand their content.

[1436] "Means for generating a response" refers to a device or software for creating an appropriate response based on an analyzed message.

[1437] "Means for sending a response" refers to the device or software necessary to send the generated response to the user's terminal and continue the conversation.

[1438] A "natural language processing algorithm" is a set of computational methods and models used to understand and generate human language.

[1439] A "generative AI model" is a type of artificial intelligence that generates new data or text based on input, and operates based on specific rules or training data.

[1440] A "prompt" is input text provided to a generative AI model to elicit a specific response.

[1441] An "idol character" is a character that imitates a fictional entertainer or celebrity, possessing specific personalities and characteristics.

[1442] "Personality" refers to the unique character, speaking style, and behavior of an idol character.

[1443] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. Specific embodiments of this system are described below.

[1444] System Configuration

[1445] This system consists of a server, a user terminal, and an idle chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission, while the user terminal is used by the user to access the system and interact with the chatbot. The chatbot runs on the server and generates responses based on the idle character.

[1446] Hardware and software to be used

[1447] server:

[1448] Database server (e.g., MySQL)

[1449] Natural language processing algorithms (e.g., Google Cloud Natural Language API)

[1450] Response generation engine (e.g., OpenAI GPT-3)

[1451] User's device:

[1452] Chat application installed on a smartphone or computer

[1453] Detailed Operation Description

[1454] 1. The user accesses from a device:

[1455] The user launches a chat application on their smartphone or computer and is redirected to the login screen.

[1456] The user enters their username and password and clicks the "Login" button.

[1457] 2. The server authenticates the login information:

[1458] The server receives the login information entered by the user, compares it with the registered information in the database, and performs authentication.

[1459] If authentication is successful, the server creates a session for the user, and the user proceeds to the main chat screen.

[1460] 3. The user enters a message:

[1461] The user types a message on the main chat screen and clicks the "Send" button.

[1462] 4. The server receives and parses the message:

[1463] The server receives messages sent by users and parses them using the Google Cloud Natural Language API.

[1464] The server understands the user's intentions and emotions from the analysis results.

[1465] 5. The server generates a response:

[1466] Based on the analysis results, the server generates an appropriate response using OpenAI GPT-3.

[1467] The generated responses are tailored based on the personality of the idol character.

[1468] 6. The server sends a response to the user's terminal:

[1469] The server sends the generated response to the user's terminal and displays it on the chat screen.

[1470] Specific example

[1471] For example, if a user says to the chatbot, "The weather is nice today," the following process will occur.

[1472] The user types the message "The weather's nice today" into the chat app on their device and presses the send button.

[1473] The server receives this message and parses it using the Google Cloud Natural Language API.

[1474] The server identifies the topic "weather" from the analysis results and further recognizes the associated emotion as "positive."

[1475] The server uses OpenAI GPT-3 to generate the response, "It's such nice weather! Shall we go for a walk?"

[1476] The server sends this response to the user's device and displays it on the chat screen.

[1477] Example of a prompt

[1478] The following are examples of prompt statements for a generative AI model.

[1479] The user typed "The weather's nice today." Generate a response from the idol character to this message.

[1480] This system allows users to enjoy natural conversations with idol characters, providing a high level of satisfaction even in situations where opportunities to directly interact with idols are limited.

[1481] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1482] Step 1:

[1483] Users access from their devices

[1484] Input: The user launches the chat app on their smartphone or computer and enters their username and password on the login screen.

[1485] Data processing: The terminal sends the user's login information to the server.

[1486] Output: The user's login information (username and password) is sent to the server.

[1487] Specific actions:

[1488] The user launches the chat app by tapping it from the device's home screen.

[1489] Enter your username and password on the login screen and click the "Login" button.

[1490] Step 2:

[1491] The server authenticates the login information.

[1492] Input: The user's login information (username and password) is sent to the server.

[1493] Data processing: The server receives login information, compares it with the database, and performs authentication.

[1494] Output: An authentication result (success or failure) is generated.

[1495] Specific actions:

[1496] The server verifies the username and password against the database.

[1497] Once authentication is successful, the server creates a session and sends an instruction to the user to display the main chat screen.

[1498] Step 3:

[1499] The user enters a message.

[1500] Input: The user enters a message in the input field displayed on the main chat screen and clicks the "Send" button.

[1501] Data processing: The terminal sends the message entered by the user to the server.

[1502] Output: The user's message is sent to the server.

[1503] Specific actions:

[1504] The user types "The weather is nice today" into the input field on the chat screen.

[1505] The user clicks the "Send" button.

[1506] Step 4:

[1507] The server receives and parses the message.

[1508] Input: The message sent by the user is sent to the server.

[1509] Data processing: The server receives the message and analyzes its content using the Google Cloud Natural Language API. The analysis extracts the message's intent and sentiment.

[1510] Output: Analysis results (such as "weather" and "positive emotions") are generated.

[1511] Specific actions:

[1512] The server receives the user's message.

[1513] Using the Google Cloud Natural Language API, the message content is analyzed to identify "weather" and "positive emotions."

[1514] Step 5:

[1515] The server generates a response.

[1516] Input: Analysis results (such as "weather" and "positive emotions") are sent to the server.

[1517] Data processing: The server uses OpenAI GPT-3 to generate an appropriate response based on the analysis results. The generated response is adjusted based on the personality of the idol character.

[1518] Output: The generated response message is obtained.

[1519] Specific actions:

[1520] The server sends the analysis results to the response generation engine.

[1521] The response generation engine (OpenAI GPT-3) generates the response message, "It's really nice weather! Shall we go for a walk?"

[1522] Step 6:

[1523] The server sends a response to the user's terminal.

[1524] Input: The generated response message is on the server.

[1525] Data processing: The server sends the generated response message to the user's terminal.

[1526] Output: A response message is displayed on the user's terminal.

[1527] Specific actions:

[1528] The server sends the generated response message to the user's terminal.

[1529] The user's device receives this response message and displays it on the chat screen.

[1530] (Application Example 1)

[1531] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1532] Traditional chat systems are limited to text messaging, which makes it difficult for users to enjoy a more natural conversational experience. Furthermore, especially when conversing with entertainment characters, simply exchanging text messages lacks immersion and real-time interaction, resulting in lower user satisfaction. There is a need to address these issues and provide a more realistic conversational experience.

[1533] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1534] In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, means for receiving messages sent from the terminal as voice input and converting them into text using a speech recognition algorithm, means for analyzing the converted text, means for generating a response based on the analysis results, and means for converting the generated response into speech and sending it to the terminal. This enables the user to enjoy natural and real-time voice conversations with entertainment characters.

[1535] "User's device" refers to the electronic device that the user uses to access the system, and includes smartphones, tablets, personal computers, head-mounted displays, etc.

[1536] "Login information" refers to information such as usernames and passwords used to authenticate access rights to the system.

[1537] "Voice input" refers to a method of inputting messages to the system using voice.

[1538] A "speech recognition algorithm" refers to a technology that analyzes a user's voice and converts it into text data.

[1539] "Means of converting to text" refers to the processes and technologies for converting voice input into text-based data.

[1540] "Natural language processing algorithms" refer to technologies used to analyze text data and understand its meaning and context.

[1541] "Means of analysis" refers to the processes and technologies used to analyze input text data and understand the user's intentions and emotions.

[1542] "Means for generating a response" refers to the processes and techniques for generating an appropriate response based on the analysis results.

[1543] "Means of converting to speech" refers to the processes and technologies for converting generated text responses into audio data.

[1544] An "entertainment character" refers to a virtual personality or character created for interaction with users, possessing specific personality traits and characteristics.

[1545] This invention relates to a system that allows users to enjoy natural conversations with entertainment characters. The system consists of a user's terminal, a server, and a chatbot that supports entertainment characters.

[1546] System Configuration

[1547] The system works as follows:

[1548] 1. Means for accepting access from the user's terminal

[1549] Users access the system using a device (such as a smartphone, tablet, or head-mounted display). A login screen is displayed on the device, and the user enters their login information (username and password).

[1550] 2. Means by which the server authenticates login information

[1551] The server receives the user's login information, compares it with the database, and performs authentication. If authentication is successful, the user can proceed to the chat screen or virtual environment.

[1552] 3. Means by which the server receives voice input and performs speech recognition.

[1553] The user inputs a message by voice into their device. The server receives this voice message and converts the speech into text using a speech recognition algorithm (e.g., Google Speech-to-Text API).

[1554] 4. Means by which the server parses text data

[1555] The server analyzes the converted text data using natural language processing algorithms (e.g., OpenAI GPT-4) to understand the user's intent and emotions.

[1556] 5. Means by which the server generates a response

[1557] Based on the analysis results, the server uses a response generation engine (e.g., OpenAI GPT-4) to generate an appropriate response. This response is selected and generated based on the personality of the entertainment character.

[1558] 6. Means for converting the response generated by the server into speech and retransmitting it.

[1559] The generated text response is converted into speech using a speech synthesis system (e.g., Amazon Polly) and sent to the user's device. The device then plays the received audio and provides it to the user.

[1560] Specific example

[1561] For example, consider a scenario where a user wears a head-mounted display and uses voice input to say, "The weather's nice today."

[1562] 1. The user enters a voice message from their device saying, "The weather is nice today."

[1563] 2. The server receives this voice message and converts it to text using the Google Speech-to-Text API.

[1564] 3. The converted text data is analyzed using OpenAI GPT-4 to identify the topic "weather" and the emotion "positive".

[1565] 4. Based on the analysis results, the server generates a response such as, "It's really nice weather! Shall we go for a walk?"

[1566] 5. Convert this response into audio data using Amazon Polly and send it to the user's device.

[1567] 6. The user's device will play audio data, allowing the user to enjoy natural conversations with entertainment characters.

[1568] Example of a prompt

[1569] As an idol character, generate a response to the following user message: The weather is nice today.

[1570] Such a system allows users to enjoy natural, real-time conversations with entertainment characters, significantly improving the quality of the experience.

[1571] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1572] Step 1:

[1573] The user accesses the device.

[1574] Input: Users access the system using a head-mounted display (HMD) or smartphone.

[1575] Operation: The user launches the application, goes to the login screen, and enters their username and password.

[1576] Output: User login information is sent to the server.

[1577] Step 2:

[1578] The server authenticates the login information.

[1579] Enter: Username and password.

[1580] Operation: The server verifies the login information against the database and performs authentication. Authentication is successful if the information is correct, and failed if it is incorrect.

[1581] Output: If login is successful, the user proceeds to the chat screen or virtual environment. If login fails, an error message is displayed.

[1582] Step 3:

[1583] The user sends a message using voice input.

[1584] Input: The user says "The weather is nice today" in their voice.

[1585] Operation: The device records a voice message and sends it to the server.

[1586] Output: Audio data is sent to the server.

[1587] Step 4:

[1588] The server converts the audio data into text.

[1589] Input: User's voice data.

[1590] Operation: The server uses a speech recognition algorithm (e.g., Google Speech-to-Text API) to convert the audio data into text data.

[1591] Output: The text data "The weather is nice today" is generated.

[1592] Step 5:

[1593] The server parses the text data.

[1594] Input: Text data "The weather is nice today."

[1595] Operation: The server analyzes text data using natural language processing algorithms (e.g., OpenAI GPT-4) to understand the user's intent and emotions.

[1596] Output: The analysis results yield the topic "weather" and the emotion "positive".

[1597] Step 6:

[1598] The server generates a response.

[1599] Input: Analysis result - Topic "weather" and emotion "positive".

[1600] Operation: The server uses a response generation engine (e.g., OpenAI GPT-4) to generate a response such as, "It's such nice weather! Shall we go for a walk?"

[1601] Output: The response text "It's really nice weather! Shall we go for a walk?" is generated.

[1602] Step 7:

[1603] The server converts the response text into speech and resends it.

[1604] Input: Response text "It's such lovely weather! Shall we go for a walk?"

[1605] Operation: The server uses a text-to-speech system (e.g., Amazon Polly) to convert text into audio data. It then sends the generated audio data to the user's device.

[1606] Output: Audio data is sent to the user's device.

[1607] Step 8:

[1608] The user's device plays the audio data.

[1609] Input: Audio data sent from the server.

[1610] Operation: The device plays audio data and provides it to the user.

[1611] Output: Users can enjoy natural conversations with entertainment characters in audio format.

[1612] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1613] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. In particular, it describes a system that incorporates an emotion engine that recognizes the user's emotions and generates responses.

[1614] System Configuration

[1615] This system consists of a server, a user terminal, and an idle chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission. The user terminal is used by the user to access the system and interact with the chatbot. The chatbot runs on the server and generates responses based on the idle character. Furthermore, the system includes an emotion engine that recognizes the user's emotions.

[1616] Program processing

[1617] The following explains in detail how the system works.

[1618] 1. The user accesses from their device.

[1619] The user launches the chat app on their device and accesses the system. The user is then directed to the login screen and enters their username and password.

[1620] 2. The server authenticates the login information.

[1621] The server receives the user's login information, compares it with the database, and performs authentication. If authentication is successful, the user can proceed to the chat screen.

[1622] 3. The user enters a message.

[1623] The user enters a message in the input field displayed on the chat screen and clicks the send button. For example, they might type, "I'm a little tired today."

[1624] 4. The server receives and parses the message.

[1625] The server receives messages sent by users. Next, it uses natural language processing algorithms to analyze the messages and understand the user's intentions and emotions.

[1626] 5. The server recognizes emotions.

[1627] An emotion engine embedded within the server recognizes the user's emotions from the analyzed messages. For example, it recognizes "fatigue" from the message "I'm a little tired today."

[1628] 6. The server generates a response.

[1629] Based on the analysis results and recognized emotions, the server uses a response generation engine to generate an appropriate response. This response is tailored based on the user's emotions and the personality of the idol character. For example, it might generate a response such as, "Good work! Get some rest."

[1630] 7. The server sends a response to the user's terminal.

[1631] The server sends the generated response to the user's terminal and displays it on the chat screen.

[1632] Specific example

[1633] For example, consider a scenario where a user says to the chatbot, "Today was a very busy day at work."

[1634] 1. The user types the message "Today was a very busy day at work" from their device and presses the send button.

[1635] 2. The server receives this message and parses it using a natural language processing algorithm.

[1636] 3. The server's emotion engine recognizes the emotions of "busyness" and "fatigue."

[1637] 4. Based on the analysis results and recognized emotions, the server uses a response generation engine to generate a response such as, "You seem to have been very busy. You must be exhausted! Try to do something to relax."

[1638] 5. The server sends this response to the user's device and displays it on the user's chat screen.

[1639] In this way, users can not only enjoy natural conversations with idol characters but also receive responses that resonate with their emotions. This system can greatly improve user satisfaction even in situations where opportunities to directly interact with idols are limited.

[1640] The following describes the processing flow.

[1641] Step 1:

[1642] The user launches the "Idle Chatbot" app on their device and accesses the login screen. The user enters their username and password as login information.

[1643] Step 2:

[1644] The terminal sends the entered login information to the server.

[1645] Step 3:

[1646] The server compares the received login information with the database and performs authentication. If authentication is successful, the user is redirected to the chat screen.

[1647] Step 4:

[1648] The user enters a message in the input field displayed on the chat screen and clicks the send button. For example, they might type, "I'm a little tired today."

[1649] Step 5:

[1650] The terminal sends the entered message to the server.

[1651] Step 6:

[1652] The server analyzes the received messages using natural language processing algorithms. For example, it recognizes the message "I'm a little tired today" as "a feeling of fatigue."

[1653] Step 7:

[1654] An emotion engine embedded within the server recognizes the user's emotions from the analyzed messages. For example, it recognizes the emotion "fatigue" from the message "I'm a little tired today."

[1655] Step 8:

[1656] Based on the analysis results and recognized emotions, the server uses a response generation engine to generate an appropriate response. This response is tailored based on the user's emotions and the personality of the idol character. For example, it might generate a response such as, "Good work! Get some rest."

[1657] Step 9:

[1658] The server sends the generated response to the user's terminal.

[1659] Step 10:

[1660] The device displays the response it received on the chat screen. The user's device displays the message, "Good work! Get some rest."

[1661] Step 11:

[1662] The user types a new message and clicks the send button. For example, they might type, "Thank you, I'm going to rest for a bit."

[1663] Step 12:

[1664] The device sends a new message to the server.

[1665] Step 13:

[1666] The server then analyzes the new message again using a natural language processing algorithm. For example, it recognizes the intentions of "gratitude" and "rest" from the message "Thank you, I'll take a short rest."

[1667] Step 14:

[1668] The server then uses the emotion engine again to recognize the user's emotions. For example, it recognizes the emotion of "gratitude" from the message "Thank you, I'll take a break."

[1669] Step 15:

[1670] Based on the new analysis results and recognized emotions, the server uses the response generation engine again to generate a response. For example, it might generate a response like, "Okay, please continue to relax."

[1671] Step 16:

[1672] The server sends a new response to the user's device, which then displays it on the chat screen.

[1673] (Example 2)

[1674] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1675] Traditional chat systems have struggled to accurately understand users' emotions and intentions and generate appropriate responses. As a result, users were not satisfied, and the systems failed to provide a natural conversational experience. Furthermore, generating responses based on the personality of specific characters was difficult, resulting in a lack of technology to achieve realistic dialogue. Solving these challenges is highly desirable.

[1676] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1677] In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, means for receiving and analyzing messages sent from the terminal, means for recognizing the user's emotions from the analyzed messages, means for generating a response based on the analysis results and the recognized emotions, and means for sending the generated response to the terminal. This makes it possible to accurately recognize the user's emotions and provide a natural conversation experience that takes into account the character's personality.

[1678] A "user's terminal" is an electronic device used by a user to access and interact with the system.

[1679] "Login information" refers to information such as usernames and passwords that are necessary for user authentication.

[1680] A "message" is text or written information that a user sends to a system through their device.

[1681] A "natural language processing algorithm" is a computational method and process for analyzing text information and understanding its context and meaning.

[1682] An "emotion engine" is a software component that recognizes a user's emotions from analyzed messages.

[1683] A "response generation engine" is a software component that generates appropriate responses based on analysis results and recognized emotions.

[1684] "Character personality" refers to the characteristics and behavioral traits assigned to a specific fictional person or character.

[1685] This invention relates to a chat system that allows fans to enjoy natural conversations with idol characters. The system consists of a server, a user terminal, and an idol chatbot. The server is responsible for user authentication, message reception, analysis, response generation, and transmission. The user terminal is used by the user to access the system and interact with the chatbot. The chatbot operates on the server and generates responses based on the idol character. Furthermore, the system includes an emotion engine that recognizes the user's emotions.

[1686] Hardware and software to be used

[1687] The system's hardware includes smartphones and personal computers as user terminals, and high-performance computing devices as servers. The software uses chat applications, natural language processing algorithms (e.g., spaCy, NLTK), sentiment engines (e.g., IBM Watson NLU, Google Cloud Natural Language API), and response generation engines (e.g., OpenAI's GPT-4).

[1688] Program processing

[1689] The following describes the specific processing details of this system's program.

[1690] The user accesses the system from their device, launches the chat application, and is directed to the login screen. The user enters their username and password and sends them to the server. The server authenticates the user, and if successful, the user proceeds to the chat screen. Next, the user types and sends a message. The server receives the message and analyzes it using a natural language processing algorithm. Subsequently, the emotion engine recognizes the user's emotions from the analysis results, and the response generation engine generates a response based on the recognized emotions. Finally, the server sends the generated response to the user's device and displays it on the chat screen.

[1691] Specific example

[1692] For example, if a user says to the chatbot, "I was very busy at work today," the process will proceed as follows:

[1693] 1. The user types the message "Today was a very busy day at work" from their device and presses the send button.

[1694] 2. The server receives this message and parses its content using a natural language processing algorithm.

[1695] 3. The emotion engine within the server recognizes "busyness" and "fatigue."

[1696] 4. Based on the analysis results and recognized emotions, the server uses a response generation engine to generate a response such as, "You seem to have been very busy. You must be exhausted! Try to do something to relax."

[1697] 5. The server sends the generated response to the user's device and displays it on the chat screen.

[1698] In this way, users can not only enjoy natural conversations with idol characters, but also receive responses that resonate with their emotions.

[1699] Example of a prompt

[1700] Examples of prompts for a generative AI model are as follows:

[1701] Please create a system where, when a user says "Today was a very busy day at work," the emotion engine recognizes "busyness" and "fatigue," and the character then generates a response such as "Great job! Get some rest."

[1702] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1703] Step 1:

[1704] The user accesses it from their device.

[1705] Enter: Username and password.

[1706] The server displays a login screen and provides input fields for username and password.

[1707] The user enters their username and password and clicks the login button.

[1708] Output: Login information is sent from the terminal to the server.

[1709] Step 2:

[1710] The server authenticates the login information.

[1711] Input: Username and password sent from the device.

[1712] The server compares the user information stored in the database with the entered information.

[1713] If authentication is successful, the server sends an instruction to redirect the user to the chat screen.

[1714] Output: Authentication result and instructions to transition to the chat screen.

[1715] Step 3:

[1716] The user enters a message.

[1717] Input: User's message (e.g., "Today was a very busy day at work").

[1718] The user enters text into the message input field on the chat screen and clicks the send button.

[1719] Output: A message is sent from the terminal to the server.

[1720] Step 4:

[1721] The server receives and parses the message.

[1722] Input: Message sent by the user.

[1723] The server receives the message and parses it using a natural language processing algorithm (e.g., spaCy, NLTK).

[1724] As part of data processing, the text is split and keywords and context are extracted.

[1725] Output: Analysis results (prediction of user intent and emotions).

[1726] Step 5:

[1727] The server recognizes emotions.

[1728] Input: Analysis results.

[1729] The system uses an emotion engine on the server (e.g., IBM Watson NLU, Google Cloud Natural Language API) to recognize the user's emotions from the analysis results.

[1730] As a data processing technique, emotions are identified based on extracted keywords and context.

[1731] Output: Recognized emotions (e.g., "busyness," "fatigue").

[1732] Step 6:

[1733] The server generates a response.

[1734] Input: Recognized emotions and analysis results.

[1735] The server uses a response generation engine (e.g., OpenAI's GPT-4) to generate an appropriate response.

[1736] As part of data processing, text is generated that matches the user's emotions and the character's personality.

[1737] For example, a response like, "You seem to have been very busy. You must be exhausted! Try to do something to relax," might be generated.

[1738] Output: The generated response.

[1739] Step 7:

[1740] The server sends a response to the user's terminal.

[1741] Input: Generated response.

[1742] The server sends the generated response to the user's terminal.

[1743] The device receives the response and displays it on the chat screen.

[1744] Output: The response displayed on the user's chat screen. The message displayed is: "You seem to have been very busy. You must be exhausted! Try to do something to relax."

[1745] As described above, appropriate data processing and calculations are performed at each step based on the input data, and finally, a response that resonates with the user's emotions is generated and displayed.

[1746] (Application Example 2)

[1747] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1748] In modern content delivery services, users demand personalized responses that address their individual emotions. However, traditional systems struggle to understand user emotions and generate appropriate responses. Furthermore, there is a lack of methods to provide interactive experiences that allow fans to feel satisfied through natural conversations with idol characters.

[1749] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving access from a user's terminal, means for authenticating the user's login information, means for receiving and analyzing messages sent from the terminal, means for generating a response based on the analysis results, means for transmitting the generated response to the terminal, means for recognizing the user's emotions, means for adjusting the generated response based on the user's emotions, means for providing individual responses to the user in conjunction with video content, and means for including messages to alleviate the user's emotions. As a result, the user can enjoy an interactive experience with an idol character through natural dialogue that responds to their own emotions.

[1750] "User's device" refers to the device that a user uses to access the internet and utilize various services.

[1751] "Means of accepting access" refers to a function that receives connection requests from users' terminals and provides an interface for accessing the system.

[1752] "Means of authenticating login information" refers to a function that verifies authentication information such as the username and password entered by the user to confirm whether they are a legitimate user.

[1753] "Means for receiving and analyzing messages" refers to a function in which the system receives messages sent by users and analyzes their content.

[1754] "Means for generating a response based on analysis results" refers to a function that creates an appropriate response based on the analysis results of the message.

[1755] "Means for transmitting the generated response to the terminal" refers to a function that transmits the system-generated response to the user's terminal for display.

[1756] "Means of recognizing user emotions" refers to a function that analyzes and recognizes user emotions from analyzed messages.

[1757] "Means for adjusting generated responses based on the user's emotions" refers to a function that adjusts the generated responses to match the user's emotions and provides them in a more appropriate form.

[1758] "A means of providing individualized responses to users in conjunction with video content" refers to a function in which the system references video content and provides individualized and appropriate responses to users.

[1759] "Means of including messages to alleviate emotions" refers to features that include encouraging or comforting messages to reduce the emotions of the recognized user.

[1760] This invention relates to a chat system that allows users to enjoy natural conversations with idol characters. In particular, this system incorporates an emotion engine that recognizes the user's emotions and generates and adjusts responses accordingly. Detailed embodiments for carrying out this invention are described below.

[1761] This system includes the following components:

[1762] 1. Server:

[1763] Access Acceptance: Accepts access from the user's device. Specifically, it provides an interface for receiving HTTP requests.

[1764] Login Authentication: Authenticates the user's login information (username, password). Authentication is performed by comparing it with the database.

[1765] Message analysis: Receives messages sent by users and analyzes them using natural language processing algorithms.

[1766] Emotion Recognition: Recognizes the user's emotions from the analyzed message. Use an emotion engine (e.g., Transformers' emotion analysis model).

[1767] Response Generation: Generates appropriate responses based on analysis results and emotion recognition. Adjusts responses based on the personality and emotional state of the idol character.

[1768] Send response: Send the generated response to the user's device.

[1769] 2. User's device:

[1770] Access: Access the system through the chat app and enter your username and password on the login screen.

[1771] Chat interface: Provides a user interface for entering and sending messages.

[1772] Response Display: Displays the response received from the server.

[1773] 3. Emotional Engine:

[1774] Natural language processing algorithms: For example, use the Transformers library from Hugging Face.

[1775] Emotion analysis models: In particular, generative AI models such as BERT and RoBERTa are used to recognize emotions.

[1776] Specific example

[1777] For example, if a user sends a message saying "I'm a little tired today," the server will perform the following actions:

[1778] 1. The server receives the user's message and analyzes it using a natural language processing algorithm.

[1779] 2. The emotional engine recognizes "fatigue."

[1780] 3. The response generation engine generates a response based on the "feeling of fatigue," such as, "That must have been tough. Take a break and relax."

[1781] 4. The server sends this response to the user's device and displays it in the chat interface.

[1782] Examples of prompt messages include the following:

[1783] "Analyze the following user message, recognize and analyze the user's emotions, and generate a personalized response based on that: 'I'm a little tired today.'"

[1784] This invention allows users to enjoy an interactive experience with idol characters through natural dialogue that responds to their own emotions.

[1785] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1786] Step 1:

[1787] The server accepts access from the user's device. Specifically, it receives HTTP requests and grants access. At this stage, the input is a connection request from the user's device, and the output is a response granting access permission.

[1788] Step 2:

[1789] The server performs login authentication. The user enters their username and password and submits them from their terminal. The server compares this authentication information with the database to verify that the user is legitimate. At this stage, the input is the user's login information, and the output is the result of successful or unsuccessful authentication.

[1790] Step 3:

[1791] The user types a message into the chat interface and sends it. The input content is the message entered by the user. The message is sent from the terminal to the server.

[1792] Step 4:

[1793] The server receives messages sent from the user's terminal and analyzes them using natural language processing algorithms. The input here is the user's message, and the analyzed information (such as intent and emotion) is output.

[1794] Step 5:

[1795] The emotion engine within the server recognizes the user's emotions from the analyzed messages. The input is the analyzed information, and the output is the recognized emotion (e.g., "fatigue").

[1796] Step 6:

[1797] The server's response generation engine generates an appropriate response based on the analysis results and recognized emotions. The personality of the idol character is also taken into consideration during response generation. At this stage, the input is the analysis results and recognized emotions, and the output is the generated response.

[1798] Step 7:

[1799] The server sends the generated response to the user's terminal. The input is the generated response, and the output is the transmission of the response to the terminal.

[1800] Step 8:

[1801] The terminal displays the received response in the chat interface. The input is the response received from the server, and the output is the displayed message.

[1802] In this way, users can enjoy natural conversations with idol characters.

[1803] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1804] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1805] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1806] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1807] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1808] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1809] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1810] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1811] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1812] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1813] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1814] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1815] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1816] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1817] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1818] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1819] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1820] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1821] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1822] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1823] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1824] The following is further disclosed regarding the embodiments described above.

[1825] (Claim 1)

[1826] A means of accepting access from the user's device,

[1827] A means for authenticating the user's login information,

[1828] means for receiving and analyzing messages sent from the aforementioned terminal,

[1829] Means for generating a response based on the analysis results,

[1830] Means for transmitting the generated response to the terminal,

[1831] A system that includes this.

[1832] (Claim 2)

[1833] The system according to claim 1, wherein the analysis means is a means for analyzing a message using a natural language processing algorithm.

[1834] (Claim 3)

[1835] The system according to claim 1, wherein the response generation means is a means for selecting and generating a response based on the personality of an idol character.

[1836] "Example 1"

[1837] (Claim 1)

[1838] A means of accepting access from the user's device,

[1839] A means for authenticating the user's login information,

[1840] means for receiving and analyzing messages sent from the aforementioned terminal,

[1841] Means for generating a response based on the analysis results,

[1842] Means for transmitting the generated response to the terminal,

[1843] Means for adjusting the aforementioned response based on the personality of the idol character,

[1844] A system that includes this.

[1845] (Claim 2)

[1846] The system according to claim 1, wherein the analysis means is a means for analyzing a message using a natural language processing algorithm.

[1847] (Claim 3)

[1848] The system according to claim 1, wherein the response generation means is a means for generating a response using a generation AI model and adjusting the response based on the prompt sentence.

[1849] "Application Example 1"

[1850] (Claim 1)

[1851] A means of accepting access from the user's device,

[1852] A means for authenticating the user's login information,

[1853] A means for receiving a message sent from the aforementioned terminal as voice input and converting it into text using a speech recognition algorithm,

[1854] A means of analyzing the converted text,

[1855] Means for generating a response based on the analysis results,

[1856] Means for converting the generated response into speech and transmitting it to the terminal,

[1857] A system that includes this.

[1858] (Claim 2)

[1859] The system according to claim 1, wherein the analysis means is a means for analyzing a message using a natural language processing algorithm.

[1860] (Claim 3)

[1861] The system according to claim 1, wherein the response generation means is a means for selecting and generating a response based on the personality of an entertainment character.

[1862] "Example 2 of combining an emotion engine"

[1863] (Claim 1)

[1864] A means of accepting access from the user's device,

[1865] A means for authenticating the user's login information,

[1866] means for receiving and analyzing messages sent from the aforementioned terminal,

[1867] A means for recognizing the user's emotions from the analyzed message,

[1868] Means for generating a response based on analysis results and recognized emotions,

[1869] Means for transmitting the generated response to the terminal,

[1870] A system that includes this.

[1871] (Claim 2)

[1872] The system according to claim 1, wherein the analysis means is a means for analyzing a message using a natural language processing algorithm.

[1873] (Claim 3)

[1874] The system according to claim 1, wherein the response generation means is a means for selecting and generating a response based on the character's personality.

[1875] "Application example 2 when combining with an emotional engine"

[1876] (Claim 1)

[1877] A means of accepting access from the user's device,

[1878] A means for authenticating the user's login information,

[1879] means for receiving and analyzing messages sent from the aforementioned terminal,

[1880] Means for generating a response based on the analysis results,

[1881] Means for transmitting the generated response to the terminal,

[1882] Means for recognizing the user's emotions,

[1883] A means of adjusting the generated response based on the user's emotions,

[1884] A system that includes this.

[1885] (Claim 2)

[1886] The system according to claim 1, wherein the analysis means is a means for analyzing a message using a natural language processing algorithm.

[1887] (Claim 3)

[1888] The system according to claim 1, wherein the response generation means is a means for selecting and generating a response based on the personality of an idol character.

[1889] (Claim 4)

[1890] The system according to claim 1, wherein the response generation means is a means that works in conjunction with video content to provide individual responses to users.

[1891] (Claim 5)

[1892] The system according to claim 4, wherein the response generated based on the user's emotions includes a message to soothe the user's emotions. [Explanation of Symbols]

[1893] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of accepting access from the user's device, A means for authenticating the user's login information, means for receiving and analyzing messages sent from the aforementioned terminal, Means for generating a response based on the analysis results, Means for transmitting the generated response to the terminal, A system that includes this.

2. The system according to claim 1, wherein the analysis means is a means for analyzing a message using a natural language processing algorithm.

3. The system according to claim 1, wherein the response generation means is a means for selecting and generating a response based on the personality of an idol character.

Images