system

Abstract

Provide a system. A method for receiving a message from a user includes: A means for sending the received message to a generative AI to generate a response; means for returning the generated response to the user; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional systems for simulating conversations with characters, idols, or actors have limited the quality and naturalness of responses, making it difficult to provide a satisfying dialogue experience for users. Furthermore, the response generation process is complex, making the system difficult to implement and use. New technological solutions are needed to address these issues. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system that includes a means for receiving messages from users, a means for sending the received messages to a generative AI to generate a response, and a means for returning the generated response to the user. Furthermore, the system includes a means for generating prompts containing character information and sending them to the generative AI, and a means for providing a user interface using a web application framework. This allows users to enjoy natural conversations with characters, idols, and actors, making the system easy to use.

[0006] "User" refers to the entity that uses the system to enjoy interacting with characters, idols, and actors.

[0007] "Message" refers to data such as text or voice that a user sends through the system.

[0008] "Generative AI" refers to artificial intelligence technology that generates responses in natural language in response to messages entered by users.

[0009] "Response" refers to output data such as text or audio generated by generative AI based on the user's message.

[0010] "System" refers to a set of hardware and software configurations that link users and generative AI to provide an interactive experience.

[0011] "Means" refers to specific methods, devices, and programs for realizing each function of the system.

[0012] A "prompt" refers to text data containing instructions and information given to a generative AI to generate a response.

[0013] A "web application framework" refers to a development environment and libraries for developing and operating web-based applications.

[0014] "User interface" refers to the screens and operating means through which a user interacts with a system.

[0015] "Character information" refers to information about a specific character, idol, or actor selected by the user. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention relates to a system that allows users to enjoy conversations with characters, idols, and actors. Specific operations and processing flows of the server, terminal, and user in the embodiment of the invention will be described below.

[0038] Server Roles and Program Processing

[0039] First, the server is built using a web application framework such as Flask. The server receives a message and character name from the user, generates a response by sending it as a prompt to a generative AI (e.g., the OpenAI (registered trademark) API), and then returns the generated response to the user.

[0040] Specific working example:

[0041] The server uses a web application framework to define a / chat endpoint to handle HTTP POST requests.

[0042] Upon receiving a request from a user, the server extracts the message and character name from the request body.

[0043] The server generates a prompt of the form "You are [character name], respond to this message: [user message]".

[0044] The server sends this prompt to the generative AI and receives a response.

[0045] The server formats the response and sends it back to the user.

[0046] Terminal roles and program processing

[0047] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[0048] Specific working example:

[0049] The user opens the chat interface on the device and types a message.

[0050] The user selects a particular character from a drop-down menu within the chat interface.

[0051] When the user presses the "Send" button, the terminal sends an HTTP POST request containing the message and the character name to the server.

[0052] The terminal that receives the response from the server displays the response in the chat interface.

[0053] User roles and examples

[0054] Users can easily use this system to enjoy conversations with specific characters, idols, and actors. The operations performed by users are very simple.

[0055] Specific working example:

[0056] Suppose the user inputs the message "How are you feeling today?" and selects "Hatsune Miku" as the character.

[0057] When the user presses the send button, the device sends the message "How are you feeling today?" and the character name "Hatsune Miku" to the server.

[0058] The server sends a prompt to the generative AI, which generates the text in response: "I'm feeling great today! I'm even working on a new song!"

[0059] Responses are displayed on the terminal, allowing the user to experience a natural and enjoyable conversation.

[0060] This allows users to seamlessly enjoy interactions with specific characters, idols, and actors. In addition, server-side processing is performed automatically, providing a stress-free experience for users.

[0061] The processing flow will be explained below.

[0062] Step 1:

[0063] The user opens the chat interface on the terminal and types a message.

[0064] Specific behavior:

[0065] The user types a message into a text box in the chat interface.

[0066] The user selects a particular character from a drop-down menu or the like.

[0067] Step 2:

[0068] The user presses the "Send" button to send the entered message and selected character name.

[0069] Specific behavior:

[0070] The user presses the "send" button.

[0071] The terminal packages the entered message and the selected character name to form an HTTP POST request.

[0072] Step 3:

[0073] The device sends the constructed HTTP POST request to the server.

[0074] Specific behavior:

[0075] The device sends a request to the server's / chat endpoint, including the user's message and character name.

[0076] Step 4:

[0077] The server receives the HTTP POST request and extracts the message and character name from the request body.

[0078] Specific behavior:

[0079] The server receives requests through the Flask application at the / chat endpoint.

[0080] The server parses the request body and extracts the message and character name.

[0081] Step 5:

[0082] The server generates a prompt for the generative AI based on the user's message and the character name.

[0083] Specific behavior:

[0084] The server creates a prompt saying "You are [character name], respond to this message: [user message]".

[0085] Prepare a request to send the generated prompt to the generative AI API.

[0086] Step 6:

[0087] The server sends prompts to the generative AI, causing it to generate a response.

[0088] Specific behavior:

[0089] The server calls a generative AI API such as OpenAI and sends the prepared prompts.

[0090] The server waits for the generative AI to generate a response.

[0091] Step 7:

[0092] The server receives the response from the generative AI and formats it as necessary.

[0093] Specific behavior:

[0094] The server analyzes the response data received from the generative AI.

[0095] Remove unnecessary whitespace and newlines from the response and format the text.

[0096] Step 8:

[0097] The server sends a formatted response back to the terminal.

[0098] Specific behavior:

[0099] The server generates a JSON object containing the formatted response.

[0100] The generated JSON object is sent to the terminal as an HTTP response.

[0101] Step 9:

[0102] The terminal receives the response from the server and displays it to the user.

[0103] Specific behavior:

[0104] The terminal parses the JSON data received from the server.

[0105] The analysis results are displayed in the chat interface so that the user can check them.

[0106] Step 10:

[0107] The user checks the response displayed on the terminal and continues the conversation by entering the next message.

[0108] Specific behavior:

[0109] The user reads the response displayed on the terminal and enters a new message.

[0110] If necessary, select a different character and submit again.

[0111] Example 1

[0112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] Providing a system that allows users to seamlessly and enjoy real-time interactions with unrealistic characters or celebrities is an important challenge in the entertainment industry. There is also a need for an interface that allows users to easily select specific characters and enjoy natural interactions. However, current dialogue systems require complex configurations to achieve the desired dialogue experience, and are not intuitive.

[0114] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0115] In this invention, the server includes a means for receiving a message from a user, a means for generating a prompt including the received message and character information and sending it to the generative AI, and a means for returning the generated response to the user, thereby enabling the user to intuitively enjoy a conversation with a specific character or celebrity.

[0116] A "message" is text data that indicates the content of a conversation entered by a user.

[0117] "Character information" is data about the particular character that the user selects to interact with.

[0118] A "prompt" is input text sent to a generative AI to guide it to a specific output.

[0119] "Generative AI" is a system that uses artificial intelligence technology to generate responses to input text.

[0120] A "server" is a computer system whose role is to receive messages and character information from the user, send prompts to the generative AI, obtain responses, and return them to the user.

[0121] A "user interface" is a part of the software that provides a screen and operating means for the user to input messages and select character information.

[0122] A "web application framework" is a set of software tools for building web-based user interfaces.

[0123] A "response" is a reply text that the generative AI generates based on a prompt.

[0124] An "HTTP POST request" is a protocol-based request method for sending data to a server over the Internet.

[0125] A "terminal" is a computing or mobile device that allows a user to enter and send messages to a server.

[0126] This invention relates to a system that allows users to enjoy conversations with specific characters or celebrities. To implement this invention, the server, terminal, and user each have specific roles and work in cooperation with each other.

[0127] Server configuration and roles

[0128] The server is built using a web application framework such as Flask. The server receives messages and character information sent by the user and generates prompts for the generative AI model based on that information. It then sends the prompts to the generative AI model (e.g., OpenAI's API) and returns the responses it receives to the user.

[0129] Specific working example:

[0130] The server uses Flask to define a / chat endpoint and handles HTTP POST requests.

[0131] Upon receiving a request from a user, the server extracts the message and character information from the request body.

[0132] The server generates a prompt of the form "You are [character name], respond to this message: [user message]".

[0133] The server sends this prompt to the generative AI and receives a response.

[0134] The server formats the response and sends it back to the user.

[0135] Examples of prompts:

[0136] Prompt: "You are Hatsune Miku, respond to this message: How are you feeling today?"

[0137] Example response: "I'm feeling great today! I'm even working on some new music!"

[0138] Device configuration and role

[0139] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[0140] Specific working example:

[0141] The user opens the chat interface on the device and types a message.

[0142] The user selects a particular character from a drop-down menu within the chat interface.

[0143] When the user presses the "Send" button, the terminal sends an HTTP POST request containing the message and the character name to the server.

[0144] The terminal that receives the response from the server displays the response in the chat interface.

[0145] User operations

[0146] Users can easily use this system to enjoy conversations with specific characters or celebrities. The operation is very intuitive and does not require complicated settings.

[0147] Specific working example:

[0148] The user inputs the message "How are you feeling today?" and selects "Hatsune Miku" as the character.

[0149] When the user presses the send button, the device sends the message "How are you feeling today?" and the character name "Hatsune Miku" to the server.

[0150] The server sends a prompt to the generative AI, which generates the text in response: "I'm feeling great today! I'm even working on a new song!"

[0151] The response is displayed on the terminal, allowing the user to enjoy natural dialogue.

[0152] In this way, the system of the present invention allows users to interact with characters and celebrities in real time, creating a seamless experience.

[0153] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0154] Step 1:

[0155] User enters message and selects character

[0156] Users access the chat interface, type the message they want to communicate, and then select the character they want to interact with from a drop-down menu.

[0157] input:

[0158] Message: "How are you feeling today?"

[0159] Character name: "Hatsune Miku"

[0160] output:

[0161] Information that the send button has been pressed is passed to the terminal.

[0162] Specific behavior:

[0163] The user opens the chat interface.

[0164] The user types a message: "How are you feeling today?"

[0165] Select "Hatsune Miku" from the drop-down menu.

[0166] Step 2:

[0167] The device sends a request to the server

[0168] When the user presses the send button, the terminal sends the entered message and the selected character name to the server as an HTTP POST request.

[0169] input:

[0170] The event of the submit button pressed by the user

[0171] Entered message and character name

[0172] output:

[0173] An HTTP POST request is sent to the server.

[0174] Specific behavior:

[0175] The user presses the "Send" button.

[0176] The device sends an HTTP POST request to the server containing the message "How are you feeling today?" and the character name "Hatsune Miku."

[0177] Step 3:

[0178] The server receives the request and generates a prompt.

[0179] The server processes the request received from the device, extracts the message and character name from the request body, and then creates a prompt for the generative AI model.

[0180] input:

[0181] Received HTTP POST request

[0182] output:

[0183] Generated prompt statement

[0184] Specific behavior:

[0185] The server receives an HTTP POST request.

[0186] The server extracts the message "How are you feeling today?" and the character name "Hatsune Miku" from the request body.

[0187] The server generates the prompt "You are Hatsune Miku, respond to this message: How are you feeling today?"

[0188] Step 4:

[0189] The server sends prompts to the generative AI and gets a response

[0190] The server sends a prompt to the generative AI (e.g., OpenAI API) and receives the generated response.

[0191] input:

[0192] Generated prompt statement

[0193] output:

[0194] Response text from generative AI

[0195] Specific behavior:

[0196] The server sends the generative AI a prompt: "You are Hatsune Miku, respond to this message: How are you feeling today?"

[0197] The generative AI generates a response such as "I'm feeling great today! I'm even working on a new song!" and sends it back to the server.

[0198] Step 5:

[0199] The server formats the response it receives and sends it back to the device.

[0200] The server formats the response received from the generative AI, makes it easy for the user to understand, and sends it back to the device.

[0201] input:

[0202] Response text from generative AI

[0203] output:

[0204] Formatted response text

[0205] Specific behavior:

[0206] The server receives the generated response "I'm feeling great today! I'm even working on a new song!"

[0207] The server formats the response and sends it back to the device as an HTTP response.

[0208] Step 6:

[0209] The device receives the response from the server and displays it to the user.

[0210] The terminal receives the response from the server and displays it in the chat interface.

[0211] input:

[0212] HTTP response from the server

[0213] output:

[0214] The displayed response text

[0215] Specific behavior:

[0216] The device receives a response saying, "I'm feeling great today! I'm even working on a new song!"

[0217] The terminal displays the received response on the chat interface and notifies the user.

[0218] The user checks the response and enjoys the interaction.

[0219] (Application example 1)

[0220] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0221] To improve the effectiveness of advertising, interactive communication with users is required. However, with conventional advertising methods, users only passively receive advertising content, and two-way communication is insufficient. It is also difficult for users to obtain information tailored to their own interests in real time. This has resulted in limited advertising effectiveness and low user engagement.

[0222] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0223] In this invention, the server includes a means for receiving a message from a user, a means for sending the received message to a generative AI model to generate a response, a means for returning the generated response to the user, and a means for generating and sending a prompt including advertising character information. This allows users to obtain information about new products in real time through interactions with specific advertising characters or idols. This enables two-way communication with users, improving advertising effectiveness and user engagement.

[0224] "Means for receiving messages from users" refers to the interface for receiving messages entered by users, which is mainly realized through web applications or mobile applications.

[0225] "Means for sending received messages to a generative AI model to generate a response" refers to the process of passing received messages to a generative AI model and generating an appropriate response. Specifically, this is the part that calls the AI's API and receives the generated response.

[0226] "Means for returning the generated response to the user" refers to the process for displaying the response obtained from the generative AI model to the user. This information is typically returned to the user via a web browser or smartphone application.

[0227] "Means for generating and transmitting prompts including advertising character information" refers to a process for generating prompts for a user based on a particular advertising character and transmitting the prompts to a generative AI model, which allows the character to interact with the user.

[0228] "Means for providing new product information" refers to the process of using a generative AI model to provide related new product information based on messages from users. Specifically, it provides users with detailed information and explanations related to new products.

[0229] "Means for utilizing a web application framework to provide an advertising interface" means a process for utilizing a web application framework to provide an interface that allows users to interact with advertisements in an interactive manner, thereby enabling users to easily send messages and receive responses.

[0230] A "generative AI model" is an AI that generates appropriate responses based on prompts from the user. Specifically, it is an AI model that can learn from a large amount of data and generate natural language.

[0231] A "prompt sentence" is a sentence used to generate a response from a generative AI model. It is written in a format that includes specific character information and a message from the user.

[0232] MODE FOR CARRYING OUT THE INVENTION

[0233] This invention is a system that allows users to obtain information about new products in real time through conversations with advertising characters and idols. The specific operations of the server, terminal, and user are described below.

[0234] Server Roles and Program Processing

[0235] The server is built using a web application framework such as Flask. It receives messages and advertising character information from users, generates responses by sending them as prompts to a generative AI model (such as the OpenAI API), and then sends the generated responses back to the user.

[0236] Specific working example:

[0237] The server uses a web application framework to define a / chat endpoint to handle HTTP POST requests.

[0238] Upon receiving a request from a user, the server extracts the message and advertising character information from the request body.

[0239] The server generates a prompt in the form "You are [advertising character name], introduce and explain the new product: [user message]".

[0240] The server sends this prompt to the generative AI model and receives a response.

[0241] The server formats the response and sends it back to the user.

[0242] Terminal roles and program processing

[0243] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[0244] Specific working example:

[0245] The user opens the advertising interface on the terminal and enters a message.

[0246] The user selects a particular advertising character from a drop-down menu within the advertising interface.

[0247] When the user presses the "Send" button, the terminal sends an HTTP POST request including the message and advertising character information to the server.

[0248] The terminal that receives the response from the server displays the response on the advertising interface.

[0249] User roles and examples

[0250] Users can easily use this system to enjoy interacting with specific advertising characters and idols. The operations performed by users are very simple.

[0251] Specific working example:

[0252] Suppose a user inputs the message "How is the battery life of this smartphone?" and selects "Ambassador for home appliances" as the advertising character.

[0253] When the user presses the send button, the device sends the message "How is the battery life of this smartphone?" along with character information about the "home appliance ambassador" to the server.

[0254] The server sends a prompt to the generative AI model, which generates the following text in response: "This smartphone has up to 12 hours of battery life and fast charging."

[0255] The responses are displayed on the terminal, providing the user with detailed and interesting product information in real time.

[0256] Prompt Sentence Examples

[0257] If the user selects an ambassador for a consumer electronics product and asks about smartphone battery life, the prompt might look like this:

[0258] "You are an ambassador for a consumer electronics product. You introduce and explain the new product: How is the battery life of this smartphone?"

[0259] This invention allows users to obtain information about new products while interactively communicating with specific advertising characters or idols, which is expected to improve advertising effectiveness and user engagement.

[0260] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0261] Step 1:

[0262] The user opens the advertising interface on the device and inputs a message. The input message and character information are sent to the server in the form of an HTTP POST request when the user presses the send button.

[0263] Input: User-entered message and selected character information

[0264] Output: Sends message and character information to the server as an HTTP POST request

[0265] Specific operation: The user enters the message "How is the battery life of this smartphone?" and selects "Home appliance ambassador" as the character.

[0266] Step 2:

[0267] The server receives the HTTP POST request and extracts the user's message and character information from the request body.

[0268] Input: HTTP POST request

[0269] Output: Message and character information extracted from the request body

[0270] Specific operation: The server extracts the message "How is the battery life of this smartphone?" and the character information "Ambassador for home appliances" from the received request.

[0271] Step 3:

[0272] The server generates a prompt sentence to send to the generative AI model based on the extracted message and character information.

[0273] Input: Extracted message and character information

[0274] Output: Generated prompt statement

[0275] Specific operation: The server generates the prompt sentence, "You are an ambassador for home appliances, introduce and explain the new product: How is the battery life of this smartphone?"

[0276] Step 4:

[0277] The server sends the generated prompt sentence to the generative AI model, which then generates a response.

[0278] Input: Generated prompt text

[0279] Output: Response from the generative AI model

[0280] Specific operation: The server calls the OpenAI API, sends a prompt, and receives a response.

[0281] Step 5:

[0282] The server formats the response received from the generative AI model and prepares it to be sent back to the user.

[0283] Input: Response from a generative AI model

[0284] Output: The formatted response

[0285] Specific behavior: The server formats the response it receives: "This smartphone has a battery life of up to 12 hours and also has fast charging capabilities."

[0286] Step 6:

[0287] The server returns the formatted response to the user's terminal as an HTTP response.

[0288] Input: Formatted response

[0289] Output: Returned to the user's device as an HTTP response

[0290] Specific operation: The server sends the formatted response to the user's terminal as an HTTP response.

[0291] Step 7:

[0292] The user's device receives the HTTP response from the server and displays the response in the advertising interface.

[0293] Input: HTTP response from the server

[0294] Output: Response displayed in the advertising interface

[0295] Specific behavior: The user's device displays the response "This smartphone has up to 12 hours of battery life and fast charging capabilities" in the advertising interface.

[0296] The operation of the user, terminal, and server at each step is explained in detail, and specific inputs and outputs are shown, making it clear how the invention is implemented.

[0297] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0298] The present invention relates to a system that allows users to enjoy conversations with characters, idols, and actors, and in particular, by combining an emotion engine that recognizes the user's emotions, it is possible to generate more natural responses that correspond to the emotions. The specific operations and processing flows of the server, terminal, and user for the embodiments of this invention are described below.

[0299] Server Roles and Program Processing

[0300] The server is built using a web application framework such as Flask. The role of the server is to receive messages from users, analyze the messages with an emotion engine to generate emotion tags, send prompts to the generative AI to generate responses, and return the generated responses to the user.

[0301] Specific working example:

[0302] The server uses a web application framework to define a / chat endpoint to handle HTTP POST requests.

[0303] Upon receiving a request from a user, the server extracts the message and character name from the request body.

[0304] The server uses an emotion engine to generate emotion tags from the extracted messages.

[0305] The server creates a prompt in the form of "You are [character name], respond to this message: [user's message (with emotion tag)]" and sends it to the generative AI.

[0306] The server receives the response from the generative AI, formats it as necessary, and sends it back to the user.

[0307] Terminal roles and program processing

[0308] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[0309] Specific working example:

[0310] The user opens the chat interface on the terminal and types a message.

[0311] The user selects a particular character from a drop-down menu within the chat interface.

[0312] When the user presses the "Send" button, the terminal sends an HTTP POST request containing the message and the character name to the server.

[0313] The terminal that receives the response from the server displays the response in the chat interface.

[0314] User roles and examples

[0315] Users can easily use this system to enjoy conversations with specific characters, idols, and actors. The operations performed by users are very simple.

[0316] Specific working example:

[0317] Suppose the user inputs the message "How are you feeling today?" and selects "Hatsune Miku" as the character.

[0318] When the user presses the send button, the device sends the message "How are you feeling today?" and the character name "Hatsune Miku" to the server.

[0319] The server receives the message, analyzes the message with an emotion engine, and generates an emotion tag, such as "positive."

[0320] The server sends a prompt to the generative AI, tagging it with the emotion "positive" and asking it to generate a response. The generative AI might generate the response "I'm feeling great today! I'm even working on a new song!"

[0321] Responses are displayed on the terminal, allowing the user to experience a natural and emotionally appropriate conversation.

[0322] This allows users to seamlessly enjoy conversations with specific characters, idols, and actors. In addition, by using an emotion engine, responses are made natural according to the user's emotions, providing a more satisfying conversational experience.

[0323] The processing flow will be explained below.

[0324] Step 1:

[0325] The user opens the chat interface on the terminal and types a message.

[0326] Specific behavior:

[0327] The user types a message into a text box in the chat interface.

[0328] The user selects a particular character from a drop-down menu or the like.

[0329] Step 2:

[0330] The user presses the "Send" button to send the entered message and selected character name.

[0331] Specific behavior:

[0332] The user presses the "send" button.

[0333] The terminal packages the entered message and the selected character name to form an HTTP POST request.

[0334] Step 3:

[0335] The device sends the constructed HTTP POST request to the server.

[0336] Specific behavior:

[0337] The device sends a request to the server's / chat endpoint, including the user's message and character name.

[0338] Step 4:

[0339] The server receives the HTTP POST request and extracts the message and character name from the request body.

[0340] Specific behavior:

[0341] The server receives requests through the Flask application at the / chat endpoint.

[0342] The server parses the request body and extracts the message and character name.

[0343] Step 5:

[0344] The server sends the extracted message to the emotion engine, which generates emotion tags.

[0345] Specific behavior:

[0346] The server requests the emotion engine to analyze the message.

[0347] The emotion engine analyzes the message and generates emotion tags such as "positive" or "negative."

[0348] The server receives emotion tags from the emotion engine.

[0349] Step 6:

[0350] The server generates prompts for the generative AI based on the user's message and emotion tag.

[0351] Specific behavior:

[0352] The server generates a prompt of the form "You are [character name], respond to this message with the emotion [emotion tag]: [user message]".

[0353] Prepare a request to send the generated prompt to the generative AI API.

[0354] Step 7:

[0355] The server sends prompts to the generative AI, causing it to generate a response.

[0356] Specific behavior:

[0357] The server calls a generative AI API such as OpenAI and sends the prepared prompts.

[0358] The server waits for the generative AI to generate a response.

[0359] Step 8:

[0360] The server receives the response from the generative AI and formats it as necessary.

[0361] Specific behavior:

[0362] The server analyzes the response data received from the generative AI.

[0363] Remove unnecessary whitespace and newlines from the response and format the text.

[0364] Step 9:

[0365] The server sends a formatted response back to the terminal.

[0366] Specific behavior:

[0367] The server generates a JSON object containing the formatted response.

[0368] The generated JSON object is sent to the terminal as an HTTP response.

[0369] Step 10:

[0370] The terminal receives the response from the server and displays it to the user.

[0371] Specific behavior:

[0372] The terminal parses the JSON data received from the server.

[0373] The analysis results are displayed in the chat interface so that the user can check them.

[0374] Step 11:

[0375] The user checks the response displayed on the terminal and continues the conversation by entering the next message.

[0376] Specific behavior:

[0377] The user reads the response displayed on the terminal and enters a new message.

[0378] If necessary, select a different character and submit again.

[0379] Example 2

[0380] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0381] Conventional dialogue systems do not properly recognize the user's emotions and simply respond to input messages. This results in unnatural responses and a poor user experience. Furthermore, they lack a mechanism for naturally engaging in dialogue with specific characters or idols, limiting their appeal to users. Furthermore, they lack the technology to generate responses based on emotions, making it impossible to realize conversations that are sensitive to the user's emotions.

[0382] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving a message from a user, a means for analyzing the received message with an emotion engine and generating an emotion tag, a means for sending the emotion-tagged message to a generative AI model to generate a response, and a means for returning the generated response to the user. This makes it possible to recognize emotions in response to messages from users and generate appropriate and natural responses. Furthermore, it is possible to have conversations with specific characters or idols according to emotions, thereby improving the user experience.

[0383] A "user" is a person who utilizes a dialogue system to input messages and receive responses.

[0384] A "message" is text data entered by a user and received by a server.

[0385] "Emotion engine" is a general term for algorithms and programs that analyze received messages and generate emotion tags from their contents.

[0386] "Emotion tags" are labels generated by the emotion engine that indicate the emotional characteristics of a message. Examples include "positive" and "negative."

[0387] A "generative AI model" is an artificial intelligence model that generates natural language responses based on received prompts.

[0388] A "prompt" is input information given to a generative AI model to generate a response. It includes character information and emotion tags.

[0389] The "server" is a computer system that receives messages from users, analyzes them using an emotion engine, sends prompts to a generative AI model, and executes a series of operations to generate a response.

[0390] "Character information" refers to information about a specific character, idol, or actor selected by the user.

[0391] A "web application framework" is a software framework for building applications that run on the web. Examples include Flask.

[0392] A "response" is the text data that a generative AI model generates based on a prompt and sends back to the user.

[0393] The present invention relates to a system that allows users to enjoy conversations with specific characters, idols, or actors. In particular, it utilizes a combination of an emotion engine and a generative AI model to recognize the user's emotions and generate natural responses accordingly. The following describes the embodiments of the invention based on the scope of the claims.

[0394] 1. Specific Embodiment of the Server

[0395] The server is built using Flask as a web application framework. The main roles of the server are as follows:

[0396] First, the server receives a message from the user. Next, it analyzes the received message using the emotion engine and generates an emotion tag. Based on the generated emotion tag, it integrates it with character information to generate a prompt and sends it to the generative AI model. Then it receives a response from the generative AI model and sends it back to the user.

[0397] As a specific use case, if a user types the message "How are you feeling today?" and selects "Hatsune Miku" as the character name, this message is sent to the server. The server uses its emotion engine to analyze the message and generate a "positive" emotion tag. It then sends the following prompt to the generative AI model:

[0398] "You are Hatsune Miku, respond to this message: How are you feeling today? (Positive)"

[0399] Based on this prompt, the generative AI model generates a response such as, "I'm feeling great today! I'm even working on a new song!", which the server sends back to the user.

[0400] 2. Specific Embodiments of the Terminal

[0401] The terminal provides an interface for users to input messages and send them to the server, using a web browser or a mobile application. The terminal's main roles are as follows:

[0402] First, the user enters a message using the chat interface. Then, they select a specific character from a drop-down menu. When the user presses the "Send" button, the device sends an HTTP POST request to the server containing the message and the character's name. The response is then received from the server and displayed in the chat interface.

[0403] 3. User's specific embodiment

[0404] Users can enjoy the interaction by performing extremely simple operations. The specific steps are shown below.

[0405] First, the user types a message in the chat interface, such as "How are you feeling today?", and then selects "Hatsune Miku" from the drop-down menu. When the user presses the "Send" button, the message and the character name are sent to the server. The server then responds with "I'm feeling great today! I'm even working on a new song!", which is displayed in the chat interface, allowing the user to enjoy the conversation.

[0406] As described above, this invention combines an emotion engine with a generative AI model to generate natural responses that correspond to the user's emotions, making it possible to seamlessly enjoy conversations with specific characters, idols, or actors.

[0407] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0408] Step 1: The user types a message into the chat interface

[0409] The user opens the chat interface on the device and types "How are you feeling today?" into the text field. Input: The message "How are you feeling today?" typed by the user. Output: The text that appears in the message input field on the device display.

[0410] Step 2: User selects a character

[0411] The user selects "Hatsune Miku" from a drop-down menu in the chat interface. Input: The character name "Hatsune Miku". Output: The selection displayed in the drop-down menu.

[0412] Step 3: User presses "Submit" button

[0413] The user presses the "Send" button, triggering the sending of the message and character name to the server. Input: Click on the "Send" button. Output: HTTP POST request containing the message and character name.

[0414] Step 4: The device sends an HTTP POST request to the server

[0415] The device sends an HTTP POST request to the server containing the user's message "How are you feeling today?" and the character name "Hatsune Miku". Input: Message and character name. Output: HTTP request sent to the server.

[0416] Step 5: The server receives the message

[0417] The server receives the HTTP POST request and extracts the message "How are you feeling today?" and the character name "Hatsune Miku" from the request body. Input: HTTP POST request body. Output: Extracted message and character name.

[0418] Step 6: The server parses the message using the emotion engine

[0419] The server passes the message "How are you feeling today?" to the emotion engine and generates the emotion tag "Positive". Input: The extracted message. Output: The generated emotion tag.

[0420] Step 7: The server creates a prompt and sends it to the generative AI model

[0421] The server generates a prompt, "You are Hatsune Miku, respond to this message: How are you feeling today? (Positive)," and sends it to the generative AI model. Input: Message, character name, emotion tag. Output: Prompt sent to the generative AI model.

[0422] Step 8: The generative AI model generates a response

[0423] Based on the prompt, the generative AI model generates a response such as "I'm feeling great today! I'm even working on a new song!". Input: Prompt. Output: Generated response.

[0424] Step 9: The server receives and formats the generated response

[0425] The server receives the response from the generative AI model, "I'm feeling great today! I'm even working on a new song!", and formats it as needed. Input: Generated response. Output: Formatted response.

[0426] Step 10: The server sends the formatted response back to the user

[0427] The server sends the formatted response back to the device as an HTTP response. Input: Formatted response. Output: Response sent to the user's device.

[0428] Step 11: The device receives the response from the server and displays it in the chat interface.

[0429] The device receives the response from the server, "I'm feeling great today! I'm even working on a new song!", and displays it in the chat interface. Input: Response from the server. Output: Response displayed in the chat interface.

[0430] Step 12: User checks the response and enjoys the interaction

[0431] The user can see the responses displayed on the chat interface of the device and enjoy natural dialogue with the character. Input: Responses displayed on the device. Output: Improved user dialogue experience.

[0432] (Application example 2)

[0433] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0434] Conventional dialogue systems have difficulty generating responses that accurately reflect the user's emotions, resulting in a limited user experience. Furthermore, when proposing products or recommending services in virtual stores, they often fail to provide appropriate suggestions that take the user's emotions into account. This has led to issues such as insufficient improvement of user satisfaction and insufficient stimulation of purchasing motivation.

[0435] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0436] In this invention, the server includes means for receiving a message from a user, means for analyzing the received message with an emotion engine and generating an emotion tag, means for generating a prompt with an emotion tag and sending it to a generative AI to generate a response, means for returning the generated response to the user, and means for providing a user interface using smart glasses or a head-mounted display. This enables more natural dialogue according to the user's emotions and makes it possible to propose products and services suitable for the user in a virtual store.

[0437] A "user" is a person who uses the system to input messages and receive proposals for services and products.

[0438] A "message" is text or voice data that a user inputs to the system.

[0439] An "emotion engine" is a software application that analyzes a user's message, identifies the emotion contained in the message, and generates an emotion tag.

[0440] An "emotion tag" is a label that represents an emotion and is generated by the emotion engine and assigned to a user's message.

[0441] A "prompt" is an instruction given to a generative AI to generate a response, and may include emotion tags and character information.

[0442] "Generative AI" is artificial intelligence that generates natural language responses in response to given prompts.

[0443] A "response" is text or audio data that a generative AI generates based on a prompt and sends back to the user.

[0444] "Smart glasses" are wearable devices with internet connectivity and other computing capabilities that are used to display information in the user's field of vision.

[0445] A "head-mounted display" is a device worn on the user's head that displays information in the user's field of vision.

[0446] "User interface" is a term that refers to an interface such as a screen, display, or input device that allows a user to interact with a system.

[0447] A "virtual store" is a virtual store built on the Internet where users can browse and purchase products online.

[0448] "Suggestion" refers to the act of selecting products or services suitable for a user in a virtual store and presenting them to the user.

[0449] This invention is a system that realizes customer service in a virtual store, and in particular enables natural dialogue and product suggestions that correspond to the user's emotions. The system is mainly composed of three entities: a server, a terminal, and a user.

[0450] Server Roles and Program Processing

[0451] The server is built using a web application framework such as Flask. The role of the server is to receive messages from users, analyze the messages with an emotion engine to generate emotion tags, create prompts and send them to the generative AI, and then send the generated responses back to the user.

[0452] The server does the following:

[0453] Receives a message from the user as an HTTP POST request.

[0454] The received message is analyzed by an emotion engine and an emotion tag is generated. Emotion engines are often provided as external APIs.

[0455] A prompt containing the generated emotion tag is created and sent to the generative AI in the format "You are a virtual shop assistant, respond to this message: 'Message content' (Emotion: emotion tag)".

[0456] Receives responses from the generative AI and displays them to the user via the user interface.

[0457] Terminal roles and program processing

[0458] The terminal provides a user interface using smart glasses or a head-mounted display (HMD). The terminal's role is to allow the user to input messages and send them to the server, and to visually present the responses sent back from the server to the user.

[0459] The terminal does the following:

[0460] The user types a message through smart glasses or an HMD.

[0461] Provides an interface for sending input messages to the server.

[0462] Responses from the server are displayed in real time on the user interface.

[0463] User roles and examples

[0464] Users can easily use the system to enjoy conversations in a virtual store. The user's operations are very simple: they simply input messages through smart glasses or an HMD and receive responses from the server.

[0465] Specific working example:

[0466] Let's say a user types a message like "I've been feeling tired lately." The user sends this message to the system using smart glasses or an HMD. The server receives this message, analyzes it with its emotion engine, and generates an emotion tag called "tired." It then sends the following prompt to the generative AI:

[0467] You are a virtual shop assistant, respond to this message: "I've been feeling tired lately." (Emotion: tiredness)

[0468] The generative AI generates a response such as "Why not try a relaxing massage chair?", which is then displayed on the smart glasses or HMD.

[0469] This allows users to receive appropriate product and service recommendations in the virtual store through natural dialogue based on their emotions, improving the user experience and significantly increasing satisfaction in the virtual store.

[0470] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0471] Step 1:

[0472] The terminal receives messages from the user as input. The user uses smart glasses or a head-mounted display (HMD) to input text or voice messages in natural language. This message contains information including the user's wishes and emotions expressed to the system. The terminal stores the message as is and prepares it for transmission to the server.

[0473] Step 2:

[0474] The device sends the message entered by the user to the server as an HTTP POST request, along with any additional information related to the message (e.g., character selection). The entered data is passed to the server through the interface of the smart glasses or HMD.

[0475] Step 3:

[0476] The server receives messages sent from the devices as input. To analyze these messages, the server first sends them to the emotion engine. The emotion engine analyzes the messages and generates emotion tags based on their content. Emotion tags express the emotions contained in the messages and are generated using natural language processing techniques on the message text. The resulting emotion tags are then returned to the server.

[0477] Step 4:

[0478] The server generates a prompt based on the received emotion tag and the user message. The generated prompt is in the format "You are a virtual shop assistant, respond to this message: 'Message content' (Emotion: emotion tag)". This prompt is used as an instruction for the generative AI, and by incorporating the emotion tag, it provides input data for generating a more appropriate response.

[0479] Step 5:

[0480] The server sends the prompt to a generative AI, which takes the prompt as input and generates an appropriate natural language response based on its content. The generative AI then analyzes the prompt using large-scale language models and machine learning algorithms, and outputs an appropriate response to the user's message. This output response is then sent back to the server.

[0481] Step 6:

[0482] The server formats the response received from the generative AI, returns it to the device in its final form, and in some cases deletes unnecessary information during the formatting process. The response is then sent back to the device as an HTTP POST request.

[0483] Step 7:

[0484] The device displays the response received from the server. The received response message is visually displayed on the display of the smart glasses or head-mounted display. This allows the user to receive product and service suggestions based on their emotions through natural dialogue in real time.

[0485] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0486] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0487] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0488] [Second embodiment]

[0489] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0490] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0491] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0492] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0493] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0494] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0495] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0496] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0497] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0498] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0499] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0500] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0501] This invention relates to a system that allows users to enjoy conversations with characters, idols, and actors. Specific operations and processing flows of the server, terminal, and user in the embodiment of the invention will be described below.

[0502] Server Roles and Program Processing

[0503] First, the server is built using a web application framework such as Flask. The server receives a message and character name from the user, generates a response by sending it as a prompt to the generative AI (e.g., OpenAI's API), and then sends the generated response back to the user.

[0504] Specific working example:

[0505] The server uses a web application framework to define a / chat endpoint to handle HTTP POST requests.

[0506] Upon receiving a request from a user, the server extracts the message and character name from the request body.

[0507] The server generates a prompt of the form "You are [character name], respond to this message: [user message]".

[0508] The server sends this prompt to the generative AI and receives a response.

[0509] The server formats the response and sends it back to the user.

[0510] Terminal roles and program processing

[0511] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[0512] Specific working example:

[0513] The user opens the chat interface on the device and types a message.

[0514] The user selects a particular character from a drop-down menu within the chat interface.

[0515] When the user presses the "Send" button, the terminal sends an HTTP POST request containing the message and the character name to the server.

[0516] The terminal that receives the response from the server displays the response in the chat interface.

[0517] User roles and examples

[0518] Users can easily use this system to enjoy conversations with specific characters, idols, and actors. The operations performed by users are very simple.

[0519] Specific working example:

[0520] Suppose the user inputs the message "How are you feeling today?" and selects "Hatsune Miku" as the character.

[0521] When the user presses the send button, the device sends the message "How are you feeling today?" and the character name "Hatsune Miku" to the server.

[0522] The server sends a prompt to the generative AI, which generates the text in response: "I'm feeling great today! I'm even working on a new song!"

[0523] Responses are displayed on the terminal, allowing the user to experience a natural and enjoyable conversation.

[0524] This allows users to seamlessly enjoy interactions with specific characters, idols, and actors. In addition, server-side processing is performed automatically, providing a stress-free experience for users.

[0525] The processing flow will be explained below.

[0526] Step 1:

[0527] The user opens the chat interface on the terminal and types a message.

[0528] Specific behavior:

[0529] The user types a message into a text box in the chat interface.

[0530] The user selects a particular character from a drop-down menu or the like.

[0531] Step 2:

[0532] The user presses the "Send" button to send the entered message and selected character name.

[0533] Specific behavior:

[0534] The user presses the "send" button.

[0535] The terminal packages the entered message and the selected character name to form an HTTP POST request.

[0536] Step 3:

[0537] The device sends the constructed HTTP POST request to the server.

[0538] Specific behavior:

[0539] The device sends a request to the server's / chat endpoint, including the user's message and character name.

[0540] Step 4:

[0541] The server receives the HTTP POST request and extracts the message and character name from the request body.

[0542] Specific behavior:

[0543] The server receives requests through the Flask application at the / chat endpoint.

[0544] The server parses the request body and extracts the message and character name.

[0545] Step 5:

[0546] The server generates a prompt for the generative AI based on the user's message and the character name.

[0547] Specific behavior:

[0548] The server creates a prompt saying "You are [character name], respond to this message: [user message]".

[0549] Prepare a request to send the generated prompt to the generative AI API.

[0550] Step 6:

[0551] The server sends prompts to the generative AI, causing it to generate a response.

[0552] Specific behavior:

[0553] The server calls a generative AI API such as OpenAI and sends the prepared prompts.

[0554] The server waits for the generative AI to generate a response.

[0555] Step 7:

[0556] The server receives the response from the generative AI and formats it as necessary.

[0557] Specific behavior:

[0558] The server analyzes the response data received from the generative AI.

[0559] Remove unnecessary whitespace and newlines from the response and format the text.

[0560] Step 8:

[0561] The server sends a formatted response back to the terminal.

[0562] Specific behavior:

[0563] The server generates a JSON object containing the formatted response.

[0564] The generated JSON object is sent to the terminal as an HTTP response.

[0565] Step 9:

[0566] The terminal receives the response from the server and displays it to the user.

[0567] Specific behavior:

[0568] The terminal parses the JSON data received from the server.

[0569] The analysis results are displayed in the chat interface so that the user can check them.

[0570] Step 10:

[0571] The user checks the response displayed on the terminal and continues the conversation by entering the next message.

[0572] Specific behavior:

[0573] The user reads the response displayed on the terminal and enters a new message.

[0574] If necessary, select a different character and submit again.

[0575] Example 1

[0576] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0577] Providing a system that allows users to seamlessly and enjoy real-time interactions with unrealistic characters or celebrities is an important challenge in the entertainment industry. There is also a need for an interface that allows users to easily select specific characters and enjoy natural interactions. However, current dialogue systems require complex configurations to achieve the desired dialogue experience, and are not intuitive.

[0578] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0579] In this invention, the server includes a means for receiving a message from a user, a means for generating a prompt including the received message and character information and sending it to the generative AI, and a means for returning the generated response to the user, thereby enabling the user to intuitively enjoy a conversation with a specific character or celebrity.

[0580] A "message" is text data that indicates the content of a conversation entered by a user.

[0581] "Character information" is data about the particular character that the user selects to interact with.

[0582] A "prompt" is input text sent to a generative AI to guide it to a specific output.

[0583] "Generative AI" is a system that uses artificial intelligence technology to generate responses to input text.

[0584] A "server" is a computer system whose role is to receive messages and character information from the user, send prompts to the generative AI, obtain responses, and return them to the user.

[0585] A "user interface" is a part of the software that provides a screen and operating means for the user to input messages and select character information.

[0586] A "web application framework" is a set of software tools for building web-based user interfaces.

[0587] A "response" is a reply text that the generative AI generates based on a prompt.

[0588] An "HTTP POST request" is a protocol-based request method for sending data to a server over the Internet.

[0589] A "terminal" is a computing or mobile device that allows a user to enter and send messages to a server.

[0590] This invention relates to a system that allows users to enjoy conversations with specific characters or celebrities. To implement this invention, the server, terminal, and user each have specific roles and work in cooperation with each other.

[0591] Server configuration and roles

[0592] The server is built using a web application framework such as Flask. The server receives messages and character information sent by the user and generates prompts for the generative AI model based on that information. It then sends the prompts to the generative AI model (e.g., OpenAI's API) and returns the responses it receives to the user.

[0593] Specific working example:

[0594] The server uses Flask to define a / chat endpoint and handles HTTP POST requests.

[0595] Upon receiving a request from a user, the server extracts the message and character information from the request body.

[0596] The server generates a prompt of the form "You are [character name], respond to this message: [user message]".

[0597] The server sends this prompt to the generative AI and receives a response.

[0598] The server formats the response and sends it back to the user.

[0599] Examples of prompts:

[0600] Prompt: "You are Hatsune Miku, respond to this message: How are you feeling today?"

[0601] Example response: "I'm feeling great today! I'm even working on some new music!"

[0602] Device configuration and role

[0603] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[0604] Specific working example:

[0605] The user opens the chat interface on the device and types a message.

[0606] The user selects a particular character from a drop-down menu within the chat interface.

[0607] When the user presses the "Send" button, the terminal sends an HTTP POST request containing the message and the character name to the server.

[0608] The terminal that receives the response from the server displays the response in the chat interface.

[0609] User operations

[0610] Users can easily use this system to enjoy conversations with specific characters or celebrities. The operation is very intuitive and does not require complicated settings.

[0611] Specific working example:

[0612] The user inputs the message "How are you feeling today?" and selects "Hatsune Miku" as the character.

[0613] When the user presses the send button, the device sends the message "How are you feeling today?" and the character name "Hatsune Miku" to the server.

[0614] The server sends a prompt to the generative AI, which generates the text in response: "I'm feeling great today! I'm even working on a new song!"

[0615] The response is displayed on the terminal, allowing the user to enjoy natural dialogue.

[0616] In this way, the system of the present invention allows users to interact with characters and celebrities in real time, creating a seamless experience.

[0617] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0618] Step 1:

[0619] User enters message and selects character

[0620] Users access the chat interface, type the message they want to communicate, and then select the character they want to interact with from a drop-down menu.

[0621] input:

[0622] Message: "How are you feeling today?"

[0623] Character name: "Hatsune Miku"

[0624] output:

[0625] Information that the send button has been pressed is passed to the terminal.

[0626] Specific behavior:

[0627] The user opens the chat interface.

[0628] The user types a message: "How are you feeling today?"

[0629] Select "Hatsune Miku" from the drop-down menu.

[0630] Step 2:

[0631] The device sends a request to the server

[0632] When the user presses the send button, the terminal sends the entered message and the selected character name to the server as an HTTP POST request.

[0633] input:

[0634] The event of the submit button pressed by the user

[0635] Entered message and character name

[0636] output:

[0637] An HTTP POST request is sent to the server.

[0638] Specific behavior:

[0639] The user presses the "Send" button.

[0640] The device sends an HTTP POST request to the server containing the message "How are you feeling today?" and the character name "Hatsune Miku."

[0641] Step 3:

[0642] The server receives the request and generates a prompt.

[0643] The server processes the request received from the device, extracts the message and character name from the request body, and then creates a prompt for the generative AI model.

[0644] input:

[0645] Received HTTP POST request

[0646] output:

[0647] Generated prompt statement

[0648] Specific behavior:

[0649] The server receives an HTTP POST request.

[0650] The server extracts the message "How are you feeling today?" and the character name "Hatsune Miku" from the request body.

[0651] The server generates the prompt "You are Hatsune Miku, respond to this message: How are you feeling today?"

[0652] Step 4:

[0653] The server sends prompts to the generative AI and gets a response

[0654] The server sends a prompt to the generative AI (e.g., OpenAI API) and receives the generated response.

[0655] input:

[0656] Generated prompt statement

[0657] output:

[0658] Response text from generative AI

[0659] Specific behavior:

[0660] The server sends the generative AI a prompt: "You are Hatsune Miku, respond to this message: How are you feeling today?"

[0661] The generative AI generates a response such as "I'm feeling great today! I'm even working on a new song!" and sends it back to the server.

[0662] Step 5:

[0663] The server formats the response it receives and sends it back to the device.

[0664] The server formats the response received from the generative AI, makes it easy for the user to understand, and sends it back to the device.

[0665] input:

[0666] Response text from generative AI

[0667] output:

[0668] Formatted response text

[0669] Specific behavior:

[0670] The server receives the generated response "I'm feeling great today! I'm even working on a new song!"

[0671] The server formats the response and sends it back to the device as an HTTP response.

[0672] Step 6:

[0673] The device receives the response from the server and displays it to the user.

[0674] The terminal receives the response from the server and displays it in the chat interface.

[0675] input:

[0676] HTTP response from the server

[0677] output:

[0678] The displayed response text

[0679] Specific behavior:

[0680] The device receives a response saying, "I'm feeling great today! I'm even working on a new song!"

[0681] The terminal displays the received response on the chat interface and notifies the user.

[0682] The user checks the response and enjoys the interaction.

[0683] (Application example 1)

[0684] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0685] To improve the effectiveness of advertising, interactive communication with users is required. However, with conventional advertising methods, users only passively receive advertising content, and two-way communication is insufficient. It is also difficult for users to obtain information tailored to their own interests in real time. This has resulted in limited advertising effectiveness and low user engagement.

[0686] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0687] In this invention, the server includes a means for receiving a message from a user, a means for sending the received message to a generative AI model to generate a response, a means for returning the generated response to the user, and a means for generating and sending a prompt including advertising character information. This allows users to obtain information about new products in real time through interactions with specific advertising characters or idols. This enables two-way communication with users, improving advertising effectiveness and user engagement.

[0688] "Means for receiving messages from users" refers to the interface for receiving messages entered by users, which is mainly realized through web applications or mobile applications.

[0689] "Means for sending received messages to a generative AI model to generate a response" refers to the process of passing received messages to a generative AI model and generating an appropriate response. Specifically, this is the part that calls the AI's API and receives the generated response.

[0690] "Means for returning the generated response to the user" refers to the process for displaying the response obtained from the generative AI model to the user. This information is typically returned to the user via a web browser or smartphone application.

[0691] "Means for generating and transmitting prompts including advertising character information" refers to a process for generating prompts for a user based on a particular advertising character and transmitting the prompts to a generative AI model, which allows the character to interact with the user.

[0692] "Means for providing new product information" refers to the process of using a generative AI model to provide related new product information based on messages from users. Specifically, it provides users with detailed information and explanations related to new products.

[0693] "Means for utilizing a web application framework to provide an advertising interface" means a process for utilizing a web application framework to provide an interface that allows users to interact with advertisements in an interactive manner, thereby enabling users to easily send messages and receive responses.

[0694] A "generative AI model" is an AI that generates appropriate responses based on prompts from the user. Specifically, it is an AI model that can learn from a large amount of data and generate natural language.

[0695] A "prompt sentence" is a sentence used to generate a response from a generative AI model. It is written in a format that includes specific character information and a message from the user.

[0696] MODE FOR CARRYING OUT THE INVENTION

[0697] This invention is a system that allows users to obtain information about new products in real time through conversations with advertising characters and idols. The specific operations of the server, terminal, and user are described below.

[0698] Server Roles and Program Processing

[0699] The server is built using a web application framework such as Flask. It receives messages and advertising character information from users, generates responses by sending them as prompts to a generative AI model (such as the OpenAI API), and then sends the generated responses back to the user.

[0700] Specific working example:

[0701] The server uses a web application framework to define a / chat endpoint to handle HTTP POST requests.

[0702] Upon receiving a request from a user, the server extracts the message and advertising character information from the request body.

[0703] The server generates a prompt in the form "You are [advertising character name], introduce and explain the new product: [user message]".

[0704] The server sends this prompt to the generative AI model and receives a response.

[0705] The server formats the response and sends it back to the user.

[0706] Terminal roles and program processing

[0707] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[0708] Specific working example:

[0709] The user opens the advertising interface on the terminal and enters a message.

[0710] The user selects a particular advertising character from a drop-down menu within the advertising interface.

[0711] When the user presses the "Send" button, the terminal sends an HTTP POST request including the message and advertising character information to the server.

[0712] The terminal that receives the response from the server displays the response on the advertising interface.

[0713] User roles and examples

[0714] Users can easily use this system to enjoy interacting with specific advertising characters and idols. The operations performed by users are very simple.

[0715] Specific working example:

[0716] Suppose a user inputs the message "How is the battery life of this smartphone?" and selects "Ambassador for home appliances" as the advertising character.

[0717] When the user presses the send button, the device sends the message "How is the battery life of this smartphone?" along with character information about the "home appliance ambassador" to the server.

[0718] The server sends a prompt to the generative AI model, which generates the following text in response: "This smartphone has up to 12 hours of battery life and fast charging."

[0719] The responses are displayed on the terminal, providing the user with detailed and interesting product information in real time.

[0720] Prompt Sentence Examples

[0721] If the user selects an ambassador for a consumer electronics product and asks about smartphone battery life, the prompt might look like this:

[0722] "You are an ambassador for a consumer electronics product. You introduce and explain the new product: How is the battery life of this smartphone?"

[0723] This invention allows users to obtain information about new products while interactively communicating with specific advertising characters or idols, which is expected to improve advertising effectiveness and user engagement.

[0724] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0725] Step 1:

[0726] The user opens the advertising interface on the device and inputs a message. The input message and character information are sent to the server in the form of an HTTP POST request when the user presses the send button.

[0727] Input: User-entered message and selected character information

[0728] Output: Sends message and character information to the server as an HTTP POST request

[0729] Specific operation: The user enters the message "How is the battery life of this smartphone?" and selects "Home appliance ambassador" as the character.

[0730] Step 2:

[0731] The server receives the HTTP POST request and extracts the user's message and character information from the request body.

[0732] Input: HTTP POST request

[0733] Output: Message and character information extracted from the request body

[0734] Specific operation: The server extracts the message "How is the battery life of this smartphone?" and the character information "Ambassador for home appliances" from the received request.

[0735] Step 3:

[0736] The server generates a prompt sentence to send to the generative AI model based on the extracted message and character information.

[0737] Input: Extracted message and character information

[0738] Output: Generated prompt statement

[0739] Specific operation: The server generates the prompt sentence, "You are an ambassador for home appliances, introduce and explain the new product: How is the battery life of this smartphone?"

[0740] Step 4:

[0741] The server sends the generated prompt sentence to the generative AI model, which then generates a response.

[0742] Input: Generated prompt text

[0743] Output: Response from the generative AI model

[0744] Specific operation: The server calls the OpenAI API, sends a prompt, and receives a response.

[0745] Step 5:

[0746] The server formats the response received from the generative AI model and prepares it to be sent back to the user.

[0747] Input: Response from a generative AI model

[0748] Output: The formatted response

[0749] Specific behavior: The server formats the response it receives: "This smartphone has a battery life of up to 12 hours and also has fast charging capabilities."

[0750] Step 6:

[0751] The server returns the formatted response to the user's terminal as an HTTP response.

[0752] Input: Formatted response

[0753] Output: Returned to the user's device as an HTTP response

[0754] Specific operation: The server sends the formatted response to the user's terminal as an HTTP response.

[0755] Step 7:

[0756] The user's device receives the HTTP response from the server and displays the response in the advertising interface.

[0757] Input: HTTP response from the server

[0758] Output: Response displayed in the advertising interface

[0759] Specific behavior: The user's device displays the response "This smartphone has up to 12 hours of battery life and fast charging capabilities" in the advertising interface.

[0760] The operation of the user, terminal, and server at each step is explained in detail, and specific inputs and outputs are shown, making it clear how the invention is implemented.

[0761] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0762] The present invention relates to a system that allows users to enjoy conversations with characters, idols, and actors, and in particular, by combining an emotion engine that recognizes the user's emotions, it is possible to generate more natural responses that correspond to the emotions. The specific operations and processing flows of the server, terminal, and user for the embodiments of this invention are described below.

[0763] Server Roles and Program Processing

[0764] The server is built using a web application framework such as Flask. The role of the server is to receive messages from users, analyze the messages with an emotion engine to generate emotion tags, send prompts to the generative AI to generate responses, and return the generated responses to the user.

[0765] Specific working example:

[0766] The server uses a web application framework to define a / chat endpoint to handle HTTP POST requests.

[0767] Upon receiving a request from a user, the server extracts the message and character name from the request body.

[0768] The server uses an emotion engine to generate emotion tags from the extracted messages.

[0769] The server creates a prompt in the form of "You are [character name], respond to this message: [user's message (with emotion tag)]" and sends it to the generative AI.

[0770] The server receives the response from the generative AI, formats it as necessary, and sends it back to the user.

[0771] Terminal roles and program processing

[0772] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[0773] Specific working example:

[0774] The user opens the chat interface on the terminal and types a message.

[0775] The user selects a particular character from a drop-down menu within the chat interface.

[0776] When the user presses the "Send" button, the terminal sends an HTTP POST request containing the message and the character name to the server.

[0777] The terminal that receives the response from the server displays the response in the chat interface.

[0778] User roles and examples

[0779] Users can easily use this system to enjoy conversations with specific characters, idols, and actors. The operations performed by users are very simple.

[0780] Specific working example:

[0781] Suppose the user inputs the message "How are you feeling today?" and selects "Hatsune Miku" as the character.

[0782] When the user presses the send button, the device sends the message "How are you feeling today?" and the character name "Hatsune Miku" to the server.

[0783] The server receives the message, analyzes the message with an emotion engine, and generates an emotion tag, such as "positive."

[0784] The server sends a prompt to the generative AI, tagging it with the emotion "positive" and asking it to generate a response. The generative AI might generate the response "I'm feeling great today! I'm even working on a new song!"

[0785] Responses are displayed on the terminal, allowing the user to experience a natural and emotionally appropriate conversation.

[0786] This allows users to seamlessly enjoy conversations with specific characters, idols, and actors. In addition, by using an emotion engine, responses are made natural according to the user's emotions, providing a more satisfying conversational experience.

[0787] The processing flow will be explained below.

[0788] Step 1:

[0789] The user opens the chat interface on the terminal and types a message.

[0790] Specific behavior:

[0791] The user types a message into a text box in the chat interface.

[0792] The user selects a particular character from a drop-down menu or the like.

[0793] Step 2:

[0794] The user presses the "Send" button to send the entered message and selected character name.

[0795] Specific behavior:

[0796] The user presses the "send" button.

[0797] The terminal packages the entered message and the selected character name to form an HTTP POST request.

[0798] Step 3:

[0799] The device sends the constructed HTTP POST request to the server.

[0800] Specific behavior:

[0801] The device sends a request to the server's / chat endpoint, including the user's message and character name.

[0802] Step 4:

[0803] The server receives the HTTP POST request and extracts the message and character name from the request body.

[0804] Specific behavior:

[0805] The server receives requests through the Flask application at the / chat endpoint.

[0806] The server parses the request body and extracts the message and character name.

[0807] Step 5:

[0808] The server sends the extracted message to the emotion engine, which generates emotion tags.

[0809] Specific behavior:

[0810] The server requests the emotion engine to analyze the message.

[0811] The emotion engine analyzes the message and generates emotion tags such as "positive" or "negative."

[0812] The server receives emotion tags from the emotion engine.

[0813] Step 6:

[0814] The server generates prompts for the generative AI based on the user's message and emotion tag.

[0815] Specific behavior:

[0816] The server generates a prompt of the form "You are [character name], respond to this message with the emotion [emotion tag]: [user message]".

[0817] Prepare a request to send the generated prompt to the generative AI API.

[0818] Step 7:

[0819] The server sends prompts to the generative AI, causing it to generate a response.

[0820] Specific behavior:

[0821] The server calls a generative AI API such as OpenAI and sends the prepared prompts.

[0822] The server waits for the generative AI to generate a response.

[0823] Step 8:

[0824] The server receives the response from the generative AI and formats it as necessary.

[0825] Specific behavior:

[0826] The server analyzes the response data received from the generative AI.

[0827] Remove unnecessary whitespace and newlines from the response and format the text.

[0828] Step 9:

[0829] The server sends a formatted response back to the terminal.

[0830] Specific behavior:

[0831] The server generates a JSON object containing the formatted response.

[0832] The generated JSON object is sent to the terminal as an HTTP response.

[0833] Step 10:

[0834] The terminal receives the response from the server and displays it to the user.

[0835] Specific behavior:

[0836] The terminal parses the JSON data received from the server.

[0837] The analysis results are displayed in the chat interface so that the user can check them.

[0838] Step 11:

[0839] The user checks the response displayed on the terminal and continues the conversation by entering the next message.

[0840] Specific behavior:

[0841] The user reads the response displayed on the terminal and enters a new message.

[0842] If necessary, select a different character and submit again.

[0843] Example 2

[0844] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0845] Conventional dialogue systems do not properly recognize the user's emotions and simply respond to input messages. This results in unnatural responses and a poor user experience. Furthermore, they lack a mechanism for naturally engaging in dialogue with specific characters or idols, limiting their appeal to users. Furthermore, they lack the technology to generate responses based on emotions, making it impossible to realize conversations that are sensitive to the user's emotions.

[0846] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving a message from a user, a means for analyzing the received message with an emotion engine and generating an emotion tag, a means for sending the emotion-tagged message to a generative AI model to generate a response, and a means for returning the generated response to the user. This makes it possible to recognize emotions in response to messages from users and generate appropriate and natural responses. Furthermore, it is possible to have conversations with specific characters or idols according to emotions, thereby improving the user experience.

[0847] A "user" is a person who utilizes a dialogue system to input messages and receive responses.

[0848] A "message" is text data entered by a user and received by a server.

[0849] "Emotion engine" is a general term for algorithms and programs that analyze received messages and generate emotion tags from their contents.

[0850] "Emotion tags" are labels generated by the emotion engine that indicate the emotional characteristics of a message. Examples include "positive" and "negative."

[0851] A "generative AI model" is an artificial intelligence model that generates natural language responses based on received prompts.

[0852] A "prompt" is input information given to a generative AI model to generate a response. It includes character information and emotion tags.

[0853] The "server" is a computer system that receives messages from users, analyzes them using an emotion engine, sends prompts to a generative AI model, and executes a series of operations to generate a response.

[0854] "Character information" refers to information about a specific character, idol, or actor selected by the user.

[0855] A "web application framework" is a software framework for building applications that run on the web. Examples include Flask.

[0856] A "response" is the text data that a generative AI model generates based on a prompt and sends back to the user.

[0857] The present invention relates to a system that allows users to enjoy conversations with specific characters, idols, or actors. In particular, it utilizes a combination of an emotion engine and a generative AI model to recognize the user's emotions and generate natural responses accordingly. The following describes the embodiments of the invention based on the scope of the claims.

[0858] 1. Specific Embodiment of the Server

[0859] The server is built using Flask as a web application framework. The main roles of the server are as follows:

[0860] First, the server receives a message from the user. Next, it analyzes the received message using the emotion engine and generates an emotion tag. Based on the generated emotion tag, it integrates it with character information to generate a prompt and sends it to the generative AI model. Then it receives a response from the generative AI model and sends it back to the user.

[0861] As a specific use case, if a user types the message "How are you feeling today?" and selects "Hatsune Miku" as the character name, this message is sent to the server. The server uses its emotion engine to analyze the message and generate a "positive" emotion tag. It then sends the following prompt to the generative AI model:

[0862] "You are Hatsune Miku, respond to this message: How are you feeling today? (Positive)"

[0863] Based on this prompt, the generative AI model generates a response such as, "I'm feeling great today! I'm even working on a new song!", which the server sends back to the user.

[0864] 2. Specific Embodiments of the Terminal

[0865] The terminal provides an interface for users to input messages and send them to the server, using a web browser or a mobile application. The terminal's main roles are as follows:

[0866] First, the user enters a message using the chat interface. Then, they select a specific character from a drop-down menu. When the user presses the "Send" button, the device sends an HTTP POST request to the server containing the message and the character's name. The response is then received from the server and displayed in the chat interface.

[0867] 3. User's specific embodiment

[0868] Users can enjoy the interaction by performing extremely simple operations. The specific steps are shown below.

[0869] First, the user types a message in the chat interface, such as "How are you feeling today?", and then selects "Hatsune Miku" from the drop-down menu. When the user presses the "Send" button, the message and the character name are sent to the server. The server then responds with "I'm feeling great today! I'm even working on a new song!", which is displayed in the chat interface, allowing the user to enjoy the conversation.

[0870] As described above, this invention combines an emotion engine with a generative AI model to generate natural responses that correspond to the user's emotions, making it possible to seamlessly enjoy conversations with specific characters, idols, or actors.

[0871] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0872] Step 1: The user types a message into the chat interface

[0873] The user opens the chat interface on the device and types "How are you feeling today?" into the text field. Input: The message "How are you feeling today?" typed by the user. Output: The text that appears in the message input field on the device display.

[0874] Step 2: User selects a character

[0875] The user selects "Hatsune Miku" from a drop-down menu in the chat interface. Input: The character name "Hatsune Miku". Output: The selection displayed in the drop-down menu.

[0876] Step 3: User presses "Submit" button

[0877] The user presses the "Send" button, triggering the sending of the message and character name to the server. Input: Click on the "Send" button. Output: HTTP POST request containing the message and character name.

[0878] Step 4: The device sends an HTTP POST request to the server

[0879] The device sends an HTTP POST request to the server containing the user's message "How are you feeling today?" and the character name "Hatsune Miku". Input: Message and character name. Output: HTTP request sent to the server.

[0880] Step 5: The server receives the message

[0881] The server receives the HTTP POST request and extracts the message "How are you feeling today?" and the character name "Hatsune Miku" from the request body. Input: HTTP POST request body. Output: Extracted message and character name.

[0882] Step 6: The server parses the message using the emotion engine

[0883] The server passes the message "How are you feeling today?" to the emotion engine and generates the emotion tag "Positive". Input: The extracted message. Output: The generated emotion tag.

[0884] Step 7: The server creates a prompt and sends it to the generative AI model

[0885] The server generates a prompt, "You are Hatsune Miku, respond to this message: How are you feeling today? (Positive)," and sends it to the generative AI model. Input: Message, character name, emotion tag. Output: Prompt sent to the generative AI model.

[0886] Step 8: The generative AI model generates a response

[0887] Based on the prompt, the generative AI model generates a response such as "I'm feeling great today! I'm even working on a new song!". Input: Prompt. Output: Generated response.

[0888] Step 9: The server receives and formats the generated response

[0889] The server receives the response from the generative AI model, "I'm feeling great today! I'm even working on a new song!", and formats it as needed. Input: Generated response. Output: Formatted response.

[0890] Step 10: The server sends the formatted response back to the user

[0891] The server sends the formatted response back to the device as an HTTP response. Input: Formatted response. Output: Response sent to the user's device.

[0892] Step 11: The device receives the response from the server and displays it in the chat interface.

[0893] The device receives the response from the server, "I'm feeling great today! I'm even working on a new song!", and displays it in the chat interface. Input: Response from the server. Output: Response displayed in the chat interface.

[0894] Step 12: User checks the response and enjoys the interaction

[0895] The user can see the responses displayed on the chat interface of the device and enjoy natural dialogue with the character. Input: Responses displayed on the device. Output: Improved user dialogue experience.

[0896] (Application example 2)

[0897] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0898] Conventional dialogue systems have difficulty generating responses that accurately reflect the user's emotions, resulting in a limited user experience. Furthermore, when proposing products or recommending services in virtual stores, they often fail to provide appropriate suggestions that take the user's emotions into account. This has led to issues such as insufficient improvement of user satisfaction and insufficient stimulation of purchasing motivation.

[0899] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0900] In this invention, the server includes means for receiving a message from a user, means for analyzing the received message with an emotion engine and generating an emotion tag, means for generating a prompt with an emotion tag and sending it to a generative AI to generate a response, means for returning the generated response to the user, and means for providing a user interface using smart glasses or a head-mounted display. This enables more natural dialogue according to the user's emotions and makes it possible to propose products and services suitable for the user in a virtual store.

[0901] A "user" is a person who uses the system to input messages and receive proposals for services and products.

[0902] A "message" is text or voice data that a user inputs to the system.

[0903] An "emotion engine" is a software application that analyzes a user's message, identifies the emotion contained in the message, and generates an emotion tag.

[0904] An "emotion tag" is a label that represents an emotion and is generated by the emotion engine and assigned to a user's message.

[0905] A "prompt" is an instruction given to a generative AI to generate a response, and may include emotion tags and character information.

[0906] "Generative AI" is artificial intelligence that generates natural language responses in response to given prompts.

[0907] A "response" is text or audio data that a generative AI generates based on a prompt and sends back to the user.

[0908] "Smart glasses" are wearable devices with internet connectivity and other computing capabilities that are used to display information in the user's field of vision.

[0909] A "head-mounted display" is a device worn on the user's head that displays information in the user's field of vision.

[0910] "User interface" is a term that refers to an interface such as a screen, display, or input device that allows a user to interact with a system.

[0911] A "virtual store" is a virtual store built on the Internet where users can browse and purchase products online.

[0912] "Suggestion" refers to the act of selecting products or services suitable for a user in a virtual store and presenting them to the user.

[0913] This invention is a system that realizes customer service in a virtual store, and in particular enables natural dialogue and product suggestions that correspond to the user's emotions. The system is mainly composed of three entities: a server, a terminal, and a user.

[0914] Server Roles and Program Processing

[0915] The server is built using a web application framework such as Flask. The role of the server is to receive messages from users, analyze the messages with an emotion engine to generate emotion tags, create prompts and send them to the generative AI, and then send the generated responses back to the user.

[0916] The server does the following:

[0917] Receives a message from the user as an HTTP POST request.

[0918] The received message is analyzed by an emotion engine and an emotion tag is generated. Emotion engines are often provided as external APIs.

[0919] A prompt containing the generated emotion tag is created and sent to the generative AI in the format "You are a virtual shop assistant, respond to this message: 'Message content' (Emotion: emotion tag)".

[0920] Receives responses from the generative AI and displays them to the user via the user interface.

[0921] Terminal roles and program processing

[0922] The terminal provides a user interface using smart glasses or a head-mounted display (HMD). The terminal's role is to allow the user to input messages and send them to the server, and to visually present the responses sent back from the server to the user.

[0923] The terminal does the following:

[0924] The user types a message through smart glasses or an HMD.

[0925] Provides an interface for sending input messages to the server.

[0926] Responses from the server are displayed in real time on the user interface.

[0927] User roles and examples

[0928] Users can easily use the system to enjoy conversations in a virtual store. The user's operations are very simple: they simply input messages through smart glasses or an HMD and receive responses from the server.

[0929] Specific working example:

[0930] Let's say a user types a message like "I've been feeling tired lately." The user sends this message to the system using smart glasses or an HMD. The server receives this message, analyzes it with its emotion engine, and generates an emotion tag called "tired." It then sends the following prompt to the generative AI:

[0931] You are a virtual shop assistant, respond to this message: "I've been feeling tired lately." (Emotion: tiredness)

[0932] The generative AI generates a response such as "Why not try a relaxing massage chair?", which is then displayed on the smart glasses or HMD.

[0933] This allows users to receive appropriate product and service recommendations in the virtual store through natural dialogue based on their emotions, improving the user experience and significantly increasing satisfaction in the virtual store.

[0934] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0935] Step 1:

[0936] The terminal receives messages from the user as input. The user uses smart glasses or a head-mounted display (HMD) to input text or voice messages in natural language. This message contains information including the user's wishes and emotions expressed to the system. The terminal stores the message as is and prepares it for transmission to the server.

[0937] Step 2:

[0938] The device sends the message entered by the user to the server as an HTTP POST request, along with any additional information related to the message (e.g., character selection). The entered data is passed to the server through the interface of the smart glasses or HMD.

[0939] Step 3:

[0940] The server receives messages sent from the devices as input. To analyze these messages, the server first sends them to the emotion engine. The emotion engine analyzes the messages and generates emotion tags based on their content. Emotion tags express the emotions contained in the messages and are generated using natural language processing techniques on the message text. The resulting emotion tags are then returned to the server.

[0941] Step 4:

[0942] The server generates a prompt based on the received emotion tag and the user message. The generated prompt is in the format "You are a virtual shop assistant, respond to this message: 'Message content' (Emotion: emotion tag)". This prompt is used as an instruction for the generative AI, and by incorporating the emotion tag, it provides input data for generating a more appropriate response.

[0943] Step 5:

[0944] The server sends the prompt to a generative AI, which takes the prompt as input and generates an appropriate natural language response based on its content. The generative AI then analyzes the prompt using large-scale language models and machine learning algorithms, and outputs an appropriate response to the user's message. This output response is then sent back to the server.

[0945] Step 6:

[0946] The server formats the response received from the generative AI, returns it to the device in its final form, and in some cases deletes unnecessary information during the formatting process. The response is then sent back to the device as an HTTP POST request.

[0947] Step 7:

[0948] The device displays the response received from the server. The received response message is visually displayed on the display of the smart glasses or head-mounted display. This allows the user to receive product and service suggestions based on their emotions through natural dialogue in real time.

[0949] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0950] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0951] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0952] [Third embodiment]

[0953] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0954] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0955] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0956] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0957] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0958] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0959] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0960] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0961] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0962] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0963] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0964] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0965] This invention relates to a system that allows users to enjoy conversations with characters, idols, and actors. Specific operations and processing flows of the server, terminal, and user in the embodiment of the invention will be described below.

[0966] Server Roles and Program Processing

[0967] First, the server is built using a web application framework such as Flask. The server receives a message and character name from the user, generates a response by sending it as a prompt to the generative AI (e.g., OpenAI's API), and then sends the generated response back to the user.

[0968] Specific working example:

[0969] The server uses a web application framework to define a / chat endpoint to handle HTTP POST requests.

[0970] Upon receiving a request from a user, the server extracts the message and character name from the request body.

[0971] The server generates a prompt of the form "You are [character name], respond to this message: [user message]".

[0972] The server sends this prompt to the generative AI and receives a response.

[0973] The server formats the response and sends it back to the user.

[0974] Terminal roles and program processing

[0975] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[0976] Specific working example:

[0977] The user opens the chat interface on the device and types a message.

[0978] The user selects a particular character from a drop-down menu within the chat interface.

[0979] When the user presses the "Send" button, the terminal sends an HTTP POST request containing the message and the character name to the server.

[0980] The terminal that receives the response from the server displays the response in the chat interface.

[0981] User roles and examples

[0982] Users can easily use this system to enjoy conversations with specific characters, idols, and actors. The operations performed by users are very simple.

[0983] Specific working example:

[0984] Suppose the user inputs the message "How are you feeling today?" and selects "Hatsune Miku" as the character.

[0985] When the user presses the send button, the device sends the message "How are you feeling today?" and the character name "Hatsune Miku" to the server.

[0986] The server sends a prompt to the generative AI, which generates the text in response: "I'm feeling great today! I'm even working on a new song!"

[0987] Responses are displayed on the terminal, allowing the user to experience a natural and enjoyable conversation.

[0988] This allows users to seamlessly enjoy interactions with specific characters, idols, and actors. In addition, server-side processing is performed automatically, providing a stress-free experience for users.

[0989] The processing flow will be explained below.

[0990] Step 1:

[0991] The user opens the chat interface on the terminal and types a message.

[0992] Specific behavior:

[0993] The user types a message into a text box in the chat interface.

[0994] The user selects a particular character from a drop-down menu or the like.

[0995] Step 2:

[0996] The user presses the "Send" button to send the entered message and selected character name.

[0997] Specific behavior:

[0998] The user presses the "send" button.

[0999] The terminal packages the entered message and the selected character name to form an HTTP POST request.

[1000] Step 3:

[1001] The device sends the constructed HTTP POST request to the server.

[1002] Specific behavior:

[1003] The device sends a request to the server's / chat endpoint, including the user's message and character name.

[1004] Step 4:

[1005] The server receives the HTTP POST request and extracts the message and character name from the request body.

[1006] Specific behavior:

[1007] The server receives requests through the Flask application at the / chat endpoint.

[1008] The server parses the request body and extracts the message and character name.

[1009] Step 5:

[1010] The server generates a prompt for the generative AI based on the user's message and the character name.

[1011] Specific behavior:

[1012] The server creates a prompt saying "You are [character name], respond to this message: [user message]".

[1013] Prepare a request to send the generated prompt to the generative AI API.

[1014] Step 6:

[1015] The server sends prompts to the generative AI, causing it to generate a response.

[1016] Specific behavior:

[1017] The server calls a generative AI API such as OpenAI and sends the prepared prompts.

[1018] The server waits for the generative AI to generate a response.

[1019] Step 7:

[1020] The server receives the response from the generative AI and formats it as necessary.

[1021] Specific behavior:

[1022] The server analyzes the response data received from the generative AI.

[1023] Remove unnecessary whitespace and newlines from the response and format the text.

[1024] Step 8:

[1025] The server sends a formatted response back to the terminal.

[1026] Specific behavior:

[1027] The server generates a JSON object containing the formatted response.

[1028] The generated JSON object is sent to the terminal as an HTTP response.

[1029] Step 9:

[1030] The terminal receives the response from the server and displays it to the user.

[1031] Specific behavior:

[1032] The terminal parses the JSON data received from the server.

[1033] The analysis results are displayed in the chat interface so that the user can check them.

[1034] Step 10:

[1035] The user checks the response displayed on the terminal and continues the conversation by entering the next message.

[1036] Specific behavior:

[1037] The user reads the response displayed on the terminal and enters a new message.

[1038] If necessary, select a different character and submit again.

[1039] Example 1

[1040] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1041] Providing a system that allows users to seamlessly and enjoy real-time interactions with unrealistic characters or celebrities is an important challenge in the entertainment industry. There is also a need for an interface that allows users to easily select specific characters and enjoy natural interactions. However, current dialogue systems require complex configurations to achieve the desired dialogue experience, and are not intuitive.

[1042] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1043] In this invention, the server includes a means for receiving a message from a user, a means for generating a prompt including the received message and character information and sending it to the generative AI, and a means for returning the generated response to the user, thereby enabling the user to intuitively enjoy a conversation with a specific character or celebrity.

[1044] A "message" is text data that indicates the content of a conversation entered by a user.

[1045] "Character information" is data about the particular character that the user selects to interact with.

[1046] A "prompt" is input text sent to a generative AI to guide it to a specific output.

[1047] "Generative AI" is a system that uses artificial intelligence technology to generate responses to input text.

[1048] A "server" is a computer system whose role is to receive messages and character information from the user, send prompts to the generative AI, obtain responses, and return them to the user.

[1049] A "user interface" is a part of the software that provides a screen and operating means for the user to input messages and select character information.

[1050] A "web application framework" is a set of software tools for building web-based user interfaces.

[1051] A "response" is a reply text that the generative AI generates based on a prompt.

[1052] An "HTTP POST request" is a protocol-based request method for sending data to a server over the Internet.

[1053] A "terminal" is a computing or mobile device that allows a user to enter and send messages to a server.

[1054] This invention relates to a system that allows users to enjoy conversations with specific characters or celebrities. To implement this invention, the server, terminal, and user each have specific roles and work in cooperation with each other.

[1055] Server configuration and roles

[1056] The server is built using a web application framework such as Flask. The server receives messages and character information sent by the user and generates prompts for the generative AI model based on that information. It then sends the prompts to the generative AI model (e.g., OpenAI's API) and returns the responses it receives to the user.

[1057] Specific working example:

[1058] The server uses Flask to define a / chat endpoint and handles HTTP POST requests.

[1059] Upon receiving a request from a user, the server extracts the message and character information from the request body.

[1060] The server generates a prompt of the form "You are [character name], respond to this message: [user message]".

[1061] The server sends this prompt to the generative AI and receives a response.

[1062] The server formats the response and sends it back to the user.

[1063] Examples of prompts:

[1064] Prompt: "You are Hatsune Miku, respond to this message: How are you feeling today?"

[1065] Example response: "I'm feeling great today! I'm even working on some new music!"

[1066] Device configuration and role

[1067] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[1068] Specific working example:

[1069] The user opens the chat interface on the device and types a message.

[1070] The user selects a particular character from a drop-down menu within the chat interface.

[1071] When the user presses the "Send" button, the terminal sends an HTTP POST request containing the message and the character name to the server.

[1072] The terminal that receives the response from the server displays the response in the chat interface.

[1073] User operations

[1074] Users can easily use this system to enjoy conversations with specific characters or celebrities. The operation is very intuitive and does not require complicated settings.

[1075] Specific working example:

[1076] The user inputs the message "How are you feeling today?" and selects "Hatsune Miku" as the character.

[1077] When the user presses the send button, the device sends the message "How are you feeling today?" and the character name "Hatsune Miku" to the server.

[1078] The server sends a prompt to the generative AI, which generates the text in response: "I'm feeling great today! I'm even working on a new song!"

[1079] The response is displayed on the terminal, allowing the user to enjoy natural dialogue.

[1080] In this way, the system of the present invention allows users to interact with characters and celebrities in real time, creating a seamless experience.

[1081] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1082] Step 1:

[1083] User enters message and selects character

[1084] Users access the chat interface, type the message they want to communicate, and then select the character they want to interact with from a drop-down menu.

[1085] input:

[1086] Message: "How are you feeling today?"

[1087] Character name: "Hatsune Miku"

[1088] output:

[1089] Information that the send button has been pressed is passed to the terminal.

[1090] Specific behavior:

[1091] The user opens the chat interface.

[1092] The user types a message: "How are you feeling today?"

[1093] Select "Hatsune Miku" from the drop-down menu.

[1094] Step 2:

[1095] The device sends a request to the server

[1096] When the user presses the send button, the terminal sends the entered message and the selected character name to the server as an HTTP POST request.

[1097] input:

[1098] The event of the submit button pressed by the user

[1099] Entered message and character name

[1100] output:

[1101] An HTTP POST request is sent to the server.

[1102] Specific behavior:

[1103] The user presses the "Send" button.

[1104] The device sends an HTTP POST request to the server containing the message "How are you feeling today?" and the character name "Hatsune Miku."

[1105] Step 3:

[1106] The server receives the request and generates a prompt.

[1107] The server processes the request received from the device, extracts the message and character name from the request body, and then creates a prompt for the generative AI model.

[1108] input:

[1109] Received HTTP POST request

[1110] output:

[1111] Generated prompt statement

[1112] Specific behavior:

[1113] The server receives an HTTP POST request.

[1114] The server extracts the message "How are you feeling today?" and the character name "Hatsune Miku" from the request body.

[1115] The server generates the prompt "You are Hatsune Miku, respond to this message: How are you feeling today?"

[1116] Step 4:

[1117] The server sends prompts to the generative AI and gets a response

[1118] The server sends a prompt to the generative AI (e.g., OpenAI API) and receives the generated response.

[1119] input:

[1120] Generated prompt statement

[1121] output:

[1122] Response text from generative AI

[1123] Specific behavior:

[1124] The server sends the generative AI a prompt: "You are Hatsune Miku, respond to this message: How are you feeling today?"

[1125] The generative AI generates a response such as "I'm feeling great today! I'm even working on a new song!" and sends it back to the server.

[1126] Step 5:

[1127] The server formats the response it receives and sends it back to the device.

[1128] The server formats the response received from the generative AI, makes it easy for the user to understand, and sends it back to the device.

[1129] input:

[1130] Response text from generative AI

[1131] output:

[1132] Formatted response text

[1133] Specific behavior:

[1134] The server receives the generated response "I'm feeling great today! I'm even working on a new song!"

[1135] The server formats the response and sends it back to the device as an HTTP response.

[1136] Step 6:

[1137] The device receives the response from the server and displays it to the user.

[1138] The terminal receives the response from the server and displays it in the chat interface.

[1139] input:

[1140] HTTP response from the server

[1141] output:

[1142] The displayed response text

[1143] Specific behavior:

[1144] The device receives a response saying, "I'm feeling great today! I'm even working on a new song!"

[1145] The terminal displays the received response on the chat interface and notifies the user.

[1146] The user checks the response and enjoys the interaction.

[1147] (Application example 1)

[1148] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1149] To improve the effectiveness of advertising, interactive communication with users is required. However, with conventional advertising methods, users only passively receive advertising content, and two-way communication is insufficient. It is also difficult for users to obtain information tailored to their own interests in real time. This has resulted in limited advertising effectiveness and low user engagement.

[1150] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1151] In this invention, the server includes a means for receiving a message from a user, a means for sending the received message to a generative AI model to generate a response, a means for returning the generated response to the user, and a means for generating and sending a prompt including advertising character information. This allows users to obtain information about new products in real time through interactions with specific advertising characters or idols. This enables two-way communication with users, improving advertising effectiveness and user engagement.

[1152] "Means for receiving messages from users" refers to the interface for receiving messages entered by users, which is mainly realized through web applications or mobile applications.

[1153] "Means for sending received messages to a generative AI model to generate a response" refers to the process of passing received messages to a generative AI model and generating an appropriate response. Specifically, this is the part that calls the AI's API and receives the generated response.

[1154] "Means for returning the generated response to the user" refers to the process for displaying the response obtained from the generative AI model to the user. This information is typically returned to the user via a web browser or smartphone application.

[1155] "Means for generating and transmitting prompts including advertising character information" refers to a process for generating prompts for a user based on a particular advertising character and transmitting the prompts to a generative AI model, which allows the character to interact with the user.

[1156] "Means for providing new product information" refers to the process of using a generative AI model to provide related new product information based on messages from users. Specifically, it provides users with detailed information and explanations related to new products.

[1157] "Means for utilizing a web application framework to provide an advertising interface" means a process for utilizing a web application framework to provide an interface that allows users to interact with advertisements in an interactive manner, thereby enabling users to easily send messages and receive responses.

[1158] A "generative AI model" is an AI that generates appropriate responses based on prompts from the user. Specifically, it is an AI model that can learn from a large amount of data and generate natural language.

[1159] A "prompt sentence" is a sentence used to generate a response from a generative AI model. It is written in a format that includes specific character information and a message from the user.

[1160] MODE FOR CARRYING OUT THE INVENTION

[1161] This invention is a system that allows users to obtain information about new products in real time through conversations with advertising characters and idols. The specific operations of the server, terminal, and user are described below.

[1162] Server Roles and Program Processing

[1163] The server is built using a web application framework such as Flask. It receives messages and advertising character information from users, generates responses by sending them as prompts to a generative AI model (such as the OpenAI API), and then sends the generated responses back to the user.

[1164] Specific working example:

[1165] The server uses a web application framework to define a / chat endpoint to handle HTTP POST requests.

[1166] Upon receiving a request from a user, the server extracts the message and advertising character information from the request body.

[1167] The server generates a prompt in the form "You are [advertising character name], introduce and explain the new product: [user message]".

[1168] The server sends this prompt to the generative AI model and receives a response.

[1169] The server formats the response and sends it back to the user.

[1170] Terminal roles and program processing

[1171] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[1172] Specific working example:

[1173] The user opens the advertising interface on the terminal and enters a message.

[1174] The user selects a particular advertising character from a drop-down menu within the advertising interface.

[1175] When the user presses the "Send" button, the terminal sends an HTTP POST request including the message and advertising character information to the server.

[1176] The terminal that receives the response from the server displays the response on the advertising interface.

[1177] User roles and examples

[1178] Users can easily use this system to enjoy interacting with specific advertising characters and idols. The operations performed by users are very simple.

[1179] Specific working example:

[1180] Suppose a user inputs the message "How is the battery life of this smartphone?" and selects "Ambassador for home appliances" as the advertising character.

[1181] When the user presses the send button, the device sends the message "How is the battery life of this smartphone?" along with character information about the "home appliance ambassador" to the server.

[1182] The server sends a prompt to the generative AI model, which generates the following text in response: "This smartphone has up to 12 hours of battery life and fast charging."

[1183] The responses are displayed on the terminal, providing the user with detailed and interesting product information in real time.

[1184] Prompt Sentence Examples

[1185] If the user selects an ambassador for a consumer electronics product and asks about smartphone battery life, the prompt might look like this:

[1186] "You are an ambassador for a consumer electronics product. You introduce and explain the new product: How is the battery life of this smartphone?"

[1187] This invention allows users to obtain information about new products while interactively communicating with specific advertising characters or idols, which is expected to improve advertising effectiveness and user engagement.

[1188] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1189] Step 1:

[1190] The user opens the advertising interface on the device and inputs a message. The input message and character information are sent to the server in the form of an HTTP POST request when the user presses the send button.

[1191] Input: User-entered message and selected character information

[1192] Output: Sends message and character information to the server as an HTTP POST request

[1193] Specific operation: The user enters the message "How is the battery life of this smartphone?" and selects "Home appliance ambassador" as the character.

[1194] Step 2:

[1195] The server receives the HTTP POST request and extracts the user's message and character information from the request body.

[1196] Input: HTTP POST request

[1197] Output: Message and character information extracted from the request body

[1198] Specific operation: The server extracts the message "How is the battery life of this smartphone?" and the character information "Ambassador for home appliances" from the received request.

[1199] Step 3:

[1200] The server generates a prompt sentence to send to the generative AI model based on the extracted message and character information.

[1201] Input: Extracted message and character information

[1202] Output: Generated prompt statement

[1203] Specific operation: The server generates the prompt sentence, "You are an ambassador for home appliances, introduce and explain the new product: How is the battery life of this smartphone?"

[1204] Step 4:

[1205] The server sends the generated prompt sentence to the generative AI model, which then generates a response.

[1206] Input: Generated prompt text

[1207] Output: Response from the generative AI model

[1208] Specific operation: The server calls the OpenAI API, sends a prompt, and receives a response.

[1209] Step 5:

[1210] The server formats the response received from the generative AI model and prepares it to be sent back to the user.

[1211] Input: Response from a generative AI model

[1212] Output: The formatted response

[1213] Specific behavior: The server formats the response it receives: "This smartphone has a battery life of up to 12 hours and also has fast charging capabilities."

[1214] Step 6:

[1215] The server returns the formatted response to the user's terminal as an HTTP response.

[1216] Input: Formatted response

[1217] Output: Returned to the user's device as an HTTP response

[1218] Specific operation: The server sends the formatted response to the user's terminal as an HTTP response.

[1219] Step 7:

[1220] The user's device receives the HTTP response from the server and displays the response in the advertising interface.

[1221] Input: HTTP response from the server

[1222] Output: Response displayed in the advertising interface

[1223] Specific behavior: The user's device displays the response "This smartphone has up to 12 hours of battery life and fast charging capabilities" in the advertising interface.

[1224] The operation of the user, terminal, and server at each step is explained in detail, and specific inputs and outputs are shown, making it clear how the invention is implemented.

[1225] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1226] The present invention relates to a system that allows users to enjoy conversations with characters, idols, and actors, and in particular, by combining an emotion engine that recognizes the user's emotions, it is possible to generate more natural responses that correspond to the emotions. The specific operations and processing flows of the server, terminal, and user for the embodiments of this invention are described below.

[1227] Server Roles and Program Processing

[1228] The server is built using a web application framework such as Flask. The role of the server is to receive messages from users, analyze the messages with an emotion engine to generate emotion tags, send prompts to the generative AI to generate responses, and return the generated responses to the user.

[1229] Specific working example:

[1230] The server uses a web application framework to define a / chat endpoint to handle HTTP POST requests.

[1231] Upon receiving a request from a user, the server extracts the message and character name from the request body.

[1232] The server uses an emotion engine to generate emotion tags from the extracted messages.

[1233] The server creates a prompt in the form of "You are [character name], respond to this message: [user's message (with emotion tag)]" and sends it to the generative AI.

[1234] The server receives the response from the generative AI, formats it as necessary, and sends it back to the user.

[1235] Terminal roles and program processing

[1236] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[1237] Specific working example:

[1238] The user opens the chat interface on the terminal and types a message.

[1239] The user selects a particular character from a drop-down menu within the chat interface.

[1240] When the user presses the "Send" button, the terminal sends an HTTP POST request containing the message and the character name to the server.

[1241] The terminal that receives the response from the server displays the response in the chat interface.

[1242] User roles and examples

[1243] Users can easily use this system to enjoy conversations with specific characters, idols, and actors. The operations performed by users are very simple.

[1244] Specific working example:

[1245] Suppose the user inputs the message "How are you feeling today?" and selects "Hatsune Miku" as the character.

[1246] When the user presses the send button, the device sends the message "How are you feeling today?" and the character name "Hatsune Miku" to the server.

[1247] The server receives the message, analyzes the message with an emotion engine, and generates an emotion tag, such as "positive."

[1248] The server sends a prompt to the generative AI, tagging it with the emotion "positive" and asking it to generate a response. The generative AI might generate the response "I'm feeling great today! I'm even working on a new song!"

[1249] Responses are displayed on the terminal, allowing the user to experience a natural and emotionally appropriate conversation.

[1250] This allows users to seamlessly enjoy conversations with specific characters, idols, and actors. In addition, by using an emotion engine, responses are made natural according to the user's emotions, providing a more satisfying conversational experience.

[1251] The processing flow will be explained below.

[1252] Step 1:

[1253] The user opens the chat interface on the terminal and types a message.

[1254] Specific behavior:

[1255] The user types a message into a text box in the chat interface.

[1256] The user selects a particular character from a drop-down menu or the like.

[1257] Step 2:

[1258] The user presses the "Send" button to send the entered message and selected character name.

[1259] Specific behavior:

[1260] The user presses the "send" button.

[1261] The terminal packages the entered message and the selected character name to form an HTTP POST request.

[1262] Step 3:

[1263] The device sends the constructed HTTP POST request to the server.

[1264] Specific behavior:

[1265] The device sends a request to the server's / chat endpoint, including the user's message and character name.

[1266] Step 4:

[1267] The server receives the HTTP POST request and extracts the message and character name from the request body.

[1268] Specific behavior:

[1269] The server receives requests through the Flask application at the / chat endpoint.

[1270] The server parses the request body and extracts the message and character name.

[1271] Step 5:

[1272] The server sends the extracted message to the emotion engine, which generates emotion tags.

[1273] Specific behavior:

[1274] The server requests the emotion engine to analyze the message.

[1275] The emotion engine analyzes the message and generates emotion tags such as "positive" or "negative."

[1276] The server receives emotion tags from the emotion engine.

[1277] Step 6:

[1278] The server generates prompts for the generative AI based on the user's message and emotion tag.

[1279] Specific behavior:

[1280] The server generates a prompt of the form "You are [character name], respond to this message with the emotion [emotion tag]: [user message]".

[1281] Prepare a request to send the generated prompt to the generative AI API.

[1282] Step 7:

[1283] The server sends prompts to the generative AI, causing it to generate a response.

[1284] Specific behavior:

[1285] The server calls a generative AI API such as OpenAI and sends the prepared prompts.

[1286] The server waits for the generative AI to generate a response.

[1287] Step 8:

[1288] The server receives the response from the generative AI and formats it as necessary.

[1289] Specific behavior:

[1290] The server analyzes the response data received from the generative AI.

[1291] Remove unnecessary whitespace and newlines from the response and format the text.

[1292] Step 9:

[1293] The server sends a formatted response back to the terminal.

[1294] Specific behavior:

[1295] The server generates a JSON object containing the formatted response.

[1296] The generated JSON object is sent to the terminal as an HTTP response.

[1297] Step 10:

[1298] The terminal receives the response from the server and displays it to the user.

[1299] Specific behavior:

[1300] The terminal parses the JSON data received from the server.

[1301] The analysis results are displayed in the chat interface so that the user can check them.

[1302] Step 11:

[1303] The user checks the response displayed on the terminal and continues the conversation by entering the next message.

[1304] Specific behavior:

[1305] The user reads the response displayed on the terminal and enters a new message.

[1306] If necessary, select a different character and submit again.

[1307] Example 2

[1308] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1309] Conventional dialogue systems do not properly recognize the user's emotions and simply respond to input messages. This results in unnatural responses and a poor user experience. Furthermore, they lack a mechanism for naturally engaging in dialogue with specific characters or idols, limiting their appeal to users. Furthermore, they lack the technology to generate responses based on emotions, making it impossible to realize conversations that are sensitive to the user's emotions.

[1310] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving a message from a user, a means for analyzing the received message with an emotion engine and generating an emotion tag, a means for sending the emotion-tagged message to a generative AI model to generate a response, and a means for returning the generated response to the user. This makes it possible to recognize emotions in response to messages from users and generate appropriate and natural responses. Furthermore, it is possible to have conversations with specific characters or idols according to emotions, thereby improving the user experience.

[1311] A "user" is a person who utilizes a dialogue system to input messages and receive responses.

[1312] A "message" is text data entered by a user and received by a server.

[1313] "Emotion engine" is a general term for algorithms and programs that analyze received messages and generate emotion tags from their contents.

[1314] "Emotion tags" are labels generated by the emotion engine that indicate the emotional characteristics of a message. Examples include "positive" and "negative."

[1315] A "generative AI model" is an artificial intelligence model that generates natural language responses based on received prompts.

[1316] A "prompt" is input information given to a generative AI model to generate a response. It includes character information and emotion tags.

[1317] The "server" is a computer system that receives messages from users, analyzes them using an emotion engine, sends prompts to a generative AI model, and executes a series of operations to generate a response.

[1318] "Character information" refers to information about a specific character, idol, or actor selected by the user.

[1319] A "web application framework" is a software framework for building applications that run on the web. Examples include Flask.

[1320] A "response" is the text data that a generative AI model generates based on a prompt and sends back to the user.

[1321] The present invention relates to a system that allows users to enjoy conversations with specific characters, idols, or actors. In particular, it utilizes a combination of an emotion engine and a generative AI model to recognize the user's emotions and generate natural responses accordingly. The following describes the embodiments of the invention based on the scope of the claims.

[1322] 1. Specific Embodiment of the Server

[1323] The server is built using Flask as a web application framework. The main roles of the server are as follows:

[1324] First, the server receives a message from the user. Next, it analyzes the received message using the emotion engine and generates an emotion tag. Based on the generated emotion tag, it integrates it with character information to generate a prompt and sends it to the generative AI model. Then it receives a response from the generative AI model and sends it back to the user.

[1325] As a specific use case, if a user types the message "How are you feeling today?" and selects "Hatsune Miku" as the character name, this message is sent to the server. The server uses its emotion engine to analyze the message and generate a "positive" emotion tag. It then sends the following prompt to the generative AI model:

[1326] "You are Hatsune Miku, respond to this message: How are you feeling today? (Positive)"

[1327] Based on this prompt, the generative AI model generates a response such as, "I'm feeling great today! I'm even working on a new song!", which the server sends back to the user.

[1328] 2. Specific Embodiments of the Terminal

[1329] The terminal provides an interface for users to input messages and send them to the server, using a web browser or a mobile application. The terminal's main roles are as follows:

[1330] First, the user enters a message using the chat interface. Then, they select a specific character from a drop-down menu. When the user presses the "Send" button, the device sends an HTTP POST request to the server containing the message and the character's name. The response is then received from the server and displayed in the chat interface.

[1331] 3. User's specific embodiment

[1332] Users can enjoy the interaction by performing extremely simple operations. The specific steps are shown below.

[1333] First, the user types a message in the chat interface, such as "How are you feeling today?", and then selects "Hatsune Miku" from the drop-down menu. When the user presses the "Send" button, the message and the character name are sent to the server. The server then responds with "I'm feeling great today! I'm even working on a new song!", which is displayed in the chat interface, allowing the user to enjoy the conversation.

[1334] As described above, this invention combines an emotion engine with a generative AI model to generate natural responses that correspond to the user's emotions, making it possible to seamlessly enjoy conversations with specific characters, idols, or actors.

[1335] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1336] Step 1: The user types a message into the chat interface

[1337] The user opens the chat interface on the device and types "How are you feeling today?" into the text field. Input: The message "How are you feeling today?" typed by the user. Output: The text that appears in the message input field on the device display.

[1338] Step 2: User selects a character

[1339] The user selects "Hatsune Miku" from a drop-down menu in the chat interface. Input: The character name "Hatsune Miku". Output: The selection displayed in the drop-down menu.

[1340] Step 3: User presses "Submit" button

[1341] The user presses the "Send" button, triggering the sending of the message and character name to the server. Input: Click on the "Send" button. Output: HTTP POST request containing the message and character name.

[1342] Step 4: The device sends an HTTP POST request to the server

[1343] The device sends an HTTP POST request to the server containing the user's message "How are you feeling today?" and the character name "Hatsune Miku". Input: Message and character name. Output: HTTP request sent to the server.

[1344] Step 5: The server receives the message

[1345] The server receives the HTTP POST request and extracts the message "How are you feeling today?" and the character name "Hatsune Miku" from the request body. Input: HTTP POST request body. Output: Extracted message and character name.

[1346] Step 6: The server parses the message using the emotion engine

[1347] The server passes the message "How are you feeling today?" to the emotion engine and generates the emotion tag "Positive". Input: The extracted message. Output: The generated emotion tag.

[1348] Step 7: The server creates a prompt and sends it to the generative AI model

[1349] The server generates a prompt, "You are Hatsune Miku, respond to this message: How are you feeling today? (Positive)," and sends it to the generative AI model. Input: Message, character name, emotion tag. Output: Prompt sent to the generative AI model.

[1350] Step 8: The generative AI model generates a response

[1351] Based on the prompt, the generative AI model generates a response such as "I'm feeling great today! I'm even working on a new song!". Input: Prompt. Output: Generated response.

[1352] Step 9: The server receives and formats the generated response

[1353] The server receives the response from the generative AI model, "I'm feeling great today! I'm even working on a new song!", and formats it as needed. Input: Generated response. Output: Formatted response.

[1354] Step 10: The server sends the formatted response back to the user

[1355] The server sends the formatted response back to the device as an HTTP response. Input: Formatted response. Output: Response sent to the user's device.

[1356] Step 11: The device receives the response from the server and displays it in the chat interface.

[1357] The device receives the response from the server, "I'm feeling great today! I'm even working on a new song!", and displays it in the chat interface. Input: Response from the server. Output: Response displayed in the chat interface.

[1358] Step 12: User checks the response and enjoys the interaction

[1359] The user can see the responses displayed on the chat interface of the device and enjoy natural dialogue with the character. Input: Responses displayed on the device. Output: Improved user dialogue experience.

[1360] (Application example 2)

[1361] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1362] Conventional dialogue systems have difficulty generating responses that accurately reflect the user's emotions, resulting in a limited user experience. Furthermore, when proposing products or recommending services in virtual stores, they often fail to provide appropriate suggestions that take the user's emotions into account. This has led to issues such as insufficient improvement of user satisfaction and insufficient stimulation of purchasing motivation.

[1363] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1364] In this invention, the server includes means for receiving a message from a user, means for analyzing the received message with an emotion engine and generating an emotion tag, means for generating a prompt with an emotion tag and sending it to a generative AI to generate a response, means for returning the generated response to the user, and means for providing a user interface using smart glasses or a head-mounted display. This enables more natural dialogue according to the user's emotions and makes it possible to propose products and services suitable for the user in a virtual store.

[1365] A "user" is a person who uses the system to input messages and receive proposals for services and products.

[1366] A "message" is text or voice data that a user inputs to the system.

[1367] An "emotion engine" is a software application that analyzes a user's message, identifies the emotion contained in the message, and generates an emotion tag.

[1368] An "emotion tag" is a label that represents an emotion and is generated by the emotion engine and assigned to a user's message.

[1369] A "prompt" is an instruction given to a generative AI to generate a response, and may include emotion tags and character information.

[1370] "Generative AI" is artificial intelligence that generates natural language responses in response to given prompts.

[1371] A "response" is text or audio data that a generative AI generates based on a prompt and sends back to the user.

[1372] "Smart glasses" are wearable devices with internet connectivity and other computing capabilities that are used to display information in the user's field of vision.

[1373] A "head-mounted display" is a device worn on the user's head that displays information in the user's field of vision.

[1374] "User interface" is a term that refers to an interface such as a screen, display, or input device that allows a user to interact with a system.

[1375] A "virtual store" is a virtual store built on the Internet where users can browse and purchase products online.

[1376] "Suggestion" refers to the act of selecting products or services suitable for a user in a virtual store and presenting them to the user.

[1377] This invention is a system that realizes customer service in a virtual store, and in particular enables natural dialogue and product suggestions that correspond to the user's emotions. The system is mainly composed of three entities: a server, a terminal, and a user.

[1378] Server Roles and Program Processing

[1379] The server is built using a web application framework such as Flask. The role of the server is to receive messages from users, analyze the messages with an emotion engine to generate emotion tags, create prompts and send them to the generative AI, and then send the generated responses back to the user.

[1380] The server does the following:

[1381] Receives a message from the user as an HTTP POST request.

[1382] The received message is analyzed by an emotion engine and an emotion tag is generated. Emotion engines are often provided as external APIs.

[1383] A prompt containing the generated emotion tag is created and sent to the generative AI in the format "You are a virtual shop assistant, respond to this message: 'Message content' (Emotion: emotion tag)".

[1384] Receives responses from the generative AI and displays them to the user via the user interface.

[1385] Terminal roles and program processing

[1386] The terminal provides a user interface using smart glasses or a head-mounted display (HMD). The terminal's role is to allow the user to input messages and send them to the server, and to visually present the responses sent back from the server to the user.

[1387] The terminal does the following:

[1388] The user types a message through smart glasses or an HMD.

[1389] Provides an interface for sending input messages to the server.

[1390] Responses from the server are displayed in real time on the user interface.

[1391] User roles and examples

[1392] Users can easily use the system to enjoy conversations in a virtual store. The user's operations are very simple: they simply input messages through smart glasses or an HMD and receive responses from the server.

[1393] Specific working example:

[1394] Let's say a user types a message like "I've been feeling tired lately." The user sends this message to the system using smart glasses or an HMD. The server receives this message, analyzes it with its emotion engine, and generates an emotion tag called "tired." It then sends the following prompt to the generative AI:

[1395] You are a virtual shop assistant, respond to this message: "I've been feeling tired lately." (Emotion: tiredness)

[1396] The generative AI generates a response such as "Why not try a relaxing massage chair?", which is then displayed on the smart glasses or HMD.

[1397] This allows users to receive appropriate product and service recommendations in the virtual store through natural dialogue based on their emotions, improving the user experience and significantly increasing satisfaction in the virtual store.

[1398] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1399] Step 1:

[1400] The terminal receives messages from the user as input. The user uses smart glasses or a head-mounted display (HMD) to input text or voice messages in natural language. This message contains information including the user's wishes and emotions expressed to the system. The terminal stores the message as is and prepares it for transmission to the server.

[1401] Step 2:

[1402] The device sends the message entered by the user to the server as an HTTP POST request, along with any additional information related to the message (e.g., character selection). The entered data is passed to the server through the interface of the smart glasses or HMD.

[1403] Step 3:

[1404] The server receives messages sent from the devices as input. To analyze these messages, the server first sends them to the emotion engine. The emotion engine analyzes the messages and generates emotion tags based on their content. Emotion tags express the emotions contained in the messages and are generated using natural language processing techniques on the message text. The resulting emotion tags are then returned to the server.

[1405] Step 4:

[1406] The server generates a prompt based on the received emotion tag and the user message. The generated prompt is in the format "You are a virtual shop assistant, respond to this message: 'Message content' (Emotion: emotion tag)". This prompt is used as an instruction for the generative AI, and by incorporating the emotion tag, it provides input data for generating a more appropriate response.

[1407] Step 5:

[1408] The server sends the prompt to a generative AI, which takes the prompt as input and generates an appropriate natural language response based on its content. The generative AI then analyzes the prompt using large-scale language models and machine learning algorithms, and outputs an appropriate response to the user's message. This output response is then sent back to the server.

[1409] Step 6:

[1410] The server formats the response received from the generative AI, returns it to the device in its final form, and in some cases deletes unnecessary information during the formatting process. The response is then sent back to the device as an HTTP POST request.

[1411] Step 7:

[1412] The device displays the response received from the server. The received response message is visually displayed on the display of the smart glasses or head-mounted display. This allows the user to receive product and service suggestions based on their emotions through natural dialogue in real time.

[1413] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1414] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1415] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1416] [Fourth embodiment]

[1417] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1418] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1419] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1420] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1421] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1422] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1423] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1424] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1425] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1426] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1427] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1428] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1429] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1430] This invention relates to a system that allows users to enjoy conversations with characters, idols, and actors. Specific operations and processing flows of the server, terminal, and user in the embodiment of the invention will be described below.

[1431] Server Roles and Program Processing

[1432] First, the server is built using a web application framework such as Flask. The server receives a message and character name from the user, generates a response by sending it as a prompt to the generative AI (e.g., OpenAI's API), and then sends the generated response back to the user.

[1433] Specific working example:

[1434] The server uses a web application framework to define a / chat endpoint to handle HTTP POST requests.

[1435] Upon receiving a request from a user, the server extracts the message and character name from the request body.

[1436] The server generates a prompt of the form "You are [character name], respond to this message: [user message]".

[1437] The server sends this prompt to the generative AI and receives a response.

[1438] The server formats the response and sends it back to the user.

[1439] Terminal roles and program processing

[1440] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[1441] Specific working example:

[1442] The user opens the chat interface on the device and types a message.

[1443] The user selects a particular character from a drop-down menu within the chat interface.

[1444] When the user presses the "Send" button, the terminal sends an HTTP POST request containing the message and the character name to the server.

[1445] The terminal that receives the response from the server displays the response in the chat interface.

[1446] User roles and examples

[1447] Users can easily use this system to enjoy conversations with specific characters, idols, and actors. The operations performed by users are very simple.

[1448] Specific working example:

[1449] Suppose the user inputs the message "How are you feeling today?" and selects "Hatsune Miku" as the character.

[1450] When the user presses the send button, the device sends the message "How are you feeling today?" and the character name "Hatsune Miku" to the server.

[1451] The server sends a prompt to the generative AI, which generates the text in response: "I'm feeling great today! I'm even working on a new song!"

[1452] Responses are displayed on the terminal, allowing the user to experience a natural and enjoyable conversation.

[1453] This allows users to seamlessly enjoy interactions with specific characters, idols, and actors. In addition, server-side processing is performed automatically, providing a stress-free experience for users.

[1454] The processing flow will be explained below.

[1455] Step 1:

[1456] The user opens the chat interface on the terminal and types a message.

[1457] Specific behavior:

[1458] The user types a message into a text box in the chat interface.

[1459] The user selects a particular character from a drop-down menu or the like.

[1460] Step 2:

[1461] The user presses the "Send" button to send the entered message and selected character name.

[1462] Specific behavior:

[1463] The user presses the "send" button.

[1464] The terminal packages the entered message and the selected character name to form an HTTP POST request.

[1465] Step 3:

[1466] The device sends the constructed HTTP POST request to the server.

[1467] Specific behavior:

[1468] The device sends a request to the server's / chat endpoint, including the user's message and character name.

[1469] Step 4:

[1470] The server receives the HTTP POST request and extracts the message and character name from the request body.

[1471] Specific behavior:

[1472] The server receives requests through the Flask application at the / chat endpoint.

[1473] The server parses the request body and extracts the message and character name.

[1474] Step 5:

[1475] The server generates a prompt for the generative AI based on the user's message and the character name.

[1476] Specific behavior:

[1477] The server creates a prompt saying "You are [character name], respond to this message: [user message]".

[1478] Prepare a request to send the generated prompt to the generative AI API.

[1479] Step 6:

[1480] The server sends prompts to the generative AI, causing it to generate a response.

[1481] Specific behavior:

[1482] The server calls a generative AI API such as OpenAI and sends the prepared prompts.

[1483] The server waits for the generative AI to generate a response.

[1484] Step 7:

[1485] The server receives the response from the generative AI and formats it as necessary.

[1486] Specific behavior:

[1487] The server analyzes the response data received from the generative AI.

[1488] Remove unnecessary whitespace and newlines from the response and format the text.

[1489] Step 8:

[1490] The server sends a formatted response back to the terminal.

[1491] Specific behavior:

[1492] The server generates a JSON object containing the formatted response.

[1493] The generated JSON object is sent to the terminal as an HTTP response.

[1494] Step 9:

[1495] The terminal receives the response from the server and displays it to the user.

[1496] Specific behavior:

[1497] The terminal parses the JSON data received from the server.

[1498] The analysis results are displayed in the chat interface so that the user can check them.

[1499] Step 10:

[1500] The user checks the response displayed on the terminal and continues the conversation by entering the next message.

[1501] Specific behavior:

[1502] The user reads the response displayed on the terminal and enters a new message.

[1503] If necessary, select a different character and submit again.

[1504] Example 1

[1505] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1506] Providing a system that allows users to seamlessly and enjoy real-time interactions with unrealistic characters or celebrities is an important challenge in the entertainment industry. There is also a need for an interface that allows users to easily select specific characters and enjoy natural interactions. However, current dialogue systems require complex configurations to achieve the desired dialogue experience, and are not intuitive.

[1507] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1508] In this invention, the server includes a means for receiving a message from a user, a means for generating a prompt including the received message and character information and sending it to the generative AI, and a means for returning the generated response to the user, thereby enabling the user to intuitively enjoy a conversation with a specific character or celebrity.

[1509] A "message" is text data that indicates the content of a conversation entered by a user.

[1510] "Character information" is data about the particular character that the user selects to interact with.

[1511] A "prompt" is input text sent to a generative AI to guide it to a specific output.

[1512] "Generative AI" is a system that uses artificial intelligence technology to generate responses to input text.

[1513] A "server" is a computer system whose role is to receive messages and character information from the user, send prompts to the generative AI, obtain responses, and return them to the user.

[1514] A "user interface" is a part of the software that provides a screen and operating means for the user to input messages and select character information.

[1515] A "web application framework" is a set of software tools for building web-based user interfaces.

[1516] A "response" is a reply text that the generative AI generates based on a prompt.

[1517] An "HTTP POST request" is a protocol-based request method for sending data to a server over the Internet.

[1518] A "terminal" is a computing or mobile device that allows a user to enter and send messages to a server.

[1519] This invention relates to a system that allows users to enjoy conversations with specific characters or celebrities. To implement this invention, the server, terminal, and user each have specific roles and work in cooperation with each other.

[1520] Server configuration and roles

[1521] The server is built using a web application framework such as Flask. The server receives messages and character information sent by the user and generates prompts for the generative AI model based on that information. It then sends the prompts to the generative AI model (e.g., OpenAI's API) and returns the responses it receives to the user.

[1522] Specific working example:

[1523] The server uses Flask to define a / chat endpoint and handles HTTP POST requests.

[1524] Upon receiving a request from a user, the server extracts the message and character information from the request body.

[1525] The server generates a prompt of the form "You are [character name], respond to this message: [user message]".

[1526] The server sends this prompt to the generative AI and receives a response.

[1527] The server formats the response and sends it back to the user.

[1528] Examples of prompts:

[1529] Prompt: "You are Hatsune Miku, respond to this message: How are you feeling today?"

[1530] Example response: "I'm feeling great today! I'm even working on some new music!"

[1531] Device configuration and role

[1532] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[1533] Specific working example:

[1534] The user opens the chat interface on the device and types a message.

[1535] The user selects a particular character from a drop-down menu within the chat interface.

[1536] When the user presses the "Send" button, the terminal sends an HTTP POST request containing the message and the character name to the server.

[1537] The terminal that receives the response from the server displays the response in the chat interface.

[1538] User operations

[1539] Users can easily use this system to enjoy conversations with specific characters or celebrities. The operation is very intuitive and does not require complicated settings.

[1540] Specific working example:

[1541] The user inputs the message "How are you feeling today?" and selects "Hatsune Miku" as the character.

[1542] When the user presses the send button, the device sends the message "How are you feeling today?" and the character name "Hatsune Miku" to the server.

[1543] The server sends a prompt to the generative AI, which generates the text in response: "I'm feeling great today! I'm even working on a new song!"

[1544] The response is displayed on the terminal, allowing the user to enjoy natural dialogue.

[1545] In this way, the system of the present invention allows users to interact with characters and celebrities in real time, creating a seamless experience.

[1546] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1547] Step 1:

[1548] User enters message and selects character

[1549] Users access the chat interface, type the message they want to communicate, and then select the character they want to interact with from a drop-down menu.

[1550] input:

[1551] Message: "How are you feeling today?"

[1552] Character name: "Hatsune Miku"

[1553] output:

[1554] Information that the send button has been pressed is passed to the terminal.

[1555] Specific behavior:

[1556] The user opens the chat interface.

[1557] The user types a message: "How are you feeling today?"

[1558] Select "Hatsune Miku" from the drop-down menu.

[1559] Step 2:

[1560] The device sends a request to the server

[1561] When the user presses the send button, the terminal sends the entered message and the selected character name to the server as an HTTP POST request.

[1562] input:

[1563] The event of the submit button pressed by the user

[1564] Entered message and character name

[1565] output:

[1566] An HTTP POST request is sent to the server.

[1567] Specific behavior:

[1568] The user presses the "Send" button.

[1569] The device sends an HTTP POST request to the server containing the message "How are you feeling today?" and the character name "Hatsune Miku."

[1570] Step 3:

[1571] The server receives the request and generates a prompt.

[1572] The server processes the request received from the device, extracts the message and character name from the request body, and then creates a prompt for the generative AI model.

[1573] input:

[1574] Received HTTP POST request

[1575] output:

[1576] Generated prompt statement

[1577] Specific behavior:

[1578] The server receives an HTTP POST request.

[1579] The server extracts the message "How are you feeling today?" and the character name "Hatsune Miku" from the request body.

[1580] The server generates the prompt "You are Hatsune Miku, respond to this message: How are you feeling today?"

[1581] Step 4:

[1582] The server sends prompts to the generative AI and gets a response

[1583] The server sends a prompt to the generative AI (e.g., OpenAI API) and receives the generated response.

[1584] input:

[1585] Generated prompt statement

[1586] output:

[1587] Response text from generative AI

[1588] Specific behavior:

[1589] The server sends the generative AI a prompt: "You are Hatsune Miku, respond to this message: How are you feeling today?"

[1590] The generative AI generates a response such as "I'm feeling great today! I'm even working on a new song!" and sends it back to the server.

[1591] Step 5:

[1592] The server formats the response it receives and sends it back to the device.

[1593] The server formats the response received from the generative AI, makes it easy for the user to understand, and sends it back to the device.

[1594] input:

[1595] Response text from generative AI

[1596] output:

[1597] Formatted response text

[1598] Specific behavior:

[1599] The server receives the generated response "I'm feeling great today! I'm even working on a new song!"

[1600] The server formats the response and sends it back to the device as an HTTP response.

[1601] Step 6:

[1602] The device receives the response from the server and displays it to the user.

[1603] The terminal receives the response from the server and displays it in the chat interface.

[1604] input:

[1605] HTTP response from the server

[1606] output:

[1607] The displayed response text

[1608] Specific behavior:

[1609] The device receives a response saying, "I'm feeling great today! I'm even working on a new song!"

[1610] The terminal displays the received response on the chat interface and notifies the user.

[1611] The user checks the response and enjoys the interaction.

[1612] (Application example 1)

[1613] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1614] To improve the effectiveness of advertising, interactive communication with users is required. However, with conventional advertising methods, users only passively receive advertising content, and two-way communication is insufficient. It is also difficult for users to obtain information tailored to their own interests in real time. This has resulted in limited advertising effectiveness and low user engagement.

[1615] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1616] In this invention, the server includes a means for receiving a message from a user, a means for sending the received message to a generative AI model to generate a response, a means for returning the generated response to the user, and a means for generating and sending a prompt including advertising character information. This allows users to obtain information about new products in real time through interactions with specific advertising characters or idols. This enables two-way communication with users, improving advertising effectiveness and user engagement.

[1617] "Means for receiving messages from users" refers to the interface for receiving messages entered by users, which is mainly realized through web applications or mobile applications.

[1618] "Means for sending received messages to a generative AI model to generate a response" refers to the process of passing received messages to a generative AI model and generating an appropriate response. Specifically, this is the part that calls the AI's API and receives the generated response.

[1619] "Means for returning the generated response to the user" refers to the process for displaying the response obtained from the generative AI model to the user. This information is typically returned to the user via a web browser or smartphone application.

[1620] "Means for generating and transmitting prompts including advertising character information" refers to a process for generating prompts for a user based on a particular advertising character and transmitting the prompts to a generative AI model, which allows the character to interact with the user.

[1621] "Means for providing new product information" refers to the process of using a generative AI model to provide related new product information based on messages from users. Specifically, it provides users with detailed information and explanations related to new products.

[1622] "Means for utilizing a web application framework to provide an advertising interface" means a process for utilizing a web application framework to provide an interface that allows users to interact with advertisements in an interactive manner, thereby enabling users to easily send messages and receive responses.

[1623] A "generative AI model" is an AI that generates appropriate responses based on prompts from the user. Specifically, it is an AI model that can learn from a large amount of data and generate natural language.

[1624] A "prompt sentence" is a sentence used to generate a response from a generative AI model. It is written in a format that includes specific character information and a message from the user.

[1625] MODE FOR CARRYING OUT THE INVENTION

[1626] This invention is a system that allows users to obtain information about new products in real time through conversations with advertising characters and idols. The specific operations of the server, terminal, and user are described below.

[1627] Server Roles and Program Processing

[1628] The server is built using a web application framework such as Flask. It receives messages and advertising character information from users, generates responses by sending them as prompts to a generative AI model (such as the OpenAI API), and then sends the generated responses back to the user.

[1629] Specific working example:

[1630] The server uses a web application framework to define a / chat endpoint to handle HTTP POST requests.

[1631] Upon receiving a request from a user, the server extracts the message and advertising character information from the request body.

[1632] The server generates a prompt in the form "You are [advertising character name], introduce and explain the new product: [user message]".

[1633] The server sends this prompt to the generative AI model and receives a response.

[1634] The server formats the response and sends it back to the user.

[1635] Terminal roles and program processing

[1636] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[1637] Specific working example:

[1638] The user opens the advertising interface on the terminal and enters a message.

[1639] The user selects a particular advertising character from a drop-down menu within the advertising interface.

[1640] When the user presses the "Send" button, the terminal sends an HTTP POST request including the message and advertising character information to the server.

[1641] The terminal that receives the response from the server displays the response on the advertising interface.

[1642] User roles and examples

[1643] Users can easily use this system to enjoy interacting with specific advertising characters and idols. The operations performed by users are very simple.

[1644] Specific working example:

[1645] Suppose a user inputs the message "How is the battery life of this smartphone?" and selects "Ambassador for home appliances" as the advertising character.

[1646] When the user presses the send button, the device sends the message "How is the battery life of this smartphone?" along with character information about the "home appliance ambassador" to the server.

[1647] The server sends a prompt to the generative AI model, which generates the following text in response: "This smartphone has up to 12 hours of battery life and fast charging."

[1648] The responses are displayed on the terminal, providing the user with detailed and interesting product information in real time.

[1649] Prompt Sentence Examples

[1650] If the user selects an ambassador for a consumer electronics product and asks about smartphone battery life, the prompt might look like this:

[1651] "You are an ambassador for a consumer electronics product. You introduce and explain the new product: How is the battery life of this smartphone?"

[1652] This invention allows users to obtain information about new products while interactively communicating with specific advertising characters or idols, which is expected to improve advertising effectiveness and user engagement.

[1653] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1654] Step 1:

[1655] The user opens the advertising interface on the device and inputs a message. The input message and character information are sent to the server in the form of an HTTP POST request when the user presses the send button.

[1656] Input: User-entered message and selected character information

[1657] Output: Sends message and character information to the server as an HTTP POST request

[1658] Specific operation: The user enters the message "How is the battery life of this smartphone?" and selects "Home appliance ambassador" as the character.

[1659] Step 2:

[1660] The server receives the HTTP POST request and extracts the user's message and character information from the request body.

[1661] Input: HTTP POST request

[1662] Output: Message and character information extracted from the request body

[1663] Specific operation: The server extracts the message "How is the battery life of this smartphone?" and the character information "Ambassador for home appliances" from the received request.

[1664] Step 3:

[1665] The server generates a prompt sentence to send to the generative AI model based on the extracted message and character information.

[1666] Input: Extracted message and character information

[1667] Output: Generated prompt statement

[1668] Specific operation: The server generates the prompt sentence, "You are an ambassador for home appliances, introduce and explain the new product: How is the battery life of this smartphone?"

[1669] Step 4:

[1670] The server sends the generated prompt sentence to the generative AI model, which then generates a response.

[1671] Input: Generated prompt text

[1672] Output: Response from the generative AI model

[1673] Specific operation: The server calls the OpenAI API, sends a prompt, and receives a response.

[1674] Step 5:

[1675] The server formats the response received from the generative AI model and prepares it to be sent back to the user.

[1676] Input: Response from a generative AI model

[1677] Output: The formatted response

[1678] Specific behavior: The server formats the response it receives: "This smartphone has a battery life of up to 12 hours and also has fast charging capabilities."

[1679] Step 6:

[1680] The server returns the formatted response to the user's terminal as an HTTP response.

[1681] Input: Formatted response

[1682] Output: Returned to the user's device as an HTTP response

[1683] Specific operation: The server sends the formatted response to the user's terminal as an HTTP response.

[1684] Step 7:

[1685] The user's device receives the HTTP response from the server and displays the response in the advertising interface.

[1686] Input: HTTP response from the server

[1687] Output: Response displayed in the advertising interface

[1688] Specific behavior: The user's device displays the response "This smartphone has up to 12 hours of battery life and fast charging capabilities" in the advertising interface.

[1689] The operation of the user, terminal, and server at each step is explained in detail, and specific inputs and outputs are shown, making it clear how the invention is implemented.

[1690] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1691] The present invention relates to a system that allows users to enjoy conversations with characters, idols, and actors, and in particular, by combining an emotion engine that recognizes the user's emotions, it is possible to generate more natural responses that correspond to the emotions. The specific operations and processing flows of the server, terminal, and user for the embodiments of this invention are described below.

[1692] Server Roles and Program Processing

[1693] The server is built using a web application framework such as Flask. The role of the server is to receive messages from users, analyze the messages with an emotion engine to generate emotion tags, send prompts to the generative AI to generate responses, and return the generated responses to the user.

[1694] Specific working example:

[1695] The server uses a web application framework to define a / chat endpoint to handle HTTP POST requests.

[1696] Upon receiving a request from a user, the server extracts the message and character name from the request body.

[1697] The server uses an emotion engine to generate emotion tags from the extracted messages.

[1698] The server creates a prompt in the form of "You are [character name], respond to this message: [user's message (with emotion tag)]" and sends it to the generative AI.

[1699] The server receives the response from the generative AI, formats it as necessary, and sends it back to the user.

[1700] Terminal roles and program processing

[1701] The terminal provides an interface for users to input messages and send them to the server. The terminal can be implemented as a web browser or a mobile application.

[1702] Specific working example:

[1703] The user opens the chat interface on the terminal and types a message.

[1704] The user selects a particular character from a drop-down menu within the chat interface.

[1705] When the user presses the "Send" button, the terminal sends an HTTP POST request containing the message and the character name to the server.

[1706] The terminal that receives the response from the server displays the response in the chat interface.

[1707] User roles and examples

[1708] Users can easily use this system to enjoy conversations with specific characters, idols, and actors. The operations performed by users are very simple.

[1709] Specific working example:

[1710] Suppose the user inputs the message "How are you feeling today?" and selects "Hatsune Miku" as the character.

[1711] When the user presses the send button, the device sends the message "How are you feeling today?" and the character name "Hatsune Miku" to the server.

[1712] The server receives the message, analyzes the message with an emotion engine, and generates an emotion tag, such as "positive."

[1713] The server sends a prompt to the generative AI, tagging it with the emotion "positive" and asking it to generate a response. The generative AI might generate the response "I'm feeling great today! I'm even working on a new song!"

[1714] Responses are displayed on the terminal, allowing the user to experience a natural and emotionally appropriate conversation.

[1715] This allows users to seamlessly enjoy conversations with specific characters, idols, and actors. In addition, by using an emotion engine, responses are made natural according to the user's emotions, providing a more satisfying conversational experience.

[1716] The processing flow will be explained below.

[1717] Step 1:

[1718] The user opens the chat interface on the terminal and types a message.

[1719] Specific behavior:

[1720] The user types a message into a text box in the chat interface.

[1721] The user selects a particular character from a drop-down menu or the like.

[1722] Step 2:

[1723] The user presses the "Send" button to send the entered message and selected character name.

[1724] Specific behavior:

[1725] The user presses the "send" button.

[1726] The terminal packages the entered message and the selected character name to form an HTTP POST request.

[1727] Step 3:

[1728] The device sends the constructed HTTP POST request to the server.

[1729] Specific behavior:

[1730] The device sends a request to the server's / chat endpoint, including the user's message and character name.

[1731] Step 4:

[1732] The server receives the HTTP POST request and extracts the message and character name from the request body.

[1733] Specific behavior:

[1734] The server receives requests through the Flask application at the / chat endpoint.

[1735] The server parses the request body and extracts the message and character name.

[1736] Step 5:

[1737] The server sends the extracted message to the emotion engine, which generates emotion tags.

[1738] Specific behavior:

[1739] The server requests the emotion engine to analyze the message.

[1740] The emotion engine analyzes the message and generates emotion tags such as "positive" or "negative."

[1741] The server receives emotion tags from the emotion engine.

[1742] Step 6:

[1743] The server generates prompts for the generative AI based on the user's message and emotion tag.

[1744] Specific behavior:

[1745] The server generates a prompt of the form "You are [character name], respond to this message with the emotion [emotion tag]: [user message]".

[1746] Prepare a request to send the generated prompt to the generative AI API.

[1747] Step 7:

[1748] The server sends prompts to the generative AI, causing it to generate a response.

[1749] Specific behavior:

[1750] The server calls a generative AI API such as OpenAI and sends the prepared prompts.

[1751] The server waits for the generative AI to generate a response.

[1752] Step 8:

[1753] The server receives the response from the generative AI and formats it as necessary.

[1754] Specific behavior:

[1755] The server analyzes the response data received from the generative AI.

[1756] Remove unnecessary whitespace and newlines from the response and format the text.

[1757] Step 9:

[1758] The server sends a formatted response back to the terminal.

[1759] Specific behavior:

[1760] The server generates a JSON object containing the formatted response.

[1761] The generated JSON object is sent to the terminal as an HTTP response.

[1762] Step 10:

[1763] The terminal receives the response from the server and displays it to the user.

[1764] Specific behavior:

[1765] The terminal parses the JSON data received from the server.

[1766] The analysis results are displayed in the chat interface so that the user can check them.

[1767] Step 11:

[1768] The user checks the response displayed on the terminal and continues the conversation by entering the next message.

[1769] Specific behavior:

[1770] The user reads the response displayed on the terminal and enters a new message.

[1771] If necessary, select a different character and submit again.

[1772] Example 2

[1773] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1774] Conventional dialogue systems do not properly recognize the user's emotions and simply respond to input messages. This results in unnatural responses and a poor user experience. Furthermore, they lack a mechanism for naturally engaging in dialogue with specific characters or idols, limiting their appeal to users. Furthermore, they lack the technology to generate responses based on emotions, making it impossible to realize conversations that are sensitive to the user's emotions.

[1775] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving a message from a user, a means for analyzing the received message with an emotion engine and generating an emotion tag, a means for sending the emotion-tagged message to a generative AI model to generate a response, and a means for returning the generated response to the user. This makes it possible to recognize emotions in response to messages from users and generate appropriate and natural responses. Furthermore, it is possible to have conversations with specific characters or idols according to emotions, thereby improving the user experience.

[1776] A "user" is a person who utilizes a dialogue system to input messages and receive responses.

[1777] A "message" is text data entered by a user and received by a server.

[1778] "Emotion engine" is a general term for algorithms and programs that analyze received messages and generate emotion tags from their contents.

[1779] "Emotion tags" are labels generated by the emotion engine that indicate the emotional characteristics of a message. Examples include "positive" and "negative."

[1780] A "generative AI model" is an artificial intelligence model that generates natural language responses based on received prompts.

[1781] A "prompt" is input information given to a generative AI model to generate a response. It includes character information and emotion tags.

[1782] The "server" is a computer system that receives messages from users, analyzes them using an emotion engine, sends prompts to a generative AI model, and executes a series of operations to generate a response.

[1783] "Character information" refers to information about a specific character, idol, or actor selected by the user.

[1784] A "web application framework" is a software framework for building applications that run on the web. Examples include Flask.

[1785] A "response" is the text data that a generative AI model generates based on a prompt and sends back to the user.

[1786] The present invention relates to a system that allows users to enjoy conversations with specific characters, idols, or actors. In particular, it utilizes a combination of an emotion engine and a generative AI model to recognize the user's emotions and generate natural responses accordingly. The following describes the embodiments of the invention based on the scope of the claims.

[1787] 1. Specific Embodiment of the Server

[1788] The server is built using Flask as a web application framework. The main roles of the server are as follows:

[1789] First, the server receives a message from the user. Next, it analyzes the received message using the emotion engine and generates an emotion tag. Based on the generated emotion tag, it integrates it with character information to generate a prompt and sends it to the generative AI model. Then it receives a response from the generative AI model and sends it back to the user.

[1790] As a specific use case, if a user types the message "How are you feeling today?" and selects "Hatsune Miku" as the character name, this message is sent to the server. The server uses its emotion engine to analyze the message and generate a "positive" emotion tag. It then sends the following prompt to the generative AI model:

[1791] "You are Hatsune Miku, respond to this message: How are you feeling today? (Positive)"

[1792] Based on this prompt, the generative AI model generates a response such as, "I'm feeling great today! I'm even working on a new song!", which the server sends back to the user.

[1793] 2. Specific Embodiments of the Terminal

[1794] The terminal provides an interface for users to input messages and send them to the server, using a web browser or a mobile application. The terminal's main roles are as follows:

[1795] First, the user enters a message using the chat interface. Then, they select a specific character from a drop-down menu. When the user presses the "Send" button, the device sends an HTTP POST request to the server containing the message and the character's name. The response is then received from the server and displayed in the chat interface.

[1796] 3. User's specific embodiment

[1797] Users can enjoy the interaction by performing extremely simple operations. The specific steps are shown below.

[1798] First, the user types a message in the chat interface, such as "How are you feeling today?", and then selects "Hatsune Miku" from the drop-down menu. When the user presses the "Send" button, the message and the character name are sent to the server. The server then responds with "I'm feeling great today! I'm even working on a new song!", which is displayed in the chat interface, allowing the user to enjoy the conversation.

[1799] As described above, this invention combines an emotion engine with a generative AI model to generate natural responses that correspond to the user's emotions, making it possible to seamlessly enjoy conversations with specific characters, idols, or actors.

[1800] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1801] Step 1: The user types a message into the chat interface

[1802] The user opens the chat interface on the device and types "How are you feeling today?" into the text field. Input: The message "How are you feeling today?" typed by the user. Output: The text that appears in the message input field on the device display.

[1803] Step 2: User selects a character

[1804] The user selects "Hatsune Miku" from a drop-down menu in the chat interface. Input: The character name "Hatsune Miku". Output: The selection displayed in the drop-down menu.

[1805] Step 3: User presses "Submit" button

[1806] The user presses the "Send" button, triggering the sending of the message and character name to the server. Input: Click on the "Send" button. Output: HTTP POST request containing the message and character name.

[1807] Step 4: The device sends an HTTP POST request to the server

[1808] The device sends an HTTP POST request to the server containing the user's message "How are you feeling today?" and the character name "Hatsune Miku". Input: Message and character name. Output: HTTP request sent to the server.

[1809] Step 5: The server receives the message

[1810] The server receives the HTTP POST request and extracts the message "How are you feeling today?" and the character name "Hatsune Miku" from the request body. Input: HTTP POST request body. Output: Extracted message and character name.

[1811] Step 6: The server parses the message using the emotion engine

[1812] The server passes the message "How are you feeling today?" to the emotion engine and generates the emotion tag "Positive". Input: The extracted message. Output: The generated emotion tag.

[1813] Step 7: The server creates a prompt and sends it to the generative AI model

[1814] The server generates a prompt, "You are Hatsune Miku, respond to this message: How are you feeling today? (Positive)," and sends it to the generative AI model. Input: Message, character name, emotion tag. Output: Prompt sent to the generative AI model.

[1815] Step 8: The generative AI model generates a response

[1816] Based on the prompt, the generative AI model generates a response such as "I'm feeling great today! I'm even working on a new song!". Input: Prompt. Output: Generated response.

[1817] Step 9: The server receives and formats the generated response

[1818] The server receives the response from the generative AI model, "I'm feeling great today! I'm even working on a new song!", and formats it as needed. Input: Generated response. Output: Formatted response.

[1819] Step 10: The server sends the formatted response back to the user

[1820] The server sends the formatted response back to the device as an HTTP response. Input: Formatted response. Output: Response sent to the user's device.

[1821] Step 11: The device receives the response from the server and displays it in the chat interface.

[1822] The device receives the response from the server, "I'm feeling great today! I'm even working on a new song!", and displays it in the chat interface. Input: Response from the server. Output: Response displayed in the chat interface.

[1823] Step 12: User checks the response and enjoys the interaction

[1824] The user can see the responses displayed on the chat interface of the device and enjoy natural dialogue with the character. Input: Responses displayed on the device. Output: Improved user dialogue experience.

[1825] (Application example 2)

[1826] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1827] Conventional dialogue systems have difficulty generating responses that accurately reflect the user's emotions, resulting in a limited user experience. Furthermore, when proposing products or recommending services in virtual stores, they often fail to provide appropriate suggestions that take the user's emotions into account. This has led to issues such as insufficient improvement of user satisfaction and insufficient stimulation of purchasing motivation.

[1828] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1829] In this invention, the server includes means for receiving a message from a user, means for analyzing the received message with an emotion engine and generating an emotion tag, means for generating a prompt with an emotion tag and sending it to a generative AI to generate a response, means for returning the generated response to the user, and means for providing a user interface using smart glasses or a head-mounted display. This enables more natural dialogue according to the user's emotions and makes it possible to propose products and services suitable for the user in a virtual store.

[1830] A "user" is a person who uses the system to input messages and receive proposals for services and products.

[1831] A "message" is text or voice data that a user inputs to the system.

[1832] An "emotion engine" is a software application that analyzes a user's message, identifies the emotion contained in the message, and generates an emotion tag.

[1833] An "emotion tag" is a label that represents an emotion and is generated by the emotion engine and assigned to a user's message.

[1834] A "prompt" is an instruction given to a generative AI to generate a response, and may include emotion tags and character information.

[1835] "Generative AI" is artificial intelligence that generates natural language responses in response to given prompts.

[1836] A "response" is text or audio data that a generative AI generates based on a prompt and sends back to the user.

[1837] "Smart glasses" are wearable devices with internet connectivity and other computing capabilities that are used to display information in the user's field of vision.

[1838] A "head-mounted display" is a device worn on the user's head that displays information in the user's field of vision.

[1839] "User interface" is a term that refers to an interface such as a screen, display, or input device that allows a user to interact with a system.

[1840] A "virtual store" is a virtual store built on the Internet where users can browse and purchase products online.

[1841] "Suggestion" refers to the act of selecting products or services suitable for a user in a virtual store and presenting them to the user.

[1842] This invention is a system that realizes customer service in a virtual store, and in particular enables natural dialogue and product suggestions that correspond to the user's emotions. The system is mainly composed of three entities: a server, a terminal, and a user.

[1843] Server Roles and Program Processing

[1844] The server is built using a web application framework such as Flask. The role of the server is to receive messages from users, analyze the messages with an emotion engine to generate emotion tags, create prompts and send them to the generative AI, and then send the generated responses back to the user.

[1845] The server does the following:

[1846] Receives a message from the user as an HTTP POST request.

[1847] The received message is analyzed by an emotion engine and an emotion tag is generated. Emotion engines are often provided as external APIs.

[1848] A prompt containing the generated emotion tag is created and sent to the generative AI in the format "You are a virtual shop assistant, respond to this message: 'Message content' (Emotion: emotion tag)".

[1849] Receives responses from the generative AI and displays them to the user via the user interface.

[1850] Terminal roles and program processing

[1851] The terminal provides a user interface using smart glasses or a head-mounted display (HMD). The terminal's role is to allow the user to input messages and send them to the server, and to visually present the responses sent back from the server to the user.

[1852] The terminal does the following:

[1853] The user types a message through smart glasses or an HMD.

[1854] Provides an interface for sending input messages to the server.

[1855] Responses from the server are displayed in real time on the user interface.

[1856] User roles and examples

[1857] Users can easily use the system to enjoy conversations in a virtual store. The user's operations are very simple: they simply input messages through smart glasses or an HMD and receive responses from the server.

[1858] Specific working example:

[1859] Let's say a user types a message like "I've been feeling tired lately." The user sends this message to the system using smart glasses or an HMD. The server receives this message, analyzes it with its emotion engine, and generates an emotion tag called "tired." It then sends the following prompt to the generative AI:

[1860] You are a virtual shop assistant, respond to this message: "I've been feeling tired lately." (Emotion: tiredness)

[1861] The generative AI generates a response such as "Why not try a relaxing massage chair?", which is then displayed on the smart glasses or HMD.

[1862] This allows users to receive appropriate product and service recommendations in the virtual store through natural dialogue based on their emotions, improving the user experience and significantly increasing satisfaction in the virtual store.

[1863] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1864] Step 1:

[1865] The terminal receives messages from the user as input. The user uses smart glasses or a head-mounted display (HMD) to input text or voice messages in natural language. This message contains information including the user's wishes and emotions expressed to the system. The terminal stores the message as is and prepares it for transmission to the server.

[1866] Step 2:

[1867] The device sends the message entered by the user to the server as an HTTP POST request, along with any additional information related to the message (e.g., character selection). The entered data is passed to the server through the interface of the smart glasses or HMD.

[1868] Step 3:

[1869] The server receives messages sent from the devices as input. To analyze these messages, the server first sends them to the emotion engine. The emotion engine analyzes the messages and generates emotion tags based on their content. Emotion tags express the emotions contained in the messages and are generated using natural language processing techniques on the message text. The resulting emotion tags are then returned to the server.

[1870] Step 4:

[1871] The server generates a prompt based on the received emotion tag and the user message. The generated prompt is in the format "You are a virtual shop assistant, respond to this message: 'Message content' (Emotion: emotion tag)". This prompt is used as an instruction for the generative AI, and by incorporating the emotion tag, it provides input data for generating a more appropriate response.

[1872] Step 5:

[1873] The server sends the prompt to a generative AI, which takes the prompt as input and generates an appropriate natural language response based on its content. The generative AI then analyzes the prompt using large-scale language models and machine learning algorithms, and outputs an appropriate response to the user's message. This output response is then sent back to the server.

[1874] Step 6:

[1875] The server formats the response received from the generative AI, returns it to the device in its final form, and in some cases deletes unnecessary information during the formatting process. The response is then sent back to the device as an HTTP POST request.

[1876] Step 7:

[1877] The device displays the response received from the server. The received response message is visually displayed on the display of the smart glasses or head-mounted display. This allows the user to receive product and service suggestions based on their emotions through natural dialogue in real time.

[1878] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1879] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1880] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1881] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1882] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1883] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1884] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1885] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1886] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1887] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1888] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1889] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1890] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1891] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1892] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1893] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1894] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1895] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1896] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1897] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1898] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1899] The following is further disclosed regarding the above embodiment.

[1900] (Claim 1)

[1901] means for receiving a message from a user;

[1902] A means for sending the received message to a generative AI to generate a response;

[1903] means for returning the generated response to the user;

[1904] A system including:

[1905] (Claim 2)

[1906] 2. The system of claim 1, further comprising means for generating a prompt including character information and transmitting the prompt to the generative AI.

[1907] (Claim 3)

[1908] 10. The system of claim 1, further comprising means for providing a user interface utilizing a web application framework.

[1909] "Example 1"

[1910] (Claim 1)

[1911] means for receiving a message from a user;

[1912] a means for generating a prompt including the received message and character information and transmitting the prompt to the generative AI;

[1913] means for returning the generated response to the user;

[1914] A system including:

[1915] (Claim 2)

[1916] 10. The system of claim 1, further comprising means for providing a user interface utilizing a web application framework.

[1917] (Claim 3)

[1918] 10. The system of claim 1, further comprising means for providing an interface for a user to input a message and select a character name.

[1919] "Application Example 1"

[1920] (Claim 1)

[1921] means for receiving a message from a user;

[1922] a means for transmitting the received message to a generative AI model to generate a response;

[1923] means for returning the generated response to the user;

[1924] means for generating and transmitting a prompt including advertising character information;

[1925] A system including:

[1926] (Claim 2)

[1927] 2. The system according to claim 1, further comprising means for providing information on new products based on the received message.

[1928] (Claim 3)

[1929] 10. The system of claim 1, further comprising means for providing an advertising interface utilizing a web application framework.

[1930] "Example 2: Combining Emotion Engines"

[1931] (Claim 1)

[1932] means for receiving a message from a user;

[1933] A means for analyzing received messages with an emotion engine to generate emotion tags;

[1934] a means of sending emotion-tagged messages to a generative AI model to generate a response; and

[1935] means for returning the generated response to the user;

[1936] A system including:

[1937] (Claim 2)

[1938] 10. The system of claim 1, further comprising means for generating and transmitting a prompt including character information and an emotion tag to the generative AI model.

[1939] (Claim 3)

[1940] 10. The system of claim 1, further comprising means for providing a user interface utilizing a web application framework.

[1941] "Application example 2 when combining emotion engines"

[1942] (Claim 1)

[1943] means for receiving a message from a user;

[1944] A means for analyzing received messages with an emotion engine and generating emotion tags;

[1945] A means for generating emotion-tagged prompts and sending them to a generative AI to generate a response; and

[1946] means for returning the generated response to the user;

[1947] A means for providing a user interface using smart glasses or a head-mounted display;

[1948] A system including:

[1949] (Claim 2)

[1950] a means for generating a prompt including character information and transmitting the prompt to the generative AI;

[1951] 2. The system according to claim 1, further comprising means for suggesting suitable products and services to the user based on the emotion tag.

[1952] (Claim 3)

[1953] 10. The system of claim 1, further comprising means for providing a user interface utilizing a web application framework. [Explanation of symbols]

[1954] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving a message from a user; A means for sending the received message to a generative AI to generate a response; means for returning the generated response to the user; A system including:

2. 2. The system of claim 1, further comprising means for generating a prompt including character information and transmitting the prompt to the generative AI.

3. The system of claim 1 further comprising means for providing a user interface utilizing a web application framework.

Images